Skip to content

Native: later result batches carry vectors in the float32 tail (arcadedb_n_next_bin) (draft, stacked on #520) - #521

Closed
tae898 wants to merge 4 commits into
native-previewfrom
native-next-float-tail
Closed

tae898 wants to merge 4 commits into
native-previewfrom
native-next-float-tail

Conversation

@tae898

@tae898 tae898 commented Oct 25, 2026 •

Copy link
Copy Markdown

DRAFT. Built into one image and measured untimed (section Measured below: wide SELECTs about 2x faster, decoded rows bit-identical to the JVM wheel); a timed ab.py A/B is queued. Do not merge before it.

STACKED ON #520: the vector-tail code in ArcadeNative.replyOf is the block #520 changes to a bulk copy, so this branch starts from native-result-copy; its first two commits are #520's. Review the last two commits (test, implementation); when #520 merges this PR needs only a rebase.

Problem (found by the study in .notes/bench/repros/native-result-transport-20261025, local notes repo): arcadedb_n_exec with flag 4 sends float[] values of 8 or more elements in a binary float32 tail, but only for the first reply (256 rows). arcadedb_n_next (every later batch, 2048 rows) has no sink, so an embedding column comes back as JSON number text from row 257 on, and json.loads with the object_hook is about half of the wall time of a 10k-100k row SELECT with a 32-float vector (untimed: 77 of 159 ms, 873 of 1,680 ms).

Change:

  • New entry point arcadedb_n_next_bin(rs, max): the same batch as arcadedb_n_next, encoded with the float sink. The tail code of replyOf is one helper, withFloatTail, used by both. arcadedb_n_next is unchanged, so an older Python on this image keeps working.
  • lib.py: NEXT_BIN (on when the library has the entry point; ARCADEDB_NATIVE_NEXT_BIN=0 turns it off for A/B runs in one image). NativeResultSet._fill reads batches with lib.call_bin("arcadedb_n_next_bin", ...) and loads(text, floats) when NEXT_BIN and the binary reply are on, else as before.
  • Tests (written first, commit 3; they pass against the current library and must pass against the new image): more than 256 rows with an ARRAY_OF_FLOATS column, compared bit for bit (struct.pack('<d'), so -0.0 differs from 0.0) on the first and on later batches; vectors of 0, 3, 8 and 32 floats, float32 extremes (1e-45, 3.4028235e38, +/-inf), and rows with no vector at all; the same read through the row-by-row cursor.

The untimed numbers are in the Measured section below (notes folder native-next-float-tail-20261025). Still to come: the timed A/B.

Measured (2026-10-25, untimed, laptop, 2 runs per arm, one image built from this PR's tree)

Untimed: these are stage clocks from a probe, not an ab.py A/B; a timed A/B is queued before any merge.

Wide SELECT with a 32-float vector, microseconds per warm query (mean of runs; pass B stage clocks)

query arm runs wall A wall B foreign calls (exec + next) string_at + tail + decode json.loads (hook) reply KB of which float tail KB loads isolated (E) columnar floor (E)
wide10k_proj unmodified wheel 2 307,101 247,475 119,256 2,287 120,198 6,888 32 99,266 22,827
wide10k_proj new image, NEXT_BIN=0 (text) 2 243,903 245,626 115,444 2,203 122,293 6,888 32 104,042 24,324
wide10k_proj new image, NEXT_BIN on (tail) 2 127,427 124,559 61,891 809 55,878 2,465 1,250 43,257 24,508
wide100k_proj unmodified wheel 2 2,543,688 2,507,009 1,119,369 20,682 1,312,760 70,020 32 1,354,133 551,104
wide100k_proj new image, NEXT_BIN=0 (text) 2 2,849,452 2,738,727 1,214,425 24,465 1,439,988 70,020 32 1,458,806 574,336
wide100k_proj new image, NEXT_BIN on (tail) 2 1,518,897 1,316,007 582,464 7,347 668,941 24,749 12,500 731,433 681,737

sha256 of decoded rows (first 20 hex), floats compared as the 8 bytes of the double

query rows before off on jvm all equal
rows10k 10000 fdd8079c1ded6725d711 fdd8079c1ded6725d711 fdd8079c1ded6725d711 fdd8079c1ded6725d711 yes
rows100k 100004 4ec9f5396649225fdb4e 4ec9f5396649225fdb4e 4ec9f5396649225fdb4e 4ec9f5396649225fdb4e yes
tail_rows 14 e2a8bc86abe0d844f2bd e2a8bc86abe0d844f2bd e2a8bc86abe0d844f2bd e2a8bc86abe0d844f2bd yes

🤖 Generated with Claude Code

tae898 and others added 4 commits October 25, 2026 15:32
…op turn

out() and the float32-tail reply wrote the reply with p.write(i, b[i]) in a Java loop. Both now fill a direct ByteBuffer view of the
malloc'd block (Unsafe.copyMemory); the bytes written are the same. New test: replies of one, several and many batches, multi-byte text,
megabyte rows and vectors round-trip exactly.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The previous commit widened off + 4 * sink.n to long, which UnmanagedMemory.malloc does not accept (found by the first image build).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…hort, empty and missing vectors

Passes against the current library (vectors of later batches travel as number text); it must keep passing when arcadedb_n_next carries them in a float32 tail.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…e float32 tail

arcadedb_n_exec flag 4 already sends float[] values of 8 or more elements in a binary tail, but only for the first reply (256 rows). Every
later batch (arcadedb_n_next, 2048 rows) sent them as number text, which Python parses with json.loads. New entry point arcadedb_n_next_bin
encodes a batch with the same sink; the tail code of replyOf is now one helper (withFloatTail) used by both. lib.py uses it when the library has it
(ARCADEDB_NATIVE_NEXT_BIN=0 and ARCADEDB_NATIVE_BIN_REPLY=0 turn it off); arcadedb_n_next is unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@tae898

tae898 commented Oct 26, 2026

Copy link
Copy Markdown
Author

Closing here: the bindings moved to ArcadeData/arcadedb-python, and this change is merged there as ArcadeData/arcadedb-python#76 (later result batches carry vectors in the float32 tail). The branch stays.

@tae898 tae898 closed this Oct 26, 2026
@tae898
tae898 deleted the native-next-float-tail branch November 8, 2026 11:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant