Skip to content

Native: async create_record hands records over as columnar frames with per-record callbacks (refs #388, #315) - #393

Merged
tae898 merged 20 commits into
mainfrom
perf/388-native-async-create-batch
Nov 16, 2026
Merged

tae898 merged 20 commits into
mainfrom
perf/388-native-async-create-batch

Conversation

@tae898

@tae898 tae898 commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

What does this PR do?

On the native wheel, create_record of the async executor sent one JSON op ({type, kind, props, cb, err}) per record: one library crossing, one json.dumps and one JSON parse in the image for every record, and the same again for the next. This is rank 8 of #388.

Now create_record collects the records of one type and kind (a copy of their properties) and hands _ASYNC_BATCH (1,000) of them over at once (or after _ASYNC_LINGER, 50 ms, see below) as one columnar frame, the layout of arcadedb_n_insert_colp, through arcadedb_n_async_bin with a new op create_records. The header carries the callback id and the error callback id of every record, and the image hands record i to the async writers with its own callbacks, so each record still gets its own callback(record) and its own error_callback(error), on the writer thread, in the order of the calls. The image reads the frame with the column reader of BulkOps that insert_parallel already uses. No new library entry point.

What keeps the order and loses nothing:

  • Every other call on the executor (wait_completion, is_processing and the getters, query, command, new_edge, transaction, kill, close, append_samples) hands the collected records over first, and a change of type or kind does too. So the engine queue sees the calls in the order they were made, and wait_completion covers every create_record that returned before it.
  • The properties are copied at the call (scalars as they are, anything else with copy.deepcopy), as the JSON op copied them by encoding them. A document that the caller changes or reuses afterwards is stored as it was.
  • A shape the frame cannot carry (rows with different keys, nested lists or maps, a mixed-type column, an integer beyond 64 bits) and a batch of one keep the per-record op, unchanged. The old op is not removed for them.
  • A record that the image refuses before it is queued (the frame and the type are checked before the first record is handed over; a record that cannot be built): the call reports how many records the writers hold and the error of the next one. That record's error_callback gets the error (called from the thread that hands the batch over); without one the error is raised by the call that hands the batch over, after the rest of the batch was handed over in order. A refusal of the whole batch (an unknown type, an edge type) gives the ids back and raises. A record that a writer refuses later (a duplicate key) reaches its own error_callback on the writer, as before.

A thread of the executor, started with the first collected record and stopped by close(), hands a batch over when its first record has waited _ASYNC_LINGER (50 ms). The first native CI run showed why it is needed: the existing test_per_record_callback_runs_before_the_batch_commits_and_on_ok_after creates one record and waits for its callback with no other call on the executor, and the record sat in the Python buffer for good. With the linger time a created record waits at most 50 ms in Python, and a program that creates records and reads after a pause sees what it saw before (a read without wait_completion was a race before as well). A failure of the linger thread's hand-over has no caller to raise to, so the next call that hands records over raises it.

One behaviour that differs from the per-record op, which I want the owner to see: the refusal of a record at queue time, including an unknown type, is now reported by the call that hands the batch over (or the next one after the linger thread met it), not by the create_record call of that record.

Related issues

Refs #388 and #315.

Stacking with the draft #390: the Java change is one line in the parallel block of BulkOps.insertColumns (the hand-over of record i goes to NativeExtras.RecordHooks.submit when the error callback is one of those) and a small handColumns entry beside insertColumnsReply. #390 rewrites the head of the same function, so whichever merges second has a trivial textual conflict there (the line next to its setRow(doc, handed, table)) and one at the end of the collect_ignore list in tests/conftest.py. The from . import bulk, columnar, ... line in extras.py is the same in both.

Performance changes

Not timed. The change removes one crossing, one json.dumps and one JSON parse per record, which the tests count; the speed and memory claim waits for the pre-registered A/B.

Figure definition for the A/B (no numbers yet):

  • Operation: executor.create_record(doc, callback, error_callback) for N documents of one type on the native arcadedb wheel, then wait_completion(); one operation is one record, and the figure is the wall time of the N calls plus the wait, divided by N. Peak RSS and anon growth are information figures.
  • Sizes: N = 1 (the no-change control: a batch of one keeps the per-record op), 100, 1,000 (one full batch), 100,000. Two shapes: records with callbacks (record callback and error callback) and without (the callbacks run an upcall per record, so they set the floor), and one wide record shape (10 typed properties) beside a two-property one.
  • Parity digest: the stored records of the batch and of the per-record op (17 shapes: ints, ints beyond 2^53, floats, NaN and infinity, strings with non-BMP characters, empty strings, bools, bytes, datetimes, dates, Decimals, all-null and some-null columns, numpy scalars, nested lists and maps, key order, rows with different keys) compared through to_dict, with key order and None against a missing key; the callbacks received (one per record, with its RID, in call order with one writer); a duplicate key in the middle of a batch compared with the per-record route.
  • No-change control: the A/A of the released 26.11.1.post3 against itself, and the arm of records whose shape keeps the per-record op (different keys, nested values), which must not move.

Memory: one daemon thread per executor that has collected a record (blocked on a condition, no work between batches). Per flush the change holds the buffered property copies (at most 1,000 records), one Python list per column, then one numpy array per column (strings as an offsets array and a UTF-8 byte buffer), the address table, and the two id lists. All of it is released when the call returns; nothing grows with N. The image builds one MutableDocument per record as before, plus one callback object for a record that has a callback.

How I checked it

The tests come first in the first commit and the change in the second.

  • tests/test_native_async_create_batch_unit.py (a recorded library, no JVM and no native library): 25 cases. Run on the parent commit (26.11.1.post3 plus the test commit) 15 of the first 22 fail and 7 pass; the three linger cases failed before the linger change; the 7 that pass pin what must not change (the per-record op for shapes the frame cannot carry and for a batch of one, the ids released by wait_completion, an empty executor). All 25 pass on the change. They count crossings, pin the order against every other call, the copy of the properties, the bounded buffer, a record refused in the middle of a batch (its own callback, the rest handed over in a second batch, ids given back), the same without an error callback (raised, after the rest), and a refused batch.
  • tests/test_native_async_create_batch.py (the real library, native parity run): the stored values of a batch against the per-record op for 17 shapes, the empty and single batch, each record's own callback with its RID in call order for a document and a vertex type, a duplicate key in the middle of a batch compared with the per-record route, and an unknown type.
  • Run locally, light only: compile, pyflakes, uvx pre-commit run --files on the changed files, and Bandit as CI runs it (no issue left). The unit file ran as plain Python (the queue guard blocks the test runner on the laptop). Not run locally: the Java compile and the native tests, which need the native image build in CI.

CI: the first native run (89c44d9) built the image and failed 10 of 25 jobs, all on two files: the existing test_per_record_callback_runs_before_the_batch_commits_and_on_ok_after (a real consequence of the buffering, fixed by the linger thread, not by changing the test) and my parity tests, which closed the database's async executor between the two phases (close() shuts it down for good, a test bug, fixed). A second run is pending on the fix.

Docs

The docstring of NativeAsyncExecutor.create_record describes the buffering, the flush points and the queue-time refusal. No page under docs/ or the README describes the async executor's batching, so none changes.

Checklist

  • A regression test fails on the old code and passes on the new (the unit file, run on the parent: 15 fail)
  • The test suite that the change can affect passes (CI, native run: a second run is pending)
  • uvx pre-commit run --files <changed files> passes

🤖 Generated with Claude Code

tae898 and others added 11 commits November 14, 2026 07:34
…ject_hook and without reading its side array (refs #315)

to_list() and iter_dicts() of the JVM wheel build a json object_hook for every batch and read the
side array of every batch, whether or not the batch holds a tagged value. These tests count the hook
factory, the object_hook argument of json.loads and the batch elements that are read, for a result of
plain values (no hook, no read of the side array) and for every kind of value the bridge tags (the
hook stays). They also compare to_list() and iter_dicts() with the decoding that always builds the
hook, over single shapes, a mixed result and batch boundaries.

They fail on the current decoder (every batch gets a hook and a read of the side array) and pass once
the decoder looks for the tag escape in the batch text first.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…s text without an object_hook and without reading its side array (refs #315)

Every batch was decoded with json.loads(text, object_hook=_typed_hook(side)) and its side array was
read first, also when the batch held only plain values. The hook is a Python call per JSON object and
the read of the side array is a JPype operation per batch, which for iter_dicts (one row per crossing)
is one per row.

TypedRows writes every tag as {"\u0001": [kind, payload]} with the escape spelled out, writes a control
character of a string as the same escape, and never writes the raw character, and every side array
entry is referred to by a tag in the text. A text without the escape therefore holds no tag and no side
index, the hook would return each object unchanged, and plain json.loads gives the same value. The
check is a substring scan of a str that already exists (_loads_typed_row did it for one row). A string
that holds U+0001 or the text of the escape only keeps the hook, which then changes nothing.

No Java change, and the native wheel is untouched. The speed effect is not measured: it waits for a
timed A/B of a candidate wheel against 26.11.1.post2.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…taframe on the JVM wheel (refs #315)

Round trips of every Arrow temporal type through a real load on both wheels (every timestamp unit with and without a zone, to the nanosecond; date32 and date64 before 1970; nulls; batches; vertex and parallel loads; pandas and Polars frames), and JVM-only tests that count what one load does: no java.time call and no convert_python_to_java per cell, and each timestamp or date column reaching the bridge as one TemporalColumn per batch. The counting tests fail on the code before, which builds one LocalDateTime or LocalDate per cell in Python and converts every element again.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…te column crosses as one long[] of epoch ticks, not one JPype call per cell (refs #315)

_temporal_to_java made one LocalDateTime.ofEpochSecond or LocalDate.ofEpochDay call per cell, and insert_columns converted every element of that list again with convert_python_to_java: a million-row timestamp column was two million JPype calls. The column is now the integer epoch values in one long[] (plus one boolean[] when a row is null) wrapped in the new bridge class TemporalColumn (dateTimes(long[] ticks, long ticksPerSecond, boolean[] nulls) and dates(long[] ticks, long ticksPerDay, boolean[] nulls)), and DocumentBatcher builds each LocalDateTime or LocalDate as it fills the document of that row, as StringColumn does for text. The reading rules are the ones of the code before: UTC wall clock, a zoned timestamp is already UTC epoch time, a naive one is the clock it shows, the unit's precision is kept (a nanosecond tick is not cut to microseconds), a tick before the epoch floors, date64 is milliseconds divided by 86,400,000 with a floor. The factory checks the smallest and the largest tick, so a value past the years of java.time still raises before a row is written. The per-cell path is retired: the wheel carries its own bridge jar, so no fallback for an older one. Not measured: the speed claim waits for a timed A/B against 26.11.1.post2.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ly text, no free and no JSON decode (refs #315)

arcadedb_n_tx returns the transaction state as one integer (bit 0 active, bits 1 to 31 depth, bits 32 to 62 commits, -1 for an
error whose text lib.last_error holds), as arcadedb_n_cmd returns a cursor id. These tests pin that and that an error still
reaches Python as the same typed exception with the same message:

- with db.transaction() is three foreign calls (mark, begin, commit), all arcadedb_n_tx: no strlen, no copy into a str, no
  arcadedb_free (was twelve);
- begin, commit, rollback, is_transaction_active and the mark each make exactly one foreign call;
- no JSON decode in a transaction, not even for the mark that reads active and depth;
- the layout of the integer, -1 for an unknown handle with the error in last_error, commit without begin, rollback with nothing to
  roll back, a failed statement in a block, nested with blocks, an injected failure on each entry point, a database closed with a
  transaction open.

They fail on the old code: lib.tx does not exist there, and the foreign-call and decode counts are higher.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…not a JSON text (refs #315)

A begin, a commit, a rollback and a mark each crossed into the library and got back a malloc'd JSON text of about 60 bytes that
Python read with strlen, a copy into a str and arcadedb_free, and mostly never looked at. with db.transaction() is three crossings
(the mark, begin, commit), so twelve foreign calls and three strings built, copied and freed.

arcadedb_n_tx now returns a long: bit 0 active, bits 1 to 31 depth, bits 32 to 62 the low 31 bits of the commit count, -1 for an
error. It is the way arcadedb_n_cmd returns a cursor id: the error text is kept per thread and read with arcadedb_n_last_error, built
by the same errorJson as before, so the typed exception and its message do not change. Each crossing is one foreign call, and
with db.transaction() is three. The Python side reads the fields of the integer inline, so the mark, the begin and rollback states
the unwind and run_in_transaction read, and is_transaction_active no longer decode anything.

Not measured: the speed claim waits for a timed A/B of a candidate wheel against 26.11.1.post2.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ossings, not three (refs #317)

The transaction check of a saved record's save() cannot refuse while auto-transaction is on, yet it
still crossed into the library (arcadedb_n_tx) before the SELECT and the UPDATE. These tests pin two
crossings there, bare and inside a transaction, with and without the read-back of the row, and pin
what must not move: the check is back the moment auto-transaction is off (first, before any read),
a stale record is refused under auto-transaction too, and the stored values and the next save's
baseline are those of an update inside a transaction for NULL, float, text, date-time, date,
Decimal, nested values, a float32 array and a value set unchanged.

The two-crossing tests fail on the code before the change (three crossings); the rest pass on both.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…te when auto-transaction is on (refs #317)

_save_update asked is_transaction_active() (an arcadedb_n_tx crossing) first and only then looked at
the auto-transaction flag, so the answer was thrown away whenever the flag was on. The operands are
swapped: the flag is a Python attribute, the state is a crossing. With auto-transaction on, an update
of a saved record is two crossings (the SELECT of the stale-record check and the UPDATE), not three.
With it off nothing changes: the check still comes first, outside a transaction and inside one.

The check is not replaced by the engine's refusal for the default mode: the engine runs the
before-update callbacks and the property validation before it looks for a transaction (read in
LocalDatabase.updateRecord of the 26.12.1-SNAPSHOT source), and an UPDATE of a record that is gone
raises nothing, so a save outside a transaction would run callbacks, or pass silently, instead of
raising "Transaction not begun" before it reads. A Python-side record of the transaction state is not
exact either: a SQL COMMIT or ROLLBACK statement ends the transaction without passing the wheel's
transaction calls.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… no failed assert leaves a transaction open (refs #315)

CI of the first push (run 43478436168) failed two tests on a wrong assumption of mine: I expected the outer begin to raise the depth
by one. LocalDatabase.getNestedTransactions() is the size of the thread's stack of transaction contexts
(DatabaseContext.DatabaseContextTL.transactions). The stack always holds the thread's base context once the thread has used the
database, so the depth is 1 while idle. The outer begin reuses that base context (depth 1, active). Only a begin while a transaction
is active pushes a context (2), the commit or rollback of that level pops it (1, still active), and the outermost end leaves the base
context on the stack (1, idle). popIfNotLastTransaction never removes the last one. The native tx entry point returns exactly that
number, as the old JSON reply did, so the change is right and the two expectations were not.

The fixture now rolls back whatever a test left open before it drops the database, so a failed assert is one failure and not also a
"Cannot drop the database in transaction" teardown error. The close-and-reopen test does the same in a finally.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ach record keeps its own callbacks (refs #388, #315)

The records of one type cross as one columnar frame with the callback ids of each record, not one JSON op per record. The recorded-library
tests count crossings and pin the order, the copy of the properties at the call, the shapes that keep the per-record op and a record that
is refused in the middle of a batch; the native tests compare the stored values with the per-record op and run the real callbacks.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…nar frame with per-record callbacks (refs #388, #315)

create_record collects the records of one type and kind (a copy of their properties) and hands _ASYNC_BATCH of them over at once through
arcadedb_n_async_bin, op create_records, with the callback ids of each record. Every other call on the executor, wait_completion, close and
kill hand the collected records over first, so the engine queue keeps the order of the calls. A shape the frame cannot carry and a batch of
one keep the per-record op. A record that is refused before it is queued reaches its own error callback, or is raised by the call that
hands the batch over, after the records around it were handed over.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@tae898 tae898 added python Pull requests that update python code performance Speed or memory of the bindings, with a measurement native The arcadedb package (native library build) labels Nov 14, 2026
@tae898 tae898 self-assigned this Nov 14, 2026
tae898 and others added 2 commits November 14, 2026 15:51
…ssing numpy fails and never skips (refs #388)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…atements, so Bandit B608 has nothing to flag (refs #388)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
tae898 and others added 6 commits November 14, 2026 17:22
…nc executor between phases, and a collected record is handed over without another call (refs #388)

CI of 89c44d9: close() shuts the engine's async executor down for good, so the second phase of each parity test failed with 'Async executor
has been shut down'; DELETE needs a transaction. The new linger tests pin that a record created with no other call on the executor reaches
the library after the linger time, that a failure of that hand-over is raised by the next call, and that close() stops the thread.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…other call on the executor (refs #388)

test_async_executor.py::test_per_record_callback_runs_before_the_batch_commits_and_on_ok_after creates one record and waits for its callback,
with no other call on the executor, so the record sat in the Python buffer for ever. A thread of the executor (started with the first collected
record, stopped by close()) hands a batch over when its first record has waited _ASYNC_LINGER; a failure of that hand-over is raised by the
next call that hands records over.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… values, not the order of the properties, and the refused records are proved by what is stored (refs #388)

The parity run on the installed wheel failed 16 of 21 tests of test_native_async_create_batch.py on every platform. The values of the batch and of the per-record op were equal in all 14 shapes that were compared; two facts of the real library that the test had guessed wrong:

- The per-record op parses the properties of a record into a hash map (Json.java), so it stores them in hash order ({v, i} for the call {i, v}), and the batch stores them in the order of the call. The order of the properties of a document is no promise, so the dump sorts the keys of each row.
- The writer reports a record stored once it is in the transaction; a refusal by a unique index comes at the commit, after that. The records 4 and 7 of the refusal test reach their callback and then their error callback, on both routes (the equal results say so). The test now needs the callback of every stored record and checks the stored (key, index) pairs.

A record with numpy scalars is not a shape the column reader carries (Python scalars only, as for insert_many), so it crosses as the per-record op; the test lists it with the other shapes without a frame and the docstring of create_record names it. The shape key_order is renamed unsorted_keys.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…e op each, as in post3 (refs #388, #401)

The laptop A/B of this pull request measured a single record 1.57 times slower than post3 (n_async_nar_1) and batches of 100 records and more faster. Collecting has a fixed cost per executor (a linger thread, the locks, the copy of the properties, then the columnar frame) that a small load does not repay. An untimed micro of one executor call (a new executor, N records, wait_completion) on the wheel of this branch against post3, native, measured: slower for 1, 4, 8 and 16 records (1.6x, 2.7x, 1.7x, 1.1x), faster for 32, 64 and 100 records (0.74, 0.54, 0.42); the crossover is near 20 records.

The first _ASYNC_DIRECT (32) records of an executor now take the post3 path exactly: no copy, no buffer, no linger thread, no frame. From the 33rd record on the records are collected and handed over as frames, as before. An executor of up to 32 records cannot be slower than post3; the pre-registered figures n_async_nar_1 and n_async_nar_100 verify the change. A fallback at flush time was considered and not used: it would not remove the linger thread and the copy, which are most of the cost of a small load.

Tests: the existing tests pin the collecting route with the direct phase off; new tests pin that 1, 2, 31 and 32 records send one op each and collect nothing, that 33, 34, 64 and 100 keep every record and their order, that 0 turns it off, and the default; the native parity test stores the same rows with the default and with the collecting route at 1, 31, 32, 33, 64 and 100 records and compares every stored record. The new tests fail when the direct branch is removed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…cords goes one op each, as in post3 (refs #388, #401)

The untimed micro of the first fix head (the first 32 records of an executor one op each) read executors of 33 to 64 records 1.2x to 1.7x slower than post3: a columnar frame costs about 270 us per crossing, a record in a frame about 3 us, a per-record op about 12 us. A frame repays itself from about 30 records; the decided floor is 64 collected records at the hand-over, below it the post3 op per record.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…e op each, as in post3 (refs #388, #393, #401)

The first fix head collected from the 33rd record and read 1.2x to 1.7x slower than post3 for executors of 33 to 64 records; the frame floor of 64 left 1.2x to 1.3x. The direct phase is now 96 records, so an executor of up to 96 records is exactly post3 and the frame (from 64 collected records) serves executors beyond about 160 records. An executor of 97 to 159 records still pays the per-executor collection cost without a frame (1.1x to 1.2x slower in the untimed micro); a linger thread shared by the whole process would remove it and is a follow-up.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ne op each, as in post3 (refs #388, #393, #401, #406)

A direct phase of 96 put the band of the collection cost on the 100-record case: the untimed micro read 1.19 at 100 records, 1.12 to 1.19 for 97 to 159. The cost of the band is a near-constant amount per executor, about 0.2 ms, so its relative size falls as the direct phase grows (about 20 percent at 96, about 15 percent at 128). The band is not removed, it moves to 129 to 191 records (the frame starts at 128 + 64 = 192); a linger thread shared by the whole process is the real fix (#406).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@tae898

tae898 commented Nov 15, 2026

Copy link
Copy Markdown
Collaborator Author

I raised the direct phase to 128 records (_ASYNC_DIRECT, head b77f887) and kept the frame floor at 64 collected records (_ASYNC_FRAME_MIN), so a frame appears from 192 records. An executor of up to 128 records takes the 26.11.1.post3 path: one op per record, no copy, no buffer, no linger thread.

Why 128: a direct phase of 96 put a slow band on the 100-record case of the laptop A/B (the untimed micro read 1.19 at 100 records against post3). I expect the extra cost of the band to be about a fixed amount per executor (the few collected records go one op each after a fixed collection cost), so its relative size should fall as the direct phase grows. I did not pick a larger number and I did not tune it to the sizes of the study.

The band is not removed, it moved to 129 to 191 records. In the untimed micro (two sessions) an executor there takes 1.11x to 1.19x the time of post3, 198 to 388 us more. From 192 records the executor is faster than post3 (0.95 at 192, 0.73 at 256, 0.36 at 1000). The band lies outside the four sizes of the study (1, 100, 1k and 100k), so the study does not measure it, and the release notes will say so. The micro is too noisy to show the relative size falling with the threshold.

The real fix is one linger thread shared by the whole process, tracked in #406 with the micro table.

@tae898
tae898 merged commit 43e1e52 into main Nov 16, 2026
27 checks passed
@tae898
tae898 deleted the perf/388-native-async-create-batch branch November 16, 2026 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

native The arcadedb package (native library build) performance Speed or memory of the bindings, with a measurement python Pull requests that update python code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant