Skip to content

gen-orchestrator: emit the per-node bookkeeping once, not per node - #15

Merged
nicolas-maman merged 1 commit into
mainfrom
fix/orchestrator-token-budget
Sep 18, 2026
Merged

nicolas-maman merged 1 commit into
mainfrom
fix/orchestrator-token-budget

Conversation

@nicolas-maman

Copy link
Copy Markdown
Collaborator

Why

aether-lang-dev/aether-ui#140 adds one .build.ae (tests/min_size) and the whole fan-out fails on both CI legs with:

error: source file exceeds maximum token limit (50000 tokens)
aeb-link: FATAL — failed to compile the fan-out orchestrator

No file is named because the file is aeb's own generated _orchestrator.ae. gen-orchestrator emitted ~360 tokens of bookkeeping inline per node (selector gate, rc capture, status, the telemetry record). Measured with the aetherc lexer over aether-ui's node list: main = 49,412 tokens, #140 = 49,770, against aetherc's 50,000-token cap on a main file (modules get 100,000). The tree was one node from the wall.

What

The bookkeeping is emitted once as two helpers, _aeb_selected(sel, label) and _aeb_record(s, root, records, label, type_word, nrc, start); each node becomes:

if _aeb_selected(_sel, "apps/aevg_anim") == 1 {
    _start = clock_ns()
    _nrc = apps_aevg_anim__D_build_D_ae(s)
    _aeb_record(s, _root, _records, "apps/aevg_anim", "build", _nrc, _start)
}

The extern call stays inline because a fn passed as a value from another TU is DCE'd (aether #2037). Same 138 nodes: 49,770 → 7,227 tokens. Semantics unchanged: same selector gate, same rc→_mark_failed, same record fields, same TU so bldr's cloned statics resolve.

Verified

  • Regenerated aether-ui's orchestrator (138 nodes) with the new tool; aetherc --check --lib lib accepts it with the same two bldr-internal warnings as the old one.
  • tests/test_gen_orchestrator_budget.ae: generates a 500-node orchestrator (~180k tokens under the old scheme) and requires ae check to accept it against lib/bldr, plus counts that the helpers appear once and every node goes through them. Passes with the new generator, fails 4/4 against the old one.

Follow-up for aether-ui: bump AEB_REF once this is released.

A fan-out of ~140 nodes could not build: the generated orchestrator
carried ~360 tokens of inline bookkeeping per node (selector gate, rc
capture, status, telemetry record), so aether-ui's 138-node tree sat at
49,800 tokens and one more .build.ae tripped aetherc's 50,000-token cap
on a main file — "source file exceeds maximum token limit" with no file
named (aether-ui #140).

The bookkeeping is now two helpers emitted once (_aeb_selected,
_aeb_record); each node is a guarded extern call plus one record call,
~35 tokens. Same 138 nodes: 49,770 → 7,227 tokens. Semantics unchanged.

tests/test_gen_orchestrator_budget.ae generates a 500-node orchestrator
(~180k tokens under the old scheme) and requires `ae check` to accept it
against lib/bldr; it fails 4/4 against the previous generator.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant