Skip to content

Add planner-backed devtools simulation - #170

Merged
SandroMaglione merged 3 commits into
mainfrom
codex/planner-simulation
Aug 24, 2026
Merged

Add planner-backed devtools simulation#170
SandroMaglione merged 3 commits into
mainfrom
codex/planner-simulation

Conversation

@SandroMaglione

Copy link
Copy Markdown
Member

Summary

  • Add isolated planner-backed simulation sessions to the interactive text-tree visualizer.
  • Generate machine and event controls from Effect schemas, including visible field metadata and inline validation errors.
  • Show state changes, microsteps, selected branches, raised and emitted events, planned commands, completion, and output without committing side effects.

Changeset

  • Added or updated for a library or package-metadata change
  • Not required because this PR does not change a publishable package

Validation

  • pnpm check
  • Automated type-performance measurement passed or was not required
  • Automated runtime- and memory-performance measurement passed or was not required

Local type and runtime performance smoke tests passed. The schema-derived forms, authoritative field errors, and Idle -> Finished planner trace were also verified in the local browser.

@github-actions

Copy link
Copy Markdown
Contributor

Type performance

Measured with TypeScript 6.0.3 and skipLibCheck=true.

Scenario Base PR Difference
Effect only 55 55 0 (0.0%)
Import effect-machine 55 55 0 (0.0%)
Machine.states (3 states) 3,039 3,039 0 (0.0%)
Machine.make (3 states, 2 events) 10,787 10,787 0 (0.0%)
machine.handle (3 states, 2 transitions) 28,551 28,551 0 (0.0%)
fluent transition (10 named branches) 122,775 122,775 0 (0.0%)
fluent invocation (state-dependent Effect) 108,429 108,429 0 (0.0%)
machine.handle (depth 24) 207,970 207,970 0 (0.0%)
machine.handle (wide depth 16) 236,332 236,332 0 (0.0%)
machine.handle (parallel/history/choice) 144,812 144,812 0 (0.0%)
machine definition (3 independent implementations) 143,617 143,617 0 (0.0%)
machine exact input/output/error/services 129,873 129,873 0 (0.0%)
execution adapter readiness 136,469 136,469 0 (0.0%)

Marginal instantiations are measured against the matching setup without that API call:

Scenario Base PR Difference
Import effect-machine 0 0 0
Machine.states (3 states) 2,984 2,984 0 (0.0%)
Machine.make (3 states, 2 events) 7,740 7,740 0 (0.0%)
machine.handle (3 states, 2 transitions) 17,764 17,764 0 (0.0%)
fluent transition (10 named branches) 112,498 112,498 0 (0.0%)
fluent invocation (state-dependent Effect) 99,018 99,018 0 (0.0%)
machine.handle (depth 24) 187,474 187,474 0 (0.0%)
machine.handle (wide depth 16) 217,043 217,043 0 (0.0%)
machine.handle (parallel/history/choice) 124,167 124,167 0 (0.0%)
machine definition (3 independent implementations) 127,576 127,576 0 (0.0%)
machine exact input/output/error/services 115,217 115,217 0 (0.0%)
execution adapter readiness 106,299 106,299 0 (0.0%)
Check times (informational)
Scenario Base PR
Effect only 0.02s 0.02s
Import effect-machine 0.02s 0.02s
Machine.states (3 states) 0.08s 0.08s
Machine.make (3 states, 2 events) 0.13s 0.13s
machine.handle (3 states, 2 transitions) 0.21s 0.21s
fluent transition (10 named branches) 0.45s 0.46s
fluent invocation (state-dependent Effect) 0.46s 0.44s
machine.handle (depth 24) 0.65s 0.64s
machine.handle (wide depth 16) 0.67s 0.65s
machine.handle (parallel/history/choice) 0.54s 0.55s
machine definition (3 independent implementations) 0.53s 0.57s
machine exact input/output/error/services 0.50s 0.52s
execution adapter readiness 0.51s 0.51s

Type instantiations are the comparison metric. Check time varies with runner load and is informational only.

@github-actions

Copy link
Copy Markdown
Contributor

Runtime performance

Median of 5 independent benchmark processes on AMD EPYC 7763 64-Core Processor with Node v24.19.0.

Pull request baseline

Scenario Effect Machine
Plan counter transitions 110,768 transitions/s
Drain burst with terminal fence 334,163 increments/s
Drain burst with a change observer 313,626 increments/s
Lookup and send to one child 280,209 increments/s
Start and stop a machine 135,980 machines/s
Start and stop a parent with one child 27,780 families/s
Plan transitions through a compound state 95,350 transitions/s
Plan transitions through parallel regions 78,685 transitions/s
Drain burst through a compound state 279,692 events/s
Drain burst through two parallel regions 280,942 events/s
Drain a compound-state burst with a change observer 266,606 events/s

Process runtime reference points

Scenario Effect Machine
Start and stop a raw generic process 15,434 processes/s
Start and stop a raw compiled process 61,843 processes/s
Memory profile Effect Machine
Idle machine 1.8 KiB
Raw generic managed process 13.9 KiB
Raw compiled process 3.2 KiB
Two independent idle machines 3.4 KiB
Idle parent with one child 5.8 KiB
Parent with observed child registry 10.0 KiB
Parent with observed invoked child snapshots 6.3 KiB

Effect Machine change from base

Metric Base Base variability PR PR variability Difference
Plan counter transitions 112,547 transitions/s 2.0% MAD 110,768 transitions/s 2.1% MAD -1.6%
Drain burst with terminal fence 338,966 increments/s 0.4% MAD 334,163 increments/s 2.8% MAD -1.4%
Drain burst with a change observer 320,181 increments/s 0.2% MAD 313,626 increments/s 3.0% MAD -2.0%
Lookup and send to one child 283,802 increments/s 0.4% MAD 280,209 increments/s 2.9% MAD -1.3%
Start and stop a machine 134,517 machines/s 0.7% MAD 135,980 machines/s 1.1% MAD +1.1%
Start and stop a parent with one child 28,276 families/s 2.6% MAD 27,780 families/s 1.0% MAD -1.8%
Plan transitions through a compound state 96,483 transitions/s 1.3% MAD 95,350 transitions/s 2.3% MAD -1.2%
Plan transitions through parallel regions 78,838 transitions/s 2.9% MAD 78,685 transitions/s 2.6% MAD -0.2%
Drain burst through a compound state 286,306 events/s 0.7% MAD 279,692 events/s 0.3% MAD -2.3%
Drain burst through two parallel regions 286,780 events/s 1.5% MAD 280,942 events/s 1.0% MAD -2.0%
Drain a compound-state burst with a change observer 272,815 events/s 0.9% MAD 266,606 events/s 1.5% MAD -2.3%
Idle machine heap per unit 1.8 KiB 0.1% MAD 1.8 KiB 0.0% MAD -0.2%
Raw generic managed process heap per unit 13.9 KiB 0.0% MAD 13.9 KiB 0.0% MAD -0.0%
Raw compiled process heap per unit 3.1 KiB 0.4% MAD 3.2 KiB 0.9% MAD +0.6%
Two independent idle machines heap per unit 3.4 KiB 0.0% MAD 3.4 KiB 0.0% MAD -0.0%
Idle parent with one child heap per unit 5.8 KiB 0.0% MAD 5.8 KiB 0.0% MAD +0.0%
Parent with observed child registry heap per unit 10.0 KiB 0.0% MAD 10.0 KiB 0.0% MAD -0.0%
Parent with observed invoked child snapshots heap per unit 6.3 KiB 0.0% MAD 6.3 KiB 0.0% MAD +0.0%

Process runtime reference change from base

Metric Base Base variability PR PR variability Difference
Start and stop a raw generic process 15,970 processes/s 0.7% MAD 15,434 processes/s 1.5% MAD -3.4%
Start and stop a raw compiled process 62,267 processes/s 0.6% MAD 61,843 processes/s 0.8% MAD -0.7%

Regression guard

No large, noise-adjusted throughput or heap regressions detected.

Versions and interpretation
  • Effect Machine: 0.23.0

Higher throughput is better; lower heap is better. Variability is the median absolute deviation across independent processes, relative to their median. Small differences on shared GitHub-hosted hardware remain informational; the required guard rejects only large changes beyond the measured noise allowance.

@SandroMaglione
SandroMaglione merged commit eda432c into main Aug 24, 2026
8 checks passed
@SandroMaglione
SandroMaglione deleted the codex/planner-simulation branch August 24, 2026 10:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant