chore(benchmarks): regenerate public node/bun baseline at b77aba6343 - #11073
proggeramlug wants to merge 1 commit into
Conversation
Measure b77aba6 (v0.5.1639) in a fresh clone on perry-macos.local: Apple M1, arm64, 8 CPUs, 8 GiB RAM, macOS 26.5.1 / Darwin 25.5.0, AC power. Use Node v22.23.1, Bun 1.3.14, esbuild 0.28.1 and Zig 0.15.2. Run the original public-baseline entry point uninterrupted, without build/profile overrides, and retain all raw measurements and losing rows. Preflight: 13 CPU-active readings over 60 seconds, 1.37%-1.81%. Per-component quiet confirmations: 1.4%, 1.5%, 1.5%, 1.7%, 1.5%; each requires CPU active <=25% for 60 consecutive seconds. Validate 50 workloads (24+9+2+12+3), 1443 positive raw timing samples across Perry/Node/Bun, and 5/11/11/15/20 samples per workload/peer. All correctness checks pass; both freshness checkers and all 15 public baseline tests pass. Assembly completed 2026-09-23T01:34:16Z. Preserve warn-only historical regressions: loop overhead +28.9%, object creation +300.0%, binary trees +333.3%. Optional auxiliary Go produced no measurements; it is not part of the public artifact. Commit only the artifact, generated README section, and suite RESULTS. Claude-Session: https://claude.ai/code/session_01EQdCw7BN4AAnn2hAbNXg33
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (3)
Included review availability: Your plan provides up to 8 included reviews per hour; 6 remain after this review. 📝 WalkthroughWalkthroughThe pull request refreshes benchmark measurements in the README and results document. It updates the evidence commit, Perry version, median timings, result classifications, and summary counts. ChangesBenchmark results refresh
Priority: ⬇️ Low Estimated code review effort: 1 (Trivial) | ~3 minutes Change: Other Merge Risk: ⚪ Minimal · up to No actionable merge-blocking risk was identified in the benchmark refresh. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Superseded by #11075, which landed the same artifact rebuilt on current main (v0.5.1640) so it validates on the tree it merges into — your branch predated train 257. Your measurement run is credited in the commit message with its provenance: one uninterrupted run at b77aba6 on the quiet mini, node v22.23.1 / bun 1.3.14 exact, esbuild 0.28.1, Zig 0.15.2, quiet gate re-checked per component, 50 workloads and 1,443 samples, all correctness passed. lint's public-baseline step is now GREEN for the first time since 2026-09-01. |
Regenerate the public Perry/Node/Bun baseline at
b77aba63433b100b9482d47181b5ccc5e724ff81(v0.5.1639) so thelintpublic-baseline freshness step has conforming evidence for the root manifest after #10354 (turnloop). #10977 fingerprints the root manifest except its workspace release version; a real manifest change requires measurement.This PR contains only the freshly generated
benchmarks/results/public-node-bun-v1.json, the generated README section, andbenchmarks/suite/results/RESULTS.md. No timings, hashes, benchmark sources, correctness oracles, sample counts, or thresholds were edited. The three files match the mini's generated files byte for byte. README content outside the generated markers is unchanged.Measurement:
perry-macos.local(perry-macos.fritz.box), Apple M1 / arm64, 8 CPUs, 8 GiB RAM, macOS 26.5.1 / Darwin 25.5.0, AC power, initially 32 GiB disk free.targetdirectory. Original./benchmarks/run_public_baseline.shran detached and uninterrupted, exiting 0. It started at2026-09-22T23:55:39Z; artifact assembly completed at2026-09-23T01:34:16Z. Fresh local clone and PR branch also start at that pinned commit.~/nodebinwas untouched. Both polyglot banners confirmed untimed esbuild precompilation to.mjs; all nine compute-polyglot.mjsfiles and required Perry executables were freshly generated. Rust/Cargo: nightly 1.100.0, toolchain dated 2026-08-20. No inheritedCARGO_PROFILE_*,RUSTFLAGS, encoded Rust flags,CARGO_TARGET_DIR, orPERRY_*overrides.taskpolicyscheduler hint, not hard CPU affinity.All 50 workloads / 1,443 positive raw timing samples passed correctness and validation, with all components naming the measured commit:
Honest-bench additionally measured Rust and Zig: all 300 total rows passed the strict Bun-reference check. App patterns retained 3 warmups and honest-bench retained 5 warmups.
Verification:
python3 benchmarks/ci_public_baseline_check.py— pass (the original artifact failed withbenchmark inputs changedbefore regeneration).python3 benchmarks/public_baseline.py check— pass, including generated documentation.PYTHONPATH=. python3 tests/test_public_baseline.py— 15 tests pass.git diff --checkandbash scripts/check_file_size.sh— pass.9c87723d7cedca511b1dba158fdd505bdbabd667367ee41152f82370c9ceeae5; harness fingerprint:513dba8ff9eaf931edc8a5fc01a0093a1156c31b0b6c9cc1218f710405e315e0.311cd8efa1865e8bc700cd476843f0535f2c76d42f0fe19d60bb9886e8ad78c7.Performance notes retained without suppressing losing rows:
--warn-onlyregressions: loop overhead +28.9% (RAM −3.8%, noise allowance 1.0 ms), object creation +300.0% (RAM +34.7%, noise 1.0 ms), binary trees +333.3% (RAM +33.9%, noise 4.4 ms). The harness explicitly reported the comparison failure without failing the build. These are historical comparisons, not freshness failures or a controlled cross-host performance claim.Observed anomalies and extra generated files:
b77aba6343was the September 1 Ubuntu artifact measured at827a92bad5a5, not the September 22 mini artifact present on prior branch commit6d9286d3. This PR replaces it with the new measurement; it does not copy that earlier branch's evidence.go: doneprogress line). Go is outside the exported public Perry/Node/Bun evidence. Hermes was explicitly skipped because it was absent. No optional-language results are claimed as passing.relevant_box_roots; compilation succeeded. Auto-optimized runtimes, including the regex-specific feature set, were rebuilt within this fresh clone.git statusalso listedbenchmarks/honest_bench/REPORT.md,benchmarks/honest_bench/results/metadata.json,benchmarks/honest_bench/results/results.json,benchmarks/honest_bench/results/summary.txt,benchmarks/json_polyglot/RESULTS.md,benchmarks/polyglot/RESULTS_AUTO.md, and untrackedbenchmarks/app-patterns/results/matrix-20260922-182300.md. These seven extra generated files were not copied or committed. No other tracked source/input modifications were reported.990b3eeada01406e40a9d99ecdf87f9f91d39db3(v0.5.1640). Across the fingerprinted input sets, the only change is the exempt workspace release version. This PR remains based on requestedb77aba6343, and the artifact accurately names that measured compiler, not the later main compiler.CI at head
21a210735df50bfe469f589225dcc95a3e2fc05f: lint job is still type-checking Windows runtime/stdlib; the public-baseline freshness step is pending. Local freshness and all 15 tests pass; CI success is not yet claimed.https://claude.ai/code/session_01EQdCw7BN4AAnn2hAbNXg33