Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/configs/feature.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ monad:
monad_runloop:
evm-type: eels
# Like `monad`, but `--monad-runloop` and eestnet chain id `30143`
fill-params: -m blockchain_test --from=MONAD_EIGHT --until=MONAD_ELEVEN --chain-id=30143 --monad-runloop -k "not invalid_header"
fill-params: -m blockchain_test --from=MONAD_TEN --until=MONAD_ELEVEN --chain-id=30143 --monad-runloop -k "not invalid_header"
Comment thread
pdobacz marked this conversation as resolved.

monad_benchmark:
evm-type: eels
Expand All @@ -15,4 +15,4 @@ monad_benchmark:
# transaction. The excluded `page_spread` cases size their work from `m`
# rather than the gas budget, so they need ~29M and ~177M whatever is
# passed here.
fill-params: -m blockchain_test tests/benchmark/stateful/mip8_pageified_storage/ --from=MONAD_NINE --until=MONAD_TEN --chain-id=30143 --monad-runloop --gas-benchmark-values=10 -k "not m4096 and not m1024"
fill-params: -m blockchain_test tests/benchmark/stateful/mip8_pageified_storage/ --fork=MONAD_ELEVEN --chain-id=30143 --monad-runloop --gas-benchmark-values=10 -k "not m4096 and not m1024"
43 changes: 16 additions & 27 deletions MONAD_RUNLOOP_TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ leaves a stale library that fails silently).

```sh
uv run fill --clean -m blockchain_test <test paths...> \
--fork MONAD_NINE --chain-id 30143 --monad-runloop \
--fork MONAD_TEN --chain-id 30143 --monad-runloop \
--output ../fixtures_eestnet

uv run consume direct --input ../fixtures_eestnet \
Expand All @@ -64,12 +64,12 @@ uv run consume direct --input ../fixtures_eestnet \
across ~14 threads), so budget ~5 vCPUs per worker (`-n N` needs
roughly `5 * N` cores).

## MIP-8 perf-regression tests
## Storage perf-regression tests

`tests/benchmark/stateful/mip8_pageified_storage/test_perf_regression.py`
fills SLOAD/SSTORE workloads at both forks and times block execution on
the runloop to compare MONAD_NINE (slot-encoded) vs MONAD_TEN
(page-encoded).
the runloop to compare MONAD_TEN vs MONAD_ELEVEN, both on page-encoded
(MIP-8) storage.

### Setup

Expand All @@ -91,7 +91,7 @@ From the repo root:
tmux new -s perf 'TAG=v4 scripts/perf_cycle.sh'
```

Fills once, consumes `RUNS` times, and writes the NINE-vs-TEN table to
Fills once, consumes `RUNS` times, and writes the TEN-vs-ELEVEN table to
`../timing_${TAG}_<utc>_table.md` (headed with the
cycle time and the four repo SHAs). Knobs:

Expand All @@ -111,35 +111,29 @@ Run on a quiet host; timings are noisy under contention.
- Fixtures containing expected-invalid blocks are skipped: the ledger
machine validates blocks at proposal time and cannot write invalid
ones.
- Fixtures with blocks before MONAD_TEN fail before the runloop starts,
transitions into MONAD_TEN included: monad executes only on a
page-encoded (MIP-8) triedb, and pre-MIP-8 blocks need the
slot-encoded one.
Comment thread
greptile-apps[bot] marked this conversation as resolved.
- The runloop runs in a privileged container (io_uring) and needs
`vm.nr_hugepages >= 2048` on the host; the `bin/` wrappers
re-provision this automatically (it resets on host reboot).

## Fork-transition flow (MIP-8 dual-db)

MONAD_NINE storage is slot-encoded; MONAD_TEN (MIP-8) is
page-encoded. A transition fixture (e.g.
`MONAD_NINEToMONAD_TENAtTime15k`) crosses that boundary mid-run, so
the run keeps both encodings live in one triedb: a slot-encoded
primary timeline and a page-encoded secondary timeline.
## Execution flow

- **Consumer** (`clis/monad.py`): loads the fixture, maps its
`network` to a `(revision, from_timestamp)` schedule, skips
`expectException` blocks, and provisions the db. Revisions spanning
the fork activate the secondary timeline via `monad-mpt
--activate-secondary`.
`expectException` blocks, and provisions the db with `monad-mpt
--create --state-machine monad`.
- **Harness** (`eest-runner`): fakes consensus with a Ledger
simulator (`propose` then `finalize` per block) that writes BFT
headers/bodies to `--ledger-dir` — the same artifacts monad-bft
persists on a real network — then drives the runloop over the FFI
and writes `output.json`.
- **Execution** (`runloop_monad`): reads each block from the ledger
dir and dispatches on `get_monad_revision(timestamp)` via
`SWITCH_MONAD_TRAITS` — block 1 runs MONAD_NINE, block 2 MONAD_TEN.
- **Dual-write** (`commit_block`): both timelines get the same
`StateDeltas` every block. Slot is canonical before the fork, page
after; the canonical root flips to page at the first MONAD_TEN
block, matching the fixture's final `blockHeader.stateRoot`.
`SWITCH_MONAD_TRAITS`, e.g. MONAD_TEN then MONAD_ELEVEN in a
`MONAD_TENToMONAD_ELEVENAtTime15k` fixture.
- **Assertions**: `state_root` and every `postState` account
(balance, nonce, code, per-slot storage) match `output`, with no
unexpected non-zero slots.
Expand All @@ -156,15 +150,10 @@ two differences: the block source and the revision schedule.
| Execution loop | `runloop_monad` | the same `runloop_monad` |
| Per-block dispatch | `SWITCH_MONAD_TRAITS` on `get_monad_revision(timestamp)` | the same |
| Block execution | `execute_block<traits>` | the same |
| Commit | `commit_block<traits>` with dual-write | the same function; dual-write engages whenever a secondary timeline is active |
| State root / db | `mpt::Db` + `TrieDb`, optional secondary | the same |
| Commit | `commit_block<traits>` | the same |
| State root / db | page-encoded `mpt::Db` + `TrieDb` | the same |

eest-only shims: the C ABI in `runloop_interface_monad.cpp`, the `EestNet`
chain (runtime schedule), `MonadRunloopDbCache` / `set_balance` (balance
overrides, unused here), genesis loading from the fixture, and the Rust
Ledger simulator that fabricates the BFT blocks consensus would deliver.
The production fork migration uses the same dual-db mechanism: an operator
runs `monad-mpt --activate-secondary --state-machine monad` on the live
slot-encoded db before the fork, the node opens the secondary timeline and
dual-writes both encodings across the boundary, and the authoritative root
flips from slot to page at the first MONAD_TEN block.
5 changes: 2 additions & 3 deletions monad-runloop/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,8 +83,7 @@ for block N:
eest-runner ─ runloop.run(1) ─▶ C++ engine ─ reads ledger dir ─ executes ─▶ triedb

after the last block:
eest-runner ─ runloop.get_primary_state_root() or
get_secondary_state_root() (page canonical from MONAD_TEN)
eest-runner ─ runloop.get_state_root() (page-encoded)
+ dump_json()
─▶ normalize_dump() ─▶ output.json ─▶ (Python compares vs fixture)
```
Expand All @@ -102,7 +101,7 @@ monad-runloop/bin/eest-runner --version

A plain `git clone` leaves all of this empty, which costs nothing for
contributors who never run the harness. `git clone --recurse-submodules`
does populate it, but pulls the full 3.6 GB rather than the 300 MB
does populate it, but pulls the full 6.5 GB rather than the 300 MB
`init-stack.sh` selects; re-running the script afterwards is harmless
but will not shrink an already-fetched checkout.

Expand Down
4 changes: 3 additions & 1 deletion monad-runloop/bin/eest-runner
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,11 @@ fi
# --name (when the caller sets it): a stable handle so the consumer can
# `docker kill` the container if its client is killed without forwarding
# a stop, which would otherwise leave the runloop spinning past pytest.
# --ulimit core=0: an aborting runloop would otherwise leave a multi-GB
# core dump on the host for every failing fixture.
exec docker run --rm --init \
${EEST_RUNNER_CONTAINER_NAME:+--name "$EEST_RUNNER_CONTAINER_NAME"} \
--privileged --ulimit nofile=16384:16384 --ulimit memlock=-1:-1 --user $(id -u):$(id -g) \
--privileged --ulimit nofile=16384:16384 --ulimit memlock=-1:-1 --ulimit core=0 --user $(id -u):$(id -g) \
-v /tmp:/tmp \
-v "$ROOT":"$ROOT" \
-e LD_LIBRARY_PATH="$ROOT"/install/lib \
Expand Down
4 changes: 2 additions & 2 deletions monad-runloop/init-stack.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# monad-execution submodule, and the third_party submodules the CMake
# build needs.
#
# monad declares 23 third_party submodules, 3.3 GB, not all needed.
# monad declares 21 third_party submodules, 6.5 GB, not all needed.
#
# SKIP is a denylist on purpose. A submodule the build turns out to need
# fails as a confusing CMake error deep into the build, while an extra
Expand Down Expand Up @@ -42,7 +42,7 @@ EXEC="$ROOT/monad-bft/monad-execution"

SKIP=(
third_party/ethereum-tests
third_party/yellowpaper
third_party/monad-execution-specs
third_party/zkevm-standards
)

Expand Down
2 changes: 1 addition & 1 deletion monad-runloop/monad-bft
Submodule monad-bft updated 79 files
+4 −0 .cargo/audit.toml
+3 −0 .dockerignore
+1 −0 .gitignore
+224 −142 Cargo.lock
+6 −4 Cargo.toml
+52 −0 Makefile
+10 −0 debian/DEBIAN/control.in
+12 −6 debian/DEBIAN/postrm
+1 −0 debian/opt/monad/scripts/reset-workspace.sh
+1 −0 debian/usr/lib/systemd/system/monad-bft.service
+3 −0 debian/usr/lib/systemd/system/monad-execution.service
+55 −0 docker/debian-package/Dockerfile
+1 −0 docker/devnet/monad/config/node.toml
+1 −1 docker/single-node/hive/compose.yaml
+2 −2 docker/single-node/nets/compose.yaml
+1 −2 monad-archive/Cargo.toml
+20 −3 monad-archive/src/bin/monad-archive-checker/cli.rs
+7 −10 monad-archive/src/bin/monad-archive-checker/main.rs
+58 −12 monad-archive/src/bin/monad-archiver/cli.rs
+10 −8 monad-archive/src/bin/monad-archiver/main.rs
+19 −9 monad-archive/src/bin/monad-indexer/cli.rs
+11 −7 monad-archive/src/bin/monad-indexer/main.rs
+3 −50 monad-archive/src/metrics.rs
+5 −1 monad-dataplane/Cargo.toml
+69 −0 monad-dataplane/benches/pacing.rs
+36 −7 monad-dataplane/examples/throughput.rs
+67 −7 monad-dataplane/src/lib.rs
+13 −2 monad-dataplane/src/metrics.rs
+1,132 −0 monad-dataplane/src/pacing.rs
+253 −0 monad-dataplane/src/tcp/conn.rs
+697 −71 monad-dataplane/src/tcp/mod.rs
+164 −269 monad-dataplane/src/tcp/rx.rs
+375 −249 monad-dataplane/src/tcp/tx.rs
+190 −210 monad-dataplane/src/udp.rs
+110 −34 monad-dataplane/tests/tests.rs
+1 −1 monad-execution
+33 −4 monad-executor-glue/src/lib.rs
+16 −0 monad-metrics/Cargo.toml
+178 −0 monad-metrics/src/lib.rs
+39 −0 monad-node-config/src/network.rs
+1 −0 monad-node/Cargo.toml
+2 −0 monad-node/examples/ledger-tail.rs
+6 −0 monad-node/src/cli.rs
+0 −5 monad-node/src/error.rs
+21 −18 monad-node/src/main.rs
+301 −6 monad-node/src/metrics.rs
+3 −0 monad-node/src/state.rs
+2 −4 monad-raptorcast/src/auth/socket.rs
+168 −5 monad-raptorcast/src/decoding.rs
+38 −12 monad-raptorcast/src/packet/deterministic.rs
+26 −4 monad-raptorcast/src/parser/packet_parser.rs
+46 −56 monad-raptorcast/src/round_info.rs
+124 −1 monad-raptorcast/src/udp.rs
+1 −1 monad-rpc/Cargo.toml
+2 −2 monad-rpc/README.md
+10 −3 monad-rpc/src/cli.rs
+3 −1 monad-rpc/src/handlers/eth/simulate.rs
+2 −2 monad-rpc/src/handlers/eth/txn.rs
+1 −0 monad-rpc/src/handlers/mod.rs
+8 −7 monad-rpc/src/main.rs
+0 −34 monad-rpc/src/middleware/metrics.rs
+47 −9 monad-secp/src/secp.rs
+0 −1 monad-statesync-executor/Cargo.toml
+7 −14 monad-statesync-executor/src/client/mod.rs
+5 −0 monad-statesync-executor/src/ipc.rs
+2 −31 monad-statesync-executor/src/lib.rs
+1 −1 monad-triedb-utils/src/lib.rs
+49 −10 monad-triedb-utils/src/triedb_env.rs
+1 −1 monad-types/src/lib.rs
+1 −1 monad-wireauth/README.md
+4 −7 monad-wireauth/src/api.rs
+5 −1 monad-wireauth/src/config/mod.rs
+18 −2 monad-wireauth/src/metrics.rs
+5 −2 monad-wireauth/src/session/initiator.rs
+5 −1 monad-wireauth/src/state.rs
+186 −0 monad-wireauth/tests/tests.rs
+55 −0 scripts/build-binaries
+132 −0 scripts/build-deb
+11 −0 scripts/package-version
15 changes: 1 addition & 14 deletions monad-runloop/rust-harness/eest-runner/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -241,20 +241,7 @@ fn main() {
runloop.run(1);
}

// The canonical state root is the page-encoded secondary timeline's
// from MONAD_ELEVEN (MIP-8, revision 10) on, the slot-encoded
// primary's before. The schedule is ascending, so the last entry is
// the revision the run ends in.
let final_revision = input
.revision_schedule
.last()
.expect("non-empty revision schedule")
.revision;
let state_root = if final_revision >= 10 {
runloop.get_secondary_state_root()
} else {
runloop.get_primary_state_root()
};
let state_root = runloop.get_state_root();
let post_state = normalize_dump(&runloop.dump_json());

let output = json!({
Expand Down
3 changes: 1 addition & 2 deletions monad-runloop/rust-harness/runloop/src/runloop.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,5 @@ use alloy_primitives::U256;

pub trait Runloop {
fn run(&mut self, n_blocks: u64);
fn get_primary_state_root(&mut self) -> U256;
fn get_secondary_state_root(&mut self) -> U256;
fn get_state_root(&mut self) -> U256;
}
Original file line number Diff line number Diff line change
Expand Up @@ -38,14 +38,8 @@ extern "C" {
// Execute and finalize `nblocks` number of blocks.
fn monad_runloop_run(runloop: *mut RawMonadRunloop, nblocks: u64);

// Store current primary (slot-encoded) state root.
fn monad_runloop_get_primary_state_root(
runloop: *mut RawMonadRunloop,
result_state_root: *mut RawMonadRunloopWord,
);

// Store current secondary (page-encoded) state root.
fn monad_runloop_get_secondary_state_root(
// Store current (page-encoded) state root.
fn monad_runloop_get_state_root(
runloop: *mut RawMonadRunloop,
result_state_root: *mut RawMonadRunloopWord,
);
Expand Down Expand Up @@ -130,22 +124,12 @@ impl Runloop for MonadRunloop {
}
}

fn get_primary_state_root(&mut self) -> U256 {
let mut state_root = RawMonadRunloopWord{
bytes: [0; 32],
};
unsafe {
monad_runloop_get_primary_state_root(self.raw, &mut state_root)
};
U256::from_be_bytes(state_root.bytes)
}

fn get_secondary_state_root(&mut self) -> U256 {
fn get_state_root(&mut self) -> U256 {
let mut state_root = RawMonadRunloopWord{
bytes: [0; 32],
};
unsafe {
monad_runloop_get_secondary_state_root(self.raw, &mut state_root)
monad_runloop_get_state_root(self.raw, &mut state_root)
};
U256::from_be_bytes(state_root.bytes)
}
Expand Down
42 changes: 21 additions & 21 deletions packages/testing/src/execution_testing/cli/perf_regression.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
"""
Build a NINE-vs-TEN significance table from perf timing runs.
Build a TEN-vs-ELEVEN significance table from perf timing runs.

Reads the `timing_consume.csv` produced by `consume direct
--timing-report` for several identical runs (one dir each) and, per
(test, params) case, reports each measure's mean +/- sd over the runs for
both forks, the two-sided Mann-Whitney U p-value comparing the forks,
and (for significant measures) the NINE->TEN average change.
and (for significant measures) the TEN->ELEVEN average change.

Writes GitHub-flavored Markdown to `--md`, or stdout if omitted; when
`--now` is given, prefixes a provenance header.
Expand All @@ -29,7 +29,7 @@
# A case, identified by (test, params).
CaseKey = Tuple[str, str]

# One metric's paired samples: [(NINE, TEN) per run measuring both forks].
# One metric's paired samples: [(TEN, ELEVEN) per run measuring both forks].
PairedSamples = List[Tuple[int, int]]

METRICS = ["tx_exec_us", "commit_us", "total_us"]
Expand All @@ -42,7 +42,7 @@
# wall time is what a retry count belongs to: retries are a property of
# executing the whole block, not of one of its phases.
REFERENCE_METRIC = "total_us"
FORKS = ["MONAD_NINE", "MONAD_TEN"]
FORKS = ["MONAD_TEN", "MONAD_ELEVEN"]
# A measure is significant at this Benjamini-Hochberg adjusted p-value.
# The table runs one test per (case, measure), so the unadjusted rate
# would produce a steady trickle of false positives across a cycle. The
Expand All @@ -52,8 +52,8 @@
# signed-rank test can reach scales with 2^RUNS.
ALPHA = 0.05

UP = "🔴⬆️" # significant measures all rise NINE->TEN (TEN slower)
DOWN = "🟢⬇️" # significant measures all fall NINE->TEN (TEN faster)
UP = "🔴⬆️" # significant measures all rise TEN->ELEVEN (ELEVEN slower)
DOWN = "🟢⬇️" # significant measures all fall TEN->ELEVEN (ELEVEN faster)
MIXED = "⚠️" # significant measures move both up and down

# Per-case workload descriptions live here; pinned to the execution-specs
Expand Down Expand Up @@ -151,7 +151,7 @@ def _wilcoxon_p(diffs: List[int]) -> float:
Two-sided Wilcoxon signed-rank p-value for paired differences.

Both forks are measured inside the same consume pass, so run `i`
yields a matched (NINE, TEN) pair. Testing the differences cancels
yields a matched (TEN, ELEVEN) pair. Testing the differences cancels
whatever drifted between runs — host thermals, page cache, other load
— which an unpaired test would instead charge to both samples'
variance.
Expand Down Expand Up @@ -243,13 +243,13 @@ def _paired_samples(
}
missing = 0
for run in runs:
nine = run.get((*case, FORKS[0]))
ten = run.get((*case, FORKS[1]))
if nine is None or ten is None:
ten = run.get((*case, FORKS[0]))
eleven = run.get((*case, FORKS[1]))
if ten is None or eleven is None:
missing += 1
continue
for m in (*METRICS, *COUNTS):
per_metric[m].append((nine[m], ten[m]))
per_metric[m].append((ten[m], eleven[m]))
samples[case] = per_metric
if missing:
dropped[case] = missing
Expand All @@ -274,16 +274,16 @@ def build(runs: List[Run]) -> List[str]:

header = ["test-params"]
for m in METRICS:
header += [f"{m} NINE", f"{m} TEN", f"{m} q"]
header += [f"{m} TEN", f"{m} ELEVEN", f"{m} q"]
for c in COUNTS:
header += [f"{c} NINE", f"{c} TEN"]
header += ["significant", "Δ avg NINE→TEN (sig)"]
header += [f"{c} TEN", f"{c} ELEVEN"]
header += ["significant", "Δ avg TEN→ELEVEN (sig)"]

lines = [
f"Mean ± sd over up to {len(runs)} runs; durations in µs, "
f"{'/'.join(COUNTS)} a count. Bold = measure significant (paired "
f"Wilcoxon signed-rank, Benjamini–Hochberg adjusted q ≤ {ALPHA}). "
f"Significant flag: {UP} TEN slower, {DOWN} TEN faster, "
f"Significant flag: {UP} ELEVEN slower, {DOWN} ELEVEN faster, "
f"{MIXED} mixed.",
"",
"| " + " | ".join(header) + " |",
Expand All @@ -301,8 +301,8 @@ def build(runs: List[Run]) -> List[str]:
chgs = []
for m in METRICS:
pairs = samples[case][m]
n = [nine for nine, _ in pairs]
x = [ten for _, ten in pairs]
n = [ten for ten, _ in pairs]
x = [eleven for _, eleven in pairs]
q = adjusted[(case, m)]
ncell, xcell = _stat(n), _stat(x)
if q <= ALPHA:
Expand All @@ -315,8 +315,8 @@ def build(runs: List[Run]) -> List[str]:
for c in COUNTS:
pairs = samples[case][c]
cells += [
_stat([nine for nine, _ in pairs]),
_stat([ten for _, ten in pairs]),
_stat([ten for ten, _ in pairs]),
_stat([eleven for _, eleven in pairs]),
]
cells.append(_direction(chgs) if chgs else "-")
cells.append(", ".join(deltas) if deltas else "-")
Expand All @@ -325,7 +325,7 @@ def build(runs: List[Run]) -> List[str]:
lines += [
"",
"q is the Benjamini–Hochberg adjusted p-value of a two-sided "
"paired Wilcoxon signed-rank test on the per-run NINE→TEN "
"paired Wilcoxon signed-rank test on the per-run TEN→ELEVEN "
f"differences for that measure; significant at q ≤ {ALPHA}. "
f"Smallest raw p in this table: {_pfmt(min(ps)) if ps else 'n/a'}. "
f"{'/'.join(COUNTS)} is reported, not tested: it tells a block "
Expand Down Expand Up @@ -490,7 +490,7 @@ def main(
monad: Optional[str],
) -> None:
"""
Build a NINE-vs-TEN significance table from perf timing runs.
Build a TEN-vs-ELEVEN significance table from perf timing runs.

Each RUN_DIRS argument is one `consume direct --timing-report` output
directory holding a `timing_consume.csv`.
Expand Down
Loading