Allocation-free Merkle in Motoko: independent validation + supporting research

[KYounesMercatura]

Allocation-free Merkle in Motoko - independent validation + supporting research

This PR adds a small, self-contained research subdirectory (research/allocfree-merkle/).
It's offered as a contribution and a thank-you, not a replacement for anything: Timo
Hanke has already demonstrated allocation-free on-chain Merkle hashing in his Bitcoin work,
and this is us reproducing it independently, confirming it against real chain data, and
writing down a couple of extra data points we hadn't seen recorded. The credit for the
approach is his - we're validating and exploring around it to see if any of it is useful to
hand back.

What's here

A double-SHA256 Merkle implementation with O(1) allocation (one fixed scratch buffer,
no per-hash garbage), plus the streaming and naive variants for comparison, full tests, a
codegen for the unrolled hasher, and an on-chain bench canister.

What we set out to confirm / add

• Independent byte-exact verification. Roots match real Bitcoin block headers at heights
  0, 170, 546, 100000, and 350000 (1,795 tx - exercises odd-node duplication across several
  levels), and the SHA-256 core matches NIST vectors. A Python hashlib re-computation
  cross-checks the vectors against the chain before the Motoko tests run.
• Instruction-level measurements on mainnet (Motoko playground, performanceCounter) -
  an angle complementing the allocation focus. The allocation result is the important part;
  these are just extra reference numbers.
• The locals issue + the trade-off. The recent IC Wasm-locals validation (IC0505,
  since rolled back) flags fully-unrolled SHA-256 (~2,000+ locals). We include both an
  unrolled build and a loop build (~50 locals) that stays under the cap at some instruction
  cost - context in case that validation ever returns. We frame this as a speed / locals /
  allocation trade-off in RESULTS.md.
• Toolchain notes. moc leaves optimization to a wasm-opt -O3 post-pass (easy to miss
  with a prebuilt-wasm deploy); and keeping the message schedule in let locals rather than
  a [var Nat32] array measurably helps in Motoko.

Measurements (real mainnet IC, after wasm-opt -O3, for reference)

leaves	research-ag/sha2 0.2.1 (digest-reuse)	this in-place build
1,024	66.5M instr / 1.26 MB alloc	62.5M instr / 1,128 B alloc
4,096	266.3M / 5.05 MB	250.1M / 1,128 B
8,192	532.6M / ~10 MB	500.3M / 1,128 B

Both land in the same instruction ballpark; the property of interest is the flat 1,128-byte,
O(1) allocation - the zero-GC-pressure result the approach targets - reproduced here
independently. (Benchmarked against the public sha2 0.2.1; we didn't have the latest
Merkle work to compare against, so please read the numbers as validation, not a comparison.)

Use

Free to use, fork, or fold upstream into sha2. Feedback and corrections very welcome -
this only exists because of Timo's work, and we're happy to hand any of it back.