Skip to content

Consume the varve layer instead of maintaining a parallel checksum set #660

Description

@avrabe

Filed from the varve side, with measurements. The two systems currently
maintain the same knowledge twice and already disagree.

They disagree today

tool this ruleset pins the pulseengine/pulseengine-wasm layers pin
wasm-tools 1.246.2 1.259.0
wac 0.9.0 0.11.0
wkg 0.15.0 0.16.1
witness 0.22.0 0.43.0
synth 0.6.0 0.69.0
spar 0.33.0 0.40.0
loom 1.1.14 1.4.1
wsc 0.9.2 0.11.0

A project using both Bazel rules and a varve pin today builds with two
different toolchains
. That is the mixed-toolchain hazard varve exists to
remove, arriving through the back door.

The same trivia, twice

toolchains/tool_registry.bzl holds a hand-written table of how each upstream
names its assets:

"wasmtime": { "filename": "wasmtime-v{version}-{suffix}" },
"wac":      { "filename": "wac-cli-{platform_name}", "is_binary": True },
"wkg":      { "filename": "{suffix}" },   # url_suffix IS the filename

A realm's layer.toml encodes exactly the same facts, as templates the
assembler resolves against the real release listing before anything is
fetched:

[[tool]]
name  = "wasmtime"
asset = "wasmtime-%R-%U.tar.xz"

[[tool]]
name   = "wac"
layout = "raw-per-platform"
[tool.asset-for]
"x86_64-unknown-linux-gnu" = "wac-cli-x86_64-unknown-linux-musl"

Both tables drift the same way, and the drift is invisible until a download
404s.

What already works

varve export-bazel emits this ruleset's schema from a verified installed
layer
— every hash a transcription from a signed manifest rather than TOFU.
Run against a layer composing both realms, it produced 16 registries in one
pass:

kilnd loom meld ordeal rivet spar spar-aadl synth
varve-producer wac wasm-tools wit-bindgen-wrpc with-device witness wkg wsc

The export is stamped and re-checkable: varve verify --export <dir> fails if
the tree stops matching the pin, so a stale vendored registry cannot slip
through CI.

The three deltas to settle

  1. url_suffix semantics. varve writes the complete filename
    (wasm-tools-1.259.0-x86_64-macos.tar.gz); this ruleset mostly writes a
    suffix and rebuilds the name from a per-tool pattern — except for wkg,
    wrpc and wit-bindgen-wrpc, which already use "filename": "{suffix}".
    Proposal: make {suffix} the rule for every tool and delete the table.
    The layer already resolved the name; re-deriving it is where drift lives.

  2. Windows. This ruleset supports windows_amd64; varve's default
    platform set is four non-Windows triples, so a varve-sourced registry would
    silently drop it. Either varve grows a Windows platform or the ruleset
    keeps its own entry for Windows and takes the other four from the layer.
    This is a real decision, not a formatting detail.

  3. Metadata. release_date, last_checked and supported_platforms are
    not emitted. last_checked becomes meaningless under this model — the
    layer's issue date and counter replace it — but say so rather than dropping
    it silently.

Coverage

Of the 22 tools in checksums/tools/, 12 were carried by no realm. Four are
being added now (pulseengine/wasm-layers#8): wit-bindgen (the realm had
wit-bindgen-wrpc, which is not a substitute), wasmtime, binaryen,
wrpc-wasmtime. The rest need decisions of their own: wasi-sdk is an sdk
payload rather than a binary, go/nodejs/tinygo are language runtimes,
jco ships only an npm tarball, and componentize-py is pinned upstream at
the moving tag canary — which is precisely the thing a digest pin fixes.

Cross-referenced from pulseengine/varve#175.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions