POC: discover extensions bundled with the release archive

[tedkahwaji]

Why

Proof of concept for shipping a first-party extension inside every pup release, so pup setup works right after installing pup (for example, a setup extension wrapping the public @datadog/ai-setup-cli package). This is the "everyone gets it installed" option, to compare against on-demand install via pup extension install or auto-install on first use.

What

• Discovery (src/extensions/discovery.rs): extension_path falls back to libexec/pup-extensions/pup-<name>, either next to the pup binary (tarball layout) or one level up (Homebrew's bin/ + libexec/ layout). User-installed extensions still take precedence. The bundled lookup only accepts valid extension names and regular files.
• Release (.goreleaser-linux.yaml, scripts/fetch-bundled-extensions.sh):
  • A before hook downloads a pinned pup-setup-<os>-<arch> release asset per platform and verifies it against a pinned SHA-256.
  • The archive places it at libexec/pup-extensions/pup-setup with mode 0755.
  • The pin (DataDog/pup-setup, v0.0.0) and the checksums are placeholders, since no such release exists yet.
  • The script sticks to bash 3.2 features so it also runs on the macOS runner.
  • Only the Linux config is changed in this POC. macOS, Windows, and the Homebrew formula (which would install libexec/ alongside bin/pup) would follow the same pattern.
• Docs: a "Bundled extensions" note in docs/EXTENSIONS.md.

Testing

cargo test --bin pup extensions: 121 passed, including 5 new bundled-discovery tests:

• tarball layout
• Homebrew layout
• missing extension
• invalid names (../setup, Setup, empty, set/up)
• directory instead of file

cargo clippy -- -D warnings and cargo fmt --check are clean.

YAML parses (ruby -ryaml). The script passes bash -n under both bash 5 and macOS /bin/bash 3.2. goreleaser isn't installed locally, so the archive step wasn't run end to end.

Local simulation, with a stub placed in target/debug/libexec/pup-extensions/pup-setup and an empty PUP_CONFIG_DIR:

$ ./target/debug/pup setup --product apm
bundled pup-setup invoked with: --product apm
DD_SITE=datadoghq.com token_set=yes

A user-installed pup-setup takes precedence:

$ ./target/debug/pup setup
user-installed pup-setup

With the bundled stub removed:

$ ./target/debug/pup setup
error: unrecognized subcommand 'setup'

The fetch script fails closed against the placeholder pin:

$ bash scripts/fetch-bundled-extensions.sh /tmp/out
curl: (56) The requested URL returned error: 404

Size impact

A local release build of pup is about 68 MB. A real setup extension carries a native agent runtime. A separate POC measured a compiled setup extension on darwin-arm64:

Artifact	Size
Compiled extension	78 MB
Agent runtime binary it spawns	213 MB
Bundle on disk	279 MB
Bundle .tar.gz	91 MB

So bundling would add roughly 90 MB to every compressed pup download and about 280 MB on disk, per platform. Users who never run pup setup pay that cost too. The agent runtime binary alone is also above the current 100 MiB per-binary limit in pup extension install.

Open questions

• Is the per-install download size acceptable for every pup user, including CI images that only query APIs?
• Pup releases would now be tied to a pinned extension version. Should the extension self-update, or only update with pup?
• Bundled extensions have no manifest.json, so pup extension list and pup --help don't show them yet.
• Release CI would depend on another repository's release assets being present and stable.
• Should bundling be limited to Homebrew (where libexec/ is idiomatic) and left out of raw tarballs?

🤖 Generated with Claude Code