Repository navigation
Conversation
…-server The server, store and archive code moves from apps/cli/src/server into a private workspace package so a second host can embed it. No behaviour changes; the CLI imports it from @maple/local-server. Untangled first so the package never reaches back into the CLI: - the 41 historical schema snapshots move to apps/cli's schema-snapshots.ts, next to the migrations that are their only users - the migration journal envelope, the PID claim helpers and the store errors get their own modules - ensurePortFree, the chDB config path and DEFAULT_REMOTE_UI_URL move next to their users; serve.ts takes the telemetry layer from its caller - archive calibration logs through a shared debug-log instead of lib/debug Error tags stay @maple/cli/* so CLI exit handling and telemetry still group failures by tag. A no-restricted-imports override keeps the CLI out of the package; lint burndown entries move with their files.
…ned libchdb
startLocal({ dataDir, port?, libchdb?, onDirtyStore? }) owns one loopback
Maple Local server until its scope closes, with no CLI or maple binary
involved. It claims the store, checks the port, loads libchdb, recovers an
interrupted checkpoint, then refuses unsafe stores the way `maple start`
does. A store left unclean fails closed unless onDirtyStore is "wipe";
embedded stores have no checkpoints to restore from. A failed start
releases everything it acquired even when the caller keeps its scope
open, so a host can retry in the same process.
packages/local-server/libchdb.json pins the chDB release and each platform
asset's sha256. build-local-binary.sh reads it instead of its own table,
and provisionLibchdb() downloads the same asset, verifies it before
extraction and caches it under ~/.maple/cache/libchdb/<release>. The
runtime checks the loaded library's chdb_version() against the pin, so a
mismatched MAPLE_LIBCHDB is refused before any store is opened.
Local("observability") from @maple-dev/alchemy/local starts the embedded
@maple/local-server runtime in an Alchemy native sidecar. Telemetry({ local })
binds MAPLE_ENDPOINT to it under alchemy dev and skips the ingest key;
deploys ignore it. Every Worker in the stack shares one loopback server, the
sidecar keeps running across stack reloads, and the store lives per stack,
stage and resource under .alchemy/maple-local/<hash>/data.
The sidecar provisions libchdb from the pinned manifest unless
MAPLE_LIBCHDB is set, and LocalProps exposes dataDir, port and onDirtyStore.
@maple/local-server is bundled into the package; effect, @effect/* and
alchemy stay external, with @effect/platform-bun as an optional peer, so the
sidecar runs on the user's own effect version.
Adds a two-Worker example (examples/alchemy-maple/local.alchemy.ts), docs in
the package README and the hosted Alchemy page, and a changeset.
Maple review🟡 Confidence 6/10 · needs attention. Held at 6 because 13 reviewed files were not read. Moves the Maple Local server out of
Production impactOpen errors in the changed files (7)
After this merges, Maple checks whether they stop. Files not reviewed (13)The review ended before it read these diffs, so nothing above vouches for them.
|
|
Important Review skippedToo many files! This PR contains 211 files, which is 61 over the limit of 150. To get a review, reduce the PR to 150 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.
 |
…runtime # Conflicts: # apps/cli/src/server/schema-snapshots.ts
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A team running Workers under
alchemy devcan now see their traces and logs in Maple without a Maple account, an ingest key or a separatemaple start: oneLocalresource runs Maple Local inside the stack, and every Worker exports to it.Problem
On
main,Telemetryonly knows hosted Maple. Underalchemy deva Worker either exports to production with a real ingest key, or runs the SDK in no-op mode and exports nothing. To see local telemetry you install themaplebinary, runmaple startin another terminal, and point each Worker'sMAPLE_ENDPOINTat it by hand. Nothing ties that server's lifetime or data to the stack.Fix
Under
alchemy dev,Telemetry({ local })bindsMAPLE_ENDPOINTto the local server and needs no ingest key; deploys ignorelocal. The dashboard islocal.maple.devpointed at the loopback server, as withmaple start.Localstarts the server in an Alchemy native sidecar. Workers share it, it survives stack reloads (HMR keeps the same sidecar process), and Ctrl-C closes the store cleanly..alchemy/maple-local/<sha256(stack, stage, fqn)>/dataand persists across runs.LocalPropstakesdataDir,port(default4318, an occupied port fails the start) andonDirtyStore.startLocal()in@maple/local-server; it never spawnsmaple.~/.maple/cache/libchdb/<release>/.MAPLE_LIBCHDBuses a copy you already have. The runtime checks the loaded library'schdb_version()against the pin.alchemy devexit I could produce (Ctrl-C during ingest, closing the terminal,kill -9ofalchemy dev) closed the store cleanly; only killing the sidecar itself, an OOM or power loss leaves it dirty. These stores take no checkpoints, so there is nothing to restore: the start fails closed with a message namingonDirtyStore: "wipe", which discards the live data. Changing the prop applies in the running session.How it's built (three commits, review in order)
refactor(local-server): move the Maple Local server into@maple/local-server. A pure move: 56 files fromapps/cli/src/serverinto a private workspace package, import rewrites, and the untangling that lets the package build without the CLI. The 41 historical schema snapshots stay in the CLI next to the migrations (keeps the sidecar about 1 MB instead of 5 MB). The PID claim, migration journal and store errors move into their own modules. Ano-restricted-importsoverride keeps the CLI out of the package, and lint burndown entries move with their files. Error tags stay@maple/cli/*so CLI exit handling and telemetry group failures the same. No behaviour change.feat(local-server): embeddable scoped runtime with a pinned, provisioned libchdb.startLocal();packages/local-server/libchdb.json, the one manifestscripts/build-local-binary.shand the sidecar both read (it replaces the script's hash table);provisionLibchdb(). A failed start releases its claim even when the caller keeps its scope open, because Alchemy retries a failed resource in the same process without closing it.feat(alchemy): run Maple Local inside alchemy dev.Local,localProviders(), the sidecar,Telemetry({ local }), a two-Worker example (examples/alchemy-maple/local.alchemy.ts), docs (package README, hosted Alchemy page,docs/local-mode.md) and a minor changeset.@maple/local-serveris bundled;effect, every@effect/*package andalchemystay external so the sidecar runs on the user's owneffect, with@effect/platform-bunas an optional peer (Alchemy needs it under Bun already).tsconfig.alchemy-maple.json(from #2006) gainstypeRootsso the declaration build finds@types/bun, which@maple/local-serverneeds.Verification
Each commit was checked out on its own and built: typecheck, tests and lint pass at every commit.
packages/local-server,apps/cli,packages/alchemy-maple,examples/alchemy-maplepackages/local-servertestsapps/clitestspackages/alchemy-mapletestsbun run clickhouse:schema:check, oxlint (.oxlintrc.effect.json), oxfmtscripts/build-local-binary.shon commit 2chdb: v26.7.3from the manifestSmoke runs on macOS arm64, Bun 1.4:
alchemy dev: two Workers export to one server, the traces are queryable at/local/query, the sidecar PID survives a reload, data survives a restart, and Ctrl-C closes the store cleanly.HOME, onalchemy@2.0.0-beta.84andeffect@4.0.7: libchdb downloads (byte-identical to the one the release binary ships), a dirty store is refused, then settingonDirtyStore: "wipe"in the same session recovers it.Limits
bun:ffi), on macOS and Linux, x64 and arm64. Only macOS arm64 was run; the other three take the same code path with pinned hashes.maple start.effect4.0.x a fresh install resolves@effect/platform-node-sharedto 4.1.x, which breaks@effect/platform-bun@4.0.7itself (Alchemy included). Users pin it to 4.0.7. Not something this package can fix.Open questions
Need help on this PR? Tag
@codesmith-botwith what you need.