feat: DSC support for mongodb runner by mpobrien · Pull Request #822 · mongodb-js/devtools-shared

mpobrien · 2026-07-15T19:33:34Z

Adds the ability to launch DSC clusters backed by SLS.

New disaggregatedStorage cluster option (all topologies): starts a docker compose project for the storage backend, waits for readiness, runs per-shard setup, and injects disaggregatedStorageConfig/disaggregatedStorageEnabled into every mongod. Compose project is shared across shards and torn down with the cluster (including on failed startup).
SLS helpers: createSLSDisaggregatedStorageOptions() handles the full flow for the SLS multi-cell compose file from a server checkout — env/port allocation (services parsed from the compose file itself), readiness polling, per-shard StartLog, encryption key file, and auto-generated disaggregatedStorageConfig (incl. replSetConfig from pre-allocated member ports).
DSC-specific behavior: replset defaults to 2 nodes, replSetInitiate is skipped (config comes from startup parameters), and local-db metadata tracking is disabled (not writable on DSC).
New CLI flags: --slsCompose/--slsImageTag (full SLS flow), --disaggregatedStorageCompose/--disaggregatedStorageConfig (custom backends), and --downloadUrl for using a direct mongod tarball URL
(also added to mongodb-downloader, cached by URL).
Misc: docker compose output + download progress in debug logs, mongodb-runner start prints allocated SLS service ports.

Sample invocation:

node bin/runner.js start -t replset \
  --slsCompose=/home/ubuntu/mongo/buildscripts/modules/atlas/sls-multicell-docker-compose.yml \
  --slsImageTag=dfb12c1092695ae79f6ff3c5b6c31ff7417e4843 \
  --binDir=/mnt/mac/Users/mikeo/ubuntu-mongod/dist-test/bin/ \
  --debug --logDir=/tmp/mongodb-runner-logs

Open Questions

Checklist

I have signed the Contributor License Agreement (https://www.mongodb.com/legal/contributor-agreement)
New tests and/or benchmarks are included
Documentation is changed or added

Copilot

Pull request overview

Adds disaggregated storage (SLS) support to mongodb-runner, including orchestration of a docker compose-backed storage layer, pre-allocated ports for mongod members, and new CLI/config surface to drive it. It also extends mongodb-downloader to support downloading MongoDB binaries from a direct tarball URL (with progress logging).

Changes:

Add disaggregated storage abstractions (DisaggregatedStorageOptions, SLS helpers) and wire them into cluster startup/teardown.
Add docker compose lifecycle helper and CLI flags for SLS/disaggregated storage setup.
Add downloadUrl support to the MongoDB downloader and runner, enabling custom server builds.

Reviewed changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
packages/saslprep/src/code-points-data.ts	Updates embedded saslprep code point data blob.
packages/mongodb-runner/src/util.ts	Adds `allocatePort()` helper for pre-allocating member ports.
packages/mongodb-runner/src/util.js	Adds JS runtime utility helpers (parallel helpers, safePromiseAll, connection string helpers).
packages/mongodb-runner/src/tls-helpers.js	Adds TLS client key material helper for runner-managed TLS setups.
packages/mongodb-runner/src/sls.ts	Introduces SLS compose parsing, env/port allocation, readiness, and config generation helpers.
packages/mongodb-runner/src/runner-helpers.js	Adds JS runtime helpers for CLI commands (start/stop/prune/exec).
packages/mongodb-runner/src/oidc.js	Adds JS runtime OIDC mock provider subprocess management.
packages/mongodb-runner/src/mongoserver.ts	Adds disagg detection and skips local-db metadata tracking for disagg servers.
packages/mongodb-runner/src/mongoserver.js	Adds JS runtime implementation of MongoServer (process management, metadata, logging).
packages/mongodb-runner/src/mongologreader.js	Adds JS runtime log parsing/stream filtering for port/build info extraction.
packages/mongodb-runner/src/mongocluster.ts	Wires in disaggregated storage (compose lifecycle, per-shard setup, port preallocation) and `downloadUrl`.
packages/mongodb-runner/src/mongocluster.spec.js	Adds JS test coverage for cluster behaviors and CLI plumbing.
packages/mongodb-runner/src/mongocluster.js	Adds JS runtime implementation of MongoCluster including docker compose integration.
packages/mongodb-runner/src/index.ts	Exports new disagg/SLS and docker compose APIs from the package entrypoint.
packages/mongodb-runner/src/index.js	Adds JS runtime entrypoint exports for main APIs and CLI helpers.
packages/mongodb-runner/src/docker-compose.ts	Adds typed docker compose project wrapper (env + project naming + down --volumes).
packages/mongodb-runner/src/docker-compose.js	Adds JS runtime docker compose wrapper.
packages/mongodb-runner/src/cli.ts	Adds CLI flags for `downloadUrl`, SLS compose/image tag, and generic disagg compose/config.
packages/mongodb-runner/src/cli.spec.js	Adds JS tests for CLI workflows (start/stop/ls/prune/exec/config).
packages/mongodb-runner/src/cli.js	Adds JS runtime CLI implementation via yargs.
packages/mongodb-runner/README.md	Adds documentation pointer for disaggregated storage clusters.
packages/mongodb-runner/examples/sls-replset.js	Adds example for starting an SLS-backed replica set with a custom mongod build.
packages/mongodb-downloader/src/index.ts	Adds `downloadUrl` support, better error handling, and download progress logging.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mpobrien · 2026-07-16T13:16:06Z

+import { spawn } from 'child_process';
+import { once } from 'events';
+import { parseCLIArgs, OIDCMockProvider } from '@mongodb-js/oidc-mock-provider';
+import { debug } from './util';
+if (process.env.RUN_OIDC_MOCK_PROVIDER !== undefined) {
+  (async function main() {
+    const uuid = crypto.randomUUID();
+    debug('starting OIDC mock provider with UUID', uuid);


Fixed in ecef1cf — the compiled .js files in src/ were committed accidentally and have been removed; the .ts source now imports randomUUID explicitly from 'crypto'.

mpobrien · 2026-07-16T13:16:07Z

+        return; // process already exited
+    }
+    if (!this.issuer || !this.uuid) return;
+    await fetch(new URL(this.issuer, `/shutdown/${this.uuid}`));


Fixed in ecef1cf — arguments swapped to new URL(/shutdown/${uuid}, this.issuer) in oidc.ts (the stale compiled .js files were removed).

mpobrien · 2026-07-16T13:16:09Z

+    throw new AggregateError(
+      [rejected.map((r) => r.reason)],
+      `${rejected.length} errors: ${rejected.map((r) => r.reason).join(', ')}`,
+    );


Fixed in ecef1cf — the errors array is no longer nested; AggregateError.errors now contains the individual errors.

mpobrien · 2026-07-16T13:16:10Z

+      if (disaggregatedStorage) {
+        // Pre-allocate member ports so the storage configuration (which
+        // embeds the replica set config, including member host:ports) can be
+        // computed before the servers start.
+        const members: ShardMemberDescriptor[] = [];


Fixed in ecef1cf — ShardMemberDescriptor now carries arbiterOnly, populated from the replica set member options, and the generated DSC replSetConfig includes it for each member.

mpobrien · 2026-07-16T13:16:12Z

+  isClosed() {
+    // Return true if and only if there are no running sub-clusters/servers
+    // eslint-disable-next-line @typescript-eslint/no-unused-vars
+    for (const _ of this.children()) return true;
+    return true;
+  }


Fixed in ecef1cf — isClosed() now returns false while child servers/shards exist (the stale compiled .js copy was removed).

- flatten AggregateError errors array in safePromiseAll - fix isClosed() to return false while child servers/shards are running - fix reversed new URL() arguments in OIDC mock provider shutdown - import randomUUID explicitly in oidc.ts - include arbiterOnly in ShardMemberDescriptor and generated DSC replSetConfig - add docs/disaggregated-storage.md

mpobrien added 3 commits July 15, 2026 13:35

DSC support for mongodb-runner

1484845

wip

60e8af4

wip

5908876

Copilot AI review requested due to automatic review settings July 15, 2026 19:33

Copilot started reviewing on behalf of mpobrien July 15, 2026 19:34 View session

Copilot AI reviewed Jul 15, 2026

View reviewed changes

wip

89be729

mpobrien changed the title ~~DSC support for mongodb runner~~ feat: DSC support for mongodb runner Jul 15, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: DSC support for mongodb runner#822

feat: DSC support for mongodb runner#822
mpobrien wants to merge 5 commits into
mongodb-js:mainfrom
mpobrien:07-01-dsc_support_for_mongodb-runner

mpobrien commented Jul 15, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

mpobrien Jul 16, 2026

Uh oh!

mpobrien Jul 16, 2026

Uh oh!

mpobrien Jul 16, 2026

Uh oh!

mpobrien Jul 16, 2026

Uh oh!

mpobrien Jul 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

mpobrien commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Open Questions

Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

mpobrien Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

mpobrien Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

mpobrien Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

mpobrien Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

mpobrien Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mpobrien commented Jul 15, 2026 •

edited

Loading