BackupForge

BackupForge is a self-hosted backup control plane for Convex deployments.

It is a development-stage open source project that is intentionally a little overengineered: the current MVP backs up self-hosted Convex deployments, but the architecture is being shaped for a future where BackupForge can orchestrate different backup runners for databases, BaaS platforms, object stores, and other stateful systems.

The goal is not to hide backup complexity behind a magic button. The goal is to make backup operations explicit, observable, repeatable, and boring enough to trust.

Project Status

BackupForge is under active development. It is suitable for local testing, experimentation, and early contributor work. Treat production use with care until the project has had more operational hardening, security review, and restore workflow coverage.

The first target is Convex self-hosted backups:

• register a Convex deployment
• run manual or scheduled backups
• execute convex export inside an ephemeral Docker runner
• upload backup.zip, manifest.json, and logs.txt to S3-compatible storage
• track job status, logs, checksums, and downloadable artifacts

Convex Cloud appears in the data model and UI as a future-facing shape, but the working backup path today is self-hosted Convex plus an admin key.

Contributions are welcome. See CONTRIBUTING.md.

Why BackupForge Exists

Self-hosted systems often accumulate small backup scripts that work until they do not: credentials are scattered, logs disappear, schedules are invisible, artifacts are not verified, and nobody remembers whether the last successful backup was actually downloadable.

BackupForge turns that script pile into a small control plane:

• the server owns identity, policy, scheduling, state, and audit-friendly data
• short-lived runners perform one backup job and then exit
• object storage keeps portable backup artifacts
• the UI gives operators a real-time view of backup health
• SQLite remains the source of truth for the control plane itself

Yes, this is more structure than a shell script. That is on purpose. The project is designed to grow into a runner-oriented backup platform without making the first MVP a dead end.

Current Features

• First-admin setup flow.
• Email and password login with HttpOnly cookie sessions.
• Project, deployment, policy, and backup job management.
• Manual backup jobs from the UI or API.
• Scheduled backups with cron expressions and time zones.
• Ephemeral local Docker runner containers.
• Convex self-hosted export through the Convex CLI.
• S3-compatible storage support, including MinIO for development.
• Artifact metadata for backup.zip, manifest.json, and logs.txt.
• SHA-256 tracking and artifact size metadata.
• Job state machine for queued, preparing, exporting, uploading, verifying, completed, failed, cancelled, and expired jobs.
• Server-sent events for real-time UI updates.
• OpenAPI documentation served by the backend.
• Secret storage encrypted with an application-provided AES-GCM key.
• Local Docker Compose setup for the backend and MinIO.
• TanStack Start frontend with React, TypeScript, Tailwind CSS, and Biome.

Planned Direction

BackupForge starts with Convex, but the intended shape is broader:

• additional runner types for databases such as PostgreSQL, MySQL/MariaDB, SQLite, MongoDB, and Redis
• export runners for other BaaS-like platforms where a repeatable backup API or CLI exists
• restore drills and restore documentation, not only backup creation
• retention enforcement and artifact pruning
• notification delivery for failed and recovered backup jobs
• stronger runner capability isolation
• signed release images and more production deployment examples
• richer OpenAPI and SDK-style client coverage

Ideas, issues, implementation PRs, documentation improvements, and hardening work are welcome.

Architecture

backup-server
  Go HTTP API, auth, SQLite, scheduler, SSE, Docker orchestration,
  secret encryption, backup metadata, and artifact download links

backup-runner
  ephemeral container for one backup job; currently runs Convex export,
  packages artifacts, uploads them to S3/MinIO, and reports status

SQLite
  source of truth for users, sessions, projects, deployments, policies,
  jobs, runner tokens, artifacts, logs, notifications, and audit records

MinIO / S3
  durable object storage for backup.zip, manifest.json, and logs.txt

web
  TanStack Start frontend that talks to the REST API and listens to SSE

The server controls the lifecycle. The runner does the risky, tool-specific work in a separate container. This keeps the control plane focused on state and policy while runners can evolve per backup target.

Backup Lifecycle

1. An operator creates a project and deployment.
2. The deployment stores its backup credentials through the backend secret service.
3. A manual request or scheduled policy creates a backup_job.
4. The backend mints a short-lived runner token.
5. The backend starts a local Docker container from the configured runner image.
6. The runner authenticates back to the server and receives job configuration.
7. The runner executes the backup command, currently convex export.
8. The runner packages backup.zip, manifest.json, and logs.txt.
9. The runner uploads artifacts to S3-compatible storage.
10. The backend records bucket, object key, size, checksum, status, and logs.
11. The frontend refreshes through REST and live SSE events.
12. An operator can request a temporary download URL for an artifact.

Repository Layout

.
+-- apps/
|   +-- backend/          # Go server, runner, migrations, Docker files, OpenAPI
|   +-- frontend/         # TanStack Start UI
+-- package.json          # root pnpm script wrappers
+-- pnpm-lock.yaml        # single Node lockfile for the monorepo
+-- CONTRIBUTING.md
+-- LICENSE
+-- README.md

Requirements

• Docker with access to the local Docker engine.
• Docker Compose v2.
• Go 1.26.x.
• Node.js 22 or newer.
• pnpm 11.x.
• Air for backend hot reload during manual development.
• An S3-compatible object store. MinIO is provided for local development, and manual dev mode requires either MinIO running or equivalent S3/R2 settings.

The runner image installs the Convex CLI inside its container. You do not need the Convex CLI on the host for the normal Docker runner path.

Install Air on machines that run the backend in development mode:

go install github.com/air-verse/air@latest

Quick Start With Docker Compose

The recommended local path is the root Docker Compose stack. It builds and starts the frontend, backend, MinIO, MinIO bucket initialization, and the local runner image in one command:

docker compose up --build

The services are exposed on:

Frontend: http://localhost:3000
Backend:  http://localhost:8080
MinIO:    http://localhost:9001

The backend SQLite database is stored on the host at:

apps/backend/data/backup-control.db

The whole apps/backend/data directory is ignored by Git, including SQLite WAL and SHM files.

If you already have the backend-only MinIO compose running, stop it first:

cd apps/backend
docker compose -f docker/docker-compose.minio.yml down

Manual Development Quick Start

Stop the root Compose stack before using this mode if it is already running, because the frontend also binds localhost:3000:

docker compose down

Install JavaScript dependencies from the repository root:

pnpm install

Build the runner image:

pnpm --dir apps/backend run build:runner

Start MinIO unless you have already configured S3_ENDPOINT, S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY, and related storage variables for another S3-compatible service:

pnpm run minio

This command starts the backend development MinIO compose file and creates the local convex-backups bucket used by the default backend environment.

Stop it with:

pnpm run minio:stop

Start frontend and backend together:

pnpm run dev

This runs:

Backend:  http://localhost:18080
Frontend: http://localhost:3000

The backend process is managed by Air. When pnpm run dev exits, the root script runs pnpm run backend:clean, which stops anything still listening on port 18080 and removes apps/backend/tmp.

You can still run the services separately when needed:

pnpm run backend:dev
pnpm run frontend:dev

The local backend listens on:

http://localhost:18080

Open:

http://localhost:3000

API documentation is available at:

http://localhost:18080/docs
http://localhost:18080/openapi.yaml

Create the First Admin

You can use the setup screen in the UI or call the API directly. With the root Compose stack, use port 8080:

curl -X POST http://localhost:8080/api/setup/admin \
  -H 'Content-Type: application/json' \
  -d '{"email":"admin@example.com","name":"Admin","password":"change-me-now"}' \
  -c cookies.txt

With the manual development backend, use http://localhost:18080 instead.

After setup, create a project, add a deployment, and run a manual backup from the UI.

Docker-Based Local Backend

The backend ships local Dockerfiles and compose files under apps/backend.

Start MinIO:

cd apps/backend
pnpm run minio

Build the runner image:

pnpm run build:runner

Start the backend container:

pnpm run compose

With this path, the backend API is exposed on:

http://localhost:8080

The frontend is currently run separately with:

pnpm run frontend:dev

Configuration

Backend configuration lives in environment variables. Start from:

apps/backend/.env.example

Important groups:

• HTTP_ADDR, PUBLIC_BASE_URL, and ALLOWED_ORIGINS configure HTTP serving and browser access.
• DATA_DIR and DATABASE_PATH configure SQLite persistence.
• DOCKER_HOST, RUNNER_IMAGE, BACKUP_JOB_TIMEOUT_SECONDS, and timeout/concurrency values configure local runner orchestration.
• ALLOW_MOCK_DOCKER defaults to false; use it only for development without real backup execution.
• S3_ENDPOINT, S3_BUCKET, S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY, and related values configure artifact storage.
• SESSION_SECRET, SESSION_COOKIE_NAME, and session TTL values configure authentication cookies.
• SECRETS_ENCRYPTION_KEY configures encryption for stored deployment secrets.

Production deployments must replace every development secret and MinIO default.

Switching Object Storage To S3 Or R2

Backup artifacts are written through the S3-compatible settings on the backend. The root compose file defaults to local MinIO:

S3_ENDPOINT: http://minio:9000
S3_PUBLIC_ENDPOINT: http://localhost:9000
S3_REGION: us-east-1
S3_BUCKET: convex-backups
S3_ACCESS_KEY_ID: minioadmin
S3_SECRET_ACCESS_KEY: minioadmin
S3_FORCE_PATH_STYLE: "true"

S3_ENDPOINT is the endpoint used by the backend and runner containers. S3_PUBLIC_ENDPOINT is used only for signed download URLs returned to the browser. In local Docker Compose, this must be http://localhost:9000; using http://minio:9000 in the browser will fail because minio is only resolvable inside Docker.

When deploying the compose stack on a non-local host and still using the bundled MinIO service, S3_PUBLIC_ENDPOINT must be the URL that users' browsers can reach from outside the Docker host. Do not leave it as localhost unless the browser is running on the same machine as Docker.

Examples:

# Local development on the same machine.
S3_ENDPOINT: http://minio:9000
S3_PUBLIC_ENDPOINT: http://localhost:9000

# Remote server reached by IP.
S3_ENDPOINT: http://minio:9000
S3_PUBLIC_ENDPOINT: http://203.0.113.10:9000

# Remote server behind TLS/reverse proxy.
S3_ENDPOINT: http://minio:9000
S3_PUBLIC_ENDPOINT: https://minio.example.com

The public MinIO endpoint must route to the same bucket and credentials used by S3_ENDPOINT, because the host is part of the signed URL. If the URL is signed for minio:9000 or localhost:9000 but opened from another machine, downloads will fail.

To use AWS S3, create a bucket and an access key with permission to read and write objects in that bucket. Then change the backup-server.environment values in docker-compose.yml:

S3_ENDPOINT: https://s3.us-east-1.amazonaws.com
S3_PUBLIC_ENDPOINT:
S3_REGION: us-east-1
S3_BUCKET: your-backupforge-bucket
S3_ACCESS_KEY_ID: your-access-key-id
S3_SECRET_ACCESS_KEY: your-secret-access-key
S3_FORCE_PATH_STYLE: "false"

Use the region and endpoint that match your bucket. For standard AWS S3, virtual-host style addressing is normally preferred, so S3_FORCE_PATH_STYLE can be false.

To use Cloudflare R2, create an R2 bucket and an R2 S3 API token. Cloudflare's S3-compatible API endpoint has this form:

https://<ACCOUNT_ID>.r2.cloudflarestorage.com

See Cloudflare's R2 S3 API documentation for the current endpoint and token setup details:

https://developers.cloudflare.com/r2/api/s3/api/

Then set:

S3_ENDPOINT: https://<ACCOUNT_ID>.r2.cloudflarestorage.com
S3_PUBLIC_ENDPOINT:
S3_REGION: auto
S3_BUCKET: your-r2-bucket
S3_ACCESS_KEY_ID: your-r2-access-key-id
S3_SECRET_ACCESS_KEY: your-r2-secret-access-key
S3_FORCE_PATH_STYLE: "true"

If you switch away from MinIO, you can remove minio and minio-init from the compose stack and change backup-server.depends_on so it only waits for backup-runner-image. You may also leave MinIO in place for local testing as long as the backend points at the external S3/R2 endpoint.

After changing storage settings, recreate the backend:

docker compose up -d --force-recreate backup-server

Production Deployment Notes

BackupForge is designed to be self-hosted on a single operator-controlled host first. The current repository includes the backend and runner Dockerfiles, plus local compose examples. A hardened production deployment should include:

• TLS termination through a reverse proxy.
• A stable volume for DATABASE_PATH.
• Regular backups of BackupForge's own SQLite database.
• A real S3-compatible bucket with lifecycle and access policies.
• Strong SESSION_SECRET and SECRETS_ENCRYPTION_KEY values.
• APP_ENV=production.
• ALLOW_MOCK_DOCKER=false; production must fail fast if Docker is not reachable.
• PUBLIC_BASE_URL set to the public HTTPS origin.
• ALLOWED_ORIGINS restricted to trusted origins.
• Secure cookie settings, preferably with the __Host-backup_session cookie name.
• Pinned server and runner image tags.
• A dedicated Docker host or VM for running backup jobs.
• Monitoring around failed jobs, storage upload errors, and scheduler health.

The backend currently needs access to the Docker engine so it can create runner containers. Runners must not be privileged and must not mount /var/run/docker.sock.

Development Commands

From the repository root:

pnpm run dev
pnpm run frontend:dev
pnpm run frontend:build
pnpm run frontend:test
pnpm run frontend:check
pnpm run backend:dev
pnpm run backend:test
pnpm run backend:server
pnpm run backend:clean
pnpm run minio
pnpm run minio:stop

From apps/backend:

air
go run ./cmd/server
go test ./...
sh scripts/build-runner.sh
sh scripts/build-server.sh
docker compose -f docker/docker-compose.minio.yml up -d
docker compose -f docker/docker-compose.yml up --build

From apps/frontend:

pnpm run dev
pnpm run build
pnpm run test
pnpm run check

API

The backend serves OpenAPI documentation:

/docs
/openapi.yaml

If a change modifies the API contract, update the backend implementation, frontend client/types, and apps/backend/docs/openapi.yaml in the same pull request.

Security Model

BackupForge intentionally keeps a small self-hosted security model:

• SQLite is the source of truth.
• Sessions use HttpOnly cookies.
• The frontend must not store tokens, admin keys, runner tokens, signed URLs, or secrets in browser storage.
• The backend must not log secrets or return complete secret values to the UI.
• Runner tokens are short lived and scoped to a single job.
• The UI must not allow arbitrary Docker images or arbitrary shell commands.
• Runner containers must not run privileged.
• Signed download URLs should be temporary.

Security hardening is one of the most valuable contribution areas.

License

BackupForge is released under the MIT License. See LICENSE.