Skip to content

feat: add opt-in multi-organization runner support - #5428

Open
guicaulada wants to merge 4 commits into
mainfrom
gc/feat/multi-org-runners
Open

guicaulada wants to merge 4 commits into
mainfrom
gc/feat/multi-org-runners

Conversation

@guicaulada

Copy link
Copy Markdown
Contributor

Description

Add opt-in support for a control plane serving multiple GitHub organizations. With enable_multi_org_runners = true, webhook-driven runners register in the repository's organization, and each pool schedule can select an org and target size. The flag defaults to false and preserves existing behavior in both legacy and v2 configurations.

  • Select organization-specific app installations for scale-up, job retry, pools, and scale-down instead of reusing a global installation ID.
  • Scope runner-group caching by organization so identically named groups can have different IDs.
  • Reuse existing ghr:Owner and ghr:Type tags for deregistration and orphan checks; apply idle retention independently per organization.
  • Add configuration examples and document per-organization capacity limits, default pool-owner fallback, and app-installation requirements.

Test Plan

  • 376 control-plane tests passed, including mixed-organization batches, JIT group IDs, pool isolation, retry authentication, orphan checks, and disabled-mode compatibility.
  • 29 mocked Terraform tests passed across legacy runners, runner-config, pool scheduling, and legacy/v2 multi-runner configuration forwarding. Tests used locally cached AWS provider 6.60.0; the existing dependency lock files are unchanged.
  • TypeScript, ESLint, Prettier, Terraform validation/formatting, and git diff --check passed. Terraform reports existing provider deprecation warnings.
  • Live AWS/GitHub deployment validation remains outstanding.

@guicaulada
guicaulada requested review from a team as code owners September 14, 2026 13:14
@github-actions

Copy link
Copy Markdown
Contributor

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Pool-owner validation currently accepts organization logins that GitHub rejects.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds opt-in multi-organization runner support across the control plane and Terraform interfaces.

Changes:

  • Resolves GitHub App installations, runner groups, capacity, and cleanup per organization.
  • Adds organization-specific scheduled pools across legacy and v2 configurations.
  • Adds comprehensive tests and usage documentation.
File summaries
File Description
variables.tf Adds the feature flag and pool organization input.
README.md Documents public inputs and links the guide.
modules/runners/variables.tf Extends legacy runner inputs.
modules/runners/tests/pool.tftest.hcl Tests lifecycle Lambda flag forwarding.
modules/runners/scale-up.tf Configures multi-org scale-up.
modules/runners/scale-down.tf Configures multi-org scale-down.
modules/runners/README.md Updates runner-module inputs.
modules/runners/pool/variables.tf Adds pool organization validation.
modules/runners/pool/README.md Updates pool configuration schema.
modules/runners/pool/main.tf Adds organization-aware schedule payloads.
modules/runners/pool.tf Forwards the pool feature flag.
modules/runners/job-retry/variables.tf Extends retry configuration.
modules/runners/job-retry/README.md Documents the retry flag.
modules/runners/job-retry/main.tf Configures multi-org retries.
modules/runners/job-retry.tf Forwards the retry feature flag.
modules/runner-config/variables.orchestration-provider.tf Extends the v2 provider contract.
modules/runner-config/tests/pool.tftest.hcl Tests v2 lifecycle propagation.
modules/runner-config/README.md Documents v2 configuration.
modules/orchestration-providers/webhook/variables.tf Extends webhook provider inputs.
modules/orchestration-providers/webhook/scale-runners/variables.tf Extends scaling configuration.
modules/orchestration-providers/webhook/scale-runners/scale-up.tf Configures v2 multi-org scale-up.
modules/orchestration-providers/webhook/scale-runners/scale-down.tf Configures v2 multi-org scale-down.
modules/orchestration-providers/webhook/scale-runners/README.md Documents scaling inputs.
modules/orchestration-providers/webhook/README.md Documents webhook inputs.
modules/orchestration-providers/webhook/pool/variables.tf Adds v2 pool organization validation.
modules/orchestration-providers/webhook/pool/tests/provider.tftest.hcl Tests schedule isolation and validation.
modules/orchestration-providers/webhook/pool/README.md Documents v2 pool inputs.
modules/orchestration-providers/webhook/pool/pool.tf Emits organization-aware pool events.
modules/orchestration-providers/webhook/pool.tf Forwards v2 pool settings.
modules/orchestration-providers/webhook/job-retry/variables.tf Extends v2 retry inputs.
modules/orchestration-providers/webhook/job-retry/README.md Documents v2 retry configuration.
modules/orchestration-providers/webhook/job-retry/job-retry.tf Configures v2 multi-org retries.
modules/multi-runner/variables.tf Extends legacy and v2 lane schemas.
modules/multi-runner/variables.experimental.orchestration-provider.tf Extends global pool schedules.
modules/multi-runner/tests/config-effective.tftest.hcl Tests lane configuration translation.
modules/multi-runner/runners.tf Forwards lane-specific flags.
modules/multi-runner/README.md Documents multi-runner schemas.
modules/multi-runner/config.experimental.translation.tf Translates the legacy feature flag.
mkdocs.yaml Adds the guide to navigation.
main.tf Forwards the root feature flag.
lambdas/functions/control-plane/src/scale-runners/scale-up.ts Enables organization-scoped scale-up.
lambdas/functions/control-plane/src/scale-runners/scale-up-contract.test.ts Tests mixed-organization capacity.
lambdas/functions/control-plane/src/scale-runners/scale-down.ts Adds organization-specific authentication and retention.
lambdas/functions/control-plane/src/scale-runners/scale-down.test.ts Tests multi-org cleanup behavior.
lambdas/functions/control-plane/src/scale-runners/job-retry.ts Enables organization-scoped retry checks.
lambdas/functions/control-plane/src/scale-runners/github-runner.ts Scopes installation and group caching.
lambdas/functions/control-plane/src/scale-runners/github-runner.multi-org.test.ts Tests registration isolation.
lambdas/functions/control-plane/src/pool/pool.ts Selects pool owners per event.
lambdas/functions/control-plane/src/pool/pool.test.ts Tests organization-specific pools.
lambdas/functions/control-plane/src/github/octokit.ts Resolves retry installations per organization.
lambdas/functions/control-plane/src/github/octokit.test.ts Tests multi-org retry authentication.
lambdas/functions/control-plane/src/github/multi-org.ts Centralizes feature-flag parsing.
docs/multi-org.md Adds configuration and operational guidance.
Review details
  • Files reviewed: 52/53 changed files
  • Comments generated: 3
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread lambdas/functions/control-plane/src/pool/pool.ts Outdated
Comment thread modules/orchestration-providers/webhook/pool/variables.tf Outdated
Comment thread modules/runners/pool/variables.tf Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Organization identity must be normalized consistently to prevent case variants from bypassing per-organization capacity and retention boundaries.

Get a fresh assessment by requesting another Copilot review.

Review details
  • Files reviewed: 53/54 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread lambdas/functions/control-plane/src/pool/pool.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Idle retention is incorrectly allocated per legacy repository after multi-org mode is enabled.

Get a fresh assessment by requesting another Copilot review.

Review details
  • Files reviewed: 61/62 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread lambdas/functions/control-plane/src/scale-runners/scale-down.ts

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

It changes authentication and runner lifecycle behavior across many components, while live AWS and GitHub validation remains outstanding.

Review details
  • Files reviewed: 61/62 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants