From 150638c4143447cb9315af39bf6d5cbeedbc9d8a Mon Sep 17 00:00:00 2001 From: edersonbrilhante Date: Thu, 17 Sep 2026 16:50:01 +0200 Subject: [PATCH 01/20] feat(scale-set): isolate orchestration provider module --- .../scale-set/README.md | 256 ++++ .../scale-set/cluster.tf | 14 + .../scale-set/config-store.tf | 27 + .../orchestration-providers/scale-set/data.tf | 5 + .../orchestration-providers/scale-set/iam.tf | 217 ++++ .../scale-set/locals.tf | 254 ++++ .../scale-set/logging.tf | 19 + .../scale-set/networking.tf | 27 + .../scale-set/outputs.tf | 58 + .../scale-set/service.tf | 40 + .../orchestration-providers/scale-set/task.tf | 128 ++ .../tests/computed-inputs.tftest.hcl | 49 + .../tests/fixtures/computed-inputs/README.md | 38 + .../tests/fixtures/computed-inputs/main.tf | 100 ++ .../fixtures/computed-inputs/versions.tf | 10 + .../scale-set/tests/scale-set.tftest.hcl | 1137 +++++++++++++++++ .../scale-set/validations.tf | 425 ++++++ .../scale-set/variables.tf | 200 +++ .../scale-set/versions.tf | 10 + 19 files changed, 3014 insertions(+) create mode 100644 modules/orchestration-providers/scale-set/README.md create mode 100644 modules/orchestration-providers/scale-set/cluster.tf create mode 100644 modules/orchestration-providers/scale-set/config-store.tf create mode 100644 modules/orchestration-providers/scale-set/data.tf create mode 100644 modules/orchestration-providers/scale-set/iam.tf create mode 100644 modules/orchestration-providers/scale-set/locals.tf create mode 100644 modules/orchestration-providers/scale-set/logging.tf create mode 100644 modules/orchestration-providers/scale-set/networking.tf create mode 100644 modules/orchestration-providers/scale-set/outputs.tf create mode 100644 modules/orchestration-providers/scale-set/service.tf create mode 100644 modules/orchestration-providers/scale-set/task.tf create mode 100644 modules/orchestration-providers/scale-set/tests/computed-inputs.tftest.hcl create mode 100644 modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/README.md create mode 100644 modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/main.tf create mode 100644 modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/versions.tf create mode 100644 modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl create mode 100644 modules/orchestration-providers/scale-set/validations.tf create mode 100644 modules/orchestration-providers/scale-set/variables.tf create mode 100644 modules/orchestration-providers/scale-set/versions.tf diff --git a/modules/orchestration-providers/scale-set/README.md b/modules/orchestration-providers/scale-set/README.md new file mode 100644 index 0000000000..495a8cc314 --- /dev/null +++ b/modules/orchestration-providers/scale-set/README.md @@ -0,0 +1,256 @@ +# Scale-set orchestration provider + +This internal module deploys long-running GitHub Actions runner scale-set controllers on ECS Fargate. It creates one deployment unit per resolved **controller group**: + +```text +1 ECS service +1 task definition +1 running task during normal operation +1 application container +1 ScaleSetController supervising N independent reconcilers +``` + +Each reconciler still owns exactly one GitHub scale-set identity and one message session. Grouping only packs reconcilers into a shared task; it does not merge scale-set identity, session state, or compute-provider behavior. It does, however, intentionally union task IAM permissions and failure/deployment blast radius across all members of that controller group. + +This foundation adopts scale sets that were created elsewhere. It passes the configured name to the controller, which captures the scale-set and runner-group identifiers dynamically from GitHub; it does not create or delete the GitHub scale-set resource. The complete compute-provider contract must likewise come from its Terraform adapter; until that adapter and the public runner-config selection are wired, this internal module is not an end-to-end deployment interface. + +The normalized `(githubConfigUrl, scale_set.name)` ownership tuple must be globally unique across all groups. `runner_registration_level` selects organization or repository scope; `runner_owner` supplies the corresponding path appended to the GitHub server URL. A null enterprise-server URL resolves to GitHub.com. Enterprise-level registration is not supported by this module. Duplicate detection normalizes URL case, one trailing slash, and an explicit default `:443` port, so equivalent spellings cannot accidentally deploy two services against one GitHub message session. Scale-set names may repeat under different GitHub scopes. + +## Grouping + +`grouping.strategy` selects a plan-known grouping implementation: + +- `compute_provider` (default): one group per `compute_provider_contracts[*].type`. +- `runner_config`: one group per runner-config key. +- `custom`: explicit groups whose membership covers every runner config exactly once. + +```hcl +grouping = { + strategy = "custom" + custom = { + groups = { + general = { + runner_configs = ["linux-small", "linux-large"] + } + critical = { + runner_configs = ["production"] + } + } + } +} +``` + +Group names and memberships become Terraform `for_each` identities and must be known during planning. A group may contain at most 1000 runner configs, matching the service loader limit. Additional grouping algorithms can be added later by producing the same internal `map(list(runner_config_name))` shape. + +## Compute-provider capability boundary + +`compute_provider_contracts` is keyed exactly like `runner_configs`. A compute provider implements the scale-set desired-capacity interface by returning: + +```hcl +{ + type = "ec2" # plan-known grouping and runtime registry key + capabilities = { + scale_set = { + configuration_json = local.provider_owned_runtime_configuration + environment_variables = local.provider_owned_non_secret_environment + iam_statements = local.provider_owned_compute_role_statements + } + } +} +``` + +The symbolic locals above represent outputs from the selected compute-provider Terraform adapter; callers should not recreate the provider payload by hand. The provider-specific adapter owns the runtime configuration schema and the complete IAM statement set. This orchestration module treats configuration JSON as an opaque, non-secret object and combines only the selected group's statements into the corresponding compute role. Provider-owned process environment variables are also non-secret: duplicate names within a group must resolve to the same value, and reserved runtime names cannot be overridden. Runner-config-specific values stay in the SSM reconciler document, while credentials stay behind SSM references. Wildcard IAM actions are rejected. The rendered controller and compute-role policies are each checked against AWS's 10,240-byte inline role-policy quota with an explicit split-the-group error; group splitting remains the escape hatch when the union is too large or too broad. + +## Configuration delivery + +For the current ECS deployment, each task receives one bounded `SCALE_SET_CONTROLLER_MANIFEST` environment variable. Its value is JSON with this shape: + +```text +{ + "version": 1, + "groupName": "ec2", + "revision": "", + "reconcilers": [ + { + "schemaVersion": 1, + "runnerConfigName": "linux-small", + "runnerGroupName": "Default", + "scaleSetName": "linux-small", + "githubConfigUrl": "https://github.com/example", + "githubApp": { + "appIdParameterName": "/github/app-id", + "privateKeyParameterName": "/github/private-key", + "installationIdParameterName": "/github/installation-id" + }, + "computeProvider": { + "type": "ec2", + "configuration": {} + }, + "minRunners": 0, + "maxRunners": 20, + "bootTimeoutMinutes": 10, + "sessionOwner": "ec2.linux-small", + "workFolder": "_work", + "forceGhes": false, + "sslVerify": true + } + ] +} +``` + +Each leaf is the flat `ScaleSetReconcilerConfig` consumed by the service. GitHub credential values never enter the manifest; it contains only the exact Parameter Store names used by the runtime. The manifest source is mutually exclusive with the service's SSM group-path source, so the task does not receive `SCALE_SET_CONTROLLER_GROUP_CONFIG_PATH` or `SCALE_SET_CONTROLLER_GROUP_CONFIG_REVISION`. + +The module also keeps the per-reconciler SSM parameters available for the grouped configuration path while that delivery mode is being phased in. The task currently uses the manifest environment variable. + +Terraform derives `sessionOwner` locally as `.`; if that would exceed the runtime's 256-character limit, the module truncates both readable components and appends a deterministic hash. + +The manifest must remain within the ECS task-definition size budget. Use the SSM group-path delivery mode for larger groups once it is enabled by the deployment configuration. + +The individual reconciler object has this shape: + +```json +{ + "schemaVersion": 1, + "runnerConfigName": "linux-small", + "runnerGroupName": "Default", + "githubConfigUrl": "https://github.com/example", + "scaleSetName": "linux-small", + "minRunners": 0, + "maxRunners": 20, + "bootTimeoutMinutes": 10, + "sessionOwner": "ec2.linux-small", + "workFolder": "_work", + "forceGhes": false, + "sslVerify": true, + "githubApp": { + "appIdParameterName": "/github/app-id", + "privateKeyParameterName": "/github/private-key", + "installationIdParameterName": "/github/installation-id" + }, + "computeProvider": { + "type": "ec2", + "configuration": {} + } +} +``` + +GitHub credential **values** never enter Terraform configuration, task definitions, or controller-config parameters. Each leaf carries only three Parameter Store names. The task role can read the exact credential parameter ARNs for its group and decrypt only explicitly declared KMS keys. + +## Container image + +The convenience default is: + +```text +ghcr.io/github-aws-runners/terraform-aws-github-runner-scale-set-service:latest +``` + +ECS `versionConsistency` is enabled so all tasks in a deployment resolve a tag consistently. Production callers should set `container.image` to the digest published with a release: + +```hcl +container = { + image = "ghcr.io/github-aws-runners/terraform-aws-github-runner-scale-set-service@sha256:" +} +``` + +Public registry images need no pull permission. For a private ECR override, set `container.image` to the ECR image URI. The module grants the ECS task execution role wildcard ECR layer-pull permissions plus the unavoidable resource-unscoped `ecr:GetAuthorizationToken` action. The application task role is not used for image pulls. Repository-side access policy remains owned by the ECR module that owns the repository. + +For the official GHCR default, verify an anonymous pull after the first package publish. Package visibility may inherit repository or organization settings and must not be inferred only from a successful authenticated workflow push. + +## ECS and security behavior + +- A managed ECS cluster is created by default. Set `ecs.cluster.mode = "external"` and pass `ecs.cluster.arn` to reuse a cluster. The mode must be known at plan time; the ARN may be computed. +- Every group gets a separate service, task definition, task role, execution role, log group, and security group. +- `desired_count` is fixed at one. Deployment percentages are `minimum = 0` and `maximum = 100`, preventing old and new tasks from overlapping while session leasing is unavailable. +- The ECS deployment circuit breaker and rollback are enabled. +- Tasks run in supplied private subnets with public IP assignment disabled. Managed security groups have no ingress and allow only TCP/443 egress. The IPv4 Internet default is intended for controlled NAT/firewall paths and can be narrowed. +- The application container runs with a numeric non-root UID/GID, a read-only root filesystem, init enabled, no privilege, and all Linux capabilities dropped. +- ECS probes `/healthz` for liveness, and `container.health_path` accepts only that endpoint. `/readyz` remains an application readiness signal; reconnecting to GitHub should not cause ECS to restart every reconciler in a group. +- CloudWatch encrypts logs at rest with an AWS-owned key by default. Set `logging.kms_key_arn` for a customer-managed key and ensure its key policy allows the regional CloudWatch Logs service. + +## Plan-shape requirements + +The following values control `for_each`, dynamic IAM statements, or resource ownership and must be known during planning: + +- runner-config map keys; +- compute-contract map keys and provider `type`; +- grouping strategy, custom group keys, and membership; +- IAM statement keys and optional KMS/ECR wrapper presence; +- optional ECS ephemeral-storage wrapper presence; +- managed versus external cluster mode. + +Inner values such as scale-set names, SSM/KMS ARNs, provider configuration values, IAM actions/resources, and an external cluster ARN may be computed. Nullable computed values should be placed inside a plan-known wrapper rather than used as the wrapper itself. + + +## Requirements + +| Name | Version | +|------|---------| +| [terraform](#requirement\_terraform) | >= 1.5.6 | +| [aws](#requirement\_aws) | >= 6.33 | + +## Providers + +| Name | Version | +|------|---------| +| [aws](#provider\_aws) | >= 6.33 | +| [terraform](#provider\_terraform) | n/a | + +## Modules + +No modules. + +## Resources + +| Name | Type | +|------|------| +| [aws_cloudwatch_log_group.controller](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) | resource | +| [aws_ecs_cluster.controller](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_cluster) | resource | +| [aws_ecs_service.controller](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service) | resource | +| [aws_ecs_task_definition.controller](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_task_definition) | resource | +| [aws_iam_role.compute](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource | +| [aws_iam_role.execution](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource | +| [aws_iam_role.task](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource | +| [aws_iam_role_policy.compute](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) | resource | +| [aws_iam_role_policy.execution](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) | resource | +| [aws_iam_role_policy.task](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) | resource | +| [aws_security_group.controller](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource | +| [aws_ssm_parameter.reconciler_config](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) | resource | +| [terraform_data.validate_compute_role_policy](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | +| [terraform_data.validate_config_store](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | +| [terraform_data.validate_contract](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | +| [terraform_data.validate_group_task_policy](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | +| [terraform_data.validate_grouping](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | +| [terraform_data.validate_runtime](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | +| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source | +| [aws_iam_policy_document.compute](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source | +| [aws_iam_policy_document.compute_assume_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source | +| [aws_iam_policy_document.execution](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source | +| [aws_iam_policy_document.task](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source | +| [aws_iam_policy_document.task_assume_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source | +| [aws_partition.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) | data source | +| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source | + +## Inputs + +| Name | Description | Type | Default | Required | +|------|-------------|------|---------|:--------:| +| [config\_store](#input\_config\_store) | Non-secret controller configuration storage. The module writes one SSM String parameter per reconciler below `path_prefix//`. The task receives only its group path and a SHA-256 revision, then loads the group with `GetParametersByPath`.

Standard parameters are limited to 4096 encoded bytes and Advanced parameters to 8192 encoded bytes. Null `path_prefix` resolves to `//scale-set-controller`. | 
object({
    path_prefix = optional(string, null)
    tier        = optional(string, "Standard")
    tags        = optional(map(string), {})
  })
| `{}` | no | +| [container](#input\_container) | Scale-set controller image and runtime settings. A null image uses the internal official convenience image; production callers should use the release digest. Filesystem and Linux capability hardening are enforced by the module; health\_path is fixed at /healthz, the ECS liveness endpoint. | 
object({
    image                             = optional(string, null)
    user                              = optional(string, "10001:10001")
    health_port                       = optional(number, 8080)
    health_path                       = optional(string, "/healthz")
    health_check_command              = optional(list(string), null)
    health_check_interval             = optional(number, 30)
    health_check_timeout              = optional(number, 5)
    health_check_retries              = optional(number, 3)
    health_check_start_period         = optional(number, 30)
    health_stale_after_seconds        = optional(number, 180)
    shutdown_timeout_seconds          = optional(number, 110)
    session_close_timeout_seconds     = optional(number, 10)
    reconnect_initial_backoff_seconds = optional(number, 1)
    reconnect_max_backoff_seconds     = optional(number, 30)
    stop_timeout_seconds              = optional(number, 120)
  })
| `{}` | no | +| [ecs](#input\_ecs) | ECS substrate configuration. A managed cluster is created by default. For an external cluster, set `cluster.mode = "external"` and pass its ARN; the mode must be plan-known while the ARN may be computed. | 
object({
    cluster = optional(object({
      mode               = optional(string, "managed")
      arn                = optional(string, null)
      name               = optional(string, null)
      container_insights = optional(bool, true)
    }), {})
    task = optional(object({
      cpu              = optional(number, 512)
      memory           = optional(number, 1024)
      cpu_architecture = optional(string, "X86_64")
      ephemeral_storage = optional(object({
        size_in_gib = number
      }), null)
    }), {})
    service = optional(object({
      platform_version = optional(string, "LATEST")
    }), {})
    iam = optional(object({
      path                 = optional(string, "/")
      permissions_boundary = optional(string, null)
    }), {})
  })
| `{}` | no | +| [grouping](#input\_grouping) | Packing strategy for scale-set reconcilers. `compute_provider` creates one controller group per compute-provider type and is the default. `runner_config` creates one group per runner config. `custom` uses `custom.groups`; custom membership must cover every runner config exactly once.

The strategy, custom group keys, and memberships select Terraform `for_each` instances and must be known during planning. | 
object({
    strategy = optional(string, "compute_provider")
    custom = optional(object({
      groups = map(object({
        runner_configs = set(string)
      }))
    }), null)
  })
| `{}` | no | +| [log\_level](#input\_log\_level) | Logging level for the scale-set controller container. | `string` | `"info"` | no | +| [logging](#input\_logging) | CloudWatch Logs configuration. CloudWatch encrypts logs at rest with an AWS-owned key by default; set `kms_key_arn` to use a customer-managed key. | 
object({
    retention_in_days = optional(number, 30)
    kms_key_arn       = optional(string, null)
    log_group_class   = optional(string, "STANDARD")
    tags              = optional(map(string), {})
  })
| `{}` | no | +| [network](#input\_network) | Private Fargate networking. Tasks never receive public IP addresses and the managed security groups have no ingress. HTTPS egress defaults to IPv4 Internet access because GitHub endpoints cannot be represented as security-group destinations; route it through controlled NAT, firewall, or proxy infrastructure when required. | 
object({
    vpc_id     = string
    subnet_ids = set(string)
    https_egress = optional(object({
      ipv4_cidrs = optional(set(string), ["0.0.0.0/0"])
      ipv6_cidrs = optional(set(string), [])
    }), {})
  })
| n/a | yes | +| [prefix](#input\_prefix) | Stable prefix used for scale-set controller resources. | `string` | `"github-actions"` | no | +| [runner\_configs](#input\_runner\_configs) | Normalized scale-set runner configurations keyed by stable runner-config name.

Map keys must be known during planning. Credential values are never accepted: `github.app` contains only the exact GitHub App Parameter Store references used by the runtime. `github.enterprise_server` and `github.user_agent` carry the global GitHub settings needed to render each reconciler configuration. `scale_set.runner.group_name` selects the GitHub runner group. `runner_registration_level` selects organization or repository registration, and `runner_owner` supplies the corresponding organization or owner/repository path. Enterprise-level registration is not supported by this module. `compute_provider` carries the provider-neutral scale-set capability contract for this runner configuration. Parameter and optional KMS ARNs, scale-set names, and other inner values may remain unknown until apply. | 
map(object({
    github = object({
      enterprise_server = object({
        url        = optional(string, null)
        ssl_verify = optional(bool, true)
      })
      app = object({
        app_id = object({
          name        = string
          arn         = string
          kms_key_arn = optional(string, null)
        })
        private_key = object({
          name        = string
          arn         = string
          kms_key_arn = optional(string, null)
        })
        installation_id = object({
          name        = string
          arn         = string
          kms_key_arn = optional(string, null)
        })
      })
      runner_owner              = string
      runner_registration_level = string
      user_agent                = string
    })
    scale_set = object({
      name = string
      runner = optional(object({
        labels               = optional(list(string), [])
        group_name           = optional(string, "Default")
        min_runners          = optional(number, 0)
        max_runners          = optional(number, 10)
        boot_time_in_minutes = optional(number, 10)
      }), {})
    })
    compute_provider = object({
      type = string
      capabilities = object({
        scale_set = object({
          role_arn              = optional(string, null)
          configuration_json    = optional(string, "{}")
          environment_variables = optional(map(string), {})
          iam_statements = optional(map(object({
            actions   = set(string)
            resources = set(string)
            conditions = optional(list(object({
              test     = string
              variable = string
              values   = set(string)
            })), [])
          })), {})
        })
      })
    })
  }))
| n/a | yes | +| [tags](#input\_tags) | Tags applied to scale-set orchestration resources. | `map(string)` | `{}` | no | + +## Outputs + +| Name | Description | +|------|-------------| +| [cluster](#output\_cluster) | Managed or external ECS cluster selected for all controller groups. | +| [controller\_groups](#output\_controller\_groups) | Controller-group resources keyed by stable resolved group name. | +| [reconciler\_config\_parameters](#output\_reconciler\_config\_parameters) | Non-secret SSM controller configuration parameters keyed by `/`. Values are intentionally not exposed. | +| [resolved\_container\_image](#output\_resolved\_container\_image) | Container image reference selected for the controller task definitions. | + diff --git a/modules/orchestration-providers/scale-set/cluster.tf b/modules/orchestration-providers/scale-set/cluster.tf new file mode 100644 index 0000000000..6c9a586da8 --- /dev/null +++ b/modules/orchestration-providers/scale-set/cluster.tf @@ -0,0 +1,14 @@ +resource "aws_ecs_cluster" "controller" { + count = var.ecs.cluster.mode == "managed" ? 1 : 0 + + name = coalesce(var.ecs.cluster.name, "${var.prefix}-scale-set") + + setting { + name = "containerInsights" + value = var.ecs.cluster.container_insights ? "enabled" : "disabled" + } + + tags = local.common_tags + + depends_on = [terraform_data.validate_runtime] +} diff --git a/modules/orchestration-providers/scale-set/config-store.tf b/modules/orchestration-providers/scale-set/config-store.tf new file mode 100644 index 0000000000..86cb42a73a --- /dev/null +++ b/modules/orchestration-providers/scale-set/config-store.tf @@ -0,0 +1,27 @@ +resource "aws_ssm_parameter" "reconciler_config" { + for_each = local.reconciler_configs + + name = "${local.config_store_path_prefix}/${each.value.group_name}/${each.value.runner_name}" + description = "Non-secret scale-set reconciler configuration for ${each.value.runner_name}" + type = "String" + tier = var.config_store.tier + value = local.reconciler_config_json[each.key] + + tags = merge( + local.group_tags[each.value.group_name], + var.config_store.tags, + ) + + lifecycle { + precondition { + condition = local.reconciler_config_bytes[each.key] <= local.config_store_max_bytes + error_message = "The encoded reconciler configuration exceeds the selected Parameter Store tier limit." + } + } + + depends_on = [ + terraform_data.validate_contract, + terraform_data.validate_grouping, + terraform_data.validate_config_store, + ] +} diff --git a/modules/orchestration-providers/scale-set/data.tf b/modules/orchestration-providers/scale-set/data.tf new file mode 100644 index 0000000000..99b50de05a --- /dev/null +++ b/modules/orchestration-providers/scale-set/data.tf @@ -0,0 +1,5 @@ +data "aws_caller_identity" "current" {} + +data "aws_partition" "current" {} + +data "aws_region" "current" {} diff --git a/modules/orchestration-providers/scale-set/iam.tf b/modules/orchestration-providers/scale-set/iam.tf new file mode 100644 index 0000000000..6065fff396 --- /dev/null +++ b/modules/orchestration-providers/scale-set/iam.tf @@ -0,0 +1,217 @@ +data "aws_iam_policy_document" "task_assume_role" { + statement { + sid = "AllowEcsTasks" + effect = "Allow" + actions = ["sts:AssumeRole"] + + principals { + type = "Service" + identifiers = ["ecs-tasks.amazonaws.com"] + } + + condition { + test = "StringEquals" + variable = "aws:SourceAccount" + values = [data.aws_caller_identity.current.account_id] + } + + condition { + test = "ArnLike" + variable = "aws:SourceArn" + values = [format( + "arn:%s:ecs:%s:%s:*", + data.aws_partition.current.partition, + data.aws_region.current.region, + data.aws_caller_identity.current.account_id, + )] + } + } +} + +resource "aws_iam_role" "task" { + for_each = local.controller_groups + + name = "${local.group_resource_names[each.key]}-task" + path = var.ecs.iam.path + permissions_boundary = var.ecs.iam.permissions_boundary + assume_role_policy = data.aws_iam_policy_document.task_assume_role.json + tags = local.group_tags[each.key] + + depends_on = [ + terraform_data.validate_contract, + terraform_data.validate_grouping, + terraform_data.validate_runtime, + ] +} + +data "aws_iam_policy_document" "task" { + for_each = local.controller_groups + + source_policy_documents = [local.group_github_kms_policy_json[each.key]] + + statement { + sid = "ReadControllerGroupConfig" + effect = "Allow" + actions = ["ssm:GetParametersByPath"] + resources = [local.group_config_path_arns[each.key]] + } + + statement { + sid = "ReadGitHubAppParameters" + effect = "Allow" + actions = [ + "ssm:GetParameter", + "ssm:GetParameters", + ] + resources = [for parameter in local.group_github_parameters[each.key] : parameter.arn] + } + + statement { + sid = "AssumeComputeProviderRoles" + effect = "Allow" + actions = ["sts:AssumeRole"] + resources = [for runner_name in local.controller_groups[each.key] : local.compute_role_arns["${each.key}/${runner_name}"]] + } +} + +resource "aws_iam_role_policy" "task" { + for_each = local.controller_groups + + name = "scale-set-controller" + role = aws_iam_role.task[each.key].name + policy = data.aws_iam_policy_document.task[each.key].json + + depends_on = [terraform_data.validate_group_task_policy] +} + +data "aws_iam_policy_document" "compute_assume_role" { + for_each = local.compute_role_configs + + statement { + sid = "AllowScaleSetTask" + effect = "Allow" + actions = ["sts:AssumeRole"] + + principals { + type = "AWS" + identifiers = [format( + "arn:%s:iam::%s:root", + data.aws_partition.current.partition, + data.aws_caller_identity.current.account_id, + )] + } + + condition { + test = "ArnEquals" + variable = "aws:PrincipalArn" + values = [format( + "arn:%s:iam::%s:role%s%s-task", + data.aws_partition.current.partition, + data.aws_caller_identity.current.account_id, + var.ecs.iam.path, + local.group_resource_names[each.value.group_name], + )] + } + } +} + +resource "aws_iam_role" "compute" { + for_each = local.compute_role_configs + + name = "${local.group_resource_names[each.value.group_name]}-compute-${substr(sha256(each.key), 0, 8)}" + path = var.ecs.iam.path + permissions_boundary = var.ecs.iam.permissions_boundary + assume_role_policy = data.aws_iam_policy_document.compute_assume_role[each.key].json + tags = local.group_tags[each.value.group_name] +} + +data "aws_iam_policy_document" "compute" { + for_each = local.compute_role_configs + + dynamic "statement" { + for_each = local.reconciler_compute_iam_statements[each.key] + + content { + effect = "Allow" + actions = statement.value.actions + resources = statement.value.resources + + dynamic "condition" { + for_each = statement.value.conditions + + content { + test = condition.value.test + variable = condition.value.variable + values = condition.value.values + } + } + } + } +} + +resource "aws_iam_role_policy" "compute" { + for_each = local.compute_role_configs + + name = "scale-set-compute" + role = aws_iam_role.compute[each.key].name + policy = data.aws_iam_policy_document.compute[each.key].json + + depends_on = [terraform_data.validate_compute_role_policy] +} + +resource "aws_iam_role" "execution" { + for_each = local.controller_groups + + name = "${local.group_resource_names[each.key]}-exec" + path = var.ecs.iam.path + permissions_boundary = var.ecs.iam.permissions_boundary + assume_role_policy = data.aws_iam_policy_document.task_assume_role.json + tags = local.group_tags[each.key] + + depends_on = [ + terraform_data.validate_contract, + terraform_data.validate_grouping, + terraform_data.validate_runtime, + ] +} + +data "aws_iam_policy_document" "execution" { + for_each = local.controller_groups + + statement { + sid = "WriteControllerLogs" + effect = "Allow" + actions = [ + "logs:CreateLogStream", + "logs:PutLogEvents", + ] + resources = ["${aws_cloudwatch_log_group.controller[each.key].arn}:*"] + } + + statement { + sid = "PullPrivateEcrImage" + effect = "Allow" + actions = [ + "ecr:BatchCheckLayerAvailability", + "ecr:BatchGetImage", + "ecr:GetDownloadUrlForLayer", + ] + resources = ["*"] + } + + statement { + # ECR does not support resource-level permissions for authorization tokens. + sid = "AuthorizePrivateEcrPull" + effect = "Allow" + actions = ["ecr:GetAuthorizationToken"] + resources = ["*"] + } +} + +resource "aws_iam_role_policy" "execution" { + for_each = local.controller_groups + + name = "scale-set-controller-execution" + role = aws_iam_role.execution[each.key].name + policy = data.aws_iam_policy_document.execution[each.key].json +} diff --git a/modules/orchestration-providers/scale-set/locals.tf b/modules/orchestration-providers/scale-set/locals.tf new file mode 100644 index 0000000000..523b6b5a26 --- /dev/null +++ b/modules/orchestration-providers/scale-set/locals.tf @@ -0,0 +1,254 @@ +locals { + github_config_urls = { + for runner_name, runner_config in var.runner_configs : runner_name => format( + "%s%s", + trimsuffix(coalesce(runner_config.github.enterprise_server.url, "https://github.com"), "/"), + runner_config.github.runner_owner == null ? "" : "/${runner_config.github.runner_owner}", + ) + } + declared_custom_groups = var.grouping.strategy == "custom" && var.grouping.custom != null ? { + for group_name, group in var.grouping.custom.groups : group_name => sort(tolist(group.runner_configs)) + } : {} + + controller_groups = ( + var.grouping.strategy == "compute_provider" ? { + for provider_type in distinct([ + for runner_name in keys(var.runner_configs) : var.runner_configs[runner_name].compute_provider.type + ]) : provider_type => [ + for runner_name in keys(var.runner_configs) : runner_name + if var.runner_configs[runner_name].compute_provider.type == provider_type + ] + } : + var.grouping.strategy == "runner_config" ? { + for runner_name in keys(var.runner_configs) : runner_name => [runner_name] + } : + var.grouping.strategy == "custom" ? { + for group_name, runner_names in local.declared_custom_groups : group_name => [ + for runner_name in runner_names : runner_name + if contains(keys(var.runner_configs), runner_name) + ] + } : + {} + ) + + group_resource_names = { + for group_name in keys(local.controller_groups) : group_name => format( + "%s-ss-%s-%s", + var.prefix, + substr(replace(lower(group_name), "/[^a-z0-9_-]/", "-"), 0, 14), + substr(sha256(group_name), 0, 8), + ) + } + + official_container_image = "ghcr.io/github-aws-runners/terraform-aws-github-runner-scale-set-service:latest" + resolved_container_image = coalesce(var.container.image, local.official_container_image) + + resolved_health_check_command = var.container.health_check_command != null ? var.container.health_check_command : [ + "CMD", + "node", + "-e", + "fetch('http://127.0.0.1:${var.container.health_port}${var.container.health_path}').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))", + ] + + reconciler_configs = merge([ + for group_name, runner_names in local.controller_groups : { + for runner_name in runner_names : "${group_name}/${runner_name}" => { + group_name = group_name + runner_name = runner_name + value = merge({ + schemaVersion = 1 + runnerConfigName = runner_name + runnerGroupName = var.runner_configs[runner_name].scale_set.runner.group_name + runnerLabels = var.runner_configs[runner_name].scale_set.runner.labels + githubConfigUrl = local.github_config_urls[runner_name] + scaleSetName = var.runner_configs[runner_name].scale_set.name + minRunners = var.runner_configs[runner_name].scale_set.runner.min_runners + maxRunners = var.runner_configs[runner_name].scale_set.runner.max_runners + bootTimeoutMinutes = var.runner_configs[runner_name].scale_set.runner.boot_time_in_minutes + workFolder = "_work" + sslVerify = var.runner_configs[runner_name].github.enterprise_server.ssl_verify + forceGhes = var.runner_configs[runner_name].github.enterprise_server.url != null + sessionOwner = ( + length("${group_name}.${runner_name}") <= 256 + ? "${group_name}.${runner_name}" + : "${substr(group_name, 0, 119)}.${substr(runner_name, 0, 119)}.${substr(sha256(format("%s.%s", group_name, runner_name)), 0, 16)}" + ) + githubApp = { + appIdParameterName = var.runner_configs[runner_name].github.app.app_id.name + privateKeyParameterName = var.runner_configs[runner_name].github.app.private_key.name + installationIdParameterName = var.runner_configs[runner_name].github.app.installation_id.name + } + computeProvider = { + type = var.runner_configs[runner_name].compute_provider.type + roleArn = local.compute_role_arns["${group_name}/${runner_name}"] + configuration = jsondecode(var.runner_configs[runner_name].compute_provider.capabilities.scale_set.configuration_json) + } + userAgent = var.runner_configs[runner_name].github.user_agent + }) + } + } + ]...) + + config_store_path_prefix = coalesce(var.config_store.path_prefix, "/${var.prefix}/scale-set-controller") + group_config_paths = { + for group_name in keys(local.controller_groups) : group_name => "${local.config_store_path_prefix}/${group_name}" + } + group_config_revisions = { + for group_name, runner_names in local.controller_groups : group_name => sha256(jsonencode({ + for runner_name in runner_names : runner_name => local.reconciler_configs["${group_name}/${runner_name}"].value + })) + } + + group_controller_manifests = { + for group_name, runner_names in local.controller_groups : group_name => jsonencode({ + version = 1 + groupName = group_name + revision = local.group_config_revisions[group_name] + reconcilers = [ + for runner_name in runner_names : local.reconciler_configs["${group_name}/${runner_name}"].value + ] + }) + } + group_github_parameters = { + for group_name, runner_names in local.controller_groups : group_name => flatten([ + for runner_name in runner_names : [ + { + arn = var.runner_configs[runner_name].github.app.app_id.arn + kms_key_arn = var.runner_configs[runner_name].github.app.app_id.kms_key_arn + }, + { + arn = var.runner_configs[runner_name].github.app.private_key.arn + kms_key_arn = var.runner_configs[runner_name].github.app.private_key.kms_key_arn + }, + { + arn = var.runner_configs[runner_name].github.app.installation_id.arn + kms_key_arn = var.runner_configs[runner_name].github.app.installation_id.kms_key_arn + }, + ] + ]) + } + + group_github_kms_policy_json = { + for group_name, parameters in local.group_github_parameters : group_name => jsonencode({ + Version = "2012-10-17" + Statement = length(compact([for parameter in parameters : parameter.kms_key_arn])) == 0 ? [] : [{ + Sid = "DecryptGitHubAppParameters" + Effect = "Allow" + Action = ["kms:Decrypt"] + Resource = distinct(compact([for parameter in parameters : parameter.kms_key_arn])) + }] + }) + } + + compute_role_configs = { + for config_key in flatten([ + for group_name, runner_names in local.controller_groups : [ + for runner_name in runner_names : { + key = "${group_name}/${runner_name}" + group_name = group_name + runner_name = runner_name + } + ] + ]) : config_key.key => config_key + if var.runner_configs[config_key.runner_name].compute_provider.capabilities.scale_set.role_arn == null + } + + compute_role_arns = { + for config in flatten([ + for group_name, runner_names in local.controller_groups : [ + for runner_name in runner_names : { + key = "${group_name}/${runner_name}" + group_name = group_name + runner_name = runner_name + } + ] + ]) : config.key => ( + var.runner_configs[config.runner_name].compute_provider.capabilities.scale_set.role_arn != null + ? var.runner_configs[config.runner_name].compute_provider.capabilities.scale_set.role_arn + : format( + "arn:%s:iam::%s:role%s%s-compute-%s", + data.aws_partition.current.partition, + data.aws_caller_identity.current.account_id, + var.ecs.iam.path, + local.group_resource_names[config.group_name], + substr(sha256(config.key), 0, 8), + ) + ) + } + + reconciler_compute_iam_statements = { + for config_key, config in local.compute_role_configs : config_key => { + for statement_name, statement in var.runner_configs[config.runner_name].compute_provider.capabilities.scale_set.iam_statements : + statement_name => statement + } + } + + group_compute_environment_entries = { + for group_name, runner_names in local.controller_groups : group_name => flatten([ + for runner_name in runner_names : [ + for name, value in var.runner_configs[runner_name].compute_provider.capabilities.scale_set.environment_variables : { + runner_name = runner_name + name = name + value = value + } + ] + ]) + } + + group_compute_environment_variables = { + for group_name, entries in local.group_compute_environment_entries : group_name => merge([ + for entry in entries : { (entry.name) = entry.value } + ]...) + } + + config_store_max_bytes = var.config_store.tier == "Advanced" ? 8192 : 4096 + + reconciler_config_json = { + for config_key, config in local.reconciler_configs : config_key => jsonencode(config.value) + } + reconciler_config_bytes = { + for config_key, config_json in local.reconciler_config_json : config_key => ( + floor(length(base64encode(config_json)) * 3 / 4) - + (endswith(base64encode(config_json), "==") ? 2 : endswith(base64encode(config_json), "=") ? 1 : 0) + ) + } + + cluster_arn = var.ecs.cluster.mode == "managed" ? aws_ecs_cluster.controller[0].arn : var.ecs.cluster.arn + + group_config_path_arns = { + for group_name, config_path in local.group_config_paths : group_name => format( + "arn:%s:ssm:%s:%s:parameter%s/*", + data.aws_partition.current.partition, + data.aws_region.current.region, + data.aws_caller_identity.current.account_id, + config_path, + ) + } + + fargate_memory_by_cpu = { + 256 = [512, 1024, 2048] + 512 = [1024, 2048, 3072, 4096] + 1024 = range(2048, 9216, 1024) + 2048 = range(4096, 17408, 1024) + 4096 = range(8192, 31744, 1024) + 8192 = range(16384, 65536, 4096) + 16384 = range(32768, 131072, 8192) + } + + common_tags = merge( + { + "ghr:component" = "scale-set-controller" + }, + var.tags, + ) + + group_tags = { + for group_name, resource_name in local.group_resource_names : group_name => merge( + local.common_tags, + { + Name = resource_name + "ghr:controller-group" = group_name + }, + ) + } +} diff --git a/modules/orchestration-providers/scale-set/logging.tf b/modules/orchestration-providers/scale-set/logging.tf new file mode 100644 index 0000000000..974106316a --- /dev/null +++ b/modules/orchestration-providers/scale-set/logging.tf @@ -0,0 +1,19 @@ +resource "aws_cloudwatch_log_group" "controller" { + for_each = local.controller_groups + + name = "/aws/ecs/${local.group_resource_names[each.key]}" + retention_in_days = var.logging.retention_in_days + kms_key_id = var.logging.kms_key_arn + log_group_class = var.logging.log_group_class + + tags = merge( + local.group_tags[each.key], + var.logging.tags, + ) + + depends_on = [ + terraform_data.validate_contract, + terraform_data.validate_grouping, + terraform_data.validate_runtime, + ] +} diff --git a/modules/orchestration-providers/scale-set/networking.tf b/modules/orchestration-providers/scale-set/networking.tf new file mode 100644 index 0000000000..5cd9d11203 --- /dev/null +++ b/modules/orchestration-providers/scale-set/networking.tf @@ -0,0 +1,27 @@ +resource "aws_security_group" "controller" { + for_each = local.controller_groups + + name = local.group_resource_names[each.key] + description = "Private scale-set controller ${each.key}; no ingress and HTTPS-only egress" + vpc_id = var.network.vpc_id + + ingress = [] + + egress { + description = "HTTPS to GitHub and AWS APIs" + from_port = 443 + to_port = 443 + protocol = "tcp" + cidr_blocks = sort(tolist(var.network.https_egress.ipv4_cidrs)) + ipv6_cidr_blocks = sort(tolist(var.network.https_egress.ipv6_cidrs)) + } + + revoke_rules_on_delete = true + tags = local.group_tags[each.key] + + depends_on = [ + terraform_data.validate_contract, + terraform_data.validate_grouping, + terraform_data.validate_runtime, + ] +} diff --git a/modules/orchestration-providers/scale-set/outputs.tf b/modules/orchestration-providers/scale-set/outputs.tf new file mode 100644 index 0000000000..fd9d1b8c57 --- /dev/null +++ b/modules/orchestration-providers/scale-set/outputs.tf @@ -0,0 +1,58 @@ +output "cluster" { + description = "Managed or external ECS cluster selected for all controller groups." + value = { + arn = local.cluster_arn + managed = var.ecs.cluster.mode == "managed" + } +} + +output "controller_groups" { + description = "Controller-group resources keyed by stable resolved group name." + value = { + for group_name, runner_names in local.controller_groups : group_name => { + runner_configs = runner_names + config_path = local.group_config_paths[group_name] + config_revision = local.group_config_revisions[group_name] + service = { + id = aws_ecs_service.controller[group_name].id + name = aws_ecs_service.controller[group_name].name + } + task_definition = { + arn = aws_ecs_task_definition.controller[group_name].arn + family = aws_ecs_task_definition.controller[group_name].family + } + task_role = { + arn = aws_iam_role.task[group_name].arn + name = aws_iam_role.task[group_name].name + } + execution_role = { + arn = aws_iam_role.execution[group_name].arn + name = aws_iam_role.execution[group_name].name + } + log_group = { + arn = aws_cloudwatch_log_group.controller[group_name].arn + name = aws_cloudwatch_log_group.controller[group_name].name + } + security_group = { + arn = aws_security_group.controller[group_name].arn + id = aws_security_group.controller[group_name].id + } + } + } +} + +output "reconciler_config_parameters" { + description = "Non-secret SSM controller configuration parameters keyed by `/`. Values are intentionally not exposed." + value = { + for config_key, parameter in aws_ssm_parameter.reconciler_config : config_key => { + arn = parameter.arn + name = parameter.name + tier = parameter.tier + } + } +} + +output "resolved_container_image" { + description = "Container image reference selected for the controller task definitions." + value = local.resolved_container_image +} diff --git a/modules/orchestration-providers/scale-set/service.tf b/modules/orchestration-providers/scale-set/service.tf new file mode 100644 index 0000000000..953ebe4058 --- /dev/null +++ b/modules/orchestration-providers/scale-set/service.tf @@ -0,0 +1,40 @@ +resource "aws_ecs_service" "controller" { + for_each = local.controller_groups + + name = local.group_resource_names[each.key] + cluster = local.cluster_arn + task_definition = aws_ecs_task_definition.controller[each.key].arn + desired_count = 1 + launch_type = "FARGATE" + platform_version = var.ecs.service.platform_version + + scheduling_strategy = "REPLICA" + deployment_minimum_healthy_percent = 0 + deployment_maximum_percent = 100 + enable_ecs_managed_tags = true + enable_execute_command = false + propagate_tags = "SERVICE" + + deployment_circuit_breaker { + enable = true + rollback = true + } + + deployment_controller { + type = "ECS" + } + + network_configuration { + assign_public_ip = false + security_groups = [aws_security_group.controller[each.key].id] + subnets = sort(tolist(var.network.subnet_ids)) + } + + tags = local.group_tags[each.key] + + depends_on = [ + aws_iam_role_policy.execution, + aws_iam_role_policy.task, + aws_iam_role_policy.compute, + ] +} diff --git a/modules/orchestration-providers/scale-set/task.tf b/modules/orchestration-providers/scale-set/task.tf new file mode 100644 index 0000000000..b1e6f44006 --- /dev/null +++ b/modules/orchestration-providers/scale-set/task.tf @@ -0,0 +1,128 @@ +resource "aws_ecs_task_definition" "controller" { + for_each = local.controller_groups + + family = local.group_resource_names[each.key] + requires_compatibilities = ["FARGATE"] + network_mode = "awsvpc" + cpu = tostring(var.ecs.task.cpu) + memory = tostring(var.ecs.task.memory) + task_role_arn = aws_iam_role.task[each.key].arn + execution_role_arn = aws_iam_role.execution[each.key].arn + + runtime_platform { + cpu_architecture = var.ecs.task.cpu_architecture + operating_system_family = "LINUX" + } + + dynamic "ephemeral_storage" { + for_each = var.ecs.task.ephemeral_storage == null ? [] : [var.ecs.task.ephemeral_storage] + + content { + size_in_gib = ephemeral_storage.value.size_in_gib + } + } + + container_definitions = jsonencode([ + { + name = "scale-set-controller" + image = local.resolved_container_image + essential = true + user = var.container.user + privileged = false + readonlyRootFilesystem = true + stopTimeout = var.container.stop_timeout_seconds + versionConsistency = "enabled" + linuxParameters = { + initProcessEnabled = true + capabilities = { + drop = ["ALL"] + } + } + environment = concat( + [ + { + name = "LOG_LEVEL" + value = var.log_level + }, + { + name = "POWERTOOLS_SERVICE_NAME" + value = "scale-set-controller" + }, + { + name = "POWERTOOLS_LOG_LEVEL" + value = upper(var.log_level) + }, + { + name = "SCALE_SET_CONTROLLER_MANIFEST" + value = local.group_controller_manifests[each.key] + }, + { + name = "AWS_XRAY_CONTEXT_MISSING" + value = "IGNORE_ERROR" + }, + { + name = "AWS_REGION" + value = data.aws_region.current.region + }, + { + name = "AWS_DEFAULT_REGION" + value = data.aws_region.current.region + }, + { + name = "SCALE_SET_HEALTH_PORT" + value = tostring(var.container.health_port) + }, + { + name = "SCALE_SET_HEALTH_STALE_AFTER_SECONDS" + value = tostring(var.container.health_stale_after_seconds) + }, + { + name = "SCALE_SET_SHUTDOWN_TIMEOUT_SECONDS" + value = tostring(var.container.shutdown_timeout_seconds) + }, + { + name = "SCALE_SET_SESSION_CLOSE_TIMEOUT_SECONDS" + value = tostring(var.container.session_close_timeout_seconds) + }, + { + name = "SCALE_SET_RECONNECT_INITIAL_BACKOFF_SECONDS" + value = tostring(var.container.reconnect_initial_backoff_seconds) + }, + { + name = "SCALE_SET_RECONNECT_MAX_BACKOFF_SECONDS" + value = tostring(var.container.reconnect_max_backoff_seconds) + }, + ], + [ + for name in sort(keys(local.group_compute_environment_variables[each.key])) : { + name = name + value = local.group_compute_environment_variables[each.key][name] + } + ], + ) + healthCheck = { + command = local.resolved_health_check_command + interval = var.container.health_check_interval + timeout = var.container.health_check_timeout + retries = var.container.health_check_retries + startPeriod = var.container.health_check_start_period + } + logConfiguration = { + logDriver = "awslogs" + options = { + "awslogs-group" = aws_cloudwatch_log_group.controller[each.key].name + "awslogs-region" = data.aws_region.current.region + "awslogs-stream-prefix" = "controller" + } + } + } + ]) + + tags = local.group_tags[each.key] + + depends_on = [ + aws_iam_role_policy.execution, + aws_iam_role_policy.task, + aws_iam_role_policy.compute, + ] +} diff --git a/modules/orchestration-providers/scale-set/tests/computed-inputs.tftest.hcl b/modules/orchestration-providers/scale-set/tests/computed-inputs.tftest.hcl new file mode 100644 index 0000000000..22640ccd7f --- /dev/null +++ b/modules/orchestration-providers/scale-set/tests/computed-inputs.tftest.hcl @@ -0,0 +1,49 @@ +mock_provider "aws" { + mock_data "aws_caller_identity" { + defaults = { + account_id = "123456789012" + } + } + + mock_data "aws_partition" { + defaults = { + partition = "aws" + } + } + + mock_data "aws_region" { + defaults = { + region = "eu-west-1" + } + } + + mock_data "aws_iam_policy_document" { + defaults = { + json = "{\"Version\":\"2012-10-17\",\"Statement\":[]}" + } + } + + mock_resource "aws_iam_role" { + defaults = { + arn = "arn:aws:iam::123456789012:role/computed-test" + } + } +} + +run "plans_with_computed_values_inside_known_wrappers" { + command = plan + + module { + source = "./tests/fixtures/computed-inputs" + } + + assert { + condition = ( + toset(keys(output.controller_groups)) == toset(["ec2"]) && + toset(output.controller_groups.ec2.runner_configs) == toset(["computed"]) && + !output.cluster.managed && + toset(keys(output.reconciler_config_parameters)) == toset(["ec2/computed"]) + ) + error_message = "Computed inner values and explicit nulls must not affect group, ownership, IAM-wrapper, or cluster resource shape." + } +} diff --git a/modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/README.md b/modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/README.md new file mode 100644 index 0000000000..25516ecf3a --- /dev/null +++ b/modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/README.md @@ -0,0 +1,38 @@ + +## Requirements + +| Name | Version | +|------|---------| +| [terraform](#requirement\_terraform) | >= 1.5.6 | +| [aws](#requirement\_aws) | >= 6.33 | + +## Providers + +| Name | Version | +|------|---------| +| [terraform](#provider\_terraform) | n/a | + +## Modules + +| Name | Source | Version | +|------|--------|---------| +| [subject](#module\_subject) | ../../.. | n/a | + +## Resources + +| Name | Type | +|------|------| +| [terraform_data.computed](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | + +## Inputs + +No inputs. + +## Outputs + +| Name | Description | +|------|-------------| +| [cluster](#output\_cluster) | n/a | +| [controller\_groups](#output\_controller\_groups) | n/a | +| [reconciler\_config\_parameters](#output\_reconciler\_config\_parameters) | n/a | + diff --git a/modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/main.tf b/modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/main.tf new file mode 100644 index 0000000000..db8ae2b225 --- /dev/null +++ b/modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/main.tf @@ -0,0 +1,100 @@ +resource "terraform_data" "computed" { + input = { + external_cluster_arn = "arn:aws:ecs:eu-west-1:123456789012:cluster/external" + app_id_arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/computed/app-id" + private_key_arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/computed/private-key" + installation_id_arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/computed/installation-id" + kms_key_arn = "arn:aws:kms:eu-west-1:123456789012:key/33333333-3333-3333-3333-333333333333" + launch_template_name = "lt-computed" + action = "ec2:RunInstances" + resource = "arn:aws:ec2:eu-west-1:123456789012:launch-template/lt-computed" + } +} + +module "subject" { + source = "../../.." + + prefix = "computed-test" + + runner_configs = { + computed = { + github = { + enterprise_server = {} + app = { + app_id = { + name = "/github/computed/app-id" + arn = terraform_data.computed.output.app_id_arn + } + private_key = { + name = "/github/computed/private-key" + arn = terraform_data.computed.output.private_key_arn + kms_key_arn = terraform_data.computed.output.kms_key_arn + } + installation_id = { + name = "/github/computed/installation-id" + arn = terraform_data.computed.output.installation_id_arn + } + } + runner_owner = "example" + runner_registration_level = "organization" + user_agent = "scale-set-test" + } + scale_set = { + name = "computed" + } + compute_provider = { + type = "ec2" + capabilities = { + scale_set = { + configuration_json = jsonencode({ + region = "eu-west-1" + environment = "computed-test" + runnerOwner = "example" + runnerType = "Org" + runnerNamePrefix = "computed-" + jitConfigParameterPath = "/computed-test/runners/tokens" + subnets = ["subnet-12345678"] + launchTemplateName = terraform_data.computed.output.launch_template_name + ec2instanceCriteria = { + instanceTypes = ["m7i.large"] + targetCapacityType = "on-demand" + instanceAllocationStrategy = "lowest-price" + } + scaleErrors = [] + }) + iam_statements = { + run_instances = { + actions = [terraform_data.computed.output.action] + resources = [terraform_data.computed.output.resource] + } + } + } + } + } + } + } + + ecs = { + cluster = { + mode = "external" + arn = terraform_data.computed.output.external_cluster_arn + } + } + + network = { + vpc_id = "vpc-12345678" + subnet_ids = ["subnet-12345678"] + } +} + +output "controller_groups" { + value = module.subject.controller_groups +} + +output "cluster" { + value = module.subject.cluster +} + +output "reconciler_config_parameters" { + value = module.subject.reconciler_config_parameters +} diff --git a/modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/versions.tf b/modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/versions.tf new file mode 100644 index 0000000000..0bedc91fd5 --- /dev/null +++ b/modules/orchestration-providers/scale-set/tests/fixtures/computed-inputs/versions.tf @@ -0,0 +1,10 @@ +terraform { + required_version = ">= 1.5.6" + + required_providers { + aws = { + source = "hashicorp/aws" + version = ">= 6.33" + } + } +} diff --git a/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl b/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl new file mode 100644 index 0000000000..870a73b9b2 --- /dev/null +++ b/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl @@ -0,0 +1,1137 @@ +mock_provider "aws" { + mock_data "aws_caller_identity" { + defaults = { + account_id = "123456789012" + } + } + + mock_data "aws_partition" { + defaults = { + partition = "aws" + } + } + + mock_data "aws_region" { + defaults = { + region = "eu-west-1" + } + } + + mock_data "aws_iam_policy_document" { + defaults = { + json = "{\"Version\":\"2012-10-17\",\"Statement\":[]}" + } + } + + mock_resource "aws_iam_role" { + defaults = { + arn = "arn:aws:iam::123456789012:role/scale-set-test" + } + } + + mock_resource "aws_ecs_cluster" { + defaults = { + arn = "arn:aws:ecs:eu-west-1:123456789012:cluster/scale-set-test" + } + } +} + +variables { + prefix = "scale-set-test" + + runner_configs = { + linux-small = { + github = { + enterprise_server = { + ssl_verify = false + } + app = { + app_id = { + name = "/github/linux-small/app-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/linux-small/app-id" + } + private_key = { + name = "/github/linux-small/private-key" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/linux-small/private-key" + kms_key_arn = "arn:aws:kms:eu-west-1:123456789012:key/11111111-1111-1111-1111-111111111111" + } + installation_id = { + name = "/github/linux-small/installation-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/linux-small/installation-id" + } + } + runner_owner = "example" + runner_registration_level = "organization" + user_agent = "scale-set-test" + } + scale_set = { + name = "linux-small" + runner = { + group_name = "stable-group" + min_runners = 1 + max_runners = 10 + } + } + compute_provider = { + type = "ec2" + capabilities = { + scale_set = { + configuration_json = jsonencode({ + region = "eu-west-1" + environment = "scale-set-test" + runnerOwner = "example" + runnerType = "Org" + runnerNamePrefix = "small-" + jitConfigParameterPath = "/scale-set-test/runners/tokens" + subnets = ["subnet-11111111"] + launchTemplateName = "lt-small" + ec2instanceCriteria = { + instanceTypes = ["m7i.large"] + targetCapacityType = "on-demand" + instanceAllocationStrategy = "lowest-price" + } + scaleErrors = [] + }) + environment_variables = { + EC2_CONTROLLER_MODE = "grouped" + } + iam_statements = { + run_instances = { + actions = ["ec2:RunInstances"] + resources = ["arn:aws:ec2:eu-west-1:123456789012:launch-template/lt-small"] + } + read_ami = { + actions = ["ssm:GetParameters"] + resources = ["arn:aws:ssm:eu-west-1:123456789012:parameter/scale-set-test/runners/config/ami_id"] + } + } + } + } + } + } + linux-large = { + github = { + enterprise_server = { + url = "https://github.example.test" + } + app = { + app_id = { + name = "/github/linux-large/app-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/linux-large/app-id" + } + private_key = { + name = "/github/linux-large/private-key" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/linux-large/private-key" + } + installation_id = { + name = "/github/linux-large/installation-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/linux-large/installation-id" + } + } + runner_owner = "example" + runner_registration_level = "organization" + user_agent = "scale-set-test" + } + scale_set = { + name = "linux-large" + runner = { + min_runners = 0 + max_runners = 20 + } + } + compute_provider = { + type = "ec2" + capabilities = { + scale_set = { + configuration_json = jsonencode({ + region = "eu-west-1" + environment = "scale-set-test" + runnerOwner = "example" + runnerType = "Org" + runnerNamePrefix = "large-" + jitConfigParameterPath = "/scale-set-test/runners/tokens" + subnets = ["subnet-22222222"] + launchTemplateName = "lt-large" + ec2instanceCriteria = { + instanceTypes = ["m7i.xlarge"] + targetCapacityType = "on-demand" + instanceAllocationStrategy = "lowest-price" + } + scaleErrors = [] + }) + environment_variables = { + EC2_CONTROLLER_MODE = "grouped" + } + iam_statements = { + run_instances = { + actions = ["ec2:RunInstances"] + resources = ["arn:aws:ec2:eu-west-1:123456789012:launch-template/lt-large"] + } + } + } + } + } + } + microvm = { + github = { + enterprise_server = {} + app = { + app_id = { + name = "/github/microvm/app-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/microvm/app-id" + } + private_key = { + name = "/github/microvm/private-key" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/microvm/private-key" + } + installation_id = { + name = "/github/microvm/installation-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/microvm/installation-id" + } + } + runner_owner = "example/repository" + runner_registration_level = "repository" + user_agent = "scale-set-test" + } + scale_set = { + name = "microvm" + runner = { + min_runners = 0 + max_runners = 5 + } + } + compute_provider = { + # Future provider used only to prove grouping remains provider-neutral. + type = "microvm" + capabilities = { + scale_set = { + configuration_json = jsonencode({ image_arn = "arn:aws:lambda:eu-west-1:123456789012:runtime-management-config:microvm" }) + iam_statements = { + run_microvm = { + actions = ["lambda:InvokeFunction"] + resources = ["arn:aws:lambda:eu-west-1:123456789012:function:microvm"] + } + } + } + } + } + } + } + + network = { + vpc_id = "vpc-12345678" + subnet_ids = ["subnet-11111111", "subnet-22222222"] + } + + logging = { + kms_key_arn = "arn:aws:kms:eu-west-1:123456789012:key/22222222-2222-2222-2222-222222222222" + } + + tags = { + Test = "scale-set" + } +} + +run "groups_by_compute_provider_and_hardens_each_task" { + command = plan + + assert { + condition = ( + toset(keys(output.controller_groups)) == toset(["ec2", "microvm"]) && + toset(output.controller_groups["ec2"].runner_configs) == toset(["linux-small", "linux-large"]) && + toset(output.controller_groups["microvm"].runner_configs) == toset(["microvm"]) + ) + error_message = "The default strategy must create one controller group per compute-provider type." + } + + assert { + condition = ( + length(aws_ecs_service.controller) == 2 && + length(aws_ecs_task_definition.controller) == 2 && + length(aws_iam_role.task) == 2 && + length(aws_cloudwatch_log_group.controller) == 2 && + length(aws_security_group.controller) == 2 && + length(aws_ssm_parameter.reconciler_config) == 3 + ) + error_message = "Every group must own one service, task definition, task role, log group, and security group while every reconciler gets one config parameter." + } + + assert { + condition = alltrue([ + for service in values(aws_ecs_service.controller) : ( + service.desired_count == 1 && + service.deployment_minimum_healthy_percent == 0 && + service.deployment_maximum_percent == 100 && + service.deployment_circuit_breaker[0].enable && + service.deployment_circuit_breaker[0].rollback && + !service.network_configuration[0].assign_public_ip && + length(service.network_configuration[0].security_groups) == 1 + ) + ]) + error_message = "Services must run one private task and use stop-first deployment with circuit-breaker rollback." + } + + assert { + condition = alltrue([ + for task in values(aws_ecs_task_definition.controller) : ( + length(jsondecode(task.container_definitions)) == 1 && + jsondecode(task.container_definitions)[0].image == "ghcr.io/github-aws-runners/terraform-aws-github-runner-scale-set-service:latest" && + jsondecode(task.container_definitions)[0].versionConsistency == "enabled" && + jsondecode(task.container_definitions)[0].readonlyRootFilesystem && + !jsondecode(task.container_definitions)[0].privileged && + jsondecode(task.container_definitions)[0].user == "10001:10001" && + jsondecode(task.container_definitions)[0].linuxParameters.capabilities.drop == ["ALL"] && + jsondecode(task.container_definitions)[0].healthCheck.command[3] == "fetch('http://127.0.0.1:8080/healthz').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))" && + one([for entry in jsondecode(task.container_definitions)[0].environment : entry.value if entry.name == "LOG_LEVEL"]) == "info" && + contains([for entry in jsondecode(task.container_definitions)[0].environment : entry.name], "SCALE_SET_CONTROLLER_MANIFEST") && + one([for entry in jsondecode(task.container_definitions)[0].environment : entry.value if entry.name == "AWS_XRAY_CONTEXT_MISSING"]) == "IGNORE_ERROR" && + one([for entry in jsondecode(task.container_definitions)[0].environment : entry.value if entry.name == "AWS_REGION"]) == "eu-west-1" && + one([for entry in jsondecode(task.container_definitions)[0].environment : entry.value if entry.name == "AWS_DEFAULT_REGION"]) == "eu-west-1" && + !contains([for entry in jsondecode(task.container_definitions)[0].environment : entry.name], "SCALE_SET_CONTROLLER_GROUP_CONFIG_PATH") && + !contains([for entry in jsondecode(task.container_definitions)[0].environment : entry.name], "SCALE_SET_CONTROLLER_GROUP_CONFIG_REVISION") + ) + ]) + error_message = "Each task definition must contain one hardened controller container using manifest configuration and /healthz liveness." + } + + assert { + condition = ( + contains(flatten([ + for task in values(aws_ecs_task_definition.controller) : [ + for entry in jsondecode(task.container_definitions)[0].environment : [ + for reconciler in jsondecode(entry.value).reconcilers : reconciler.runnerGroupName + ] + if entry.name == "SCALE_SET_CONTROLLER_MANIFEST" + ] + ]), "stable-group") && + alltrue([ + for task in values(aws_ecs_task_definition.controller) : alltrue([ + for entry in jsondecode(task.container_definitions)[0].environment : entry.name != "SCALE_SET_CONTROLLER_MANIFEST" || ( + jsondecode(entry.value).version == 1 && + jsondecode(entry.value).groupName == one([ + for group_name in keys(local.controller_groups) : group_name + if local.group_controller_manifests[group_name] == entry.value + ]) && + length(jsondecode(entry.value).reconcilers) > 0 && + alltrue([ + for reconciler in jsondecode(entry.value).reconcilers : ( + reconciler.schemaVersion == 1 && + reconciler.runnerConfigName != null && + reconciler.runnerGroupName != null && + reconciler.scaleSetName != null && + reconciler.githubConfigUrl != null && + reconciler.githubApp.appIdParameterName != null && + reconciler.githubApp.privateKeyParameterName != null && + reconciler.computeProvider.type != null && + reconciler.computeProvider.roleArn != null && + reconciler.computeProvider.configuration != null && + reconciler.minRunners != null && + reconciler.maxRunners != null && + reconciler.bootTimeoutMinutes != null && + reconciler.sessionOwner != null && + reconciler.workFolder != null && + reconciler.forceGhes != null && + reconciler.sslVerify != null + ) + ]) + ) + ]) + ]) + ) + error_message = "Each ECS task must receive a versioned ScaleSetControllerManifest with complete reconciler configuration." + } + + assert { + condition = one([ + for entry in jsondecode(aws_ecs_task_definition.controller["ec2"].container_definitions)[0].environment : + entry.value if entry.name == "EC2_CONTROLLER_MODE" + ]) == "grouped" + error_message = "Provider-owned non-secret environment variables must be merged into their controller group task." + } + + assert { + condition = ( + length(aws_security_group.controller["ec2"].ingress) == 0 && + length(aws_security_group.controller["ec2"].egress) == 1 && + one(aws_security_group.controller["ec2"].egress).from_port == 443 && + one(aws_security_group.controller["ec2"].egress).to_port == 443 && + aws_cloudwatch_log_group.controller["ec2"].kms_key_id == var.logging.kms_key_arn + ) + error_message = "Controller networking must have no ingress and only HTTPS egress, and logs must honor customer-managed encryption." + } + + assert { + condition = ( + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-small"].value)).schemaVersion == 1 && + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-small"].value)).runnerConfigName == "linux-small" && + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-small"].value)).githubConfigUrl == "https://github.com/example" && + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-small"].value)).scaleSetName == "linux-small" && + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-small"].value)).bootTimeoutMinutes == 10 && + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-small"].value)).sslVerify == false && + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-small"].value)).forceGhes == false && + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-small"].value)).userAgent == "scale-set-test" && + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-small"].value)).githubApp.privateKeyParameterName == "/github/linux-small/private-key" && + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-large"].value)).githubConfigUrl == "https://github.example.test/example" && + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-large"].value)).forceGhes == true && + !contains(keys(jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-small"].value))), "runnerConfig") + ) + error_message = "Each SSM leaf must use the frozen flat reconciler schema and contain references instead of GitHub credential values." + } + + assert { + condition = ( + contains(flatten([for statement in data.aws_iam_policy_document.task["ec2"].statement : statement.resources]), "arn:aws:ssm:eu-west-1:123456789012:parameter/github/linux-small/private-key") && + !contains(flatten([for statement in data.aws_iam_policy_document.task["ec2"].statement : statement.resources]), "arn:aws:ssm:eu-west-1:123456789012:parameter/github/microvm/private-key") && + contains(jsondecode(local.group_github_kms_policy_json["ec2"]).Statement[0].Resource, "arn:aws:kms:eu-west-1:123456789012:key/11111111-1111-1111-1111-111111111111") && + length(jsondecode(local.group_github_kms_policy_json["microvm"]).Statement) == 0 && + contains(flatten([for statement in data.aws_iam_policy_document.task["ec2"].statement : statement.resources]), "arn:aws:ssm:eu-west-1:123456789012:parameter/scale-set-test/scale-set-controller/ec2/*") + && contains(flatten([for statement in data.aws_iam_policy_document.task["ec2"].statement : statement.actions]), "sts:AssumeRole") && + !contains(flatten([for statement in data.aws_iam_policy_document.task["ec2"].statement : statement.resources]), "arn:aws:ssm:eu-west-1:123456789012:parameter/scale-set-test/runners/config/ami_id") && + contains(flatten([for statement in data.aws_iam_policy_document.compute["ec2/linux-small"].statement : statement.actions]), "ssm:GetParameters") && + contains(flatten([for statement in data.aws_iam_policy_document.compute["ec2/linux-small"].statement : statement.resources]), "arn:aws:ssm:eu-west-1:123456789012:parameter/scale-set-test/runners/config/ami_id") + ) + error_message = "Controller IAM must contain only controller permissions, while provider permissions such as AMI SSM reads must be attached to the compute role." + } +} + +run "supports_one_group_per_runner_config" { + command = plan + + variables { + grouping = { + strategy = "runner_config" + } + container = { + image = "ghcr.io/example/scale-set-controller@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + } + } + + assert { + condition = ( + toset(keys(output.controller_groups)) == toset(["linux-small", "linux-large", "microvm"]) && + length(aws_ecs_service.controller) == 3 && + output.resolved_container_image == "ghcr.io/example/scale-set-controller@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + ) + error_message = "runner_config grouping must create one independently deployable controller task per runner config and honor an image override." + } +} + +run "grants_execution_role_ecr_pull_permissions" { + command = plan + + variables { + container = { + image = "999999999999.dkr.ecr.eu-west-1.amazonaws.com/scale-set-controller@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + } + } + + assert { + condition = ( + contains(flatten([ + for statement in data.aws_iam_policy_document.execution["ec2"].statement : statement.resources + ]), "*") && + contains(flatten([ + for statement in data.aws_iam_policy_document.execution["ec2"].statement : statement.actions + ]), "ecr:GetAuthorizationToken") && + contains(flatten([ + for statement in data.aws_iam_policy_document.execution["ec2"].statement : statement.actions + ]), "ecr:BatchCheckLayerAvailability") && + contains(flatten([ + for statement in data.aws_iam_policy_document.execution["ec2"].statement : statement.actions + ]), "ecr:BatchGetImage") && + contains(flatten([ + for statement in data.aws_iam_policy_document.execution["ec2"].statement : statement.actions + ]), "ecr:GetDownloadUrlForLayer") + ) + error_message = "The ECS execution role must have wildcard ECR pull permissions, including the authorization-token permission." + } +} + +run "supports_exact_custom_groups" { + command = plan + + variables { + grouping = { + strategy = "custom" + custom = { + groups = { + general = { + runner_configs = ["linux-small", "microvm"] + } + isolated = { + runner_configs = ["linux-large"] + } + } + } + } + } + + assert { + condition = ( + toset(keys(output.controller_groups)) == toset(["general", "isolated"]) && + toset(output.controller_groups.general.runner_configs) == toset(["linux-small", "microvm"]) && + toset(output.controller_groups.isolated.runner_configs) == toset(["linux-large"]) + ) + error_message = "Custom grouping must preserve the exact declared assignment." + } +} + +run "rejects_duplicate_custom_membership" { + command = plan + + plan_options { + target = [terraform_data.validate_grouping] + } + + variables { + grouping = { + strategy = "custom" + custom = { + groups = { + first = { + runner_configs = ["linux-small", "linux-large"] + } + second = { + runner_configs = ["linux-small", "microvm"] + } + } + } + } + } + + expect_failures = [terraform_data.validate_grouping] +} + +run "rejects_incomplete_custom_membership" { + command = plan + + plan_options { + target = [terraform_data.validate_grouping] + } + + variables { + grouping = { + strategy = "custom" + custom = { + groups = { + partial = { + runner_configs = ["linux-small", "linux-large"] + } + } + } + } + } + + expect_failures = [terraform_data.validate_grouping] +} + +run "rejects_readiness_path_as_ecs_liveness" { + command = plan + + plan_options { + target = [terraform_data.validate_runtime] + } + + variables { + container = { + health_path = "/readyz" + } + } + + expect_failures = [terraform_data.validate_runtime] +} + +run "rejects_oversized_standard_parameter" { + command = plan + + plan_options { + target = [terraform_data.validate_config_store] + } + + variables { + runner_configs = merge(var.runner_configs, { + linux-small = merge(var.runner_configs.linux-small, { + compute_provider = merge(var.runner_configs.linux-small.compute_provider, { + capabilities = { + scale_set = merge(var.runner_configs.linux-small.compute_provider.capabilities.scale_set, { + configuration_json = jsonencode({ payload = join("", [for index in range(1000) : "xxxxxx"]) }) + }) + } + }) + }) + }) + } + + expect_failures = [terraform_data.validate_config_store] +} + +run "accepts_advanced_parameter_within_eight_kib" { + command = plan + + plan_options { + target = [terraform_data.validate_config_store] + } + + variables { + config_store = { + tier = "Advanced" + } + runner_configs = merge(var.runner_configs, { + linux-small = merge(var.runner_configs.linux-small, { + compute_provider = merge(var.runner_configs.linux-small.compute_provider, { + capabilities = { + scale_set = merge(var.runner_configs.linux-small.compute_provider.capabilities.scale_set, { + configuration_json = jsonencode({ payload = join("", [for index in range(800) : "xxxxxx"]) }) + }) + } + }) + }) + }) + } + + assert { + condition = local.reconciler_config_bytes["ec2/linux-small"] > 4096 && local.reconciler_config_bytes["ec2/linux-small"] <= 8192 + error_message = "Advanced Parameter Store tier must accept reconciler JSON between four and eight KiB." + } +} + +run "assembles_github_config_url_from_registration_scope_and_owner" { + command = apply + + variables { + runner_configs = merge(var.runner_configs, { + linux-small = merge(var.runner_configs.linux-small, { + github = merge(var.runner_configs.linux-small.github, { + runner_registration_level = "organization" + runner_owner = "example" + }) + }) + linux-large = merge(var.runner_configs.linux-large, { + github = merge(var.runner_configs.linux-large.github, { + runner_registration_level = "organization" + runner_owner = "example" + }) + }) + microvm = merge(var.runner_configs.microvm, { + github = merge(var.runner_configs.microvm.github, { + runner_registration_level = "repository" + runner_owner = "example/repository" + }) + }) + }) + } + + assert { + condition = ( + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-small"].value)).githubConfigUrl == "https://github.com/example" && + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["ec2/linux-large"].value)).githubConfigUrl == "https://github.example.test/example" && + jsondecode(nonsensitive(aws_ssm_parameter.reconciler_config["microvm/microvm"].value)).githubConfigUrl == "https://github.com/example/repository" + ) + error_message = "The reconciler config URL must combine the GitHub server with the configured organization or repository owner." + } +} + +run "rejects_duplicate_scale_set_ownership_across_groups" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + linux-small = merge(var.runner_configs.linux-small, { + github = merge(var.runner_configs.linux-small.github, { + enterprise_server = { url = "https://mygithub.com" } + }) + }) + microvm = merge(var.runner_configs.microvm, { + github = merge(var.runner_configs.microvm.github, { + enterprise_server = { url = "https://mygithub.com:443/" } + runner_registration_level = "organization" + runner_owner = "example" + }) + scale_set = merge(var.runner_configs.microvm.scale_set, { + name = "linux-small" + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_leading_zero_default_port_spelling" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + microvm = merge(var.runner_configs.microvm, { + github = merge(var.runner_configs.microvm.github, { + enterprise_server = { url = "https://github.com:0443/" } + }) + scale_set = merge(var.runner_configs.microvm.scale_set, { + name = "linux-small" + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_port_above_url_maximum" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + microvm = merge(var.runner_configs.microvm, { + github = merge(var.runner_configs.microvm.github, { + enterprise_server = { url = "https://github.com:65536/" } + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_non_ascii_scale_set_name" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + microvm = merge(var.runner_configs.microvm, { + scale_set = merge(var.runner_configs.microvm.scale_set, { + name = "microvm-☃" + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_invalid_compute_provider_type_identifier" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + microvm = merge(var.runner_configs.microvm, { + compute_provider = merge(var.runner_configs.microvm.compute_provider, { + type = "AWS.MicroVM" + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_credential_arn_name_mismatch" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + microvm = merge(var.runner_configs.microvm, { + github = merge(var.runner_configs.microvm.github, { + app = merge(var.runner_configs.microvm.github.app, { + app_id = merge(var.runner_configs.microvm.github.app.app_id, { + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/another/app-id" + }) + }) + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_cross_account_credential_parameter" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + microvm = merge(var.runner_configs.microvm, { + github = merge(var.runner_configs.microvm.github, { + app = merge(var.runner_configs.microvm.github.app, { + app_id = merge(var.runner_configs.microvm.github.app.app_id, { + arn = "arn:aws:ssm:eu-west-1:210987654321:parameter/github/microvm/app-id" + }) + }) + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "allows_same_scale_set_name_in_another_github_scope" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + microvm = merge(var.runner_configs.microvm, { + github = merge(var.runner_configs.microvm.github, { + enterprise_server = { url = "https://github.example.test" } + }) + scale_set = merge(var.runner_configs.microvm.scale_set, { + name = "linux-small" + }) + }) + }) + } + + assert { + condition = length([ + for runner_name, runner_config in var.runner_configs : format( + "%s#%s", + replace(trimsuffix(lower(local.github_config_urls[runner_name]), "/"), ":443", ""), + runner_config.scale_set.name, + ) + ]) == length(distinct([ + for runner_name, runner_config in var.runner_configs : format( + "%s#%s", + replace(trimsuffix(lower(local.github_config_urls[runner_name]), "/"), ":443", ""), + runner_config.scale_set.name, + ) + ])) + error_message = "Scale-set names are scoped to their normalized enterprise-server URL." + } +} + +run "bounds_default_session_owner_for_maximum_names" { + command = plan + + variables { + grouping = { + strategy = "runner_config" + } + runner_configs = { + (join("", [for index in range(128) : "a"])) = { + github = { + enterprise_server = {} + runner_owner = "example" + runner_registration_level = "organization" + user_agent = "scale-set-test" + app = { + app_id = { + name = "/github/max/app-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/max/app-id" + } + private_key = { + name = "/github/max/private-key" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/max/private-key" + } + installation_id = { + name = "/github/max/installation-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/max/installation-id" + } + } + } + scale_set = { + name = "maximum-name" + } + compute_provider = { + type = "ec2" + capabilities = { + scale_set = { + configuration_json = "{}" + } + } + } + } + } + } + + assert { + condition = ( + length(one(values(local.reconciler_configs)).value.sessionOwner) == 256 && + can(regex("^[A-Za-z0-9][A-Za-z0-9._:/-]{0,255}$", one(values(local.reconciler_configs)).value.sessionOwner)) + ) + error_message = "A generated session owner must remain deterministic and within the runtime's 256-character limit." + } +} + +run "rejects_controller_group_policy_above_inline_quota" { + command = plan + + plan_options { + target = [terraform_data.validate_group_task_policy["ec2"]] + } + + override_data { + target = data.aws_iam_policy_document.task["ec2"] + values = { + json = <<-JSON + {"payload":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"} + JSON + } + } + + expect_failures = [terraform_data.validate_group_task_policy["ec2"]] +} + +run "rejects_conflicting_group_environment_variables" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + linux-large = merge(var.runner_configs.linux-large, { + compute_provider = merge(var.runner_configs.linux-large.compute_provider, { + capabilities = { + scale_set = merge(var.runner_configs.linux-large.compute_provider.capabilities.scale_set, { + environment_variables = { + EC2_CONTROLLER_MODE = "isolated" + } + }) + } + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_controller_group_environment_above_task_definition_budget" { + command = plan + + plan_options { + target = [terraform_data.validate_grouping] + } + + variables { + runner_configs = merge(var.runner_configs, { + linux-small = merge(var.runner_configs.linux-small, { + compute_provider = merge(var.runner_configs.linux-small.compute_provider, { + capabilities = { + scale_set = merge(var.runner_configs.linux-small.compute_provider.capabilities.scale_set, { + environment_variables = merge( + var.runner_configs.linux-small.compute_provider.capabilities.scale_set.environment_variables, + { + for index in range(16) : format("EC2_QUOTA_%02d", index) => join("", [for part in range(1024) : "xxxx"]) + }, + ) + }) + } + }) + }) + }) + } + + expect_failures = [terraform_data.validate_grouping] +} + +run "rejects_reserved_provider_environment_variables" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + linux-small = merge(var.runner_configs.linux-small, { + compute_provider = merge(var.runner_configs.linux-small.compute_provider, { + capabilities = { + scale_set = merge(var.runner_configs.linux-small.compute_provider.capabilities.scale_set, { + environment_variables = { + SCALE_SET_OVERRIDE = "unsafe" + } + }) + } + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_invalid_boot_timeout" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + linux-small = merge(var.runner_configs.linux-small, { + scale_set = merge(var.runner_configs.linux-small.scale_set, { + runner = merge(var.runner_configs.linux-small.scale_set.runner, { + boot_time_in_minutes = 0 + }) + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_enterprise_runner_registration_level" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + linux-small = merge(var.runner_configs.linux-small, { + github = merge(var.runner_configs.linux-small.github, { + runner_registration_level = "enterprise" + runner_owner = null + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_invalid_runner_registration_level" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + linux-small = merge(var.runner_configs.linux-small, { + github = merge(var.runner_configs.linux-small.github, { + runner_registration_level = "invalid" + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} + +run "rejects_controller_group_above_runtime_reconciler_limit" { + command = plan + + plan_options { + target = [terraform_data.validate_grouping] + } + + variables { + runner_configs = { + for index in range(1001) : format("runner-%04d", index) => var.runner_configs.linux-small + } + grouping = { + strategy = "custom" + custom = { + groups = { + oversized = { + runner_configs = toset([for index in range(1001) : format("runner-%04d", index)]) + } + } + } + } + } + + expect_failures = [terraform_data.validate_grouping] +} + +run "rejects_controller_group_above_runtime_config_bytes" { + command = plan + + plan_options { + target = [terraform_data.validate_grouping] + } + + variables { + config_store = { + tier = "Advanced" + } + runner_configs = { + for index in range(900) : format("runner-%04d", index) => merge(var.runner_configs.linux-small, { + compute_provider = merge(var.runner_configs.linux-small.compute_provider, { + capabilities = { + scale_set = merge(var.runner_configs.linux-small.compute_provider.capabilities.scale_set, { + configuration_json = jsonencode({ + payload = join("", [for part in range(1000) : "xxxxx"]) + }) + }) + } + }) + }) + } + grouping = { + strategy = "custom" + custom = { + groups = { + oversized = { + runner_configs = toset([for index in range(900) : format("runner-%04d", index)]) + } + } + } + } + } + + expect_failures = [terraform_data.validate_grouping] +} + +run "rejects_runtime_invalid_credential_parameter_name" { + command = plan + + plan_options { + target = [terraform_data.validate_contract] + } + + variables { + runner_configs = merge(var.runner_configs, { + microvm = merge(var.runner_configs.microvm, { + github = merge(var.runner_configs.microvm.github, { + app = merge(var.runner_configs.microvm.github.app, { + app_id = { + name = "/github/microvm/bad app-id" + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/microvm/bad app-id" + } + }) + }) + }) + }) + } + + expect_failures = [terraform_data.validate_contract] +} diff --git a/modules/orchestration-providers/scale-set/validations.tf b/modules/orchestration-providers/scale-set/validations.tf new file mode 100644 index 0000000000..9553581597 --- /dev/null +++ b/modules/orchestration-providers/scale-set/validations.tf @@ -0,0 +1,425 @@ +locals { + group_compute_environment_base64 = { + for group_name, environment_variables in local.group_compute_environment_variables : group_name => base64encode(jsonencode([ + for name in sort(keys(environment_variables)) : { + name = name + value = environment_variables[name] + } + ])) + } + group_compute_environment_bytes = { + for group_name, encoded in local.group_compute_environment_base64 : group_name => ( + floor(length(encoded) * 3 / 4) - + (endswith(encoded, "==") ? 2 : endswith(encoded, "=") ? 1 : 0) + ) + } +} + +resource "terraform_data" "validate_contract" { + lifecycle { + precondition { + condition = ( + length(var.prefix) >= 1 && + length(var.prefix) <= 20 && + can(regex("^[a-z0-9][a-z0-9-]*$", var.prefix)) + ) + error_message = "prefix must contain 1 to 20 lowercase ASCII letters, digits, or hyphens and start with a letter or digit." + } + + precondition { + condition = alltrue([ + for runner_name in keys(var.runner_configs) : ( + length(runner_name) >= 1 && + length(runner_name) <= 128 && + can(regex("^[A-Za-z0-9][A-Za-z0-9._-]*$", runner_name)) + ) + ]) + error_message = "runner-config keys must contain 1 to 128 ASCII letters, digits, dots, underscores, or hyphens and start with a letter or digit." + } + + precondition { + condition = alltrue([ + for runner_config in values(var.runner_configs) : contains([ + "organization", + "repository", + ], runner_config.github.runner_registration_level) + ]) + error_message = "runner_registration_level must be organization or repository." + } + + precondition { + condition = alltrue([ + for runner_config in values(var.runner_configs) : ( + runner_config.github.runner_owner != null && can(regex( + runner_config.github.runner_registration_level == "organization" + ? "^[A-Za-z0-9_.-]+$" + : "^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+$", + runner_config.github.runner_owner, + )) + ) + ]) + error_message = "runner_owner must be an organization or owner/repository path for organization and repository registration levels." + } + + precondition { + condition = alltrue([ + for runner_name, runner_config in var.runner_configs : ( + can(regex("^https://[A-Za-z0-9.-]+(:[1-9][0-9]{0,4})?(/[A-Za-z0-9_.-]+(/[A-Za-z0-9_.-]+)?)?/?$", local.github_config_urls[runner_name])) && + try(tonumber(regex("^https://[A-Za-z0-9.-]+:([0-9]+)", local.github_config_urls[runner_name])[0]), 443) <= 65535 + ) + ]) + error_message = "Each assembled GitHub config URL must be an HTTPS GitHub Enterprise Server URL without credentials, query, fragment, or whitespace." + } + + precondition { + condition = length([ + for runner_name, runner_config in var.runner_configs : format( + "%s#%s", + replace(trimsuffix(lower(local.github_config_urls[runner_name]), "/"), ":443", ""), + runner_config.scale_set.name, + ) + ]) == length(distinct([ + for runner_name, runner_config in var.runner_configs : format( + "%s#%s", + replace(trimsuffix(lower(local.github_config_urls[runner_name]), "/"), ":443", ""), + runner_config.scale_set.name, + ) + ])) + error_message = "Each normalized githubConfigUrl and scale_set.name tuple must be unique across runner_configs so two controller services cannot own the same message session. URL matching ignores case, one trailing slash, and the default HTTPS port." + } + + precondition { + condition = alltrue([ + for runner_config in values(var.runner_configs) : ( + alltrue([ + for parameter in [ + runner_config.github.app.app_id, + runner_config.github.app.private_key, + runner_config.github.app.installation_id, + ] : ( + length(parameter.name) <= 2048 && + can(regex("^/[A-Za-z0-9_./-]+$", parameter.name)) && + !endswith(parameter.name, "/") && + !strcontains(parameter.name, "//") && + parameter.arn == format( + "arn:%s:ssm:%s:%s:parameter%s", + data.aws_partition.current.partition, + data.aws_region.current.region, + data.aws_caller_identity.current.account_id, + parameter.name, + ) && + (parameter.kms_key_arn == null ? true : can(regex("^arn:[^:]+:kms:[^:]+:[0-9]{12}:key/.+$", parameter.kms_key_arn))) + ) + ]) + ) + ]) + error_message = "GitHub App credentials must use valid absolute SSM parameter names and exact same-account, same-region parameter ARNs; optional KMS references must be key ARNs." + } + + precondition { + condition = alltrue([ + for runner_config in values(var.runner_configs) : ( + can(regex("^[ -~]{1,128}$", runner_config.scale_set.name)) && + runner_config.scale_set.runner.min_runners >= 0 && + floor(runner_config.scale_set.runner.min_runners) == runner_config.scale_set.runner.min_runners && + runner_config.scale_set.runner.max_runners >= 1 && + runner_config.scale_set.runner.max_runners <= 10000 && + floor(runner_config.scale_set.runner.max_runners) == runner_config.scale_set.runner.max_runners && + runner_config.scale_set.runner.min_runners <= runner_config.scale_set.runner.max_runners && + runner_config.scale_set.runner.boot_time_in_minutes >= 1 && + runner_config.scale_set.runner.boot_time_in_minutes <= 120 && + floor(runner_config.scale_set.runner.boot_time_in_minutes) == runner_config.scale_set.runner.boot_time_in_minutes && + length(runner_config.github.user_agent) <= 256 && + can(regex("^[ -~]+$", runner_config.github.user_agent)) + ) + ]) + error_message = "Scale-set names must be valid, boot_time_in_minutes must be an integer from 1 through 120, and min_runners must be between zero and max_runners (maximum 10000)." + } + + precondition { + condition = alltrue([ + for runner_config in values(var.runner_configs) : ( + can(regex("^[a-z][a-z0-9_-]{0,63}$", runner_config.compute_provider.type)) && + can(keys(jsondecode(runner_config.compute_provider.capabilities.scale_set.configuration_json))) && + length(runner_config.compute_provider.capabilities.scale_set.environment_variables) <= 64 && + alltrue([ + for name, value in runner_config.compute_provider.capabilities.scale_set.environment_variables : ( + can(regex("^[A-Z][A-Z0-9_]{0,127}$", name)) && + !contains(["PATH", "HOME", "HOSTNAME", "PWD", "SHLVL"], name) && + alltrue([ + for prefix in ["AWS_", "ECS_", "GITHUB_", "SCALE_SET_", "NODE_"] : + !startswith(name, prefix) + ]) && + length(regexall("[\\x00-\\x1F\\x7F]", value)) == 0 && + ( + floor(length(base64encode(value)) * 3 / 4) - + (endswith(base64encode(value), "==") ? 2 : endswith(base64encode(value), "=") ? 1 : 0) + ) <= 4096 + ) + ]) && + alltrue([ + for statement_name, statement in runner_config.compute_provider.capabilities.scale_set.iam_statements : ( + can(regex("^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$", statement_name)) && + length(statement.actions) > 0 && + length(statement.resources) > 0 && + alltrue([for action in statement.actions : !strcontains(action, "*")]) && + alltrue([ + for condition in statement.conditions : ( + length(condition.test) > 0 && + length(condition.variable) > 0 && + length(condition.values) > 0 + ) + ]) + ) + ]) + ) + ]) + error_message = "Each compute-provider scale-set capability must have safe identifiers, object-shaped configuration JSON, non-secret environment variables with safe unreserved names and bounded values, and non-empty least-privilege IAM statements without wildcard actions." + } + + precondition { + condition = alltrue([ + for group_name, entries in local.group_compute_environment_entries : alltrue([ + for name in distinct([for entry in entries : entry.name]) : + length(distinct([for entry in entries : entry.value if entry.name == name])) <= 1 + ]) + ]) + error_message = "Compute-provider environment variables grouped into the same controller task must use identical values for duplicate names. Use a different grouping strategy when providers require conflicting process settings." + } + } +} + +resource "terraform_data" "validate_grouping" { + lifecycle { + precondition { + condition = contains(["compute_provider", "runner_config", "custom"], var.grouping.strategy) + error_message = "grouping.strategy must be compute_provider, runner_config, or custom." + } + + precondition { + condition = ( + var.grouping.strategy == "custom" + ? var.grouping.custom != null && length(var.grouping.custom.groups) > 0 + : var.grouping.custom == null + ) + error_message = "grouping.custom must be non-null and non-empty only when grouping.strategy is custom." + } + + precondition { + condition = var.grouping.strategy != "custom" ? true : alltrue([ + for group_name, group in var.grouping.custom.groups : ( + can(regex("^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$", group_name)) && + length(group.runner_configs) > 0 + ) + ]) + error_message = "Custom group names must be stable, safe identifiers of at most 64 characters, and every group must contain at least one runner config." + } + + precondition { + condition = var.grouping.strategy != "custom" ? true : ( + length(flatten(values(local.declared_custom_groups))) == length(distinct(flatten(values(local.declared_custom_groups)))) && + length(setsubtract(toset(flatten(values(local.declared_custom_groups))), toset(keys(var.runner_configs)))) == 0 && + length(setsubtract(toset(keys(var.runner_configs)), toset(flatten(values(local.declared_custom_groups))))) == 0 + ) + error_message = "Custom groups must contain every runner config exactly once and cannot contain unknown runner configs." + } + + precondition { + condition = alltrue([ + for group_name, runner_names in local.controller_groups : ( + can(regex("^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$", group_name)) && + length(runner_names) > 0 && + length(runner_names) <= 1000 + ) + ]) + error_message = "Resolved controller groups must have safe, non-empty, plan-known names and contain at most 1000 runner configs." + } + + precondition { + condition = alltrue([ + for group_name, runner_names in local.controller_groups : sum([ + for runner_name in runner_names : local.reconciler_config_bytes["${group_name}/${runner_name}"] + ]) <= 4 * 1024 * 1024 + ]) + error_message = "A controller group's decoded reconciler configuration must not exceed the runtime's 4 MiB aggregate limit. Split the group or reduce provider configuration size." + } + + precondition { + condition = alltrue([ + for group_name, environment_bytes in local.group_compute_environment_bytes : environment_bytes <= 48 * 1024 + ]) + error_message = "A controller group's compute-provider environment JSON must not exceed 49152 bytes. This reserves 16 KiB of AWS's 64 KiB ECS task-definition quota for the fixed task definition; split the group or reduce provider environment settings." + } + + precondition { + condition = alltrue([ + for group_name, manifest_bytes in local.group_controller_manifests : ( + length(manifest_bytes) + local.group_compute_environment_bytes[group_name] <= 48 * 1024 + ) + ]) + error_message = "A controller group's manifest and compute-provider environment JSON must fit the ECS task-definition budget; split the group or reduce provider configuration." + } + } +} + +resource "terraform_data" "validate_runtime" { + lifecycle { + precondition { + condition = ( + var.container.image == null ? true : ( + length(trimspace(var.container.image)) > 0 && + length(regexall("[[:space:]]", var.container.image)) == 0 + )) + error_message = "Container image references must be non-empty and cannot contain whitespace." + } + + precondition { + condition = ( + can(regex("^[1-9][0-9]{0,9}(:[1-9][0-9]{0,9})?$", var.container.user)) && + var.container.health_port >= 1 && var.container.health_port <= 65535 && + (var.container.health_check_command == null ? true : ( + length(var.container.health_check_command) >= 2 && + contains(["CMD", "CMD-SHELL"], var.container.health_check_command[0]) + )) + ) + error_message = "The container must use a numeric non-root UID (and optional GID), a valid health port, and a valid ECS health-check command." + } + + precondition { + condition = var.container.health_path == "/healthz" + error_message = "container.health_path must be /healthz, the scale-set service liveness endpoint." + } + + precondition { + condition = ( + var.container.health_check_interval >= 5 && var.container.health_check_interval <= 300 && + var.container.health_check_timeout >= 2 && var.container.health_check_timeout <= 60 && + var.container.health_check_timeout < var.container.health_check_interval && + var.container.health_check_retries >= 1 && var.container.health_check_retries <= 10 && + var.container.health_check_start_period >= 0 && var.container.health_check_start_period <= 300 && + var.container.health_stale_after_seconds >= 30 && var.container.health_stale_after_seconds <= 3600 && + var.container.shutdown_timeout_seconds >= 1 && var.container.shutdown_timeout_seconds <= 119 && + var.container.session_close_timeout_seconds >= 1 && var.container.session_close_timeout_seconds <= 60 && + var.container.reconnect_initial_backoff_seconds >= 1 && var.container.reconnect_initial_backoff_seconds <= 300 && + var.container.reconnect_max_backoff_seconds >= 1 && var.container.reconnect_max_backoff_seconds <= 3600 && + var.container.reconnect_initial_backoff_seconds <= var.container.reconnect_max_backoff_seconds && + var.container.stop_timeout_seconds >= 2 && var.container.stop_timeout_seconds <= 120 && + var.container.shutdown_timeout_seconds < var.container.stop_timeout_seconds + ) + error_message = "Container health and shutdown timings must be within ECS limits, with health timeout below interval and application shutdown below task stop timeout." + } + + precondition { + condition = ( + contains(keys(local.fargate_memory_by_cpu), tostring(var.ecs.task.cpu)) && + contains(lookup(local.fargate_memory_by_cpu, tostring(var.ecs.task.cpu), []), var.ecs.task.memory) + ) + error_message = "ecs.task.cpu and ecs.task.memory must be a supported Fargate CPU/memory combination." + } + + precondition { + condition = ( + contains(["X86_64", "ARM64"], var.ecs.task.cpu_architecture) && + (var.ecs.task.ephemeral_storage == null ? true : ( + var.ecs.task.ephemeral_storage.size_in_gib >= 21 && var.ecs.task.ephemeral_storage.size_in_gib <= 200 + )) + ) + error_message = "ecs.task.cpu_architecture must be X86_64 or ARM64, and optional ephemeral storage must be between 21 and 200 GiB." + } + + precondition { + condition = ( + contains(["managed", "external"], var.ecs.cluster.mode) && + (var.ecs.cluster.mode == "external" ? ( + var.ecs.cluster.arn != null && can(regex("^arn:[^:]+:ecs:[^:]+:[0-9]{12}:cluster/.+$", var.ecs.cluster.arn)) + ) : ( + var.ecs.cluster.arn == null && + (var.ecs.cluster.name == null ? true : can(regex("^[A-Za-z0-9_-]{1,255}$", var.ecs.cluster.name))) + )) + ) + error_message = "Use a valid external ECS cluster ARN only with cluster.mode external; managed cluster names may contain letters, digits, underscores, and hyphens." + } + + precondition { + condition = ( + startswith(var.ecs.iam.path, "/") && + endswith(var.ecs.iam.path, "/") && + length(var.ecs.iam.path) <= 512 + ) + error_message = "ecs.iam.path must start and end with a slash and be at most 512 characters." + } + + precondition { + condition = ( + length(var.network.vpc_id) > 0 && + length(var.network.subnet_ids) > 0 && + length(var.network.https_egress.ipv4_cidrs) + length(var.network.https_egress.ipv6_cidrs) > 0 && + alltrue([for cidr in var.network.https_egress.ipv4_cidrs : can(cidrnetmask(cidr))]) && + alltrue([for cidr in var.network.https_egress.ipv6_cidrs : can(cidrhost(cidr, 0)) && strcontains(cidr, ":")]) + ) + error_message = "network must select a VPC and at least one subnet, and HTTPS egress must contain valid IPv4 or IPv6 CIDRs." + } + + precondition { + condition = ( + contains(["STANDARD", "INFREQUENT_ACCESS"], var.logging.log_group_class) && + contains([1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1096, 1827, 2192, 2557, 2922, 3288, 3653], var.logging.retention_in_days) && + (var.logging.kms_key_arn == null ? true : can(regex("^arn:[^:]+:kms:[^:]+:[0-9]{12}:key/.+$", var.logging.kms_key_arn))) + ) + error_message = "logging must use a supported class and retention period; kms_key_arn must be a KMS key ARN when set." + } + } +} + +resource "terraform_data" "validate_config_store" { + lifecycle { + precondition { + condition = ( + contains(["Standard", "Advanced"], var.config_store.tier) && + startswith(local.config_store_path_prefix, "/") && + !endswith(local.config_store_path_prefix, "/") && + length(local.config_store_path_prefix) >= 2 && + can(regex("^/[A-Za-z0-9_.\\/-]+$", local.config_store_path_prefix)) + ) + error_message = "config_store must use Standard or Advanced tier and a valid absolute SSM path prefix without a trailing slash." + } + + precondition { + condition = alltrue([ + for config_key, config in local.reconciler_configs : ( + length("${local.config_store_path_prefix}/${config.group_name}/${config.runner_name}") <= 1011 && + local.reconciler_config_bytes[config_key] <= local.config_store_max_bytes + ) + ]) + error_message = "Each reconciler SSM parameter name and encoded JSON value must fit the selected Parameter Store tier. Split large controller groups or reduce provider configuration when necessary." + } + } +} + +resource "terraform_data" "validate_group_task_policy" { + for_each = local.controller_groups + + lifecycle { + precondition { + condition = ( + floor(length(base64encode(data.aws_iam_policy_document.task[each.key].json)) * 3 / 4) - + (endswith(base64encode(data.aws_iam_policy_document.task[each.key].json), "==") ? 2 : endswith(base64encode(data.aws_iam_policy_document.task[each.key].json), "=") ? 1 : 0) + ) <= 10240 + error_message = "Controller group ${each.key} produces a task-role policy exceeding AWS's 10240-byte inline role-policy quota. Split the group or reduce controller permissions." + } + } +} + +resource "terraform_data" "validate_compute_role_policy" { + for_each = local.compute_role_configs + + lifecycle { + precondition { + condition = ( + floor(length(base64encode(data.aws_iam_policy_document.compute[each.key].json)) * 3 / 4) - + (endswith(base64encode(data.aws_iam_policy_document.compute[each.key].json), "==") ? 2 : endswith(base64encode(data.aws_iam_policy_document.compute[each.key].json), "=") ? 1 : 0) + ) <= 10240 + error_message = "Compute role ${each.key} produces an inline policy exceeding AWS's 10240-byte role-policy quota. Split the group or reduce provider IAM statements." + } + } +} diff --git a/modules/orchestration-providers/scale-set/variables.tf b/modules/orchestration-providers/scale-set/variables.tf new file mode 100644 index 0000000000..236dfc8213 --- /dev/null +++ b/modules/orchestration-providers/scale-set/variables.tf @@ -0,0 +1,200 @@ +variable "prefix" { + description = "Stable prefix used for scale-set controller resources." + type = string + default = "github-actions" + nullable = false +} + +variable "log_level" { + description = "Logging level for the scale-set controller container." + type = string + default = "info" + nullable = false +} + +variable "runner_configs" { + description = <<-EOT + Normalized scale-set runner configurations keyed by stable runner-config name. + + Map keys must be known during planning. Credential values are never accepted: `github.app` contains only the exact GitHub App Parameter Store references used by the runtime. `github.enterprise_server` and `github.user_agent` carry the global GitHub settings needed to render each reconciler configuration. `scale_set.runner.group_name` selects the GitHub runner group. `runner_registration_level` selects organization or repository registration, and `runner_owner` supplies the corresponding organization or owner/repository path. Enterprise-level registration is not supported by this module. `compute_provider` carries the provider-neutral scale-set capability contract for this runner configuration. Parameter and optional KMS ARNs, scale-set names, and other inner values may remain unknown until apply. + EOT + type = map(object({ + github = object({ + enterprise_server = object({ + url = optional(string, null) + ssl_verify = optional(bool, true) + }) + app = object({ + app_id = object({ + name = string + arn = string + kms_key_arn = optional(string, null) + }) + private_key = object({ + name = string + arn = string + kms_key_arn = optional(string, null) + }) + installation_id = object({ + name = string + arn = string + kms_key_arn = optional(string, null) + }) + }) + runner_owner = string + runner_registration_level = string + user_agent = string + }) + scale_set = object({ + name = string + runner = optional(object({ + labels = optional(list(string), []) + group_name = optional(string, "Default") + min_runners = optional(number, 0) + max_runners = optional(number, 10) + boot_time_in_minutes = optional(number, 10) + }), {}) + }) + compute_provider = object({ + type = string + capabilities = object({ + scale_set = object({ + role_arn = optional(string, null) + configuration_json = optional(string, "{}") + environment_variables = optional(map(string), {}) + iam_statements = optional(map(object({ + actions = set(string) + resources = set(string) + conditions = optional(list(object({ + test = string + variable = string + values = set(string) + })), []) + })), {}) + }) + }) + }) + })) + nullable = false +} + +variable "grouping" { + description = <<-EOT + Packing strategy for scale-set reconcilers. `compute_provider` creates one controller group per compute-provider type and is the default. `runner_config` creates one group per runner config. `custom` uses `custom.groups`; custom membership must cover every runner config exactly once. + + The strategy, custom group keys, and memberships select Terraform `for_each` instances and must be known during planning. + EOT + type = object({ + strategy = optional(string, "compute_provider") + custom = optional(object({ + groups = map(object({ + runner_configs = set(string) + })) + }), null) + }) + default = {} + nullable = false +} + +variable "container" { + description = "Scale-set controller image and runtime settings. A null image uses the internal official convenience image; production callers should use the release digest. Filesystem and Linux capability hardening are enforced by the module; health_path is fixed at /healthz, the ECS liveness endpoint." + type = object({ + image = optional(string, null) + user = optional(string, "10001:10001") + health_port = optional(number, 8080) + health_path = optional(string, "/healthz") + health_check_command = optional(list(string), null) + health_check_interval = optional(number, 30) + health_check_timeout = optional(number, 5) + health_check_retries = optional(number, 3) + health_check_start_period = optional(number, 30) + health_stale_after_seconds = optional(number, 180) + shutdown_timeout_seconds = optional(number, 110) + session_close_timeout_seconds = optional(number, 10) + reconnect_initial_backoff_seconds = optional(number, 1) + reconnect_max_backoff_seconds = optional(number, 30) + stop_timeout_seconds = optional(number, 120) + }) + default = {} + nullable = false +} + +variable "config_store" { + description = <<-EOT + Non-secret controller configuration storage. The module writes one SSM String parameter per reconciler below `path_prefix//`. The task receives only its group path and a SHA-256 revision, then loads the group with `GetParametersByPath`. + + Standard parameters are limited to 4096 encoded bytes and Advanced parameters to 8192 encoded bytes. Null `path_prefix` resolves to `//scale-set-controller`. + EOT + type = object({ + path_prefix = optional(string, null) + tier = optional(string, "Standard") + tags = optional(map(string), {}) + }) + default = {} + nullable = false +} + +variable "ecs" { + description = <<-EOT + ECS substrate configuration. A managed cluster is created by default. For an external cluster, set `cluster.mode = "external"` and pass its ARN; the mode must be plan-known while the ARN may be computed. + EOT + type = object({ + cluster = optional(object({ + mode = optional(string, "managed") + arn = optional(string, null) + name = optional(string, null) + container_insights = optional(bool, true) + }), {}) + task = optional(object({ + cpu = optional(number, 512) + memory = optional(number, 1024) + cpu_architecture = optional(string, "X86_64") + ephemeral_storage = optional(object({ + size_in_gib = number + }), null) + }), {}) + service = optional(object({ + platform_version = optional(string, "LATEST") + }), {}) + iam = optional(object({ + path = optional(string, "/") + permissions_boundary = optional(string, null) + }), {}) + }) + default = {} + nullable = false +} + +variable "network" { + description = <<-EOT + Private Fargate networking. Tasks never receive public IP addresses and the managed security groups have no ingress. HTTPS egress defaults to IPv4 Internet access because GitHub endpoints cannot be represented as security-group destinations; route it through controlled NAT, firewall, or proxy infrastructure when required. + EOT + type = object({ + vpc_id = string + subnet_ids = set(string) + https_egress = optional(object({ + ipv4_cidrs = optional(set(string), ["0.0.0.0/0"]) + ipv6_cidrs = optional(set(string), []) + }), {}) + }) + nullable = false +} + +variable "logging" { + description = "CloudWatch Logs configuration. CloudWatch encrypts logs at rest with an AWS-owned key by default; set `kms_key_arn` to use a customer-managed key." + type = object({ + retention_in_days = optional(number, 30) + kms_key_arn = optional(string, null) + log_group_class = optional(string, "STANDARD") + tags = optional(map(string), {}) + }) + default = {} + nullable = false +} + +variable "tags" { + description = "Tags applied to scale-set orchestration resources." + type = map(string) + default = {} + nullable = false +} diff --git a/modules/orchestration-providers/scale-set/versions.tf b/modules/orchestration-providers/scale-set/versions.tf new file mode 100644 index 0000000000..0bedc91fd5 --- /dev/null +++ b/modules/orchestration-providers/scale-set/versions.tf @@ -0,0 +1,10 @@ +terraform { + required_version = ">= 1.5.6" + + required_providers { + aws = { + source = "hashicorp/aws" + version = ">= 6.33" + } + } +} From c87e2b8dcf424c86f700953c06682358bbd14f9d Mon Sep 17 00:00:00 2001 From: Ederson Brilhante Date: Thu, 17 Sep 2026 17:14:10 +0200 Subject: [PATCH 02/20] fix(scale-set): validate compute task policy --- .../scale-set/tests/scale-set.tftest.hcl | 6 +++--- .../scale-set/validations.tf | 14 ++++++++++++++ 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl b/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl index 870a73b9b2..8a2d319e32 100644 --- a/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl +++ b/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl @@ -887,11 +887,11 @@ run "rejects_controller_group_policy_above_inline_quota" { command = plan plan_options { - target = [terraform_data.validate_group_task_policy["ec2"]] + target = [terraform_data.validate_group_compute_policy["ec2"]] } override_data { - target = data.aws_iam_policy_document.task["ec2"] + target = data.aws_iam_policy_document.task_compute["ec2"] values = { json = <<-JSON {"payload":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"} @@ -899,7 +899,7 @@ run "rejects_controller_group_policy_above_inline_quota" { } } - expect_failures = [terraform_data.validate_group_task_policy["ec2"]] + expect_failures = [terraform_data.validate_group_compute_policy["ec2"]] } run "rejects_conflicting_group_environment_variables" { diff --git a/modules/orchestration-providers/scale-set/validations.tf b/modules/orchestration-providers/scale-set/validations.tf index 9553581597..d956e21f2f 100644 --- a/modules/orchestration-providers/scale-set/validations.tf +++ b/modules/orchestration-providers/scale-set/validations.tf @@ -423,3 +423,17 @@ resource "terraform_data" "validate_compute_role_policy" { } } } + +resource "terraform_data" "validate_group_compute_policy" { + for_each = local.controller_groups + + lifecycle { + precondition { + condition = ( + floor(length(base64encode(data.aws_iam_policy_document.task_compute[each.key].json)) * 3 / 4) - + (endswith(base64encode(data.aws_iam_policy_document.task_compute[each.key].json), "==") ? 2 : endswith(base64encode(data.aws_iam_policy_document.task_compute[each.key].json), "=") ? 1 : 0) + ) <= 10240 + error_message = "Controller group ${each.key} produces a compute-provider task-role policy exceeding AWS's 10240-byte inline role-policy quota. Split the group or reduce provider IAM statements." + } + } +} From 344736cd0cf156e15ed6205f2175aae3defc8163 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Thu, 17 Sep 2026 15:25:37 +0000 Subject: [PATCH 03/20] docs: auto update terraform docs --- modules/orchestration-providers/scale-set/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/modules/orchestration-providers/scale-set/README.md b/modules/orchestration-providers/scale-set/README.md index 495a8cc314..cc7c90eb23 100644 --- a/modules/orchestration-providers/scale-set/README.md +++ b/modules/orchestration-providers/scale-set/README.md @@ -218,6 +218,7 @@ No modules. | [terraform_data.validate_compute_role_policy](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | | [terraform_data.validate_config_store](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | | [terraform_data.validate_contract](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | +| [terraform_data.validate_group_compute_policy](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | | [terraform_data.validate_group_task_policy](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | | [terraform_data.validate_grouping](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | | [terraform_data.validate_runtime](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | From 037920104fd16a9c031c0b27f3fab58bff0bcdd9 Mon Sep 17 00:00:00 2001 From: edersonbrilhante Date: Thu, 17 Sep 2026 23:35:43 +0200 Subject: [PATCH 04/20] fix(scale-set): use declared task policy for quota validation --- .../scale-set/tests/scale-set.tftest.hcl | 6 +++--- .../scale-set/validations.tf | 14 -------------- 2 files changed, 3 insertions(+), 17 deletions(-) diff --git a/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl b/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl index 8a2d319e32..870a73b9b2 100644 --- a/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl +++ b/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl @@ -887,11 +887,11 @@ run "rejects_controller_group_policy_above_inline_quota" { command = plan plan_options { - target = [terraform_data.validate_group_compute_policy["ec2"]] + target = [terraform_data.validate_group_task_policy["ec2"]] } override_data { - target = data.aws_iam_policy_document.task_compute["ec2"] + target = data.aws_iam_policy_document.task["ec2"] values = { json = <<-JSON {"payload":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"} @@ -899,7 +899,7 @@ run "rejects_controller_group_policy_above_inline_quota" { } } - expect_failures = [terraform_data.validate_group_compute_policy["ec2"]] + expect_failures = [terraform_data.validate_group_task_policy["ec2"]] } run "rejects_conflicting_group_environment_variables" { diff --git a/modules/orchestration-providers/scale-set/validations.tf b/modules/orchestration-providers/scale-set/validations.tf index d956e21f2f..9553581597 100644 --- a/modules/orchestration-providers/scale-set/validations.tf +++ b/modules/orchestration-providers/scale-set/validations.tf @@ -423,17 +423,3 @@ resource "terraform_data" "validate_compute_role_policy" { } } } - -resource "terraform_data" "validate_group_compute_policy" { - for_each = local.controller_groups - - lifecycle { - precondition { - condition = ( - floor(length(base64encode(data.aws_iam_policy_document.task_compute[each.key].json)) * 3 / 4) - - (endswith(base64encode(data.aws_iam_policy_document.task_compute[each.key].json), "==") ? 2 : endswith(base64encode(data.aws_iam_policy_document.task_compute[each.key].json), "=") ? 1 : 0) - ) <= 10240 - error_message = "Controller group ${each.key} produces a compute-provider task-role policy exceeding AWS's 10240-byte inline role-policy quota. Split the group or reduce provider IAM statements." - } - } -} From 98693b40b0af107f1c0a0143d813e84700fcc727 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Thu, 17 Sep 2026 21:36:23 +0000 Subject: [PATCH 05/20] docs: auto update terraform docs --- modules/orchestration-providers/scale-set/README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/modules/orchestration-providers/scale-set/README.md b/modules/orchestration-providers/scale-set/README.md index cc7c90eb23..495a8cc314 100644 --- a/modules/orchestration-providers/scale-set/README.md +++ b/modules/orchestration-providers/scale-set/README.md @@ -218,7 +218,6 @@ No modules. | [terraform_data.validate_compute_role_policy](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | | [terraform_data.validate_config_store](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | | [terraform_data.validate_contract](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | -| [terraform_data.validate_group_compute_policy](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | | [terraform_data.validate_group_task_policy](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | | [terraform_data.validate_grouping](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | | [terraform_data.validate_runtime](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | From 527dadde912a3649b1a937ca754878d12f9653e8 Mon Sep 17 00:00:00 2001 From: Ederson Brilhante Date: Thu, 17 Sep 2026 23:42:31 +0200 Subject: [PATCH 06/20] feat(scale-set): wire orchestration through runner config (#5350) ## Description Wire scale-set orchestration through the provider-aware Terraform composition. - Resolve global and per-lane scale-set settings into the canonical runner configuration. - Route scale-set lanes through `runner-config` and the scale-set orchestration provider. - Extend the EC2 provider contract with the scale-set runtime configuration, IAM fragments, and outputs consumed by the controller. - Gate webhook resources by the selected orchestration provider. - Add scale-set validation and Terraform coverage across the affected modules. This PR provides the Terraform wiring used by the scale-set example and the later MiniStack integration PRs. ## Test Plan - Terraform formatting, validation/TFLint, and merge-conflict checks passed through the repository hooks. - Scale-set routing and configuration-resolution tests passed. - Multi-runner and runner-config scale-set Terraform tests passed. - Native provider tests remain subject to the local macOS arm64 plugin-handshake limitation. ## Related Issues Depends on #5299. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- modules/compute-providers/aws/ec2/outputs.tf | 2 + .../compute-providers/aws/ec2/scale-set.tf | 257 +++++++++++ .../aws/ec2/tests/provider.tftest.hcl | 29 ++ modules/multi-runner/README.md | 10 +- .../config.experimental.effective.tf | 7 +- .../config.experimental.resolved.tf | 1 + .../config.experimental.translation.tf | 6 +- modules/multi-runner/main.tf | 5 +- .../orchestration-provider.scale-set.tf | 62 +++ modules/multi-runner/outputs.tf | 11 +- modules/multi-runner/queues.tf | 15 +- modules/multi-runner/runners.experimental.tf | 1 + .../tests/config-resolution.tftest.hcl | 399 +++++++++++++++++- modules/multi-runner/validations.tf | 41 +- .../variables.experimental.github.tf | 15 +- ...les.experimental.orchestration-provider.tf | 72 ++++ modules/multi-runner/variables.tf | 13 + .../scale-set/README.md | 1 + .../scale-set/validations.tf | 14 + modules/runner-config/README.md | 3 +- .../runner-config/orchestration-provider.tf | 7 +- modules/runner-config/outputs.tf | 25 +- modules/runner-config/validations.tf | 7 +- .../variables.orchestration-provider.tf | 11 +- modules/ssm/README.md | 3 +- modules/ssm/outputs.tf | 4 + modules/ssm/ssm.tf | 9 + modules/ssm/variables.tf | 5 + 28 files changed, 1004 insertions(+), 31 deletions(-) create mode 100644 modules/compute-providers/aws/ec2/scale-set.tf create mode 100644 modules/multi-runner/orchestration-provider.scale-set.tf diff --git a/modules/compute-providers/aws/ec2/outputs.tf b/modules/compute-providers/aws/ec2/outputs.tf index 422383df0f..83b2647fca 100644 --- a/modules/compute-providers/aws/ec2/outputs.tf +++ b/modules/compute-providers/aws/ec2/outputs.tf @@ -16,6 +16,8 @@ output "resources" { output "provider" { description = "Nested EC2 compute-provider contract consumed by runner-config." value = { + type = "ec2" + capabilities = { scale_set = local.scale_set_capability } environment_variables = local.provider_environment_variables policies = local.provider_policies resources = local.provider_resources diff --git a/modules/compute-providers/aws/ec2/scale-set.tf b/modules/compute-providers/aws/ec2/scale-set.tf new file mode 100644 index 0000000000..dcbefd55f4 --- /dev/null +++ b/modules/compute-providers/aws/ec2/scale-set.tf @@ -0,0 +1,257 @@ +# Provider-owned runtime and IAM fragments for the additive scale-set +# orchestration capability. GitHub credentials, GitHub scope, desired capacity, +# and boot timeout remain orchestration-owned and are not serialized here. +locals { + scale_set_ec2_instance_criteria = merge( + { + instanceTypes = var.config.instance_types + targetCapacityType = var.config.instance_target_capacity_type + instanceAllocationStrategy = var.config.instance_allocation_strategy + }, + var.config.instance_type_priorities == null ? {} : { + instanceTypePriorities = var.config.instance_type_priorities + }, + var.config.instance_max_spot_price == null ? {} : { + maxSpotPrice = var.config.instance_max_spot_price + }, + ) + + scale_set_runtime_configuration = merge( + { + region = var.aws_region + environment = var.prefix + runnerNamePrefix = var.runner.name_prefix + jitConfigParameterPath = "${var.ssm.paths.root}/${var.ssm.paths.tokens}" + subnets = var.config.subnet_ids + launchTemplateName = aws_launch_template.runner.name + ec2instanceCriteria = local.scale_set_ec2_instance_criteria + onDemandFailoverOnError = var.config.on_demand_failover_for_errors + useDedicatedHost = var.config.use_dedicated_host + ssmParameterTags = [ + for key in sort(keys(local.ssm_parameter_tags)) : { + Key = key + Value = local.ssm_parameter_tags[key] + } + ] + }, + local.ami_id_ssm_external ? { + amiIdSsmParameterName = local.ami_id_ssm_parameter_name + } : {}, + ) + + scale_set_owned_instance_conditions = [ + { + test = "StringEquals" + variable = "ec2:ResourceTag/ghr:Application" + values = toset(["github-action-runner"]) + }, + { + test = "StringEquals" + variable = "ec2:ResourceTag/ghr:created_by" + values = toset(["scale-set-service"]) + }, + { + test = "StringEquals" + variable = "ec2:ResourceTag/ghr:environment" + values = toset([var.prefix]) + }, + ] + + scale_set_owned_request_conditions = [ + { + test = "StringEquals" + variable = "aws:RequestTag/ghr:Application" + values = toset(["github-action-runner"]) + }, + { + test = "StringEquals" + variable = "aws:RequestTag/ghr:created_by" + values = toset(["scale-set-service"]) + }, + { + test = "StringEquals" + variable = "aws:RequestTag/ghr:environment" + values = toset([var.prefix]) + }, + ] + + scale_set_launch_dependency_resources = toset(concat( + [ + "arn:${var.aws_partition}:ec2:${var.aws_region}::image/*", + "arn:${var.aws_partition}:ec2:${var.aws_region}:*:snapshot/*", + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:dedicated-host/*", + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:network-interface/*", + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:placement-group/*", + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:security-group/*", + aws_launch_template.runner.arn, + ], + [ + for subnet_id in var.config.subnet_ids : + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:subnet/${subnet_id}" + ], + var.config.key_name == null ? [] : [ + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:key-pair/${var.config.key_name}", + ], + )) + + scale_set_create_fleet_dependency_resources = toset(concat( + [ + "arn:${var.aws_partition}:ec2:${var.aws_region}::image/*", + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:placement-group/*", + aws_launch_template.runner.arn, + ], + [ + for subnet_id in var.config.subnet_ids : + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:subnet/${subnet_id}" + ], + )) + + scale_set_iam_statements = merge( + { + describe_ec2 = { + actions = toset([ + "ec2:DescribeInstances", + "ec2:DescribeLaunchTemplateVersions", + "ec2:DescribeTags", + ]) + # These EC2 Describe APIs do not support resource-level permissions. + resources = toset(["*"]) + conditions = [] + } + create_fleet_dependencies = { + actions = toset(["ec2:CreateFleet"]) + resources = local.scale_set_create_fleet_dependency_resources + conditions = [] + } + create_owned_fleet_capacity = { + actions = toset(["ec2:CreateFleet"]) + resources = toset([ + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:fleet/*", + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:instance/*", + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:volume/*", + ]) + conditions = local.scale_set_owned_request_conditions + } + run_instances_dependencies = { + actions = toset(["ec2:RunInstances"]) + resources = local.scale_set_launch_dependency_resources + conditions = [] + } + run_owned_instances = { + actions = toset(["ec2:RunInstances"]) + resources = toset([ + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:instance/*", + "arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:volume/*", + ]) + conditions = local.scale_set_owned_request_conditions + } + tag_runners_on_create = { + actions = toset(["ec2:CreateTags"]) + resources = toset(["arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:*/*"]) + conditions = [ + { + test = "StringEquals" + variable = "ec2:CreateAction" + values = toset(["CreateFleet", "RunInstances"]) + }, + ] + } + update_owned_runner_tags = { + actions = toset(["ec2:CreateTags"]) + resources = toset(["arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:instance/*"]) + conditions = concat(local.scale_set_owned_instance_conditions, [ + { + test = "ForAllValues:StringEquals" + variable = "aws:TagKeys" + values = toset([ + "ghr:github_runner_id", + "ghr:runner_name", + "ghr:scale_set_state", + ]) + }, + ]) + } + terminate_owned_runners = { + actions = toset(["ec2:TerminateInstances"]) + resources = toset(["arn:${var.aws_partition}:ec2:${var.aws_region}:${data.aws_caller_identity.current.account_id}:instance/*"]) + conditions = local.scale_set_owned_instance_conditions + } + pass_runner_role = { + actions = toset(["iam:PassRole"]) + resources = toset([var.runner.iam.role.arn]) + conditions = [ + { + test = "StringEquals" + variable = "iam:PassedToService" + values = toset(["ec2.amazonaws.com"]) + }, + ] + } + publish_runner_jit_configuration = { + actions = toset([ + "ssm:AddTagsToResource", + "ssm:DeleteParameter", + "ssm:PutParameter", + ]) + resources = toset([ + "${local.ssm_parameter_arn_prefix}${var.ssm.paths.root}/${var.ssm.paths.tokens}/*", + ]) + conditions = [] + } + read_ami_parameter = { + actions = toset([ + "ssm:GetParameter", + "ssm:GetParameters", + ]) + resources = toset([ + local.ami_id_ssm_module_managed ? aws_ssm_parameter.runner_ami_id[0].arn : local.ami_id_ssm_parameter_arn, + ]) + conditions = [] + } + }, + local.ami_kms_key_enabled ? { + use_ami_kms_key = { + actions = toset([ + "kms:Decrypt", + "kms:DescribeKey", + "kms:ReEncryptFrom", + "kms:ReEncryptTo", + ]) + resources = toset([local.ami_kms_key_arn]) + conditions = [] + } + create_ami_kms_grant = { + actions = toset(["kms:CreateGrant"]) + resources = toset([local.ami_kms_key_arn]) + conditions = [ + { + test = "Bool" + variable = "kms:GrantIsForAWSResource" + values = toset(["true"]) + }, + ] + } + } : {}, + var.config.create_service_linked_role_spot ? { + create_spot_service_linked_role = { + actions = toset(["iam:CreateServiceLinkedRole"]) + resources = toset([ + "arn:${var.aws_partition}:iam::${data.aws_caller_identity.current.account_id}:role/aws-service-role/spot.amazonaws.com/AWSServiceRoleForEC2Spot", + ]) + conditions = [ + { + test = "StringEquals" + variable = "iam:AWSServiceName" + values = toset(["spot.amazonaws.com"]) + }, + ] + } + } : {}, + ) + + scale_set_capability = { + configuration_json = jsonencode(local.scale_set_runtime_configuration) + environment_variables = {} + iam_statements = local.scale_set_iam_statements + } +} diff --git a/modules/compute-providers/aws/ec2/tests/provider.tftest.hcl b/modules/compute-providers/aws/ec2/tests/provider.tftest.hcl index bc92537279..99feb7ef8e 100644 --- a/modules/compute-providers/aws/ec2/tests/provider.tftest.hcl +++ b/modules/compute-providers/aws/ec2/tests/provider.tftest.hcl @@ -138,6 +138,15 @@ run "separates_control_plane_contract_from_ec2_resources" { error_message = "An external AMI SSM parameter must enable the pool managed policy attachment at plan time." } + assert { + condition = ( + contains(output.provider.capabilities.scale_set.iam_statements.read_ami_parameter.actions, "ssm:GetParameter") + && contains(output.provider.capabilities.scale_set.iam_statements.read_ami_parameter.actions, "ssm:GetParameters") + && contains(output.provider.capabilities.scale_set.iam_statements.read_ami_parameter.resources, "arn:aws:ssm:eu-west-1:123456789012:parameter/github-runner/ami-id") + ) + error_message = "The scale-set compute role must read an external AMI parameter with both single and batched SSM actions." + } + assert { condition = ( contains(flatten([ @@ -199,6 +208,26 @@ run "separates_control_plane_contract_from_ec2_resources" { } +run "includes_managed_ami_read_in_scale_set_contract" { + command = plan + + variables { + config = merge(var.config, { + ami = merge(var.config.ami, { + id_ssm_parameter = null + }) + }) + } + + assert { + condition = ( + contains(output.provider.capabilities.scale_set.iam_statements.read_ami_parameter.actions, "ssm:GetParameters") + && output.provider.capabilities.scale_set.iam_statements.read_ami_parameter.resources != toset([]) + ) + error_message = "The scale-set compute role must read the module-managed AMI parameter." + } +} + run "accepts_partial_typed_compute_options" { command = plan diff --git a/modules/multi-runner/README.md b/modules/multi-runner/README.md index 2866af815e..404a281580 100644 --- a/modules/multi-runner/README.md +++ b/modules/multi-runner/README.md @@ -119,6 +119,7 @@ module "multi-runner" { |------|--------|---------| | [ami\_housekeeper](#module\_ami\_housekeeper) | ../ami-housekeeper | n/a | | [instance\_termination\_watcher](#module\_instance\_termination\_watcher) | ../termination-watcher | n/a | +| [orchestration\_scale\_set](#module\_orchestration\_scale\_set) | ../orchestration-providers/scale-set | n/a | | [runner\_binaries](#module\_runner\_binaries) | ../runner-binaries-syncer | n/a | | [runner\_configs](#module\_runner\_configs) | ../runner-config | n/a | | [runners](#module\_runners) | ../runners | n/a | @@ -160,13 +161,13 @@ module "multi-runner" { | [experimental\_features](#input\_experimental\_features) | Explicit acknowledgement for opt-in features whose schemas may change
while experimental. Set to ["multi-runner-v2"] when using the v2
provider-boundary configuration. This flag will become a deprecated no-op
for one release when the feature graduates. | `set(string)` | `[]` | no | | [ghes\_ssl\_verify](#input\_ghes\_ssl\_verify) | GitHub Enterprise SSL verification. Set to 'false' when custom certificate (chains) is used for GitHub Enterprise Server (insecure). | `bool` | `true` | no | | [ghes\_url](#input\_ghes\_url) | GitHub Enterprise Server URL. Example: https://github.internal.co - DO NOT SET IF USING PUBLIC GITHUB. .However if you are using GitHub Enterprise Cloud with data-residency (ghe.com), set the endpoint here. Example - https://companyname.ghe.com\| | `string` | `null` | no | -| [github\_app](#input\_github\_app) | GitHub app parameters for the stable v1 interface, see your github app.
Omit this value when using the experimental v2 interface and provide the
app through `global_config_github` instead.
You can optionally create the SSM parameters yourself and provide the ARN and name here, through the `*_ssm` attributes.
If you chose to provide the configuration values directly here,
please ensure the key is the base64-encoded `.pem` file (the output of `base64 app.private-key.pem`, not the content of `private-key.pem`).
Note: the provided SSM parameters arn and name have a precedence over the actual value (i.e `key_base64_ssm` has a precedence over `key_base64` etc). | 
object({
    key_base64 = optional(string)
    key_base64_ssm = optional(object({
      arn  = string
      name = string
    }))
    id = optional(string)
    id_ssm = optional(object({
      arn  = string
      name = string
    }))
    webhook_secret = optional(string)
    webhook_secret_ssm = optional(object({
      arn  = string
      name = string
    }))
  })
| `{}` | no | +| [github\_app](#input\_github\_app) | GitHub app parameters for the stable v1 interface, see your github app.
Omit this value when using the experimental v2 interface and provide the
app through `global_config_github` instead.
You can optionally create the SSM parameters yourself and provide the ARN and name here, through the `*_ssm` attributes.
If you chose to provide the configuration values directly here,
please ensure the key is the base64-encoded `.pem` file (the output of `base64 app.private-key.pem`, not the content of `private-key.pem`).
Note: the provided SSM parameters arn and name have a precedence over the actual value (i.e `key_base64_ssm` has a precedence over `key_base64` etc). | 
object({
    key_base64 = optional(string)
    key_base64_ssm = optional(object({
      arn  = string
      name = string
    }))
    id = optional(string)
    id_ssm = optional(object({
      arn  = string
      name = string
    }))
    installation_id = optional(string)
    installation_id_ssm = optional(object({
      arn  = string
      name = string
    }))
    webhook_secret = optional(string)
    webhook_secret_ssm = optional(object({
      arn  = string
      name = string
    }))
  })
| `{}` | no | | [global\_config](#input\_global\_config) | Global defaults shared by all runner lanes.

global\_config = {
tags: "Tags applied to resources created for all runner lanes."
roles: {
path: "IAM path used for roles created for runner resources."
permissions\_boundary: "Optional IAM permissions boundary ARN applied to created roles."
}
runner: {
os: "Default operating system for runners."
architecture: "Default runner architecture."
disable\_default\_labels: "Whether to omit the default operating-system, architecture, and self-hosted labels."
extra\_labels: "Additional labels applied to all runners."
group\_name: "Default GitHub runner group."
name\_prefix: "Prefix for runner names."
run\_as\_root: "Whether the GitHub Actions runner executes as root."
run\_as: "User that runs the GitHub Actions agent when it is not running as root."
auto\_update\_disabled: "Whether automatic GitHub Actions runner updates are disabled."
tags: "Tags applied to runner resources."
hooks: {
job\_started: "Script executed when a job starts on a runner."
job\_completed: "Script executed when a job completes on a runner."
}
iam: {
role.arn: "Existing IAM role ARN to use for runners."
managed\_policy\_arns: "Managed policy ARNs attached to the runner IAM role."
additional\_trust\_policy\_json: "Additional trust policy JSON merged into the runner role trust policy."
path: "IAM path used for the runner role."
permissions\_boundary: "Optional IAM permissions boundary ARN for the runner role."
}
}
} | 
object({
    tags = optional(map(string), {})

    roles = optional(object({
      path                 = optional(string, null)
      permissions_boundary = optional(string, null)
    }), {})

    runner = optional(object({
      os                     = optional(string, null)
      architecture           = optional(string, null)
      disable_default_labels = optional(bool, false)
      extra_labels           = optional(list(string), [])
      group_name             = optional(string, "Default")
      name_prefix            = optional(string, "")
      run_as_root            = optional(bool, false)
      run_as                 = optional(string, "ec2-user")
      auto_update_disabled   = optional(bool, false)
      tags                   = optional(map(string), {})
      hooks = optional(object({
        job_started   = optional(string, "")
        job_completed = optional(string, "")
      }), {})
      iam = optional(object({
        role = optional(object({
          arn = string
        }), null)
        managed_policy_arns          = optional(map(string), {})
        additional_trust_policy_json = optional(string, null)
        path                         = optional(string, null)
        permissions_boundary         = optional(string, null)
      }), {})
    }), {})
  })
| `{}` | no | | [global\_config\_compute\_provider](#input\_global\_config\_compute\_provider) | Global compute-provider configuration shared by all runner lanes.

global\_config\_compute\_provider = {
selections: "Compute-provider selections keyed by namespace."
selections.namespace: "Provider namespace used to resolve a compute implementation."
selections.type: "Compute-provider type selected for the namespace."
aws.ec2.vpc\_id: "Default VPC for EC2 runners."
aws.ec2.subnet\_ids: "Default subnets for EC2 runners."
aws.ec2.managed\_security\_group\_enabled: "Whether the module manages the default runner security group."
aws.ec2.egress\_rules: "Egress rules for the managed runner security group."
aws.ec2.egress\_rules.cidr\_blocks: "IPv4 CIDR blocks allowed by an egress rule."
aws.ec2.egress\_rules.ipv6\_cidr\_blocks: "IPv6 CIDR blocks allowed by an egress rule."
aws.ec2.egress\_rules.prefix\_list\_ids: "AWS prefix lists allowed by an egress rule."
aws.ec2.egress\_rules.from\_port: "Start of the egress port range."
aws.ec2.egress\_rules.protocol: "Protocol for the egress rule."
aws.ec2.egress\_rules.security\_groups: "Referenced security groups allowed by an egress rule."
aws.ec2.egress\_rules.self: "Whether the security group itself is allowed by an egress rule."
aws.ec2.egress\_rules.to\_port: "End of the egress port range."
aws.ec2.egress\_rules.description: "Description of the egress rule."
aws.ec2.additional\_security\_group\_ids: "Additional security groups attached to EC2 runners."
aws.ec2.cloudwatch\_agent.config: "CloudWatch Agent configuration for EC2 runners."
aws.ec2.instance\_profile\_path: "IAM path used for the EC2 instance profile."
aws.ec2.key\_name: "EC2 key pair name assigned to runner instances."
aws.ec2.associate\_public\_ipv4\_address: "Whether runner instances receive a public IPv4 address."
aws.ec2.tags: "Tags applied to EC2 runner resources."
aws.ec2.ami.housekeeper.enabled: "Whether AMI cleanup is enabled."
aws.ec2.ami.housekeeper.cleanup\_config.maxItems: "Maximum number of AMIs retained by cleanup."
aws.ec2.ami.housekeeper.cleanup\_config.minimumDaysOld: "Minimum AMI age in days before cleanup."
aws.ec2.ami.housekeeper.cleanup\_config.amiFilters: "AMI filters used to select AMIs for cleanup."
aws.ec2.ami.housekeeper.cleanup\_config.amiFilters.Name: "AMI filter name."
aws.ec2.ami.housekeeper.cleanup\_config.amiFilters.Values: "Values matched by the AMI filter."
aws.ec2.ami.housekeeper.cleanup\_config.launchTemplateNames: "Launch template names associated with AMIs eligible for cleanup."
aws.ec2.ami.housekeeper.cleanup\_config.ssmParameterNames: "SSM parameter names associated with AMIs eligible for cleanup."
aws.ec2.ami.housekeeper.cleanup\_config.dryRun: "Whether AMI cleanup reports changes without deleting AMIs."
aws.ec2.ami.housekeeper.artifact.zip: "Local ZIP artifact used for the AMI housekeeper Lambda."
aws.ec2.ami.housekeeper.artifact.s3.key: "S3 object key for the AMI housekeeper Lambda artifact."
aws.ec2.ami.housekeeper.artifact.s3.object\_version: "Optional S3 object version for the AMI housekeeper artifact."
aws.ec2.ami.housekeeper.lambda.memory\_size: "Memory allocated to the AMI housekeeper Lambda."
aws.ec2.ami.housekeeper.lambda.timeout: "Timeout in seconds for the AMI housekeeper Lambda."
aws.ec2.ami.housekeeper.schedule.expression: "Schedule expression for AMI cleanup."
aws.ec2.instance\_termination\_watcher.enabled: "Whether the instance termination watcher is enabled."
aws.ec2.instance\_termination\_watcher.features.runner\_deregistration.enabled: "Whether terminated runners are deregistered."
aws.ec2.instance\_termination\_watcher.features.spot\_termination\_handler.enabled: "Whether spot termination events trigger runner handling."
aws.ec2.instance\_termination\_watcher.features.spot\_termination\_notification\_watcher.enabled: "Whether spot termination notification monitoring is enabled."
aws.ec2.instance\_termination\_watcher.environment\_variables: "Environment variables passed to the termination watcher."
aws.ec2.instance\_termination\_watcher.artifact.zip: "Local ZIP artifact used for the termination watcher Lambda."
aws.ec2.instance\_termination\_watcher.artifact.s3.key: "S3 object key for the termination watcher Lambda artifact."
aws.ec2.instance\_termination\_watcher.artifact.s3.object\_version: "Optional S3 object version for the termination watcher artifact."
aws.ec2.instance\_termination\_watcher.lambda.memory\_size: "Memory allocated to the termination watcher Lambda."
aws.ec2.instance\_termination\_watcher.lambda.timeout: "Timeout in seconds for the termination watcher Lambda."
aws.ec2.runner\_binaries.enabled: "Whether runner binary synchronization is enabled."
aws.ec2.runner\_binaries.s3.encryption.enabled: "Whether runner-binary S3 encryption is enabled."
aws.ec2.runner\_binaries.s3.encryption.bucket\_key\_enabled: "Whether an S3 bucket key is used for KMS encryption."
aws.ec2.runner\_binaries.s3.encryption.sse\_algorithm: "S3 server-side encryption algorithm."
aws.ec2.runner\_binaries.s3.encryption.kms\_master\_key\_id: "KMS key ID used for runner-binary S3 encryption."
aws.ec2.runner\_binaries.s3.tags: "Tags applied to the runner-binary S3 bucket."
aws.ec2.runner\_binaries.s3.versioning: "S3 versioning state for the runner-binary bucket."
aws.ec2.runner\_binaries.s3.logging.bucket: "S3 bucket receiving runner-binary access logs."
aws.ec2.runner\_binaries.s3.logging.prefix: "Prefix for runner-binary S3 access logs."
aws.ec2.runner\_binaries.syncer.artifact.zip: "Local ZIP artifact used for the runner-binary syncer Lambda."
aws.ec2.runner\_binaries.syncer.artifact.s3.key: "S3 object key for the runner-binary syncer artifact."
aws.ec2.runner\_binaries.syncer.artifact.s3.object\_version: "Optional S3 object version for the runner-binary syncer artifact."
aws.ec2.runner\_binaries.syncer.lambda.memory\_size: "Memory allocated to the runner-binary syncer Lambda."
aws.ec2.runner\_binaries.syncer.lambda.timeout: "Timeout in seconds for the runner-binary syncer Lambda."
aws.ec2.runner\_binaries.syncer.schedule.expression: "Schedule expression for runner-binary synchronization."
aws.ec2.runner\_binaries.syncer.schedule.state: "EventBridge rule state for runner-binary synchronization."
} | 
object({
    selections = optional(map(object({
      namespace = string
      type      = string
    })), null)
    aws = optional(object({
      ec2 = optional(object({
        vpc_id                         = optional(string, null)
        subnet_ids                     = optional(list(string), null)
        managed_security_group_enabled = optional(bool, true)
        egress_rules = optional(list(object({
          cidr_blocks      = list(string)
          ipv6_cidr_blocks = list(string)
          prefix_list_ids  = list(string)
          from_port        = number
          protocol         = string
          security_groups  = list(string)
          self             = bool
          to_port          = number
          description      = string
          })), [{
          cidr_blocks      = ["0.0.0.0/0"]
          ipv6_cidr_blocks = ["::/0"]
          prefix_list_ids  = null
          from_port        = 0
          protocol         = "-1"
          security_groups  = null
          self             = null
          to_port          = 0
          description      = null
        }])
        additional_security_group_ids = optional(list(string), [])
        cloudwatch_agent = optional(object({
          config = optional(string, null)
        }), {})
        instance_profile_path         = optional(string, null)
        key_name                      = optional(string, null)
        associate_public_ipv4_address = optional(bool, false)
        tags                          = optional(map(string), {})
        ami = optional(object({
          housekeeper = optional(object({
            enabled = optional(bool, false)
            cleanup_config = optional(object({
              maxItems       = optional(number)
              minimumDaysOld = optional(number)
              amiFilters = optional(list(object({
                Name   = string
                Values = list(string)
              })))
              launchTemplateNames = optional(list(string))
              ssmParameterNames   = optional(list(string))
              dryRun              = optional(bool)
            }), {})
            artifact = optional(object({
              zip = optional(string, null)
              s3 = optional(object({
                key            = string
                object_version = optional(string, null)
              }), null)
            }), {})
            lambda = optional(object({
              memory_size = optional(number, 256)
              timeout     = optional(number, 300)
            }), {})
            schedule = optional(object({
              expression = optional(string, "cron(11 7 * * ? *)")
            }), {})
          }), {})
        }), {})
        instance_termination_watcher = optional(object({
          enabled = optional(bool, false)
          features = optional(object({
            runner_deregistration = optional(object({
              enabled = optional(bool, true)
            }), {})
            spot_termination_handler = optional(object({
              enabled = optional(bool, true)
            }), {})
            spot_termination_notification_watcher = optional(object({
              enabled = optional(bool, true)
            }), {})
          }), {})
          environment_variables = optional(map(string), {})
          artifact = optional(object({
            zip = optional(string, null)
            s3 = optional(object({
              key            = string
              object_version = optional(string, null)
            }), null)
          }), {})
          lambda = optional(object({
            memory_size = optional(number, null)
            timeout     = optional(number, null)
          }), {})
        }), {})
        runner_binaries = optional(object({
          enabled = optional(bool, true)
          s3 = optional(object({
            encryption = optional(object({
              enabled            = optional(bool, true)
              bucket_key_enabled = optional(bool, null)
              sse_algorithm      = optional(string, "AES256")
              kms_master_key_id  = optional(string, null)
            }), {})
            tags       = optional(map(string), {})
            versioning = optional(string, "Disabled")
            logging = optional(object({
              bucket = optional(string, null)
              prefix = optional(string, null)
            }), {})
          }), {})
          syncer = optional(object({
            artifact = optional(object({
              zip = optional(string, null)
              s3 = optional(object({
                key            = string
                object_version = optional(string, null)
              }), null)
            }), {})
            lambda = optional(object({
              memory_size = optional(number, 256)
              timeout     = optional(number, 300)
            }), {})
            schedule = optional(object({
              expression = optional(string, "cron(27 * * * ? *)")
              state      = optional(string, "ENABLED")
            }), {})
          }), {})
        }), {})
      }), {})
    }), {})
  })
| `{}` | no | -| [global\_config\_github](#input\_global\_config\_github) | Global GitHub configuration shared by all runner lanes.

global\_config\_github = {
app: {
key\_base64: "Base64-encoded GitHub App private key."
key\_base64\_ssm: "SSM parameter containing the Base64-encoded GitHub App private key."
key\_base64\_ssm.arn: "ARN of the SSM parameter containing the GitHub App private key."
key\_base64\_ssm.name: "Name of the SSM parameter containing the GitHub App private key."
id: "GitHub App ID."
id\_ssm: "SSM parameter containing the GitHub App ID."
id\_ssm.arn: "ARN of the SSM parameter containing the GitHub App ID."
id\_ssm.name: "Name of the SSM parameter containing the GitHub App ID."
webhook\_secret: "GitHub App webhook secret."
webhook\_secret\_ssm: "SSM parameter containing the GitHub App webhook secret."
webhook\_secret\_ssm.arn: "ARN of the SSM parameter containing the GitHub App webhook secret."
webhook\_secret\_ssm.name: "Name of the SSM parameter containing the GitHub App webhook secret."
}
additional\_apps: "Additional GitHub Apps used to distribute GitHub API requests."
additional\_apps.key\_base64: "Base64-encoded private key for an additional GitHub App."
additional\_apps.key\_base64\_ssm: "SSM parameter containing an additional App private key."
additional\_apps.key\_base64\_ssm.arn: "ARN of the SSM parameter containing an additional App private key."
additional\_apps.key\_base64\_ssm.name: "Name of the SSM parameter containing an additional App private key."
additional\_apps.id: "ID of an additional GitHub App."
additional\_apps.id\_ssm: "SSM parameter containing an additional GitHub App ID."
additional\_apps.id\_ssm.arn: "ARN of the SSM parameter containing an additional GitHub App ID."
additional\_apps.id\_ssm.name: "Name of the SSM parameter containing an additional GitHub App ID."
additional\_apps.installation\_id: "Optional installation ID for an additional GitHub App."
additional\_apps.installation\_id\_ssm: "SSM parameter containing an additional App installation ID."
additional\_apps.installation\_id\_ssm.arn: "ARN of the SSM parameter containing an additional App installation ID."
additional\_apps.installation\_id\_ssm.name: "Name of the SSM parameter containing an additional App installation ID."
enterprise\_server.url: "GitHub Enterprise Server URL."
enterprise\_server.ssl\_verify: "Whether to verify the GitHub Enterprise Server TLS certificate."
user\_agent: "User-Agent value sent with GitHub API requests."
} | 
object({
    app = optional(object({
      key_base64 = optional(string)
      key_base64_ssm = optional(object({
        arn  = string
        name = string
      }))
      id = optional(string)
      id_ssm = optional(object({
        arn  = string
        name = string
      }))
      webhook_secret = optional(string)
      webhook_secret_ssm = optional(object({
        arn  = string
        name = string
      }))
    }), null)
    additional_apps = optional(list(object({
      key_base64          = optional(string)
      key_base64_ssm      = optional(object({ arn = string, name = string }))
      id                  = optional(string)
      id_ssm              = optional(object({ arn = string, name = string }))
      installation_id     = optional(string)
      installation_id_ssm = optional(object({ arn = string, name = string }))
    })), [])
    enterprise_server = optional(object({
      url        = optional(string, null)
      ssl_verify = optional(bool, true)
    }), {})
    user_agent = optional(string, "github-aws-runners")
  })
| `{}` | no | +| [global\_config\_github](#input\_global\_config\_github) | Global GitHub configuration shared by all runner lanes.

global\_config\_github = {
app: {
key\_base64: "Base64-encoded GitHub App private key."
key\_base64\_ssm: "SSM parameter containing the Base64-encoded GitHub App private key."
key\_base64\_ssm.arn: "ARN of the SSM parameter containing the GitHub App private key."
key\_base64\_ssm.name: "Name of the SSM parameter containing the GitHub App private key."
id: "GitHub App ID."
id\_ssm: "SSM parameter containing the GitHub App ID."
id\_ssm.arn: "ARN of the SSM parameter containing the GitHub App ID."
id\_ssm.name: "Name of the SSM parameter containing the GitHub App ID."
installation\_id: "GitHub App installation ID for the primary scale-set installation."
installation\_id\_ssm: "SSM parameter containing the primary GitHub App installation ID."
installation\_id\_ssm.arn: "ARN of the SSM parameter containing the primary GitHub App installation ID."
installation\_id\_ssm.name: "Name of the SSM parameter containing the primary GitHub App installation ID."
webhook\_secret: "GitHub App webhook secret."
webhook\_secret\_ssm: "SSM parameter containing the GitHub App webhook secret."
webhook\_secret\_ssm.arn: "ARN of the SSM parameter containing the GitHub App webhook secret."
webhook\_secret\_ssm.name: "Name of the SSM parameter containing the GitHub App webhook secret."
}
additional\_apps: "Additional GitHub Apps used to distribute GitHub API requests."
additional\_apps.key\_base64: "Base64-encoded private key for an additional GitHub App."
additional\_apps.key\_base64\_ssm: "SSM parameter containing an additional App private key."
additional\_apps.key\_base64\_ssm.arn: "ARN of the SSM parameter containing an additional App private key."
additional\_apps.key\_base64\_ssm.name: "Name of the SSM parameter containing an additional App private key."
additional\_apps.id: "ID of an additional GitHub App."
additional\_apps.id\_ssm: "SSM parameter containing an additional GitHub App ID."
additional\_apps.id\_ssm.arn: "ARN of the SSM parameter containing an additional GitHub App ID."
additional\_apps.id\_ssm.name: "Name of the SSM parameter containing an additional GitHub App ID."
additional\_apps.installation\_id: "Optional installation ID for an additional GitHub App."
additional\_apps.installation\_id\_ssm: "SSM parameter containing an additional App installation ID."
additional\_apps.installation\_id\_ssm.arn: "ARN of the SSM parameter containing an additional App installation ID."
additional\_apps.installation\_id\_ssm.name: "Name of the SSM parameter containing an additional App installation ID."
enterprise\_server.url: "GitHub Enterprise Server URL."
enterprise\_server.ssl\_verify: "Whether to verify the GitHub Enterprise Server TLS certificate."
runner\_owner: "GitHub organization or owner/repository path for organization- or repository-level scale-set registration."
runner\_registration\_level: "GitHub scale-set registration scope: organization or repository."
user\_agent: "User-Agent value sent with GitHub API requests."
} | 
object({
    app = optional(object({
      key_base64 = optional(string)
      key_base64_ssm = optional(object({
        arn  = string
        name = string
      }))
      id = optional(string)
      id_ssm = optional(object({
        arn  = string
        name = string
      }))
      installation_id = optional(string)
      installation_id_ssm = optional(object({
        arn  = string
        name = string
      }))
      webhook_secret = optional(string)
      webhook_secret_ssm = optional(object({
        arn  = string
        name = string
      }))
    }), null)
    additional_apps = optional(list(object({
      key_base64          = optional(string)
      key_base64_ssm      = optional(object({ arn = string, name = string }))
      id                  = optional(string)
      id_ssm              = optional(object({ arn = string, name = string }))
      installation_id     = optional(string)
      installation_id_ssm = optional(object({ arn = string, name = string }))
    })), [])
    enterprise_server = optional(object({
      url        = optional(string, null)
      ssl_verify = optional(bool, true)
    }), {})
    runner_owner              = optional(string, null)
    runner_registration_level = optional(string, "organization")
    user_agent                = optional(string, "github-aws-runners")
  })
| `{}` | no | | [global\_config\_lambda](#input\_global\_config\_lambda) | Global Lambda configuration shared by all runner lanes.

global\_config\_lambda = {
artifact.s3.bucket: "S3 bucket containing Lambda deployment artifacts."
runtime: "Default Lambda runtime."
architecture: "Default Lambda instruction-set architecture."
principals: "Additional AWS principals allowed to invoke the Lambda functions."
principals.type: "Principal type, such as AWS account, service, or organization."
principals.identifiers: "Identifiers allowed for the principal type."
subnet\_ids: "Subnets used by Lambda functions."
security\_group\_ids: "Security groups attached to Lambda functions."
tags: "Tags applied to Lambda functions and related resources."
role.path: "IAM path used for Lambda execution roles."
role.permissions\_boundary: "Optional IAM permissions boundary ARN for Lambda execution roles."
} | 
object({
    artifact = optional(object({
      s3 = optional(object({
        bucket = optional(string, null)
      }), {})
    }), {})
    runtime      = optional(string, "nodejs24.x")
    architecture = optional(string, "arm64")
    principals = optional(list(object({
      type        = string
      identifiers = list(string)
    })), [])
    subnet_ids         = optional(list(string), [])
    security_group_ids = optional(list(string), [])
    tags               = optional(map(string), {})
    role = optional(object({
      path                 = optional(string, null)
      permissions_boundary = optional(string, null)
    }), {})
  })
| `{}` | no | | [global\_config\_observability](#input\_global\_config\_observability) | Global observability configuration shared by all runner lanes.

global\_config\_observability = {
logs.level: "Log level for module resources."
logs.retention\_in\_days: "CloudWatch log retention period in days."
logs.kms\_key\_id: "KMS key ID used to encrypt CloudWatch log groups."
logs.class: "CloudWatch log group class."
logs.tags: "Tags applied to CloudWatch log groups."
tracing.mode: "Tracing mode used by instrumented resources."
tracing.capture\_http\_requests: "Whether HTTP requests are captured by tracing."
tracing.capture\_error: "Whether errors are captured by tracing."
metrics.enabled: "Whether module metrics are enabled."
metrics.namespace: "CloudWatch namespace used for module metrics."
metrics.metric.github\_app\_rate\_limit.enabled: "Whether GitHub App rate-limit metrics are emitted."
metrics.metric.job\_retry.enabled: "Whether job-retry metrics are emitted."
metrics.metric.spot\_termination\_warning.enabled: "Whether spot-termination warning metrics are emitted."
} | 
object({
    logs = optional(object({
      level             = optional(string, "info")
      retention_in_days = optional(number, 180)
      kms_key_id        = optional(string, null)
      class             = optional(string, "STANDARD")
      tags              = optional(map(string), {})
    }), {})
    tracing = optional(object({
      mode                  = optional(string, null)
      capture_http_requests = optional(bool, false)
      capture_error         = optional(bool, false)
    }), {})
    metrics = optional(object({
      enabled   = optional(bool, false)
      namespace = optional(string, "GitHub Runners")
      metric = optional(object({
        github_app_rate_limit = optional(object({
          enabled = optional(bool, true)
        }), {})
        job_retry = optional(object({
          enabled = optional(bool, true)
        }), {})
        spot_termination_warning = optional(object({
          enabled = optional(bool, true)
        }), {})
      }), {})
    }), {})
  })
| `{}` | no | -| [global\_config\_orchestration\_provider](#input\_global\_config\_orchestration\_provider) | Global orchestration-provider configuration shared by all runner lanes.

global\_config\_orchestration\_provider = {
webhook: {
queue\_selection\_strategy: "Strategy used to select the build queue for a webhook event."
eventbridge.enabled: "Whether EventBridge integration is enabled for webhook events."
eventbridge.accept\_events: "Event types accepted by the EventBridge integration."
matcher\_config\_parameter\_store\_tier: "SSM Parameter Store tier used for matcher configuration."
runner.boot\_time\_in\_minutes: "Expected runner boot time used by orchestration."
runner.ephemeral: "Whether runners created by the orchestration provider are ephemeral."
runner.jit\_config\_enabled: "Whether JIT runner configuration is enabled."
runner.maximum\_count: "Maximum number of runners that orchestration may create."
github.repository\_white\_list: "Repositories allowed to use the webhook configuration."
lambda.artifact.zip: "Local ZIP artifact used for orchestration Lambda functions."
lambda.artifact.s3.key: "S3 object key for the orchestration Lambda artifact."
lambda.artifact.s3.object\_version: "Optional S3 object version for the orchestration Lambda artifact."
lambda.scale.up.memory\_size: "Memory allocated to the scale-up Lambda."
lambda.scale.up.timeout: "Timeout in seconds for the scale-up Lambda."
lambda.scale.up.reserved\_concurrent\_executions: "Reserved concurrent executions for the scale-up Lambda."
lambda.scale.up.job\_queued\_check\_enabled: "Whether the scale-up Lambda checks queued jobs."
lambda.scale.up.event\_source\_mapping.batch\_size: "Maximum records passed to one scale-up Lambda invocation."
lambda.scale.up.event\_source\_mapping.maximum\_batching\_window\_in\_seconds: "Maximum time to batch records before invoking the scale-up Lambda."
lambda.scale.up.tags: "Tags applied to the scale-up Lambda."
lambda.scale.down.memory\_size: "Memory allocated to the scale-down Lambda."
lambda.scale.down.timeout: "Timeout in seconds for the scale-down Lambda."
lambda.scale.down.schedule\_expression: "Schedule expression for scale-down processing."
lambda.scale.down.minimum\_running\_time\_in\_minutes: "Minimum runner lifetime before scale-down."
lambda.scale.down.idle\_confirmation\_seconds: "Seconds a runner must consistently report not-busy before scale-down terminates it; 0 disables the confirmation window."
lambda.scale.down.idle\_config: "Scheduled minimum idle-runner pool settings."
lambda.scale.down.idle\_config.cron: "Cron expression defining when the idle-runner count applies."
lambda.scale.down.idle\_config.timeZone: "Time zone used to evaluate the idle-runner schedule."
lambda.scale.down.idle\_config.idleCount: "Minimum number of idle runners maintained during the schedule."
lambda.scale.down.idle\_config.evictionStrategy: "Strategy used when evicting idle runners."
lambda.scale.down.tags: "Tags applied to the scale-down Lambda."
lambda.webhook.artifact.zip: "Local ZIP artifact used for the webhook Lambda."
lambda.webhook.artifact.s3.key: "S3 object key for the webhook Lambda artifact."
lambda.webhook.artifact.s3.object\_version: "Optional S3 object version for the webhook Lambda artifact."
lambda.webhook.api\_gateway\_access\_log\_settings: "API Gateway access-log destination and format."
lambda.webhook.api\_gateway\_access\_log\_settings.destination\_arn: "ARN of the API Gateway access-log destination."
lambda.webhook.api\_gateway\_access\_log\_settings.format: "API Gateway access-log format."
lambda.webhook.memory\_size: "Memory allocated to the webhook Lambda."
lambda.webhook.timeout: "Timeout in seconds for the webhook Lambda."
lambda.webhook.tags: "Tags applied to the webhook Lambda."
lambda.pool.memory\_size: "Memory allocated to the pool Lambda."
lambda.pool.timeout: "Timeout in seconds for the pool Lambda."
lambda.pool.reserved\_concurrent\_executions: "Reserved concurrent executions for the pool Lambda."
lambda.pool.config: "Scheduled runner-pool size configuration."
lambda.pool.config.schedule\_expression: "Schedule expression for the pool size."
lambda.pool.config.schedule\_expression\_timezone: "Time zone used to evaluate the pool schedule."
lambda.pool.config.size: "Runner pool size applied by the schedule."
lambda.pool.include\_busy\_runners: "Whether busy runners are included in pool sizing."
lambda.pool.runner\_owner: "GitHub organization that owns the runner pool."
lambda.pool.tags: "Tags applied to the pool Lambda."
queue.delay\_webhook\_event: "Seconds a webhook event remains invisible in the build queue before processing."
queue.job\_queue\_retention\_in\_seconds: "Seconds a queued job is retained before it is purged."
queue.visibility\_timeout\_seconds: "Build queue visibility timeout in seconds."
queue.redrive\_build\_queue.enabled: "Whether the build queue dead-letter queue is enabled."
queue.redrive\_build\_queue.maxReceiveCount: "Maximum receives before a message is moved to the dead-letter queue."
queue.tags: "Tags applied to build queues."
queue.encryption.kms\_data\_key\_reuse\_period\_seconds: "KMS data-key reuse period for queue encryption."
queue.encryption.kms\_master\_key\_id: "KMS key ID used for queue encryption."
queue.encryption.sqs\_managed\_sse\_enabled: "Whether SQS-managed server-side encryption is enabled."
}
} | 
object({
    webhook = optional(object({
      queue_selection_strategy = optional(string, "first")
      eventbridge = optional(object({
        enabled       = optional(bool, true)
        accept_events = optional(list(string), [])
      }), {})
      matcher_config_parameter_store_tier = optional(string, "Standard")
      runner = optional(object({
        boot_time_in_minutes = optional(number, 5)
        ephemeral            = optional(bool, false)
        jit_config_enabled   = optional(bool, null)
        maximum_count        = optional(number, null)
      }), {})

      github = optional(object({
        repository_white_list = optional(list(string), [])
      }), {})

      lambda = optional(object({
        artifact = optional(object({
          zip = optional(string, null)
          s3 = optional(object({
            key            = string
            object_version = optional(string, null)
          }), null)
        }), {})
        scale = optional(object({
          up = optional(object({
            memory_size                    = optional(number, 512)
            timeout                        = optional(number, 30)
            reserved_concurrent_executions = optional(number, 1)
            job_queued_check_enabled       = optional(bool, null)
            event_source_mapping = optional(object({
              batch_size                         = optional(number, 10)
              maximum_batching_window_in_seconds = optional(number, 0)
            }), {})
            tags = optional(map(string), {})
          }), {})
          down = optional(object({
            memory_size                     = optional(number, 512)
            timeout                         = optional(number, 60)
            schedule_expression             = optional(string, "cron(*/5 * * * ? *)")
            minimum_running_time_in_minutes = optional(number, null)
            idle_confirmation_seconds       = optional(number, 0)
            idle_config = optional(list(object({
              cron             = string
              timeZone         = string
              idleCount        = number
              evictionStrategy = optional(string, "oldest_first")
            })), [])
            tags = optional(map(string), {})
          }), {})
        }), {})
        webhook = optional(object({
          artifact = optional(object({
            zip = optional(string, null)
            s3 = optional(object({
              key            = string
              object_version = optional(string, null)
            }), null)
          }), {})
          api_gateway_access_log_settings = optional(object({
            destination_arn = string
            format          = string
          }), null)
          memory_size = optional(number, 256)
          timeout     = optional(number, 10)
          tags        = optional(map(string), {})
        }), {})
        pool = optional(object({
          memory_size                    = optional(number, 512)
          timeout                        = optional(number, 60)
          reserved_concurrent_executions = optional(number, 1)
          config = optional(list(object({
            schedule_expression          = string
            schedule_expression_timezone = optional(string)
            size                         = number
          })), [])
          include_busy_runners = optional(bool, false)
          runner_owner         = optional(string, null)
          tags                 = optional(map(string), {})
        }), {})
      }), {})

      queue = optional(object({
        delay_webhook_event            = optional(number, 30)
        job_queue_retention_in_seconds = optional(number, 86400)
        visibility_timeout_seconds     = optional(number, 180)
        redrive_build_queue = optional(object({
          enabled         = optional(bool, false)
          maxReceiveCount = optional(number, null)
          }), {
          enabled         = false
          maxReceiveCount = null
        })
        tags = optional(map(string), {})
        encryption = optional(object({
          kms_data_key_reuse_period_seconds = number
          kms_master_key_id                 = string
          sqs_managed_sse_enabled           = bool
          }), {
          kms_data_key_reuse_period_seconds = null
          kms_master_key_id                 = null
          sqs_managed_sse_enabled           = true
        })
      }), {})
    }), {})
  })
| `{}` | no | +| [global\_config\_orchestration\_provider](#input\_global\_config\_orchestration\_provider) | Global orchestration-provider configuration shared by all runner lanes.

global\_config\_orchestration\_provider = {
webhook: {
queue\_selection\_strategy: "Strategy used to select the build queue for a webhook event."
eventbridge.enabled: "Whether EventBridge integration is enabled for webhook events."
eventbridge.accept\_events: "Event types accepted by the EventBridge integration."
matcher\_config\_parameter\_store\_tier: "SSM Parameter Store tier used for matcher configuration."
runner.boot\_time\_in\_minutes: "Expected runner boot time used by orchestration."
runner.ephemeral: "Whether runners created by the orchestration provider are ephemeral."
runner.jit\_config\_enabled: "Whether JIT runner configuration is enabled."
runner.maximum\_count: "Maximum number of runners that orchestration may create."
github.repository\_white\_list: "Repositories allowed to use the webhook configuration."
lambda.artifact.zip: "Local ZIP artifact used for orchestration Lambda functions."
lambda.artifact.s3.key: "S3 object key for the orchestration Lambda artifact."
lambda.artifact.s3.object\_version: "Optional S3 object version for the orchestration Lambda artifact."
lambda.scale.up.memory\_size: "Memory allocated to the scale-up Lambda."
lambda.scale.up.timeout: "Timeout in seconds for the scale-up Lambda."
lambda.scale.up.reserved\_concurrent\_executions: "Reserved concurrent executions for the scale-up Lambda."
lambda.scale.up.job\_queued\_check\_enabled: "Whether the scale-up Lambda checks queued jobs."
lambda.scale.up.event\_source\_mapping.batch\_size: "Maximum records passed to one scale-up Lambda invocation."
lambda.scale.up.event\_source\_mapping.maximum\_batching\_window\_in\_seconds: "Maximum time to batch records before invoking the scale-up Lambda."
lambda.scale.up.tags: "Tags applied to the scale-up Lambda."
lambda.scale.down.memory\_size: "Memory allocated to the scale-down Lambda."
lambda.scale.down.timeout: "Timeout in seconds for the scale-down Lambda."
lambda.scale.down.schedule\_expression: "Schedule expression for scale-down processing."
lambda.scale.down.minimum\_running\_time\_in\_minutes: "Minimum runner lifetime before scale-down."
lambda.scale.down.idle\_confirmation\_seconds: "Seconds a runner must consistently report not-busy before scale-down terminates it; 0 disables the confirmation window."
lambda.scale.down.idle\_config: "Scheduled minimum idle-runner pool settings."
lambda.scale.down.idle\_config.cron: "Cron expression defining when the idle-runner count applies."
lambda.scale.down.idle\_config.timeZone: "Time zone used to evaluate the idle-runner schedule."
lambda.scale.down.idle\_config.idleCount: "Minimum number of idle runners maintained during the schedule."
lambda.scale.down.idle\_config.evictionStrategy: "Strategy used when evicting idle runners."
lambda.scale.down.tags: "Tags applied to the scale-down Lambda."
lambda.webhook.artifact.zip: "Local ZIP artifact used for the webhook Lambda."
lambda.webhook.artifact.s3.key: "S3 object key for the webhook Lambda artifact."
lambda.webhook.artifact.s3.object\_version: "Optional S3 object version for the webhook Lambda artifact."
lambda.webhook.api\_gateway\_access\_log\_settings: "API Gateway access-log destination and format."
lambda.webhook.api\_gateway\_access\_log\_settings.destination\_arn: "ARN of the API Gateway access-log destination."
lambda.webhook.api\_gateway\_access\_log\_settings.format: "API Gateway access-log format."
lambda.webhook.memory\_size: "Memory allocated to the webhook Lambda."
lambda.webhook.timeout: "Timeout in seconds for the webhook Lambda."
lambda.webhook.tags: "Tags applied to the webhook Lambda."
lambda.pool.memory\_size: "Memory allocated to the pool Lambda."
lambda.pool.timeout: "Timeout in seconds for the pool Lambda."
lambda.pool.reserved\_concurrent\_executions: "Reserved concurrent executions for the pool Lambda."
lambda.pool.config: "Scheduled runner-pool size configuration."
lambda.pool.config.schedule\_expression: "Schedule expression for the pool size."
lambda.pool.config.schedule\_expression\_timezone: "Time zone used to evaluate the pool schedule."
lambda.pool.config.size: "Runner pool size applied by the schedule."
lambda.pool.include\_busy\_runners: "Whether busy runners are included in pool sizing."
lambda.pool.runner\_owner: "GitHub organization that owns the runner pool."
lambda.pool.tags: "Tags applied to the pool Lambda."
queue.delay\_webhook\_event: "Seconds a webhook event remains invisible in the build queue before processing."
queue.job\_queue\_retention\_in\_seconds: "Seconds a queued job is retained before it is purged."
queue.visibility\_timeout\_seconds: "Build queue visibility timeout in seconds."
queue.redrive\_build\_queue.enabled: "Whether the build queue dead-letter queue is enabled."
queue.redrive\_build\_queue.maxReceiveCount: "Maximum receives before a message is moved to the dead-letter queue."
queue.tags: "Tags applied to build queues."
queue.encryption.kms\_data\_key\_reuse\_period\_seconds: "KMS data-key reuse period for queue encryption."
queue.encryption.kms\_master\_key\_id: "KMS key ID used for queue encryption."
queue.encryption.sqs\_managed\_sse\_enabled: "Whether SQS-managed server-side encryption is enabled."
}
} | 
object({
    webhook = optional(object({
      queue_selection_strategy = optional(string, "first")
      eventbridge = optional(object({
        enabled       = optional(bool, true)
        accept_events = optional(list(string), [])
      }), {})
      matcher_config_parameter_store_tier = optional(string, "Standard")
      runner = optional(object({
        boot_time_in_minutes = optional(number, 5)
        ephemeral            = optional(bool, false)
        jit_config_enabled   = optional(bool, null)
        maximum_count        = optional(number, null)
      }), {})

      github = optional(object({
        repository_white_list = optional(list(string), [])
      }), {})

      lambda = optional(object({
        artifact = optional(object({
          zip = optional(string, null)
          s3 = optional(object({
            key            = string
            object_version = optional(string, null)
          }), null)
        }), {})
        scale = optional(object({
          up = optional(object({
            memory_size                    = optional(number, 512)
            timeout                        = optional(number, 30)
            reserved_concurrent_executions = optional(number, 1)
            job_queued_check_enabled       = optional(bool, null)
            event_source_mapping = optional(object({
              batch_size                         = optional(number, 10)
              maximum_batching_window_in_seconds = optional(number, 0)
            }), {})
            tags = optional(map(string), {})
          }), {})
          down = optional(object({
            memory_size                     = optional(number, 512)
            timeout                         = optional(number, 60)
            schedule_expression             = optional(string, "cron(*/5 * * * ? *)")
            minimum_running_time_in_minutes = optional(number, null)
            idle_confirmation_seconds       = optional(number, 0)
            idle_config = optional(list(object({
              cron             = string
              timeZone         = string
              idleCount        = number
              evictionStrategy = optional(string, "oldest_first")
            })), [])
            tags = optional(map(string), {})
          }), {})
        }), {})
        webhook = optional(object({
          artifact = optional(object({
            zip = optional(string, null)
            s3 = optional(object({
              key            = string
              object_version = optional(string, null)
            }), null)
          }), {})
          api_gateway_access_log_settings = optional(object({
            destination_arn = string
            format          = string
          }), null)
          memory_size = optional(number, 256)
          timeout     = optional(number, 10)
          tags        = optional(map(string), {})
        }), {})
        pool = optional(object({
          memory_size                    = optional(number, 512)
          timeout                        = optional(number, 60)
          reserved_concurrent_executions = optional(number, 1)
          config = optional(list(object({
            schedule_expression          = string
            schedule_expression_timezone = optional(string)
            size                         = number
          })), [])
          include_busy_runners = optional(bool, false)
          runner_owner         = optional(string, null)
          tags                 = optional(map(string), {})
        }), {})
      }), {})

      queue = optional(object({
        delay_webhook_event            = optional(number, 30)
        job_queue_retention_in_seconds = optional(number, 86400)
        visibility_timeout_seconds     = optional(number, 180)
        redrive_build_queue = optional(object({
          enabled         = optional(bool, false)
          maxReceiveCount = optional(number, null)
          }), {
          enabled         = false
          maxReceiveCount = null
        })
        tags = optional(map(string), {})
        encryption = optional(object({
          kms_data_key_reuse_period_seconds = number
          kms_master_key_id                 = string
          sqs_managed_sse_enabled           = bool
          }), {
          kms_data_key_reuse_period_seconds = null
          kms_master_key_id                 = null
          sqs_managed_sse_enabled           = true
        })
      }), {})

    }), {})

    scale_set = optional(object({
      grouping = optional(object({
        strategy = optional(string, "compute_provider")
        custom = optional(object({
          groups = map(object({
            runner_configs = set(string)
          }))
        }), null)
      }), {})
      container = optional(object({
        image                             = optional(string, null)
        user                              = optional(string, "10001:10001")
        health_port                       = optional(number, 8080)
        health_path                       = optional(string, "/healthz")
        health_check_command              = optional(list(string), null)
        health_check_interval             = optional(number, 30)
        health_check_timeout              = optional(number, 5)
        health_check_retries              = optional(number, 3)
        health_check_start_period         = optional(number, 30)
        health_stale_after_seconds        = optional(number, 180)
        shutdown_timeout_seconds          = optional(number, 110)
        session_close_timeout_seconds     = optional(number, 10)
        reconnect_initial_backoff_seconds = optional(number, 1)
        reconnect_max_backoff_seconds     = optional(number, 30)
        stop_timeout_seconds              = optional(number, 120)
      }), {})
      config_store = optional(object({
        path_prefix = optional(string, null)
        tier        = optional(string, "Standard")
        tags        = optional(map(string), {})
      }), {})
      ecs = optional(object({
        cluster = optional(object({
          mode               = optional(string, "managed")
          arn                = optional(string, null)
          name               = optional(string, null)
          container_insights = optional(bool, true)
        }), {})
        task = optional(object({
          cpu              = optional(number, 512)
          memory           = optional(number, 1024)
          cpu_architecture = optional(string, "X86_64")
          ephemeral_storage = optional(object({
            size_in_gib = number
          }), null)
        }), {})
        service = optional(object({
          platform_version = optional(string, "LATEST")
        }), {})
        iam = optional(object({
          path                 = optional(string, "/")
          permissions_boundary = optional(string, null)
        }), {})
      }), {})
      network = optional(object({
        vpc_id     = optional(string, null)
        subnet_ids = optional(set(string), null)
        https_egress = optional(object({
          ipv4_cidrs = optional(set(string), ["0.0.0.0/0"])
          ipv6_cidrs = optional(set(string), [])
        }), {})
      }), {})
      logging = optional(object({
        retention_in_days = optional(number, 30)
        kms_key_arn       = optional(string, null)
        log_group_class   = optional(string, "STANDARD")
        tags              = optional(map(string), {})
      }), {})
      tags = optional(map(string), {})
    }), {})
  })
| `{}` | no | | [global\_config\_ssm](#input\_global\_config\_ssm) | Global SSM configuration shared by all runner lanes.

global\_config\_ssm = {
paths.root: "Root path for SSM parameters."
paths.app: "Path segment for application parameters."
paths.webhook: "Path segment for webhook parameters."
paths.tokens: "Path segment for runner token parameters."
paths.config: "Path segment for runner configuration parameters."
kms\_key\_id: "KMS key ID used to encrypt SSM parameters."
tags: "Tags applied to SSM resources."
parameters.tags: "Tags applied to runner configuration parameters."
housekeeper.schedule\_expression: "Schedule for the SSM parameter housekeeper."
housekeeper.state: "EventBridge rule state for the SSM parameter housekeeper."
housekeeper.tags: "Tags applied to the SSM housekeeper resources."
housekeeper.lambda.artifact.zip: "Local ZIP artifact used for the SSM housekeeper Lambda."
housekeeper.lambda.artifact.s3.key: "S3 object key for the SSM housekeeper Lambda artifact."
housekeeper.lambda.artifact.s3.object\_version: "Optional S3 object version for the SSM housekeeper artifact."
housekeeper.lambda.memory\_size: "Memory allocated to the SSM housekeeper Lambda."
housekeeper.lambda.timeout: "Timeout in seconds for the SSM housekeeper Lambda."
housekeeper.config.tokenPath: "Parameter path containing runner tokens to clean up."
housekeeper.config.minimumDaysOld: "Minimum age in days before an old token is eligible for cleanup."
housekeeper.config.dryRun: "Whether the SSM housekeeper reports cleanup without deleting parameters."
} | 
object({
    paths = optional(object({
      root    = optional(string, null)
      app     = optional(string, "app")
      webhook = optional(string, "webhook")
      tokens  = optional(string, "runners/tokens")
      config  = optional(string, "runners/config")
    }), {})
    kms_key_id = optional(string, null)
    tags       = optional(map(string), {})
    parameters = optional(object({
      tags = optional(map(string), {})
    }), {})
    housekeeper = optional(object({
      schedule_expression = optional(string, "rate(1 day)")
      state               = optional(string, "ENABLED")
      tags                = optional(map(string), {})
      lambda = optional(object({
        artifact = optional(object({
          zip = optional(string, null)
          s3 = optional(object({
            key            = string
            object_version = optional(string, null)
          }), null)
        }), {})
        memory_size = optional(number, 512)
        timeout     = optional(number, 60)
      }), {})
      config = optional(object({
        tokenPath      = optional(string, null)
        minimumDaysOld = optional(number, 1)
        dryRun         = optional(bool, false)
      }), {})
    }), {})
  })
| `{}` | no | | [iam\_overrides](#input\_iam\_overrides) | This map provides the possibility to override some IAM defaults. The following attributes are supported: `instance_profile_name` overrides the instance profile name used in the launch template. `runner_role_arn` overrides the IAM role ARN used for the runner instances. | 
object({
    override_instance_profile = optional(bool, null)
    instance_profile_name     = optional(string, null)
    override_runner_role      = optional(bool, null)
    runner_role_arn           = optional(string, null)
  })
| 
{
  "instance_profile_name": null,
  "override_instance_profile": false,
  "override_runner_role": false,
  "runner_role_arn": null
}
| no | | [instance\_profile\_path](#input\_instance\_profile\_path) | The path that will be added to the instance\_profile, if not set the environment name will be used. | `string` | `null` | no | @@ -188,7 +189,7 @@ module "multi-runner" { | [logging\_retention\_in\_days](#input\_logging\_retention\_in\_days) | Specifies the number of days you want to retain log events for the lambda log group. Possible values are: 0, 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, and 3653. | `number` | `180` | no | | [matcher\_config\_parameter\_store\_tier](#input\_matcher\_config\_parameter\_store\_tier) | The tier of the parameter store for the matcher configuration. Valid values are `Standard`, and `Advanced`. | `string` | `"Standard"` | no | | [metrics](#input\_metrics) | Configuration for metrics created by the module, by default metrics are disabled to avoid additional costs. When metrics are enable all metrics are created unless explicit configured otherwise. | 
object({
    enable    = optional(bool, false)
    namespace = optional(string, "GitHub Runners")
    metric = optional(object({
      enable_github_app_rate_limit    = optional(bool, true)
      enable_job_retry                = optional(bool, true)
      enable_spot_termination_warning = optional(bool, true)
    }), {})
  })
| `{}` | no | -| [multi\_runner\_config](#input\_multi\_runner\_config) | Accepts either the stable v1 runner configuration shape or the provider-boundary v2 shape. Entries with `runner_config` use the v1 shape; entries without `runner_config` use the v2 shape. A v2 entry does not need matcher configuration. A v2 entry must be acknowledged with `experimental_features = ["multi-runner-v2"]`; the v2 shape is experimental and may change before graduation.

multi\_runner\_config = {
runner\_config: {
runner\_os: "The EC2 Operating System type to use for action runner instances (linux, osx, windows)."
runner\_architecture: "The platform architecture of the runner instance\_type."
runner\_metadata\_options: "(Optional) Metadata options for the ec2 runner instances."
ami: "(Optional) AMI configuration for the action runner instances. This object allows you to specify all AMI-related settings in one place."
create\_service\_linked\_role\_spot: (Optional) create the serviced linked role for spot instances that is required by the scale-up lambda.
credit\_specification: "(Optional) The credit specification of the runner instance\_type. Can be unset, `standard` or `unlimited`.
delay\_webhook\_event: "The number of seconds the event accepted by the webhook is invisible on the queue before the scale up lambda will receive the event."
disable\_runner\_autoupdate: "Disable the auto update of the github runner agent. Be aware there is a grace period of 30 days, see also the [GitHub article](https://github.blog/changelog/2022-02-01-github-actions-self-hosted-runners-can-now-disable-automatic-updates/)"
ebs\_optimized: "The EC2 EBS optimized configuration."
enable\_ephemeral\_runners: "Enable ephemeral runners, runners will only be used once."
enable\_job\_queued\_check: Enables JIT configuration for creating runners instead of registration token based registraton. JIT configuration will only be applied for ephemeral runners. By default JIT configuration is enabled for ephemeral runners an can be disabled via this override. When running on GHES without support for JIT configuration this variable should be set to true for ephemeral runners."
enable\_on\_demand\_failover\_for\_errors: "Enable on-demand failover. For example to fall back to on demand when no spot capacity is available the variable can be set to `InsufficientInstanceCapacity`. When not defined the default behavior is to retry later."
scale\_errors: "List of AWS error codes that should trigger retry during scale up. This list replaces the module default scale-up retry errors"
enable\_organization\_runners: "Register runners to organization, instead of repo level"
enable\_runner\_binaries\_syncer: "Option to disable the lambda to sync GitHub runner distribution, useful when using a pre-build AMI."
enable\_ssm\_on\_runners: "Enable to allow access the runner instances for debugging purposes via SSM. Note that this adds additional permissions to the runner instances."
enable\_userdata: "Should the userdata script be enabled for the runner. Set this to false if you are using your own prebuilt AMI."
instance\_allocation\_strategy: "The allocation strategy for creating instances. For spot, AWS recommends `price-capacity-optimized`; for on-demand, use `lowest-price` or `prioritized`. The AWS default is `lowest-price`."
instance\_type\_priorities: "A map of instance type to priority for the `prioritized` and `capacity-optimized-prioritized` allocation strategies. Lower numbers mean higher priority. If not provided, priorities are assigned based on the order of `instance_types`."
instance\_max\_spot\_price: "Max price price for spot instances per hour. This variable will be passed to the create fleet as max spot price for the fleet."
instance\_target\_capacity\_type: "Default lifecycle used for runner instances, can be either `spot` or `on-demand`."
instance\_types: "List of instance types for the action runner. Defaults are based on runner\_os (al2023 for linux, macOS Sequoia for osx, Windows Server Core for win)."
job\_queue\_retention\_in\_seconds: "The number of seconds the job is held in the queue before it is purged"
minimum\_running\_time\_in\_minutes: "The time an ec2 action runner should be running at minimum before terminated if not busy."
pool\_runner\_owner: "The pool will deploy runners to the GitHub org ID, set this value to the org to which you want the runners deployed. Repo level is not supported."
runner\_additional\_security\_group\_ids: "List of additional security groups IDs to apply to the runner. If added outside the multi\_runner\_config block, the additional security group(s) will be applied to all runner configs. If added inside the multi\_runner\_config, the additional security group(s) will be applied to the individual runner."
runner\_as\_root: "Run the action runner under the root user. Variable `runner_run_as` will be ignored."
runner\_boot\_time\_in\_minutes: "The minimum time for an EC2 runner to boot and register as a runner."
scale\_down\_idle\_confirmation\_seconds: "Number of seconds a runner must consistently report not-busy before scale-down terminates it. GitHub's busy flag can be stale, so a single not-busy reading is not sufficient evidence a runner is idle. 0 keeps the previous single-reading behaviour."
runner\_disable\_default\_labels: "Disable default labels for the runners (os, architecture and `self-hosted`). If enabled, the runner will only have the extra labels provided in `runner_extra_labels`. In case you on own start script is used, this configuration parameter needs to be parsed via SSM."
runner\_extra\_labels: "Extra (custom) labels for the runners (GitHub). Separate each label by a comma. Labels checks on the webhook can be enforced by setting `multi_runner_config.matcherConfig.exactMatch`. GitHub read-only labels should not be provided."
runner\_group\_name: "Name of the runner group."
runner\_name\_prefix: "Prefix for the GitHub runner name."
runner\_run\_as: "Run the GitHub actions agent as user."
runners\_maximum\_count: "The maximum number of runners that will be created. Setting the variable to `-1` disables the maximum check."
scale\_down\_schedule\_expression: "Scheduler expression to check every x for scale down."
scale\_up\_reserved\_concurrent\_executions: "Amount of reserved concurrent executions for the scale-up lambda function. A value of 0 disables lambda from being triggered and -1 removes any concurrency limitations."
lambda\_event\_source\_mapping\_batch\_size: "(Optional) Maximum number of records per Lambda invocation for this runner flavor. Overrides the module-level `lambda_event_source_mapping_batch_size` when set."
lambda\_event\_source\_mapping\_maximum\_batching\_window\_in\_seconds: "(Optional) Maximum seconds to gather records before invoking Lambda for this runner flavor. Overrides the module-level `lambda_event_source_mapping_maximum_batching_window_in_seconds` when set."
userdata\_template: "Alternative user-data template, replacing the default template. By providing your own user\_data you have to take care of installing all required software, including the action runner. Variables userdata\_pre/post\_install are ignored."
enable\_jit\_config: "Overwrite the default behavior for JIT configuration. By default JIT configuration is enabled for ephemeral runners and disabled for non-ephemeral runners. In case of GHES check first if the JIT config API is available. In case you are upgrading from 3.x to 4.x you can set `enable_jit_config` to `false` to avoid a breaking change when having your own AMI."
enable\_runner\_detailed\_monitoring: "Should detailed monitoring be enabled for the runner. Set this to true if you want to use detailed monitoring. See https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-cloudwatch-new.html for details."
enable\_cloudwatch\_agent: "Enabling the cloudwatch agent on the ec2 runner instances, the runner contains default config. Configuration can be overridden via `cloudwatch_config`."
cloudwatch\_config: "(optional) Replaces the module default cloudwatch log config. See https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-Configuration-File-Details.html for details."
userdata\_pre\_install: "Script to be ran before the GitHub Actions runner is installed on the EC2 instances"
userdata\_post\_install: "Script to be ran after the GitHub Actions runner is installed on the EC2 instances"
runner\_hook\_job\_started: "Script to be ran in the runner environment at the beginning of every job"
runner\_hook\_job\_completed: "Script to be ran in the runner environment at the end of every job"
runner\_ec2\_tags: "Map of tags that will be added to the launch template instance tag specifications."
runner\_iam\_role\_managed\_policy\_arns: "Attach AWS or customer-managed IAM policies (by ARN) to the runner IAM role"
vpc\_id: "The VPC for security groups of the action runners. If not set uses the value of `var.vpc_id`."
subnet\_ids: "List of subnets in which the action runners will be launched, the subnets needs to be subnets in the `vpc_id`. If not set, uses the value of `var.subnet_ids`."
idle\_config: "List of time period that can be defined as cron expression to keep a minimum amount of runners active instead of scaling down to 0. By defining this list you can ensure that in time periods that match the cron expression within 5 seconds a runner is kept idle."
license\_specifications: "Optional EC2 License Manager license configuration ARNs for the runner launch template. Required for macOS dedicated-host runners when the host resource group uses a Mac dedicated host license configuration."
use\_dedicated\_host: "Experimental! Can be removed / changed without trigger a major release. Whether to use EC2 dedicated hosts for the runners. Needed for macos runners Note that using dedicated hosts can increase cost significantly."
runner\_log\_files: "(optional) Replaces the module default cloudwatch log config. See https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-Configuration-File-Details.html for details."
block\_device\_mappings: "The EC2 instance block device configuration. Takes the following keys: `device_name`, `delete_on_termination`, `volume_type`, `volume_size`, `encrypted`, `iops`, `throughput`, `kms_key_id`, `snapshot_id`, `volume_initialization_rate`."
job\_retry: "Experimental! Can be removed / changed without trigger a major release. Configure job retries. The configuration enables job retries (for ephemeral runners). After creating the instances a message will be published to a job retry queue. The job retry check lambda is checking after a delay if the job is queued. If not the message will be published again on the scale-up (build queue). Using this feature can impact the rate limit of the GitHub app."
pool\_config: "The configuration for updating the pool. The `pool_size` to adjust to by the events triggered by the `schedule_expression`. For example you can configure a cron expression for week days to adjust the pool to 10 and another expression for the weekend to adjust the pool to 1. Use `schedule_expression_timezone` to override the schedule time zone (defaults to UTC)."
iam\_overrides: "Allows to (optionally) override the instance profile and runner role created by the module. Set `override_instance_profile` to true and provide the `instance_profile_name` to use an existing instance profile. Set `override_runner_role` to true and provide the `runner_role_arn` to use an existing role for the runner instances."
}
# V2 contract
tags: "Tags applied to resources created for this runner configuration."
runner: "Runner settings such as the operating system, architecture, labels, hooks, runner group, name prefix, and IAM role configuration."
lambda: "Lambda settings such as runtime, architecture, networking, tags, and execution-role options for this runner configuration."
# Webhook, queue, and scale-up/scale-down orchestration settings.
orchestration\_provider: {
webhook: {
matcherConfig: "Label matching and dynamic-label policy used to route workflow jobs to this runner configuration."
runner: "Runner lifecycle settings including boot time, ephemeral mode, JIT configuration, and maximum runner count."
queue: "Build queue delay, retention, visibility timeout, redrive, and tags."
}
}
ssm: "SSM parameter paths, tags, and housekeeper settings for runner configuration storage."
observability: "Logging, tracing, and metric settings for the resources in this runner configuration."
# Compute settings for the runner provider.
compute\_provider: {
aws: {
ec2: "AWS EC2 runner settings, including AMI selection, instance types, capacity strategy, VPC and subnet placement, storage, user data, and runner access."
}
}
matcherConfig: {
labelMatchers: "The list of list of labels supported by the runner configuration. `[[self-hosted, linux, x64, example]]`"
exactMatch: "DEPRECATED: Use `bidirectionalLabelMatch` instead. If set to true all labels in the workflow job must match the GitHub labels (os, architecture and `self-hosted`). When false if __any__ workflow label matches it will trigger the webhook. Note: this only checks that workflow labels are a subset of runner labels, not the reverse."
bidirectionalLabelMatch: "If set to true, the runner labels and workflow job labels must be an exact two-way match (same set, any order, no extras or missing labels). This is stricter than `exactMatch` which only checks that workflow labels are a subset of runner labels. When false, if __any__ workflow label matches it will trigger the webhook."
priority: "If set it defines the priority of the matcher, the matcher with the lowest priority will be evaluated first. Default is 999, allowed values 0-999."
enableDynamicLabels: "Experimental! When true the dispatcher allows `ghr-*` dynamic labels for jobs routed to this runner. Default false."
awsDynamicLabelsPolicy: "Optional AWS dynamic label policy evaluated by the dispatcher. Only effective when `enableDynamicLabels = true`. Jobs whose provider dynamic labels violate every matching runner's policy are rejected with a 202 (a warning is logged). Evaluation: keys in `blocked_keys` are always rejected; keys in `restricted_keys` are allowed only when their value passes the rule; unlisted keys are allowed. Schema: `{ blocked_keys = [], restricted_keys = { = { allowed = [globs], denied = [globs], max = number|string } } }`. Keys use the dynamic label suffix, e.g. `instance-type` for `ghr-ec2-instance-type`."
}
redrive\_build\_queue: "Set options to attach (optional) a dead letter queue to the build queue, the queue between the webhook and the scale up lambda. You have the following options. 1. Disable by setting `enabled` to false. 2. Enable by setting `enabled` to `true`, `maxReceiveCount` to a number of max retries."
} | 
map(object({
    # V1 contract
    runner_config = optional(object({
      runner_os           = string
      runner_architecture = string
      runner_metadata_options = optional(map(any), {
        instance_metadata_tags      = "enabled"
        http_endpoint               = "enabled"
        http_tokens                 = "required"
        http_put_response_hop_limit = 1
      })
      ami = optional(object({
        filter               = optional(map(list(string)), { state = ["available"] })
        owners               = optional(list(string), ["amazon"])
        id_ssm_parameter_arn = optional(string, null)
        kms_key_arn          = optional(string, null)
      }), null)
      create_service_linked_role_spot      = optional(bool, false)
      credit_specification                 = optional(string, null)
      delay_webhook_event                  = optional(number, 30)
      disable_runner_autoupdate            = optional(bool, false)
      ebs_optimized                        = optional(bool, false)
      enable_ephemeral_runners             = optional(bool, false)
      enable_job_queued_check              = optional(bool, null)
      enable_on_demand_failover_for_errors = optional(list(string), [])
      scale_errors = optional(list(string), [
        "UnfulfillableCapacity",
        "MaxSpotInstanceCountExceeded",
        "TargetCapacityLimitExceededException",
        "RequestLimitExceeded",
        "ResourceLimitExceeded",
        "MaxSpotInstanceCountExceeded",
        "MaxSpotFleetRequestCountExceeded",
        "InsufficientInstanceCapacity",
        "InsufficientCapacityOnHost",
      ])
      enable_organization_runners                                    = optional(bool, false)
      enable_runner_binaries_syncer                                  = optional(bool, true)
      enable_ssm_on_runners                                          = optional(bool, false)
      enable_userdata                                                = optional(bool, true)
      instance_allocation_strategy                                   = optional(string, "lowest-price")
      instance_type_priorities                                       = optional(map(number), null)
      instance_max_spot_price                                        = optional(string, null)
      instance_target_capacity_type                                  = optional(string, "spot")
      instance_types                                                 = list(string)
      job_queue_retention_in_seconds                                 = optional(number, 86400)
      minimum_running_time_in_minutes                                = optional(number, null)
      pool_runner_owner                                              = optional(string, null)
      runner_as_root                                                 = optional(bool, false)
      runner_boot_time_in_minutes                                    = optional(number, 5)
      scale_down_idle_confirmation_seconds                           = optional(number, 0)
      runner_disable_default_labels                                  = optional(bool, false)
      runner_extra_labels                                            = optional(list(string), [])
      runner_group_name                                              = optional(string, "Default")
      runner_name_prefix                                             = optional(string, "")
      runner_run_as                                                  = optional(string, "ec2-user")
      runners_maximum_count                                          = number
      runner_additional_security_group_ids                           = optional(list(string), [])
      scale_down_schedule_expression                                 = optional(string, "cron(*/5 * * * ? *)")
      scale_up_reserved_concurrent_executions                        = optional(number, 1)
      lambda_event_source_mapping_batch_size                         = optional(number, null)
      lambda_event_source_mapping_maximum_batching_window_in_seconds = optional(number, null)
      userdata_template                                              = optional(string, null)
      userdata_content                                               = optional(string, null)
      enable_jit_config                                              = optional(bool, null)
      enable_runner_detailed_monitoring                              = optional(bool, false)
      enable_cloudwatch_agent                                        = optional(bool, true)
      cloudwatch_config                                              = optional(string, null)
      userdata_pre_install                                           = optional(string, "")
      userdata_post_install                                          = optional(string, "")
      runner_hook_job_started                                        = optional(string, "")
      runner_hook_job_completed                                      = optional(string, "")
      runner_ec2_tags                                                = optional(map(string), {})
      runner_iam_role_managed_policy_arns                            = optional(list(string), [])
      vpc_id                                                         = optional(string, null)
      subnet_ids                                                     = optional(list(string), null)
      idle_config = optional(list(object({
        cron             = string
        timeZone         = string
        idleCount        = number
        evictionStrategy = optional(string, "oldest_first")
      })), [])
      cpu_options = optional(object({
        core_count            = optional(number)
        threads_per_core      = optional(number)
        amd_sev_snp           = optional(string)
        nested_virtualization = optional(string)
      }), null)
      placement = optional(object({
        affinity                = optional(string)
        availability_zone       = optional(string)
        group_id                = optional(string)
        group_name              = optional(string)
        host_id                 = optional(string)
        host_resource_group_arn = optional(string)
        spread_domain           = optional(string)
        tenancy                 = optional(string)
        partition_number        = optional(number)
      }), null)
      license_specifications = optional(list(object({
        license_configuration_arn = string
      })), [])
      use_dedicated_host = optional(bool, false)
      runner_log_files = optional(list(object({
        log_group_name   = string
        prefix_log_group = bool
        file_path        = string
        log_stream_name  = string
        log_class        = optional(string, "STANDARD")
      })), null)
      block_device_mappings = optional(list(object({
        delete_on_termination      = optional(bool, true)
        device_name                = optional(string, "/dev/xvda")
        encrypted                  = optional(bool, true)
        iops                       = optional(number)
        kms_key_id                 = optional(string)
        snapshot_id                = optional(string)
        throughput                 = optional(number)
        volume_initialization_rate = optional(number)
        volume_size                = number
        volume_type                = optional(string, "gp3")
        })), [{
        volume_size = 30
      }])
      pool_config = optional(list(object({
        schedule_expression          = string
        schedule_expression_timezone = optional(string)
        size                         = number
      })), [])
      job_retry = optional(object({
        enable             = optional(bool, false)
        delay_in_seconds   = optional(number, 300)
        delay_backoff      = optional(number, 2)
        lambda_memory_size = optional(number, 256)
        lambda_timeout     = optional(number, 30)
        max_attempts       = optional(number, 1)
      }), {})
      iam_overrides = optional(object({
        override_instance_profile = optional(bool, null)
        instance_profile_name     = optional(string, null)
        override_runner_role      = optional(bool, null)
        runner_role_arn           = optional(string, null)
        }), {
        override_instance_profile = false
        instance_profile_name     = null
        override_runner_role      = false
        runner_role_arn           = null
      })
    }), null)
    matcherConfig = optional(object({
      labelMatchers           = list(list(string))
      exactMatch              = optional(bool, false)
      bidirectionalLabelMatch = optional(bool, false)
      priority                = optional(number, 999)
      enableDynamicLabels     = optional(bool, false)
      awsDynamicLabelsPolicy  = optional(any, null)
    }), null)
    redrive_build_queue = optional(object({
      enabled         = bool
      maxReceiveCount = number
      }), {
      enabled         = false
      maxReceiveCount = null
    })

    # V2 Contract
    tags = optional(map(string), {})

    runner = optional(object({
      os                     = optional(string, null)
      architecture           = optional(string, null)
      disable_default_labels = optional(bool, null)
      extra_labels           = optional(list(string), null)
      group_name             = optional(string, null)
      name_prefix            = optional(string, null)
      run_as_root            = optional(bool, null)
      run_as                 = optional(string, null)
      auto_update_disabled   = optional(bool, null)
      tags                   = optional(map(string), {})
      hooks = optional(object({
        job_started   = optional(string, null)
        job_completed = optional(string, null)
      }), {})
      iam = optional(object({
        role = optional(object({
          arn = string
        }), null)
        managed_policy_arns          = optional(map(string), null)
        additional_trust_policy_json = optional(string, null)
        path                         = optional(string, null)
        permissions_boundary         = optional(string, null)
      }), {})
    }), {})

    lambda = optional(object({
      runtime            = optional(string, null)
      architecture       = optional(string, null)
      subnet_ids         = optional(list(string), null)
      security_group_ids = optional(list(string), null)
      tags               = optional(map(string), {})
      role = optional(object({
        path                 = optional(string, null)
        permissions_boundary = optional(string, null)
      }), {})
    }), {})

    orchestration_provider = optional(object({
      webhook = optional(object({
        runner = optional(object({
          boot_time_in_minutes = optional(number, null)
          ephemeral            = optional(bool, null)
          jit_config_enabled   = optional(bool, null)
          maximum_count        = optional(number, null)
        }), {})
        github = optional(object({
          organization_runners = optional(bool, false)
        }), {})
        matcherConfig = optional(object({
          labelMatchers           = list(list(string))
          exactMatch              = optional(bool, false)
          bidirectionalLabelMatch = optional(bool, false)
          priority                = optional(number, 999)
          dynamic_labels_enabled  = optional(bool, false)
          awsDynamicLabelsPolicy = optional(object({
            blocked_keys = optional(list(string), [])
            restricted_keys = optional(map(object({
              allowed = optional(list(string), [])
              denied  = optional(list(string), [])
              max     = optional(string, null)
            })), {})
          }), null)
        }), null)
        queue = optional(object({
          delay_webhook_event            = optional(number, null)
          job_queue_retention_in_seconds = optional(number, null)
          visibility_timeout_seconds     = optional(number, null)
          redrive_build_queue = optional(object({
            enabled         = optional(bool, null)
            maxReceiveCount = optional(number, null)
          }), null)
          tags = optional(map(string), {})
        }), {})
        lambda = optional(object({
          scale = optional(object({
            up = optional(object({
              memory_size                    = optional(number, null)
              timeout                        = optional(number, null)
              reserved_concurrent_executions = optional(number, null)
              job_queued_check_enabled       = optional(bool, null)
              event_source_mapping = optional(object({
                batch_size                         = optional(number, null)
                maximum_batching_window_in_seconds = optional(number, null)
              }), {})
              tags = optional(map(string), {})
            }), {})
            down = optional(object({
              memory_size                     = optional(number, null)
              timeout                         = optional(number, null)
              schedule_expression             = optional(string, null)
              minimum_running_time_in_minutes = optional(number, null)
              idle_confirmation_seconds       = optional(number, null)
              idle_config = optional(list(object({
                cron             = string
                timeZone         = string
                idleCount        = number
                evictionStrategy = optional(string, "oldest_first")
              })), null)
              tags = optional(map(string), {})
            }), {})
          }), {})
          pool = optional(object({
            memory_size                    = optional(number, null)
            timeout                        = optional(number, null)
            reserved_concurrent_executions = optional(number, null)
            config = optional(list(object({
              schedule_expression          = string
              schedule_expression_timezone = optional(string)
              size                         = number
            })), null)
            include_busy_runners = optional(bool, null)
            runner_owner         = optional(string, null)
            tags                 = optional(map(string), {})
          }), {})
        }), {})
        job_retry = optional(object({
          enabled          = optional(bool, false)
          delay_in_seconds = optional(number, 300)
          delay_backoff    = optional(number, 2)
          max_attempts     = optional(number, 1)
          tags             = optional(map(string), {})
          lambda = optional(object({
            memory_size                    = optional(number, 256)
            reserved_concurrent_executions = optional(number, 1)
            timeout                        = optional(number, 30)
          }), {})
        }), {})
      }), null)
    }), {})

    ssm = optional(object({
      paths = optional(object({
        root   = optional(string, null)
        tokens = optional(string, null)
        config = optional(string, null)
      }), {})
      tags = optional(map(string), {})
      parameters = optional(object({
        tags = optional(map(string), {})
      }), {})
      housekeeper = optional(object({
        schedule_expression = optional(string, null)
        state               = optional(string, null)
        tags                = optional(map(string), {})
        lambda = optional(object({
          artifact = optional(object({
            zip = optional(string, null)
            s3 = optional(object({
              key            = string
              object_version = optional(string, null)
            }), null)
          }), {})
          memory_size = optional(number, null)
          timeout     = optional(number, null)
        }), {})
        config = optional(object({
          tokenPath      = optional(string, null)
          minimumDaysOld = optional(number, null)
          dryRun         = optional(bool, null)
        }), {})
      }), {})
    }), {})

    observability = optional(object({
      logs = optional(object({
        level             = optional(string, null)
        retention_in_days = optional(number, null)
        kms_key_id        = optional(string, null)
        class             = optional(string, null)
        tags              = optional(map(string), {})
      }), {})
      tracing = optional(object({
        mode                  = optional(string, null)
        capture_http_requests = optional(bool, null)
        capture_error         = optional(bool, null)
      }), {})
      metrics = optional(object({
        enabled   = optional(bool, null)
        namespace = optional(string, null)
        metric = optional(object({
          github_app_rate_limit = optional(object({
            enabled = optional(bool, null)
          }), {})
          job_retry = optional(object({
            enabled = optional(bool, null)
          }), {})
          spot_termination_warning = optional(object({
            enabled = optional(bool, null)
          }), {})
        }), {})
      }), {})
    }), {})

    compute_provider = optional(object({
      aws = optional(object({
        ec2 = optional(object({
          metadata_options = optional(object({
            instance_metadata_tags      = optional(string, "enabled")
            http_endpoint               = optional(string, "enabled")
            http_tokens                 = optional(string, "required")
            http_put_response_hop_limit = optional(number, 1)
          }), {})
          ami = optional(object({
            filter = optional(map(list(string)), { state = ["available"] })
            owners = optional(list(string), ["amazon"])
            id_ssm_parameter = optional(object({
              arn = string
            }), null)
            kms_key = optional(object({
              arn = string
            }), null)
          }), null)
          block_device_mappings = optional(list(object({
            delete_on_termination      = optional(bool, true)
            device_name                = optional(string, "/dev/xvda")
            encrypted                  = optional(bool, true)
            iops                       = optional(number)
            kms_key_id                 = optional(string)
            snapshot_id                = optional(string)
            throughput                 = optional(number)
            volume_initialization_rate = optional(number)
            volume_size                = number
            volume_type                = optional(string, "gp3")
          })), [{ volume_size = 30 }])
          create_service_linked_role_spot = optional(bool, false)
          credit_specification            = optional(string, null)
          ebs_optimized                   = optional(bool, false)
          cloudwatch_agent = optional(object({
            enabled = optional(bool, true)
            config  = optional(string, null)
          }), {})
          binaries_syncer = optional(object({
            enabled = optional(bool, null)
          }), {})
          detailed_monitoring_enabled = optional(bool, false)
          ssm_enabled                 = optional(bool, false)
          user_data = optional(object({
            enabled               = optional(bool, true)
            template              = optional(string, null)
            content               = optional(string, null)
            pre_install           = optional(string, "")
            post_install          = optional(string, "")
            debug_logging_enabled = optional(bool, false)
          }), {})
          instance_allocation_strategy   = optional(string, "lowest-price")
          instance_max_spot_price        = optional(string, null)
          instance_target_capacity_type  = optional(string, "spot")
          instance_type_priorities       = optional(map(number), null)
          instance_types                 = optional(list(string), [])
          additional_security_group_ids  = optional(list(string), null)
          managed_security_group_enabled = optional(bool, null)
          egress_rules = optional(list(object({
            cidr_blocks      = list(string)
            ipv6_cidr_blocks = list(string)
            prefix_list_ids  = list(string)
            from_port        = number
            protocol         = string
            security_groups  = list(string)
            self             = bool
            to_port          = number
            description      = string
          })), null)
          instance_profile_path         = optional(string, null)
          key_name                      = optional(string, null)
          associate_public_ipv4_address = optional(bool, null)
          instance_profile = optional(object({
            name = string
          }), null)
          on_demand_failover_for_errors = optional(list(string), [])
          scale_errors = optional(list(string), [
            "UnfulfillableCapacity",
            "MaxSpotInstanceCountExceeded",
            "TargetCapacityLimitExceededException",
            "RequestLimitExceeded",
            "ResourceLimitExceeded",
            "MaxSpotInstanceCountExceeded",
            "MaxSpotFleetRequestCountExceeded",
            "InsufficientInstanceCapacity",
            "InsufficientCapacityOnHost",
          ])
          subnet_ids = optional(list(string), null)
          vpc_id     = optional(string, null)
          cpu_options = optional(object({
            core_count            = optional(number)
            threads_per_core      = optional(number)
            amd_sev_snp           = optional(string)
            nested_virtualization = optional(string)
          }), null)
          placement = optional(object({
            affinity                = optional(string)
            availability_zone       = optional(string)
            group_id                = optional(string)
            group_name              = optional(string)
            host_id                 = optional(string)
            host_resource_group_arn = optional(string)
            spread_domain           = optional(string)
            tenancy                 = optional(string)
            partition_number        = optional(number)
          }), null)
          license_specifications = optional(list(object({
            license_configuration_arn = string
          })), [])
          use_dedicated_host = optional(bool, false)
          log_files = optional(list(object({
            log_group_name   = string
            prefix_log_group = bool
            file_path        = string
            log_stream_name  = string
            log_class        = optional(string, "STANDARD")
          })), null)
          tags = optional(map(string), {})
        }), null)
      }), {})
    }), {})
  }))
| `{}` | no | +| [multi\_runner\_config](#input\_multi\_runner\_config) | Accepts either the stable v1 runner configuration shape or the provider-boundary v2 shape. Entries with `runner_config` use the v1 shape; entries without `runner_config` use the v2 shape. A v2 entry does not need matcher configuration. A v2 entry must be acknowledged with `experimental_features = ["multi-runner-v2"]`; the v2 shape is experimental and may change before graduation.

multi\_runner\_config = {
runner\_config: {
runner\_os: "The EC2 Operating System type to use for action runner instances (linux, osx, windows)."
runner\_architecture: "The platform architecture of the runner instance\_type."
runner\_metadata\_options: "(Optional) Metadata options for the ec2 runner instances."
ami: "(Optional) AMI configuration for the action runner instances. This object allows you to specify all AMI-related settings in one place."
create\_service\_linked\_role\_spot: (Optional) create the serviced linked role for spot instances that is required by the scale-up lambda.
credit\_specification: "(Optional) The credit specification of the runner instance\_type. Can be unset, `standard` or `unlimited`.
delay\_webhook\_event: "The number of seconds the event accepted by the webhook is invisible on the queue before the scale up lambda will receive the event."
disable\_runner\_autoupdate: "Disable the auto update of the github runner agent. Be aware there is a grace period of 30 days, see also the [GitHub article](https://github.blog/changelog/2022-02-01-github-actions-self-hosted-runners-can-now-disable-automatic-updates/)"
ebs\_optimized: "The EC2 EBS optimized configuration."
enable\_ephemeral\_runners: "Enable ephemeral runners, runners will only be used once."
enable\_job\_queued\_check: Enables JIT configuration for creating runners instead of registration token based registraton. JIT configuration will only be applied for ephemeral runners. By default JIT configuration is enabled for ephemeral runners an can be disabled via this override. When running on GHES without support for JIT configuration this variable should be set to true for ephemeral runners."
enable\_on\_demand\_failover\_for\_errors: "Enable on-demand failover. For example to fall back to on demand when no spot capacity is available the variable can be set to `InsufficientInstanceCapacity`. When not defined the default behavior is to retry later."
scale\_errors: "List of AWS error codes that should trigger retry during scale up. This list replaces the module default scale-up retry errors"
enable\_organization\_runners: "Register runners to organization, instead of repo level"
enable\_runner\_binaries\_syncer: "Option to disable the lambda to sync GitHub runner distribution, useful when using a pre-build AMI."
enable\_ssm\_on\_runners: "Enable to allow access the runner instances for debugging purposes via SSM. Note that this adds additional permissions to the runner instances."
enable\_userdata: "Should the userdata script be enabled for the runner. Set this to false if you are using your own prebuilt AMI."
instance\_allocation\_strategy: "The allocation strategy for creating instances. For spot, AWS recommends `price-capacity-optimized`; for on-demand, use `lowest-price` or `prioritized`. The AWS default is `lowest-price`."
instance\_type\_priorities: "A map of instance type to priority for the `prioritized` and `capacity-optimized-prioritized` allocation strategies. Lower numbers mean higher priority. If not provided, priorities are assigned based on the order of `instance_types`."
instance\_max\_spot\_price: "Max price price for spot instances per hour. This variable will be passed to the create fleet as max spot price for the fleet."
instance\_target\_capacity\_type: "Default lifecycle used for runner instances, can be either `spot` or `on-demand`."
instance\_types: "List of instance types for the action runner. Defaults are based on runner\_os (al2023 for linux, macOS Sequoia for osx, Windows Server Core for win)."
job\_queue\_retention\_in\_seconds: "The number of seconds the job is held in the queue before it is purged"
minimum\_running\_time\_in\_minutes: "The time an ec2 action runner should be running at minimum before terminated if not busy."
pool\_runner\_owner: "The pool will deploy runners to the GitHub org ID, set this value to the org to which you want the runners deployed. Repo level is not supported."
runner\_additional\_security\_group\_ids: "List of additional security groups IDs to apply to the runner. If added outside the multi\_runner\_config block, the additional security group(s) will be applied to all runner configs. If added inside the multi\_runner\_config, the additional security group(s) will be applied to the individual runner."
runner\_as\_root: "Run the action runner under the root user. Variable `runner_run_as` will be ignored."
runner\_boot\_time\_in\_minutes: "The minimum time for an EC2 runner to boot and register as a runner."
scale\_down\_idle\_confirmation\_seconds: "Number of seconds a runner must consistently report not-busy before scale-down terminates it. GitHub's busy flag can be stale, so a single not-busy reading is not sufficient evidence a runner is idle. 0 keeps the previous single-reading behaviour."
runner\_disable\_default\_labels: "Disable default labels for the runners (os, architecture and `self-hosted`). If enabled, the runner will only have the extra labels provided in `runner_extra_labels`. In case you on own start script is used, this configuration parameter needs to be parsed via SSM."
runner\_extra\_labels: "Extra (custom) labels for the runners (GitHub). Separate each label by a comma. Labels checks on the webhook can be enforced by setting `multi_runner_config.matcherConfig.exactMatch`. GitHub read-only labels should not be provided."
runner\_group\_name: "Name of the runner group."
runner\_name\_prefix: "Prefix for the GitHub runner name."
runner\_run\_as: "Run the GitHub actions agent as user."
runners\_maximum\_count: "The maximum number of runners that will be created. Setting the variable to `-1` disables the maximum check."
scale\_down\_schedule\_expression: "Scheduler expression to check every x for scale down."
scale\_up\_reserved\_concurrent\_executions: "Amount of reserved concurrent executions for the scale-up lambda function. A value of 0 disables lambda from being triggered and -1 removes any concurrency limitations."
lambda\_event\_source\_mapping\_batch\_size: "(Optional) Maximum number of records per Lambda invocation for this runner flavor. Overrides the module-level `lambda_event_source_mapping_batch_size` when set."
lambda\_event\_source\_mapping\_maximum\_batching\_window\_in\_seconds: "(Optional) Maximum seconds to gather records before invoking Lambda for this runner flavor. Overrides the module-level `lambda_event_source_mapping_maximum_batching_window_in_seconds` when set."
userdata\_template: "Alternative user-data template, replacing the default template. By providing your own user\_data you have to take care of installing all required software, including the action runner. Variables userdata\_pre/post\_install are ignored."
enable\_jit\_config: "Overwrite the default behavior for JIT configuration. By default JIT configuration is enabled for ephemeral runners and disabled for non-ephemeral runners. In case of GHES check first if the JIT config API is available. In case you are upgrading from 3.x to 4.x you can set `enable_jit_config` to `false` to avoid a breaking change when having your own AMI."
enable\_runner\_detailed\_monitoring: "Should detailed monitoring be enabled for the runner. Set this to true if you want to use detailed monitoring. See https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-cloudwatch-new.html for details."
enable\_cloudwatch\_agent: "Enabling the cloudwatch agent on the ec2 runner instances, the runner contains default config. Configuration can be overridden via `cloudwatch_config`."
cloudwatch\_config: "(optional) Replaces the module default cloudwatch log config. See https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-Configuration-File-Details.html for details."
userdata\_pre\_install: "Script to be ran before the GitHub Actions runner is installed on the EC2 instances"
userdata\_post\_install: "Script to be ran after the GitHub Actions runner is installed on the EC2 instances"
runner\_hook\_job\_started: "Script to be ran in the runner environment at the beginning of every job"
runner\_hook\_job\_completed: "Script to be ran in the runner environment at the end of every job"
runner\_ec2\_tags: "Map of tags that will be added to the launch template instance tag specifications."
runner\_iam\_role\_managed\_policy\_arns: "Attach AWS or customer-managed IAM policies (by ARN) to the runner IAM role"
vpc\_id: "The VPC for security groups of the action runners. If not set uses the value of `var.vpc_id`."
subnet\_ids: "List of subnets in which the action runners will be launched, the subnets needs to be subnets in the `vpc_id`. If not set, uses the value of `var.subnet_ids`."
idle\_config: "List of time period that can be defined as cron expression to keep a minimum amount of runners active instead of scaling down to 0. By defining this list you can ensure that in time periods that match the cron expression within 5 seconds a runner is kept idle."
license\_specifications: "Optional EC2 License Manager license configuration ARNs for the runner launch template. Required for macOS dedicated-host runners when the host resource group uses a Mac dedicated host license configuration."
use\_dedicated\_host: "Experimental! Can be removed / changed without trigger a major release. Whether to use EC2 dedicated hosts for the runners. Needed for macos runners Note that using dedicated hosts can increase cost significantly."
runner\_log\_files: "(optional) Replaces the module default cloudwatch log config. See https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-Configuration-File-Details.html for details."
block\_device\_mappings: "The EC2 instance block device configuration. Takes the following keys: `device_name`, `delete_on_termination`, `volume_type`, `volume_size`, `encrypted`, `iops`, `throughput`, `kms_key_id`, `snapshot_id`, `volume_initialization_rate`."
job\_retry: "Experimental! Can be removed / changed without trigger a major release. Configure job retries. The configuration enables job retries (for ephemeral runners). After creating the instances a message will be published to a job retry queue. The job retry check lambda is checking after a delay if the job is queued. If not the message will be published again on the scale-up (build queue). Using this feature can impact the rate limit of the GitHub app."
pool\_config: "The configuration for updating the pool. The `pool_size` to adjust to by the events triggered by the `schedule_expression`. For example you can configure a cron expression for week days to adjust the pool to 10 and another expression for the weekend to adjust the pool to 1. Use `schedule_expression_timezone` to override the schedule time zone (defaults to UTC)."
iam\_overrides: "Allows to (optionally) override the instance profile and runner role created by the module. Set `override_instance_profile` to true and provide the `instance_profile_name` to use an existing instance profile. Set `override_runner_role` to true and provide the `runner_role_arn` to use an existing role for the runner instances."
}
# V2 contract
tags: "Tags applied to resources created for this runner configuration."
runner: "Runner settings such as the operating system, architecture, labels, hooks, runner group, name prefix, and IAM role configuration."
lambda: "Lambda settings such as runtime, architecture, networking, tags, and execution-role options for this runner configuration."
# Webhook, queue, and scale-up/scale-down orchestration settings.
orchestration\_provider: {
webhook: {
matcherConfig: "Label matching and dynamic-label policy used to route workflow jobs to this runner configuration."
runner: "Runner lifecycle settings including boot time, ephemeral mode, JIT configuration, and maximum runner count."
queue: "Build queue delay, retention, visibility timeout, redrive, and tags."
}
}
ssm: "SSM parameter paths, tags, and housekeeper settings for runner configuration storage."
observability: "Logging, tracing, and metric settings for the resources in this runner configuration."
# Compute settings for the runner provider.
compute\_provider: {
aws: {
ec2: "AWS EC2 runner settings, including AMI selection, instance types, capacity strategy, VPC and subnet placement, storage, user data, and runner access."
}
}
matcherConfig: {
labelMatchers: "The list of list of labels supported by the runner configuration. `[[self-hosted, linux, x64, example]]`"
exactMatch: "DEPRECATED: Use `bidirectionalLabelMatch` instead. If set to true all labels in the workflow job must match the GitHub labels (os, architecture and `self-hosted`). When false if __any__ workflow label matches it will trigger the webhook. Note: this only checks that workflow labels are a subset of runner labels, not the reverse."
bidirectionalLabelMatch: "If set to true, the runner labels and workflow job labels must be an exact two-way match (same set, any order, no extras or missing labels). This is stricter than `exactMatch` which only checks that workflow labels are a subset of runner labels. When false, if __any__ workflow label matches it will trigger the webhook."
priority: "If set it defines the priority of the matcher, the matcher with the lowest priority will be evaluated first. Default is 999, allowed values 0-999."
enableDynamicLabels: "Experimental! When true the dispatcher allows `ghr-*` dynamic labels for jobs routed to this runner. Default false."
awsDynamicLabelsPolicy: "Optional AWS dynamic label policy evaluated by the dispatcher. Only effective when `enableDynamicLabels = true`. Jobs whose provider dynamic labels violate every matching runner's policy are rejected with a 202 (a warning is logged). Evaluation: keys in `blocked_keys` are always rejected; keys in `restricted_keys` are allowed only when their value passes the rule; unlisted keys are allowed. Schema: `{ blocked_keys = [], restricted_keys = { = { allowed = [globs], denied = [globs], max = number|string } } }`. Keys use the dynamic label suffix, e.g. `instance-type` for `ghr-ec2-instance-type`."
}
redrive\_build\_queue: "Set options to attach (optional) a dead letter queue to the build queue, the queue between the webhook and the scale up lambda. You have the following options. 1. Disable by setting `enabled` to false. 2. Enable by setting `enabled` to `true`, `maxReceiveCount` to a number of max retries."
} | 
map(object({
    # V1 contract
    runner_config = optional(object({
      runner_os           = string
      runner_architecture = string
      runner_metadata_options = optional(map(any), {
        instance_metadata_tags      = "enabled"
        http_endpoint               = "enabled"
        http_tokens                 = "required"
        http_put_response_hop_limit = 1
      })
      ami = optional(object({
        filter               = optional(map(list(string)), { state = ["available"] })
        owners               = optional(list(string), ["amazon"])
        id_ssm_parameter_arn = optional(string, null)
        kms_key_arn          = optional(string, null)
      }), null)
      create_service_linked_role_spot      = optional(bool, false)
      credit_specification                 = optional(string, null)
      delay_webhook_event                  = optional(number, 30)
      disable_runner_autoupdate            = optional(bool, false)
      ebs_optimized                        = optional(bool, false)
      enable_ephemeral_runners             = optional(bool, false)
      enable_job_queued_check              = optional(bool, null)
      enable_on_demand_failover_for_errors = optional(list(string), [])
      scale_errors = optional(list(string), [
        "UnfulfillableCapacity",
        "MaxSpotInstanceCountExceeded",
        "TargetCapacityLimitExceededException",
        "RequestLimitExceeded",
        "ResourceLimitExceeded",
        "MaxSpotInstanceCountExceeded",
        "MaxSpotFleetRequestCountExceeded",
        "InsufficientInstanceCapacity",
        "InsufficientCapacityOnHost",
      ])
      enable_organization_runners                                    = optional(bool, false)
      enable_runner_binaries_syncer                                  = optional(bool, true)
      enable_ssm_on_runners                                          = optional(bool, false)
      enable_userdata                                                = optional(bool, true)
      instance_allocation_strategy                                   = optional(string, "lowest-price")
      instance_type_priorities                                       = optional(map(number), null)
      instance_max_spot_price                                        = optional(string, null)
      instance_target_capacity_type                                  = optional(string, "spot")
      instance_types                                                 = list(string)
      job_queue_retention_in_seconds                                 = optional(number, 86400)
      minimum_running_time_in_minutes                                = optional(number, null)
      pool_runner_owner                                              = optional(string, null)
      runner_as_root                                                 = optional(bool, false)
      runner_boot_time_in_minutes                                    = optional(number, 5)
      scale_down_idle_confirmation_seconds                           = optional(number, 0)
      runner_disable_default_labels                                  = optional(bool, false)
      runner_extra_labels                                            = optional(list(string), [])
      runner_group_name                                              = optional(string, "Default")
      runner_name_prefix                                             = optional(string, "")
      runner_run_as                                                  = optional(string, "ec2-user")
      runners_maximum_count                                          = number
      runner_additional_security_group_ids                           = optional(list(string), [])
      scale_down_schedule_expression                                 = optional(string, "cron(*/5 * * * ? *)")
      scale_up_reserved_concurrent_executions                        = optional(number, 1)
      lambda_event_source_mapping_batch_size                         = optional(number, null)
      lambda_event_source_mapping_maximum_batching_window_in_seconds = optional(number, null)
      userdata_template                                              = optional(string, null)
      userdata_content                                               = optional(string, null)
      enable_jit_config                                              = optional(bool, null)
      enable_runner_detailed_monitoring                              = optional(bool, false)
      enable_cloudwatch_agent                                        = optional(bool, true)
      cloudwatch_config                                              = optional(string, null)
      userdata_pre_install                                           = optional(string, "")
      userdata_post_install                                          = optional(string, "")
      runner_hook_job_started                                        = optional(string, "")
      runner_hook_job_completed                                      = optional(string, "")
      runner_ec2_tags                                                = optional(map(string), {})
      runner_iam_role_managed_policy_arns                            = optional(list(string), [])
      vpc_id                                                         = optional(string, null)
      subnet_ids                                                     = optional(list(string), null)
      idle_config = optional(list(object({
        cron             = string
        timeZone         = string
        idleCount        = number
        evictionStrategy = optional(string, "oldest_first")
      })), [])
      cpu_options = optional(object({
        core_count            = optional(number)
        threads_per_core      = optional(number)
        amd_sev_snp           = optional(string)
        nested_virtualization = optional(string)
      }), null)
      placement = optional(object({
        affinity                = optional(string)
        availability_zone       = optional(string)
        group_id                = optional(string)
        group_name              = optional(string)
        host_id                 = optional(string)
        host_resource_group_arn = optional(string)
        spread_domain           = optional(string)
        tenancy                 = optional(string)
        partition_number        = optional(number)
      }), null)
      license_specifications = optional(list(object({
        license_configuration_arn = string
      })), [])
      use_dedicated_host = optional(bool, false)
      runner_log_files = optional(list(object({
        log_group_name   = string
        prefix_log_group = bool
        file_path        = string
        log_stream_name  = string
        log_class        = optional(string, "STANDARD")
      })), null)
      block_device_mappings = optional(list(object({
        delete_on_termination      = optional(bool, true)
        device_name                = optional(string, "/dev/xvda")
        encrypted                  = optional(bool, true)
        iops                       = optional(number)
        kms_key_id                 = optional(string)
        snapshot_id                = optional(string)
        throughput                 = optional(number)
        volume_initialization_rate = optional(number)
        volume_size                = number
        volume_type                = optional(string, "gp3")
        })), [{
        volume_size = 30
      }])
      pool_config = optional(list(object({
        schedule_expression          = string
        schedule_expression_timezone = optional(string)
        size                         = number
      })), [])
      job_retry = optional(object({
        enable             = optional(bool, false)
        delay_in_seconds   = optional(number, 300)
        delay_backoff      = optional(number, 2)
        lambda_memory_size = optional(number, 256)
        lambda_timeout     = optional(number, 30)
        max_attempts       = optional(number, 1)
      }), {})
      iam_overrides = optional(object({
        override_instance_profile = optional(bool, null)
        instance_profile_name     = optional(string, null)
        override_runner_role      = optional(bool, null)
        runner_role_arn           = optional(string, null)
        }), {
        override_instance_profile = false
        instance_profile_name     = null
        override_runner_role      = false
        runner_role_arn           = null
      })
    }), null)
    matcherConfig = optional(object({
      labelMatchers           = list(list(string))
      exactMatch              = optional(bool, false)
      bidirectionalLabelMatch = optional(bool, false)
      priority                = optional(number, 999)
      enableDynamicLabels     = optional(bool, false)
      awsDynamicLabelsPolicy  = optional(any, null)
    }), null)
    redrive_build_queue = optional(object({
      enabled         = bool
      maxReceiveCount = number
      }), {
      enabled         = false
      maxReceiveCount = null
    })

    # V2 Contract
    tags = optional(map(string), {})

    runner = optional(object({
      os                     = optional(string, null)
      architecture           = optional(string, null)
      disable_default_labels = optional(bool, null)
      extra_labels           = optional(list(string), null)
      group_name             = optional(string, null)
      name_prefix            = optional(string, null)
      run_as_root            = optional(bool, null)
      run_as                 = optional(string, null)
      auto_update_disabled   = optional(bool, null)
      tags                   = optional(map(string), {})
      hooks = optional(object({
        job_started   = optional(string, null)
        job_completed = optional(string, null)
      }), {})
      iam = optional(object({
        role = optional(object({
          arn = string
        }), null)
        managed_policy_arns          = optional(map(string), null)
        additional_trust_policy_json = optional(string, null)
        path                         = optional(string, null)
        permissions_boundary         = optional(string, null)
      }), {})
    }), {})

    lambda = optional(object({
      runtime            = optional(string, null)
      architecture       = optional(string, null)
      subnet_ids         = optional(list(string), null)
      security_group_ids = optional(list(string), null)
      tags               = optional(map(string), {})
      role = optional(object({
        path                 = optional(string, null)
        permissions_boundary = optional(string, null)
      }), {})
    }), {})

    orchestration_provider = optional(object({
      webhook = optional(object({
        runner = optional(object({
          boot_time_in_minutes = optional(number, null)
          ephemeral            = optional(bool, null)
          jit_config_enabled   = optional(bool, null)
          maximum_count        = optional(number, null)
        }), {})
        github = optional(object({
          organization_runners = optional(bool, false)
        }), {})
        matcherConfig = optional(object({
          labelMatchers           = list(list(string))
          exactMatch              = optional(bool, false)
          bidirectionalLabelMatch = optional(bool, false)
          priority                = optional(number, 999)
          dynamic_labels_enabled  = optional(bool, false)
          awsDynamicLabelsPolicy = optional(object({
            blocked_keys = optional(list(string), [])
            restricted_keys = optional(map(object({
              allowed = optional(list(string), [])
              denied  = optional(list(string), [])
              max     = optional(string, null)
            })), {})
          }), null)
        }), null)
        queue = optional(object({
          delay_webhook_event            = optional(number, null)
          job_queue_retention_in_seconds = optional(number, null)
          visibility_timeout_seconds     = optional(number, null)
          redrive_build_queue = optional(object({
            enabled         = optional(bool, null)
            maxReceiveCount = optional(number, null)
          }), null)
          tags = optional(map(string), {})
        }), {})
        lambda = optional(object({
          scale = optional(object({
            up = optional(object({
              memory_size                    = optional(number, null)
              timeout                        = optional(number, null)
              reserved_concurrent_executions = optional(number, null)
              job_queued_check_enabled       = optional(bool, null)
              event_source_mapping = optional(object({
                batch_size                         = optional(number, null)
                maximum_batching_window_in_seconds = optional(number, null)
              }), {})
              tags = optional(map(string), {})
            }), {})
            down = optional(object({
              memory_size                     = optional(number, null)
              timeout                         = optional(number, null)
              schedule_expression             = optional(string, null)
              minimum_running_time_in_minutes = optional(number, null)
              idle_confirmation_seconds       = optional(number, null)
              idle_config = optional(list(object({
                cron             = string
                timeZone         = string
                idleCount        = number
                evictionStrategy = optional(string, "oldest_first")
              })), null)
              tags = optional(map(string), {})
            }), {})
          }), {})
          pool = optional(object({
            memory_size                    = optional(number, null)
            timeout                        = optional(number, null)
            reserved_concurrent_executions = optional(number, null)
            config = optional(list(object({
              schedule_expression          = string
              schedule_expression_timezone = optional(string)
              size                         = number
            })), null)
            include_busy_runners = optional(bool, null)
            runner_owner         = optional(string, null)
            tags                 = optional(map(string), {})
          }), {})
        }), {})
        job_retry = optional(object({
          enabled          = optional(bool, false)
          delay_in_seconds = optional(number, 300)
          delay_backoff    = optional(number, 2)
          max_attempts     = optional(number, 1)
          tags             = optional(map(string), {})
          lambda = optional(object({
            memory_size                    = optional(number, 256)
            reserved_concurrent_executions = optional(number, 1)
            timeout                        = optional(number, 30)
          }), {})
        }), {})
      }), null)
      scale_set = optional(object({
        name = string
        runner = optional(object({
          min_runners          = optional(number, 0)
          max_runners          = optional(number, 10)
          boot_time_in_minutes = optional(number, 10)
        }), {})
      }), null)
    }), {})

    ssm = optional(object({
      paths = optional(object({
        root   = optional(string, null)
        tokens = optional(string, null)
        config = optional(string, null)
      }), {})
      tags = optional(map(string), {})
      parameters = optional(object({
        tags = optional(map(string), {})
      }), {})
      housekeeper = optional(object({
        schedule_expression = optional(string, null)
        state               = optional(string, null)
        tags                = optional(map(string), {})
        lambda = optional(object({
          artifact = optional(object({
            zip = optional(string, null)
            s3 = optional(object({
              key            = string
              object_version = optional(string, null)
            }), null)
          }), {})
          memory_size = optional(number, null)
          timeout     = optional(number, null)
        }), {})
        config = optional(object({
          tokenPath      = optional(string, null)
          minimumDaysOld = optional(number, null)
          dryRun         = optional(bool, null)
        }), {})
      }), {})
    }), {})

    observability = optional(object({
      logs = optional(object({
        level             = optional(string, null)
        retention_in_days = optional(number, null)
        kms_key_id        = optional(string, null)
        class             = optional(string, null)
        tags              = optional(map(string), {})
      }), {})
      tracing = optional(object({
        mode                  = optional(string, null)
        capture_http_requests = optional(bool, null)
        capture_error         = optional(bool, null)
      }), {})
      metrics = optional(object({
        enabled   = optional(bool, null)
        namespace = optional(string, null)
        metric = optional(object({
          github_app_rate_limit = optional(object({
            enabled = optional(bool, null)
          }), {})
          job_retry = optional(object({
            enabled = optional(bool, null)
          }), {})
          spot_termination_warning = optional(object({
            enabled = optional(bool, null)
          }), {})
        }), {})
      }), {})
    }), {})

    compute_provider = optional(object({
      aws = optional(object({
        ec2 = optional(object({
          metadata_options = optional(object({
            instance_metadata_tags      = optional(string, "enabled")
            http_endpoint               = optional(string, "enabled")
            http_tokens                 = optional(string, "required")
            http_put_response_hop_limit = optional(number, 1)
          }), {})
          ami = optional(object({
            filter = optional(map(list(string)), { state = ["available"] })
            owners = optional(list(string), ["amazon"])
            id_ssm_parameter = optional(object({
              arn = string
            }), null)
            kms_key = optional(object({
              arn = string
            }), null)
          }), null)
          block_device_mappings = optional(list(object({
            delete_on_termination      = optional(bool, true)
            device_name                = optional(string, "/dev/xvda")
            encrypted                  = optional(bool, true)
            iops                       = optional(number)
            kms_key_id                 = optional(string)
            snapshot_id                = optional(string)
            throughput                 = optional(number)
            volume_initialization_rate = optional(number)
            volume_size                = number
            volume_type                = optional(string, "gp3")
          })), [{ volume_size = 30 }])
          create_service_linked_role_spot = optional(bool, false)
          credit_specification            = optional(string, null)
          ebs_optimized                   = optional(bool, false)
          cloudwatch_agent = optional(object({
            enabled = optional(bool, true)
            config  = optional(string, null)
          }), {})
          binaries_syncer = optional(object({
            enabled = optional(bool, null)
          }), {})
          detailed_monitoring_enabled = optional(bool, false)
          ssm_enabled                 = optional(bool, false)
          user_data = optional(object({
            enabled               = optional(bool, true)
            template              = optional(string, null)
            content               = optional(string, null)
            pre_install           = optional(string, "")
            post_install          = optional(string, "")
            debug_logging_enabled = optional(bool, false)
          }), {})
          instance_allocation_strategy   = optional(string, "lowest-price")
          instance_max_spot_price        = optional(string, null)
          instance_target_capacity_type  = optional(string, "spot")
          instance_type_priorities       = optional(map(number), null)
          instance_types                 = optional(list(string), [])
          additional_security_group_ids  = optional(list(string), null)
          managed_security_group_enabled = optional(bool, null)
          egress_rules = optional(list(object({
            cidr_blocks      = list(string)
            ipv6_cidr_blocks = list(string)
            prefix_list_ids  = list(string)
            from_port        = number
            protocol         = string
            security_groups  = list(string)
            self             = bool
            to_port          = number
            description      = string
          })), null)
          instance_profile_path         = optional(string, null)
          key_name                      = optional(string, null)
          associate_public_ipv4_address = optional(bool, null)
          instance_profile = optional(object({
            name = string
          }), null)
          on_demand_failover_for_errors = optional(list(string), [])
          scale_errors = optional(list(string), [
            "UnfulfillableCapacity",
            "MaxSpotInstanceCountExceeded",
            "TargetCapacityLimitExceededException",
            "RequestLimitExceeded",
            "ResourceLimitExceeded",
            "MaxSpotInstanceCountExceeded",
            "MaxSpotFleetRequestCountExceeded",
            "InsufficientInstanceCapacity",
            "InsufficientCapacityOnHost",
          ])
          subnet_ids = optional(list(string), null)
          vpc_id     = optional(string, null)
          cpu_options = optional(object({
            core_count            = optional(number)
            threads_per_core      = optional(number)
            amd_sev_snp           = optional(string)
            nested_virtualization = optional(string)
          }), null)
          placement = optional(object({
            affinity                = optional(string)
            availability_zone       = optional(string)
            group_id                = optional(string)
            group_name              = optional(string)
            host_id                 = optional(string)
            host_resource_group_arn = optional(string)
            spread_domain           = optional(string)
            tenancy                 = optional(string)
            partition_number        = optional(number)
          }), null)
          license_specifications = optional(list(object({
            license_configuration_arn = string
          })), [])
          use_dedicated_host = optional(bool, false)
          log_files = optional(list(object({
            log_group_name   = string
            prefix_log_group = bool
            file_path        = string
            log_stream_name  = string
            log_class        = optional(string, "STANDARD")
          })), null)
          tags = optional(map(string), {})
        }), null)
      }), {})
    }), {})
  }))
| `{}` | no | | [parameter\_store\_tags](#input\_parameter\_store\_tags) | Map of tags that will be added to all the SSM Parameter Store parameters created by the Lambda function. | `map(string)` | `{}` | no | | [pool\_lambda\_reserved\_concurrent\_executions](#input\_pool\_lambda\_reserved\_concurrent\_executions) | Amount of reserved concurrent executions for the scale-up lambda function. A value of 0 disables lambda from being triggered and -1 removes any concurrency limitations. | `number` | `1` | no | | [pool\_lambda\_timeout](#input\_pool\_lambda\_timeout) | Time out for the pool lambda in seconds. | `number` | `60` | no | @@ -239,6 +240,7 @@ module "multi-runner" { | [instance\_termination\_watcher](#output\_instance\_termination\_watcher) | n/a | | [runners\_map](#output\_runners\_map) | n/a | | [runners\_map\_v2](#output\_runners\_map\_v2) | n/a | +| [scale\_set](#output\_scale\_set) | Shared scale-set orchestration resources, or null when no runner configuration selects scale\_set. | | [ssm\_parameters](#output\_ssm\_parameters) | n/a | | [webhook](#output\_webhook) | n/a | diff --git a/modules/multi-runner/config.experimental.effective.tf b/modules/multi-runner/config.experimental.effective.tf index 1a53a16520..4e54b5070b 100644 --- a/modules/multi-runner/config.experimental.effective.tf +++ b/modules/multi-runner/config.experimental.effective.tf @@ -16,8 +16,10 @@ locals { }) github = { - enterprise_server = local.normalized_config.github.enterprise_server - user_agent = local.normalized_config.github.user_agent + enterprise_server = local.normalized_config.github.enterprise_server + runner_owner = local.normalized_config.github.runner_owner + runner_registration_level = local.normalized_config.github.runner_registration_level + user_agent = local.normalized_config.github.user_agent } lambda = merge(v.lambda, { @@ -35,6 +37,7 @@ locals { artifact = local.normalized_config.orchestration_provider.webhook.lambda.artifact }) }) + scale_set = v.orchestration_provider.scale_set } ssm = merge(v.ssm, { diff --git a/modules/multi-runner/config.experimental.resolved.tf b/modules/multi-runner/config.experimental.resolved.tf index fc29e81a8d..a0643d0823 100644 --- a/modules/multi-runner/config.experimental.resolved.tf +++ b/modules/multi-runner/config.experimental.resolved.tf @@ -295,6 +295,7 @@ locals { tags = merge(local.normalized_config.orchestration_provider.webhook.queue.tags, v.orchestration_provider.webhook.queue.tags) }) }) + scale_set = v.orchestration_provider.scale_set } ssm = merge(v.ssm, { diff --git a/modules/multi-runner/config.experimental.translation.tf b/modules/multi-runner/config.experimental.translation.tf index 301d37ec4d..fb8f09b96e 100644 --- a/modules/multi-runner/config.experimental.translation.tf +++ b/modules/multi-runner/config.experimental.translation.tf @@ -38,7 +38,9 @@ locals { url = var.ghes_url ssl_verify = var.ghes_ssl_verify } - user_agent = var.user_agent + runner_owner = null + runner_registration_level = "organization" + user_agent = var.user_agent } stable_to_v2_lambda = { @@ -141,6 +143,7 @@ locals { encryption = var.queue_encryption } } + scale_set = null } stable_to_v2_ssm = { @@ -427,6 +430,7 @@ locals { } } } + scale_set = null } ssm = { diff --git a/modules/multi-runner/main.tf b/modules/multi-runner/main.tf index 23d51b7f65..99a41d5207 100644 --- a/modules/multi-runner/main.tf +++ b/modules/multi-runner/main.tf @@ -3,8 +3,9 @@ locals { "ghr:environment" = var.prefix }) - primary_app_id = coalesce(local.effective_config.github.app.id_ssm, module.ssm.parameters.github_app_id) - primary_app_key_base64 = coalesce(local.effective_config.github.app.key_base64_ssm, module.ssm.parameters.github_app_key_base64) + primary_app_id = coalesce(local.effective_config.github.app.id_ssm, module.ssm.parameters.github_app_id) + primary_app_key_base64 = coalesce(local.effective_config.github.app.key_base64_ssm, module.ssm.parameters.github_app_key_base64) + primary_app_installation_id = local.effective_config.github.app.installation_id_ssm != null ? local.effective_config.github.app.installation_id_ssm : module.ssm.parameters.github_app_installation_id github_app_parameters = { id = local.primary_app_id diff --git a/modules/multi-runner/orchestration-provider.scale-set.tf b/modules/multi-runner/orchestration-provider.scale-set.tf new file mode 100644 index 0000000000..f18a7c1230 --- /dev/null +++ b/modules/multi-runner/orchestration-provider.scale-set.tf @@ -0,0 +1,62 @@ +locals { + scale_set_runner_configs = { + for runner_name, runner_config in local.effective_config.multi_runner_config : runner_name => { + github = { + enterprise_server = local.effective_config.github.enterprise_server + app = { + app_id = { + name = local.primary_app_id.name + arn = local.primary_app_id.arn + kms_key_arn = local.effective_config.ssm.kms_key_id + } + private_key = { + name = local.primary_app_key_base64.name + arn = local.primary_app_key_base64.arn + kms_key_arn = local.effective_config.ssm.kms_key_id + } + installation_id = local.primary_app_installation_id == null ? null : { + name = local.primary_app_installation_id.name + arn = local.primary_app_installation_id.arn + kms_key_arn = local.effective_config.ssm.kms_key_id + } + } + runner_owner = local.effective_config.github.runner_owner + runner_registration_level = local.effective_config.github.runner_registration_level + user_agent = local.effective_config.github.user_agent + } + scale_set = { + name = runner_config.orchestration_provider.scale_set.name + runner = { + labels = runner_config.runner.labels + group_name = runner_config.runner.group_name + min_runners = runner_config.orchestration_provider.scale_set.runner.min_runners + max_runners = runner_config.orchestration_provider.scale_set.runner.max_runners + boot_time_in_minutes = runner_config.orchestration_provider.scale_set.runner.boot_time_in_minutes + } + } + compute_provider = module.runner_configs[runner_name].compute_provider_contract + } + if runner_config.orchestration_provider.scale_set != null + } +} + +module "orchestration_scale_set" { + source = "../orchestration-providers/scale-set" + count = length(local.scale_set_runner_configs) > 0 && local.primary_app_installation_id != null ? 1 : 0 + + prefix = var.prefix + log_level = var.global_config_observability.logs.level + runner_configs = local.scale_set_runner_configs + + grouping = try(local.effective_config.orchestration_provider.scale_set.grouping, {}) + container = try(local.effective_config.orchestration_provider.scale_set.container, {}) + config_store = try(local.effective_config.orchestration_provider.scale_set.config_store, {}) + ecs = try(local.effective_config.orchestration_provider.scale_set.ecs, {}) + network = try(local.effective_config.orchestration_provider.scale_set.network, {}) + logging = try(local.effective_config.orchestration_provider.scale_set.logging, {}) + tags = merge( + local.effective_config.tags, + try(local.effective_config.orchestration_provider.scale_set.tags, {}), + { "ghr:environment" = var.prefix }, + ) +} diff --git a/modules/multi-runner/outputs.tf b/modules/multi-runner/outputs.tf index 6b7632ffc6..2ec43c8c56 100644 --- a/modules/multi-runner/outputs.tf +++ b/modules/multi-runner/outputs.tf @@ -20,7 +20,6 @@ output "runners_map" { } } } - output "runners_map_v2" { value = { for runner_key, runner in module.runner_configs : runner_key => { runner = runner.runner @@ -33,6 +32,16 @@ output "runners_map_v2" { } } +output "scale_set" { + description = "Shared scale-set orchestration resources, or null when no runner configuration selects scale_set." + value = length(module.orchestration_scale_set) == 0 ? null : { + cluster = module.orchestration_scale_set[0].cluster + controller_groups = module.orchestration_scale_set[0].controller_groups + reconciler_config_parameters = module.orchestration_scale_set[0].reconciler_config_parameters + resolved_container_image = module.orchestration_scale_set[0].resolved_container_image + } +} + output "binaries_syncer_map" { value = { for runner_binary_key, runner_binary in module.runner_binaries : runner_binary_key => { lambda = runner_binary.lambda diff --git a/modules/multi-runner/queues.tf b/modules/multi-runner/queues.tf index 0f57020571..c969bc6881 100644 --- a/modules/multi-runner/queues.tf +++ b/modules/multi-runner/queues.tf @@ -26,8 +26,15 @@ data "aws_iam_policy_document" "deny_insecure_transport" { } } +locals { + webhook_queue_configs = { + for config, values in local.effective_config.multi_runner_config : config => values + if values.orchestration_provider.webhook != null + } +} + resource "aws_sqs_queue" "queued_builds" { - for_each = local.effective_config.multi_runner_config + for_each = local.webhook_queue_configs name = "${var.prefix}-${each.key}-queued-builds" delay_seconds = each.value.orchestration_provider.webhook.queue.delay_webhook_event visibility_timeout_seconds = each.value.orchestration_provider.webhook.queue.visibility_timeout_seconds @@ -50,14 +57,14 @@ resource "aws_sqs_queue" "queued_builds" { } resource "aws_sqs_queue_policy" "build_queue_policy" { - for_each = local.effective_config.multi_runner_config + for_each = local.webhook_queue_configs queue_url = aws_sqs_queue.queued_builds[each.key].id policy = data.aws_iam_policy_document.deny_insecure_transport.json } resource "aws_sqs_queue" "queued_builds_dlq" { for_each = { - for config, values in local.effective_config.multi_runner_config : config => values + for config, values in local.webhook_queue_configs : config => values if values.orchestration_provider.webhook.queue.redrive_build_queue.enabled } name = "${var.prefix}-${each.key}-queued-builds_dead_letter" @@ -74,7 +81,7 @@ resource "aws_sqs_queue" "queued_builds_dlq" { resource "aws_sqs_queue_policy" "build_queue_dlq_policy" { for_each = { - for config, values in local.effective_config.multi_runner_config : config => values + for config, values in local.webhook_queue_configs : config => values if values.orchestration_provider.webhook.queue.redrive_build_queue.enabled } queue_url = aws_sqs_queue.queued_builds_dlq[each.key].id diff --git a/modules/multi-runner/runners.experimental.tf b/modules/multi-runner/runners.experimental.tf index 55c850eb84..b1d504442a 100644 --- a/modules/multi-runner/runners.experimental.tf +++ b/modules/multi-runner/runners.experimental.tf @@ -37,6 +37,7 @@ module "runner_configs" { lambda = each.value.orchestration_provider.webhook.lambda job_retry = each.value.orchestration_provider.webhook.job_retry } + scale_set = each.value.orchestration_provider.scale_set } ssm = each.value.ssm observability = each.value.observability diff --git a/modules/multi-runner/tests/config-resolution.tftest.hcl b/modules/multi-runner/tests/config-resolution.tftest.hcl index 40aa71a426..ff61de79f9 100644 --- a/modules/multi-runner/tests/config-resolution.tftest.hcl +++ b/modules/multi-runner/tests/config-resolution.tftest.hcl @@ -65,9 +65,10 @@ variables { global_config_github = { app = { - key_base64 = "experimental-app-key" - id = "experimental-app-id" - webhook_secret = "experimental-webhook-secret" + key_base64 = "experimental-app-key" + id = "experimental-app-id" + installation_id = "experimental-app-installation" + webhook_secret = "experimental-webhook-secret" } } @@ -392,6 +393,8 @@ run "v2_inputs_resolve_lane_over_global" { && keys(module.runner_configs) == ["lane"] && length(output.runners_map) == 0 && keys(output.runners_map_v2) == ["lane"] + && keys(aws_sqs_queue.queued_builds) == ["lane"] + && keys(aws_sqs_queue_policy.build_queue_policy) == ["lane"] ) error_message = "Experimental v2 configurations must route through module.runner_configs and skip the legacy runners module." } @@ -513,3 +516,393 @@ run "v2_inputs_reject_legacy_runner_config" { } } } + +run "scale_set_only_lane_omits_webhook_queues" { + command = plan + + variables { + experimental_features = ["multi-runner-v2"] + + global_config_github = { + app = { + key_base64 = "experimental-app-key" + id = "experimental-app-id" + installation_id = "experimental-app-installation" + webhook_secret = "experimental-webhook-secret" + } + runner_owner = "example" + runner_registration_level = "organization" + } + + global_config_orchestration_provider = { + webhook = { + eventbridge = { + enabled = false + } + lambda = { + artifact = { + s3 = { + key = "scale-runners.zip" + } + } + webhook = { + artifact = { + s3 = { + key = "scale-webhook.zip" + } + } + } + } + } + scale_set = { + network = { + vpc_id = "vpc-scale-set" + subnet_ids = ["subnet-scale-set"] + } + } + } + + multi_runner_config = { + scale = { + runner = { + os = "linux" + architecture = "x64" + } + orchestration_provider = { + webhook = null + scale_set = { + name = "scale-only" + } + } + compute_provider = { + aws = { + ec2 = { + instance_types = ["m5.large"] + vpc_id = "vpc-scale-set" + subnet_ids = ["subnet-scale-set"] + binaries_syncer = { + enabled = false + } + } + } + } + } + } + } + + assert { + condition = ( + local.resolved_config.multi_runner_config["scale"].orchestration_provider.webhook == null + && local.resolved_config.multi_runner_config["scale"].orchestration_provider.scale_set.name == "scale-only" + && keys(aws_sqs_queue.queued_builds) == [] + && keys(aws_sqs_queue_policy.build_queue_policy) == [] + && keys(aws_sqs_queue.queued_builds_dlq) == [] + && keys(aws_sqs_queue_policy.build_queue_dlq_policy) == [] + && length(module.orchestration_scale_set) == 1 + ) + error_message = "A scale-set-only lane must not create or access webhook SQS resources." + } +} + +run "mixed_webhook_and_scale_set_lanes_create_webhook_queues_only_for_webhook" { + command = plan + + variables { + experimental_features = ["multi-runner-v2"] + + global_config_github = { + app = { + key_base64 = "experimental-app-key" + id = "experimental-app-id" + installation_id = "experimental-app-installation" + webhook_secret = "experimental-webhook-secret" + } + runner_owner = "example" + runner_registration_level = "organization" + } + + global_config_orchestration_provider = { + webhook = { + eventbridge = { + enabled = false + } + lambda = { + artifact = { + s3 = { + key = "mixed-runners.zip" + } + } + webhook = { + artifact = { + s3 = { + key = "mixed-webhook.zip" + } + } + } + } + } + scale_set = { + network = { + vpc_id = "vpc-scale-set" + subnet_ids = ["subnet-scale-set"] + } + } + } + + multi_runner_config = { + webhook = { + orchestration_provider = { + webhook = { + matcherConfig = { + labelMatchers = [["self-hosted", "linux", "x64"]] + } + } + scale_set = null + } + compute_provider = { + aws = { + ec2 = { + instance_types = ["m5.large"] + vpc_id = "vpc-webhook" + subnet_ids = ["subnet-webhook"] + binaries_syncer = { + enabled = false + } + } + } + } + } + scale = { + orchestration_provider = { + webhook = null + scale_set = { + name = "scale-mixed" + } + } + compute_provider = { + aws = { + ec2 = { + instance_types = ["m5.large"] + vpc_id = "vpc-scale-set" + subnet_ids = ["subnet-scale-set"] + binaries_syncer = { + enabled = false + } + } + } + } + } + } + } + + assert { + condition = ( + keys(aws_sqs_queue.queued_builds) == ["webhook"] + && keys(aws_sqs_queue_policy.build_queue_policy) == ["webhook"] + && keys(module.runner_configs) == ["scale", "webhook"] + ) + error_message = "Mixed provider lanes must create webhook queues only for the webhook lane while routing both lanes through v2 runner configs." + } +} + +run "scale_set_lane_requires_owner_for_non_enterprise_registration" { + command = plan + + plan_options { + target = [terraform_data.validate_v2] + } + + variables { + experimental_features = ["multi-runner-v2"] + + global_config_github = { + app = { + key_base64 = "experimental-app-key" + id = "experimental-app-id" + webhook_secret = "experimental-webhook-secret" + } + runner_registration_level = "organization" + } + + multi_runner_config = { + scale = { + orchestration_provider = { + webhook = null + scale_set = { + name = "scale-missing-owner" + } + } + compute_provider = { + aws = { + ec2 = { + instance_types = ["m5.large"] + vpc_id = "vpc-scale-set" + subnet_ids = ["subnet-scale-set"] + } + } + } + } + } + } + + expect_failures = [terraform_data.validate_v2] +} + +run "scale_set_lane_requires_installation_id" { + command = plan + + variables { + experimental_features = ["multi-runner-v2"] + + global_config_github = { + app = { + key_base64 = "experimental-app-key" + id = "experimental-app-id" + webhook_secret = "experimental-webhook-secret" + } + runner_owner = "example" + runner_registration_level = "organization" + } + + multi_runner_config = { + scale = { + runner = { + os = "linux" + architecture = "x64" + } + orchestration_provider = { + webhook = null + scale_set = { + name = "scale-missing-installation" + } + } + compute_provider = { + aws = { + ec2 = { + instance_types = ["m5.large"] + vpc_id = "vpc-scale-set" + subnet_ids = ["subnet-scale-set"] + } + } + } + } + } + } + + expect_failures = [terraform_data.validate_v2] +} + +run "scale_set_queue_for_each_keys_are_plan_known" { + command = plan + + plan_options { + target = [aws_sqs_queue.queued_builds, aws_sqs_queue.queued_builds_dlq] + } + + variables { + experimental_features = ["multi-runner-v2"] + + global_config_orchestration_provider = { + scale_set = { + network = { + vpc_id = "vpc-scale-set" + subnet_ids = ["subnet-scale-set"] + } + } + } + + multi_runner_config = { + scale = { + orchestration_provider = { + webhook = null + scale_set = { + name = "scale-plan-known" + } + } + compute_provider = { + aws = { + ec2 = { + instance_types = ["m5.large"] + vpc_id = "vpc-scale-set" + subnet_ids = ["subnet-scale-set"] + binaries_syncer = { + enabled = false + } + } + } + } + } + } + } + + assert { + condition = ( + keys(aws_sqs_queue.queued_builds) == [] + && keys(aws_sqs_queue.queued_builds_dlq) == [] + ) + error_message = "Webhook queue for_each keys must be known and empty for a scale-set-only plan." + } +} + +run "v2_lane_requires_exactly_one_orchestration_provider" { + command = plan + + plan_options { + target = [terraform_data.validate_v2] + } + + variables { + experimental_features = ["multi-runner-v2"] + + multi_runner_config = { + missing = { + orchestration_provider = {} + compute_provider = { + aws = { + ec2 = { + instance_types = ["m5.large"] + vpc_id = "vpc-missing-provider" + subnet_ids = ["subnet-missing-provider"] + } + } + } + } + } + } + + expect_failures = [terraform_data.validate_v2] +} + +run "v2_lane_rejects_multiple_orchestration_providers" { + command = plan + + plan_options { + target = [terraform_data.validate_v2] + } + + variables { + experimental_features = ["multi-runner-v2"] + + multi_runner_config = { + multiple = { + orchestration_provider = { + webhook = {} + scale_set = { + name = "multiple-providers" + } + } + compute_provider = { + aws = { + ec2 = { + instance_types = ["m5.large"] + vpc_id = "vpc-multiple-providers" + subnet_ids = ["subnet-multiple-providers"] + } + } + } + } + } + } + + expect_failures = [terraform_data.validate_v2] +} diff --git a/modules/multi-runner/validations.tf b/modules/multi-runner/validations.tf index 257d479b56..dafb757a9f 100644 --- a/modules/multi-runner/validations.tf +++ b/modules/multi-runner/validations.tf @@ -80,14 +80,51 @@ resource "terraform_data" "validate_v2" { precondition { condition = alltrue([ for config in local.resolved_config.multi_runner_config : ( - try(config.orchestration_provider.webhook != null, false) && try(config.compute_provider.aws.ec2 != null, false) && try(length(config.compute_provider.aws.ec2.instance_types) > 0, false) && try(config.compute_provider.aws.ec2.vpc_id != null, false) && try(length(config.compute_provider.aws.ec2.subnet_ids) > 0, false) ) ]) - error_message = "Each experimental v2 runner lane requires a webhook provider, EC2 instance_types, vpc_id, and at least one subnet." + error_message = "Each experimental v2 runner lane requires the supported aws.ec2 compute provider with instance_types, vpc_id, and at least one subnet." } + + precondition { + condition = alltrue([ + for config in local.resolved_config.multi_runner_config : ( + try(config.orchestration_provider.webhook != null, false) != + try(config.orchestration_provider.scale_set != null, false) + ) + ]) + error_message = "Each experimental v2 runner lane requires exactly one orchestration provider: webhook or scale_set." + } + + precondition { + condition = alltrue([ + for config in local.resolved_config.multi_runner_config : ( + try(config.orchestration_provider.scale_set, null) == null ? true : ( + contains([ + "organization", + "repository", + ], try(var.global_config_github.runner_registration_level, null)) && + try(var.global_config_github.runner_owner, null) != null + ) + ) + ]) + error_message = "Scale-set lanes require global_config_github.runner_registration_level to be organization or repository; runner_owner must be set for organization and repository registration." + } + + precondition { + condition = alltrue([ + for config in local.resolved_config.multi_runner_config : ( + try(config.orchestration_provider.scale_set, null) == null ? true : ( + try(var.global_config_github.app.installation_id, null) != null || + try(var.global_config_github.app.installation_id_ssm, null) != null + ) + ) + ]) + error_message = "Scale-set lanes require global_config_github.app.installation_id or global_config_github.app.installation_id_ssm." + } + } } diff --git a/modules/multi-runner/variables.experimental.github.tf b/modules/multi-runner/variables.experimental.github.tf index 6a783f7d59..79ea3a65cf 100644 --- a/modules/multi-runner/variables.experimental.github.tf +++ b/modules/multi-runner/variables.experimental.github.tf @@ -13,6 +13,10 @@ variable "global_config_github" { id_ssm: "SSM parameter containing the GitHub App ID." id_ssm.arn: "ARN of the SSM parameter containing the GitHub App ID." id_ssm.name: "Name of the SSM parameter containing the GitHub App ID." + installation_id: "GitHub App installation ID for the primary scale-set installation." + installation_id_ssm: "SSM parameter containing the primary GitHub App installation ID." + installation_id_ssm.arn: "ARN of the SSM parameter containing the primary GitHub App installation ID." + installation_id_ssm.name: "Name of the SSM parameter containing the primary GitHub App installation ID." webhook_secret: "GitHub App webhook secret." webhook_secret_ssm: "SSM parameter containing the GitHub App webhook secret." webhook_secret_ssm.arn: "ARN of the SSM parameter containing the GitHub App webhook secret." @@ -33,6 +37,8 @@ variable "global_config_github" { additional_apps.installation_id_ssm.name: "Name of the SSM parameter containing an additional App installation ID." enterprise_server.url: "GitHub Enterprise Server URL." enterprise_server.ssl_verify: "Whether to verify the GitHub Enterprise Server TLS certificate." + runner_owner: "GitHub organization or owner/repository path for organization- or repository-level scale-set registration." + runner_registration_level: "GitHub scale-set registration scope: organization or repository." user_agent: "User-Agent value sent with GitHub API requests." } EOT @@ -48,6 +54,11 @@ variable "global_config_github" { arn = string name = string })) + installation_id = optional(string) + installation_id_ssm = optional(object({ + arn = string + name = string + })) webhook_secret = optional(string) webhook_secret_ssm = optional(object({ arn = string @@ -66,7 +77,9 @@ variable "global_config_github" { url = optional(string, null) ssl_verify = optional(bool, true) }), {}) - user_agent = optional(string, "github-aws-runners") + runner_owner = optional(string, null) + runner_registration_level = optional(string, "organization") + user_agent = optional(string, "github-aws-runners") }) default = {} } diff --git a/modules/multi-runner/variables.experimental.orchestration-provider.tf b/modules/multi-runner/variables.experimental.orchestration-provider.tf index 9919d3645e..41a51d6204 100644 --- a/modules/multi-runner/variables.experimental.orchestration-provider.tf +++ b/modules/multi-runner/variables.experimental.orchestration-provider.tf @@ -173,6 +173,78 @@ variable "global_config_orchestration_provider" { sqs_managed_sse_enabled = true }) }), {}) + + }), {}) + + scale_set = optional(object({ + grouping = optional(object({ + strategy = optional(string, "compute_provider") + custom = optional(object({ + groups = map(object({ + runner_configs = set(string) + })) + }), null) + }), {}) + container = optional(object({ + image = optional(string, null) + user = optional(string, "10001:10001") + health_port = optional(number, 8080) + health_path = optional(string, "/healthz") + health_check_command = optional(list(string), null) + health_check_interval = optional(number, 30) + health_check_timeout = optional(number, 5) + health_check_retries = optional(number, 3) + health_check_start_period = optional(number, 30) + health_stale_after_seconds = optional(number, 180) + shutdown_timeout_seconds = optional(number, 110) + session_close_timeout_seconds = optional(number, 10) + reconnect_initial_backoff_seconds = optional(number, 1) + reconnect_max_backoff_seconds = optional(number, 30) + stop_timeout_seconds = optional(number, 120) + }), {}) + config_store = optional(object({ + path_prefix = optional(string, null) + tier = optional(string, "Standard") + tags = optional(map(string), {}) + }), {}) + ecs = optional(object({ + cluster = optional(object({ + mode = optional(string, "managed") + arn = optional(string, null) + name = optional(string, null) + container_insights = optional(bool, true) + }), {}) + task = optional(object({ + cpu = optional(number, 512) + memory = optional(number, 1024) + cpu_architecture = optional(string, "X86_64") + ephemeral_storage = optional(object({ + size_in_gib = number + }), null) + }), {}) + service = optional(object({ + platform_version = optional(string, "LATEST") + }), {}) + iam = optional(object({ + path = optional(string, "/") + permissions_boundary = optional(string, null) + }), {}) + }), {}) + network = optional(object({ + vpc_id = optional(string, null) + subnet_ids = optional(set(string), null) + https_egress = optional(object({ + ipv4_cidrs = optional(set(string), ["0.0.0.0/0"]) + ipv6_cidrs = optional(set(string), []) + }), {}) + }), {}) + logging = optional(object({ + retention_in_days = optional(number, 30) + kms_key_arn = optional(string, null) + log_group_class = optional(string, "STANDARD") + tags = optional(map(string), {}) + }), {}) + tags = optional(map(string), {}) }), {}) }) default = {} diff --git a/modules/multi-runner/variables.tf b/modules/multi-runner/variables.tf index 793e126e4e..bf47576d67 100644 --- a/modules/multi-runner/variables.tf +++ b/modules/multi-runner/variables.tf @@ -19,6 +19,11 @@ variable "github_app" { arn = string name = string })) + installation_id = optional(string) + installation_id_ssm = optional(object({ + arn = string + name = string + })) webhook_secret = optional(string) webhook_secret_ssm = optional(object({ arn = string @@ -386,6 +391,14 @@ variable "multi_runner_config" { }), {}) }), {}) }), null) + scale_set = optional(object({ + name = string + runner = optional(object({ + min_runners = optional(number, 0) + max_runners = optional(number, 10) + boot_time_in_minutes = optional(number, 10) + }), {}) + }), null) }), {}) ssm = optional(object({ diff --git a/modules/orchestration-providers/scale-set/README.md b/modules/orchestration-providers/scale-set/README.md index 495a8cc314..cc7c90eb23 100644 --- a/modules/orchestration-providers/scale-set/README.md +++ b/modules/orchestration-providers/scale-set/README.md @@ -218,6 +218,7 @@ No modules. | [terraform_data.validate_compute_role_policy](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | | [terraform_data.validate_config_store](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | | [terraform_data.validate_contract](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | +| [terraform_data.validate_group_compute_policy](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | | [terraform_data.validate_group_task_policy](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | | [terraform_data.validate_grouping](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | | [terraform_data.validate_runtime](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | diff --git a/modules/orchestration-providers/scale-set/validations.tf b/modules/orchestration-providers/scale-set/validations.tf index 9553581597..d956e21f2f 100644 --- a/modules/orchestration-providers/scale-set/validations.tf +++ b/modules/orchestration-providers/scale-set/validations.tf @@ -423,3 +423,17 @@ resource "terraform_data" "validate_compute_role_policy" { } } } + +resource "terraform_data" "validate_group_compute_policy" { + for_each = local.controller_groups + + lifecycle { + precondition { + condition = ( + floor(length(base64encode(data.aws_iam_policy_document.task_compute[each.key].json)) * 3 / 4) - + (endswith(base64encode(data.aws_iam_policy_document.task_compute[each.key].json), "==") ? 2 : endswith(base64encode(data.aws_iam_policy_document.task_compute[each.key].json), "=") ? 1 : 0) + ) <= 10240 + error_message = "Controller group ${each.key} produces a compute-provider task-role policy exceeding AWS's 10240-byte inline role-policy quota. Split the group or reduce provider IAM statements." + } + } +} diff --git a/modules/runner-config/README.md b/modules/runner-config/README.md index 7c1585dca2..41c0b1a191 100644 --- a/modules/runner-config/README.md +++ b/modules/runner-config/README.md @@ -114,7 +114,7 @@ yarn run dist | [github](#input\_github) | GitHub API and runner-registration configuration.

- `app_parameters.key_base64`: Parameter Store reference for the primary GitHub App private key.
- `app_parameters.id`: Parameter Store reference for the primary GitHub App ID.
- `app_parameters.additional_apps_manifest`: Optional Parameter Store reference containing the additional GitHub App manifest.
- `app_parameters.additional_app_parameter_arns`: ARNs of the additional GitHub App credential parameters.
- `enterprise_server.url`: Optional GitHub Enterprise Server base URL. Null selects GitHub.com.
- `enterprise_server.ssl_verify`: Enables TLS certificate verification for GitHub Enterprise Server requests.
- `user_agent`: Optional User-Agent value added to GitHub API requests. | 
object({
    app_parameters = object({
      key_base64 = map(string)
      id         = map(string)
      additional_apps_manifest = optional(object({
        name = string
        arn  = string
      }), null)
      additional_app_parameter_arns = optional(list(string), [])
    })
    enterprise_server = optional(object({
      url        = optional(string, null)
      ssl_verify = optional(bool, true)
    }), {})
    user_agent = optional(string, null)
  })
| n/a | yes | | [lambda](#input\_lambda) | Common Lambda substrate independent of the selected runner orchestration provider.

- `artifact.s3.bucket`: Optional shared S3 bucket containing component-owned Lambda artifacts. An orchestration provider selects its own object key and version; the bucket alone selects no artifact.
- `runtime`: Runtime used by the control-plane Lambda functions.
- `architecture`: Instruction-set architecture used by the control-plane Lambda functions. Supported values are `arm64` and `x86_64`.
- `subnet_ids`: Subnets used for Lambda VPC configuration.
- `security_group_ids`: Security groups used for Lambda VPC configuration.
- `tags`: Shared tags applied to Lambda function resources only. These override module-level `tags`; component `tags` override this map when keys conflict.
- `principals`: Additional principals allowed to assume the control-plane Lambda roles.
- `role.path`: IAM path for module-managed Lambda execution roles. Defaults to a path derived from `prefix`.
- `role.permissions_boundary`: Permissions-boundary ARN applied to module-managed Lambda execution roles. | 
object({
    artifact = optional(object({
      s3 = optional(object({
        bucket = optional(string, null)
      }), {})
    }), {})
    runtime            = optional(string, "nodejs24.x")
    architecture       = optional(string, "arm64")
    subnet_ids         = optional(list(string), [])
    security_group_ids = optional(list(string), [])
    tags               = optional(map(string), {})
    principals = optional(list(object({
      type        = string
      identifiers = list(string)
    })), [])
    role = optional(object({
      path                 = optional(string, null)
      permissions_boundary = optional(string, null)
    }), {})
  })
| `{}` | no | | [observability](#input\_observability) | Logging, tracing, and metrics configuration for control-plane and provider resources.

- `logs.level`: Application log level supplied to the control-plane functions.
- `logs.retention_in_days`: CloudWatch Logs retention period.
- `logs.kms_key_id`: Optional KMS key ID or ARN used to encrypt CloudWatch log groups.
- `logs.class`: CloudWatch log-group class. Supported values are `STANDARD` and `INFREQUENT_ACCESS`.
- `logs.tags`: Shared tags for CloudWatch log groups. These override module-level `tags`; component `tags` override this map when keys conflict.
- `tracing.mode`: Optional Lambda active-tracing mode. Null disables X-Ray tracing configuration.
- `tracing.capture_http_requests`: Enables HTTP request capture in the tracing helper.
- `tracing.capture_error`: Enables error capture in the tracing helper.
- `metrics.enabled`: Enables module-emitted metrics.
- `metrics.namespace`: CloudWatch namespace used for emitted metrics.
- `metrics.metric.github_app_rate_limit.enabled`: Emits GitHub App rate-limit metrics.
- `metrics.metric.job_retry.enabled`: Emits job-retry metrics.
- `metrics.metric.spot_termination_warning.enabled`: Emits spot-termination warning metrics where supported. | 
object({
    logs = optional(object({
      level             = optional(string, "info")
      retention_in_days = optional(number, 180)
      kms_key_id        = optional(string, null)
      class             = optional(string, "STANDARD")
      tags              = optional(map(string), {})
    }), {})
    tracing = optional(object({
      mode                  = optional(string, null)
      capture_http_requests = optional(bool, false)
      capture_error         = optional(bool, false)
    }), {})
    metrics = optional(object({
      enabled   = optional(bool, false)
      namespace = optional(string, "GitHub Runners")
      metric = optional(object({
        github_app_rate_limit = optional(object({
          enabled = optional(bool, true)
        }), {})
        job_retry = optional(object({
          enabled = optional(bool, true)
        }), {})
        spot_termination_warning = optional(object({
          enabled = optional(bool, true)
        }), {})
      }), {})
    }), {})
  })
| `{}` | no | -| [orchestration\_provider](#input\_orchestration\_provider) | Runner demand-orchestration provider configuration. Exactly one provider block must be non-null. Wrapper presence selects the provider and must therefore be known during planning; values inside the selected provider may remain unknown until apply.

- `webhook`: Selects the workflow-job webhook control plane. It owns runner lifecycle and capacity, the build queue reference, the runner-control artifact, scale-up, scale-down, scheduled pool, and optional job-retry controls. Future providers can be added as sibling blocks without moving this contract.
- `webhook.runner`: Runner lifecycle, boot timeout, and capacity settings owned by webhook orchestration.
- `webhook.runner.boot_time_in_minutes`: Expected runner boot duration used by scale-down and pool controls. The default is `5`.
- `webhook.runner.ephemeral`: Registers runners in ephemeral mode. The default is `false`.
- `webhook.runner.jit_config_enabled`: Explicitly enables or disables just-in-time configuration. The default is null, which follows `runner.ephemeral`.
- `webhook.runner.maximum_count`: Maximum number of runners managed for this runner configuration. The default is `3`.
- `webhook.github.organization_runners`: Registers runners at organization scope when true; otherwise registration is repository-scoped.
- `webhook.queue.build.arn`: ARN of the runner configuration's build queue.
- `webhook.queue.build.url`: URL of the runner configuration's build queue.
- `webhook.queue.kms_key_id`: Optional KMS key ARN encrypting the build queue. The default is null and is independent from the Parameter Store KMS key.
- `webhook.queue.tags`: Tags inherited by queue-related provider resources before component-specific overrides. The default is `{}`.
- `webhook.lambda.artifact`: Runner-control artifact shared by scale, pool, and job-retry components. Set at most one of `zip` or `s3`; no selection uses the packaged runner archive.
- `webhook.lambda.artifact.zip`: Optional local path to the runner-control Lambda archive. The default is null.
- `webhook.lambda.artifact.s3`: Optional S3 object selector in the common `lambda.artifact.s3.bucket`. Wrapper presence must be known during planning, selecting it requires a non-null common bucket, and the default is null.
- `webhook.lambda.artifact.s3.key`: Object key of the runner-control Lambda archive.
- `webhook.lambda.artifact.s3.object_version`: Optional object version of the runner-control Lambda archive. The default is null.
- `webhook.lambda.scale.up.memory_size`: Memory allocated to the scale-up Lambda in MB. The default is `512`.
- `webhook.lambda.scale.up.timeout`: Scale-up Lambda timeout in seconds. The default is `60`.
- `webhook.lambda.scale.up.reserved_concurrent_executions`: Reserved concurrency for scale-up. The default is `1`; use `-1` for unreserved concurrency.
- `webhook.lambda.scale.up.job_queued_check_enabled`: Enables queued-job verification before scaling. The default is null, which follows the resolved runner mode.
- `webhook.lambda.scale.up.event_source_mapping.batch_size`: Maximum build-queue records delivered per scale-up invocation. The default is `10`.
- `webhook.lambda.scale.up.event_source_mapping.maximum_batching_window_in_seconds`: Maximum batching window for build-queue records. The default is `0`.
- `webhook.lambda.scale.up.tags`: Tags applied within scale-up resource scopes after common provider tags. The default is `{}`.
- `webhook.lambda.scale.down.memory_size`: Memory allocated to the scale-down Lambda in MB. The default is `512`.
- `webhook.lambda.scale.down.timeout`: Scale-down Lambda timeout in seconds. The default is `60`.
- `webhook.lambda.scale.down.schedule_expression`: EventBridge schedule expression that invokes scale-down. The default is `cron(*/5 * * * ? *)`.
- `webhook.lambda.scale.down.minimum_running_time_in_minutes`: Optional minimum runner age before scale-down may terminate it. The default is null, which selects the operating-system default.
- `webhook.lambda.scale.down.idle_confirmation_seconds`: Number of seconds a runner must consistently report not-busy before scale-down terminates it. The default is `0`, which preserves the single-reading behavior.
- `webhook.lambda.scale.down.idle_config`: Time-based desired idle-runner configurations. The default is `[]`.
- `webhook.lambda.scale.down.idle_config[].cron`: Cron expression identifying when the idle configuration applies.
- `webhook.lambda.scale.down.idle_config[].timeZone`: IANA time zone used to evaluate the cron expression.
- `webhook.lambda.scale.down.idle_config[].idleCount`: Number of idle runners retained during the matching period.
- `webhook.lambda.scale.down.idle_config[].evictionStrategy`: Selection strategy used when excess idle runners are removed. The default is `oldest_first`.
- `webhook.lambda.scale.down.tags`: Tags applied within scale-down resource scopes after common provider tags. The default is `{}`.
- `webhook.lambda.pool.memory_size`: Memory allocated to the pool Lambda in MB. The default is `512`.
- `webhook.lambda.pool.timeout`: Pool Lambda timeout in seconds. The default is `60`.
- `webhook.lambda.pool.reserved_concurrent_executions`: Reserved concurrency for the pool Lambda. The default is `1`; use `-1` for unreserved concurrency.
- `webhook.lambda.pool.config`: Scheduled target pool sizes. The default is `[]`, which disables the pool component.
- `webhook.lambda.pool.config[].schedule_expression`: Scheduler expression that activates the target size.
- `webhook.lambda.pool.config[].schedule_expression_timezone`: Optional IANA time zone used to evaluate the schedule.
- `webhook.lambda.pool.config[].size`: Desired number of runners for the schedule.
- `webhook.lambda.pool.include_busy_runners`: Includes busy runners when reconciling scheduled pool capacity. The default is `false`.
- `webhook.lambda.pool.runner_owner`: Optional GitHub organization or repository owner used for pooled runners. The default is null.
- `webhook.lambda.pool.tags`: Tags applied within pool resource scopes after common provider tags. The default is `{}`.
- `webhook.job_retry.enabled`: Creates the retry queue, Lambda function, event-source mapping, and related IAM resources. The default is `false`.
- `webhook.job_retry.delay_in_seconds`: Initial delay before a queued-job retry check. The default is `300`.
- `webhook.job_retry.delay_backoff`: Multiplier applied to the delay after each unsuccessful check. The default is `2`.
- `webhook.job_retry.max_attempts`: Maximum retry-check attempts before the message is no longer republished. The default is `1`.
- `webhook.job_retry.tags`: Tags applied within job-retry resource scopes after common provider tags. The default is `{}`.
- `webhook.job_retry.lambda.memory_size`: Memory allocated to the job-retry Lambda in MB. The default is `256`.
- `webhook.job_retry.lambda.reserved_concurrent_executions`: Reserved concurrency for job retry. The default is `1`; use `-1` for unreserved concurrency.
- `webhook.job_retry.lambda.timeout`: Job-retry Lambda timeout in seconds and visibility timeout for its retry queue. The default is `30`. | 
object({
    webhook = optional(object({
      runner = optional(object({
        boot_time_in_minutes = optional(number, 5)
        ephemeral            = optional(bool, false)
        jit_config_enabled   = optional(bool, null)
        maximum_count        = optional(number, 3)
      }), {})
      github = object({
        organization_runners = bool
      })
      queue = object({
        build = object({
          arn = string
          url = string
        })
        kms_key_id = optional(string, null)
        tags       = optional(map(string), {})
      })
      lambda = optional(object({
        artifact = optional(object({
          zip = optional(string, null)
          s3 = optional(object({
            key            = string
            object_version = optional(string, null)
          }), null)
        }), {})
        scale = optional(object({
          up = optional(object({
            memory_size                    = optional(number, 512)
            timeout                        = optional(number, 60)
            reserved_concurrent_executions = optional(number, 1)
            job_queued_check_enabled       = optional(bool, null)
            event_source_mapping = optional(object({
              batch_size                         = optional(number, 10)
              maximum_batching_window_in_seconds = optional(number, 0)
            }), {})
            tags = optional(map(string), {})
          }), {})
          down = optional(object({
            memory_size                     = optional(number, 512)
            timeout                         = optional(number, 60)
            schedule_expression             = optional(string, "cron(*/5 * * * ? *)")
            minimum_running_time_in_minutes = optional(number, null)
            idle_confirmation_seconds       = optional(number, 0)
            idle_config = optional(list(object({
              cron             = string
              timeZone         = string
              idleCount        = number
              evictionStrategy = optional(string, "oldest_first")
            })), [])
            tags = optional(map(string), {})
          }), {})
        }), {})
        pool = optional(object({
          memory_size                    = optional(number, 512)
          timeout                        = optional(number, 60)
          reserved_concurrent_executions = optional(number, 1)
          config = optional(list(object({
            schedule_expression          = string
            schedule_expression_timezone = optional(string)
            size                         = number
          })), [])
          include_busy_runners = optional(bool, false)
          runner_owner         = optional(string, null)
          tags                 = optional(map(string), {})
        }), {})
      }), {})
      job_retry = optional(object({
        enabled          = optional(bool, false)
        delay_in_seconds = optional(number, 300)
        delay_backoff    = optional(number, 2)
        max_attempts     = optional(number, 1)
        tags             = optional(map(string), {})
        lambda = optional(object({
          memory_size                    = optional(number, 256)
          reserved_concurrent_executions = optional(number, 1)
          timeout                        = optional(number, 30)
        }), {})
      }), {})
    }), null)
  })
| n/a | yes | +| [orchestration\_provider](#input\_orchestration\_provider) | Runner demand-orchestration provider configuration. Exactly one provider block must be non-null. Wrapper presence selects the provider and must therefore be known during planning; values inside the selected provider may remain unknown until apply.

- `webhook`: Selects the workflow-job webhook control plane. It owns runner lifecycle and capacity, the build queue reference, the runner-control artifact, scale-up, scale-down, scheduled pool, and optional job-retry controls.
- `scale_set`: Selects scale-set orchestration for this runner config. The multi-runner topology owns the shared controller service and passes this plan-known selection marker to runner-config. Scale-set runners always use ephemeral JIT registration.
- `webhook.runner`: Runner lifecycle, boot timeout, and capacity settings owned by webhook orchestration.
- `webhook.runner.boot_time_in_minutes`: Expected runner boot duration used by scale-down and pool controls. The default is `5`.
- `webhook.runner.ephemeral`: Registers runners in ephemeral mode. The default is `false`.
- `webhook.runner.jit_config_enabled`: Explicitly enables or disables just-in-time configuration. The default is null, which follows `runner.ephemeral`.
- `webhook.runner.maximum_count`: Maximum number of runners managed for this runner configuration. The default is `3`.
- `webhook.github.organization_runners`: Registers runners at organization scope when true; otherwise registration is repository-scoped.
- `webhook.queue.build.arn`: ARN of the runner configuration's build queue.
- `webhook.queue.build.url`: URL of the runner configuration's build queue.
- `webhook.queue.kms_key_id`: Optional KMS key ARN encrypting the build queue. The default is null and is independent from the Parameter Store KMS key.
- `webhook.queue.tags`: Tags inherited by queue-related provider resources before component-specific overrides. The default is `{}`.
- `webhook.lambda.artifact`: Runner-control artifact shared by scale, pool, and job-retry components. Set at most one of `zip` or `s3`; no selection uses the packaged runner archive.
- `webhook.lambda.artifact.zip`: Optional local path to the runner-control Lambda archive. The default is null.
- `webhook.lambda.artifact.s3`: Optional S3 object selector in the common `lambda.artifact.s3.bucket`. Wrapper presence must be known during planning, selecting it requires a non-null common bucket, and the default is null.
- `webhook.lambda.artifact.s3.key`: Object key of the runner-control Lambda archive.
- `webhook.lambda.artifact.s3.object_version`: Optional object version of the runner-control Lambda archive. The default is null.
- `webhook.lambda.scale.up.memory_size`: Memory allocated to the scale-up Lambda in MB. The default is `512`.
- `webhook.lambda.scale.up.timeout`: Scale-up Lambda timeout in seconds. The default is `60`.
- `webhook.lambda.scale.up.reserved_concurrent_executions`: Reserved concurrency for scale-up. The default is `1`; use `-1` for unreserved concurrency.
- `webhook.lambda.scale.up.job_queued_check_enabled`: Enables queued-job verification before scaling. The default is null, which follows the resolved runner mode.
- `webhook.lambda.scale.up.event_source_mapping.batch_size`: Maximum build-queue records delivered per scale-up invocation. The default is `10`.
- `webhook.lambda.scale.up.event_source_mapping.maximum_batching_window_in_seconds`: Maximum batching window for build-queue records. The default is `0`.
- `webhook.lambda.scale.up.tags`: Tags applied within scale-up resource scopes after common provider tags. The default is `{}`.
- `webhook.lambda.scale.down.memory_size`: Memory allocated to the scale-down Lambda in MB. The default is `512`.
- `webhook.lambda.scale.down.timeout`: Scale-down Lambda timeout in seconds. The default is `60`.
- `webhook.lambda.scale.down.schedule_expression`: EventBridge schedule expression that invokes scale-down. The default is `cron(*/5 * * * ? *)`.
- `webhook.lambda.scale.down.minimum_running_time_in_minutes`: Optional minimum runner age before scale-down may terminate it. The default is null, which selects the operating-system default.
- `webhook.lambda.scale.down.idle_confirmation_seconds`: Number of seconds a runner must consistently report not-busy before scale-down terminates it. The default is `0`, which preserves the single-reading behavior.
- `webhook.lambda.scale.down.idle_config`: Time-based desired idle-runner configurations. The default is `[]`.
- `webhook.lambda.scale.down.idle_config[].cron`: Cron expression identifying when the idle configuration applies.
- `webhook.lambda.scale.down.idle_config[].timeZone`: IANA time zone used to evaluate the cron expression.
- `webhook.lambda.scale.down.idle_config[].idleCount`: Number of idle runners retained during the matching period.
- `webhook.lambda.scale.down.idle_config[].evictionStrategy`: Selection strategy used when excess idle runners are removed. The default is `oldest_first`.
- `webhook.lambda.scale.down.tags`: Tags applied within scale-down resource scopes after common provider tags. The default is `{}`.
- `webhook.lambda.pool.memory_size`: Memory allocated to the pool Lambda in MB. The default is `512`.
- `webhook.lambda.pool.timeout`: Pool Lambda timeout in seconds. The default is `60`.
- `webhook.lambda.pool.reserved_concurrent_executions`: Reserved concurrency for the pool Lambda. The default is `1`; use `-1` for unreserved concurrency.
- `webhook.lambda.pool.config`: Scheduled target pool sizes. The default is `[]`, which disables the pool component.
- `webhook.lambda.pool.config[].schedule_expression`: Scheduler expression that activates the target size.
- `webhook.lambda.pool.config[].schedule_expression_timezone`: Optional IANA time zone used to evaluate the schedule.
- `webhook.lambda.pool.config[].size`: Desired number of runners for the schedule.
- `webhook.lambda.pool.include_busy_runners`: Includes busy runners when reconciling scheduled pool capacity. The default is `false`.
- `webhook.lambda.pool.runner_owner`: Optional GitHub organization or repository owner used for pooled runners. The default is null.
- `webhook.lambda.pool.tags`: Tags applied within pool resource scopes after common provider tags. The default is `{}`.
- `webhook.job_retry.enabled`: Creates the retry queue, Lambda function, event-source mapping, and related IAM resources. The default is `false`.
- `webhook.job_retry.delay_in_seconds`: Initial delay before a queued-job retry check. The default is `300`.
- `webhook.job_retry.delay_backoff`: Multiplier applied to the delay after each unsuccessful check. The default is `2`.
- `webhook.job_retry.max_attempts`: Maximum retry-check attempts before the message is no longer republished. The default is `1`.
- `webhook.job_retry.tags`: Tags applied within job-retry resource scopes after common provider tags. The default is `{}`.
- `webhook.job_retry.lambda.memory_size`: Memory allocated to the job-retry Lambda in MB. The default is `256`.
- `webhook.job_retry.lambda.reserved_concurrent_executions`: Reserved concurrency for job retry. The default is `1`; use `-1` for unreserved concurrency.
- `webhook.job_retry.lambda.timeout`: Job-retry Lambda timeout in seconds and visibility timeout for its retry queue. The default is `30`. | 
object({
    webhook = optional(object({
      runner = optional(object({
        boot_time_in_minutes = optional(number, 5)
        ephemeral            = optional(bool, false)
        jit_config_enabled   = optional(bool, null)
        maximum_count        = optional(number, 3)
      }), {})
      github = object({
        organization_runners = bool
      })
      queue = object({
        build = object({
          arn = string
          url = string
        })
        kms_key_id = optional(string, null)
        tags       = optional(map(string), {})
      })
      lambda = optional(object({
        artifact = optional(object({
          zip = optional(string, null)
          s3 = optional(object({
            key            = string
            object_version = optional(string, null)
          }), null)
        }), {})
        scale = optional(object({
          up = optional(object({
            memory_size                    = optional(number, 512)
            timeout                        = optional(number, 60)
            reserved_concurrent_executions = optional(number, 1)
            job_queued_check_enabled       = optional(bool, null)
            event_source_mapping = optional(object({
              batch_size                         = optional(number, 10)
              maximum_batching_window_in_seconds = optional(number, 0)
            }), {})
            tags = optional(map(string), {})
          }), {})
          down = optional(object({
            memory_size                     = optional(number, 512)
            timeout                         = optional(number, 60)
            schedule_expression             = optional(string, "cron(*/5 * * * ? *)")
            minimum_running_time_in_minutes = optional(number, null)
            idle_confirmation_seconds       = optional(number, 0)
            idle_config = optional(list(object({
              cron             = string
              timeZone         = string
              idleCount        = number
              evictionStrategy = optional(string, "oldest_first")
            })), [])
            tags = optional(map(string), {})
          }), {})
        }), {})
        pool = optional(object({
          memory_size                    = optional(number, 512)
          timeout                        = optional(number, 60)
          reserved_concurrent_executions = optional(number, 1)
          config = optional(list(object({
            schedule_expression          = string
            schedule_expression_timezone = optional(string)
            size                         = number
          })), [])
          include_busy_runners = optional(bool, false)
          runner_owner         = optional(string, null)
          tags                 = optional(map(string), {})
        }), {})
      }), {})
      job_retry = optional(object({
        enabled          = optional(bool, false)
        delay_in_seconds = optional(number, 300)
        delay_backoff    = optional(number, 2)
        max_attempts     = optional(number, 1)
        tags             = optional(map(string), {})
        lambda = optional(object({
          memory_size                    = optional(number, 256)
          reserved_concurrent_executions = optional(number, 1)
          timeout                        = optional(number, 30)
        }), {})
      }), {})
    }), null)
    scale_set = optional(object({
      name = string
      runner = optional(object({
        min_runners          = optional(number, 0)
        max_runners          = optional(number, 10)
        boot_time_in_minutes = optional(number, 10)
      }), {})
    }), null)
  })
| n/a | yes | | [prefix](#input\_prefix) | The prefix used for naming resources. | `string` | `"github-actions"` | no | | [runner](#input\_runner) | Provider-neutral GitHub runner configuration.

- `os`: Runner operating system. Supported values are `linux`, `osx`, and `windows`.
- `architecture`: Runner distribution architecture, such as `x64` or `arm64`.
- `disable_default_labels`: Prevents GitHub's default self-hosted, operating-system, and architecture labels from being registered.
- `labels`: Complete set of labels supplied to the control-plane functions.
- `group_name`: GitHub runner group used during registration.
- `name_prefix`: Prefix added to registered runner names.
- `run_as_root`: Runs the runner service as root when supported by the compute provider.
- `run_as`: Operating-system user used when `run_as_root` is false.
- `auto_update_disabled`: Disables the GitHub runner application's built-in updater.
- `tags`: Additional tags for common runner resources, currently the managed runner IAM role. These override module-level `tags` with the same key.
- `hooks.job_started`: Script content installed as the runner job-started hook.
- `hooks.job_completed`: Script content installed as the runner job-completed hook.
- `iam.role.arn`: ARN of an externally managed runner role. When set, this module does not create or modify that role.
- `iam.managed_policy_arns`: Named managed-policy ARNs attached to the module-managed runner role.
- `iam.additional_trust_policy_json`: Optional IAM policy document merged with the selected compute provider's default runner-role trust policy.
- `iam.path`: IAM path for the module-managed runner role. Defaults to a path derived from `prefix`.
- `iam.permissions_boundary`: Permissions-boundary ARN for the module-managed runner role. | 
object({
    os                     = optional(string, "linux")
    architecture           = optional(string, "x64")
    disable_default_labels = optional(bool, false)
    labels                 = list(string)
    group_name             = optional(string, "Default")
    name_prefix            = optional(string, "")
    run_as_root            = optional(bool, false)
    run_as                 = optional(string, "ec2-user")
    auto_update_disabled   = optional(bool, false)
    tags                   = optional(map(string), {})
    hooks = optional(object({
      job_started   = optional(string, "")
      job_completed = optional(string, "")
    }), {})
    iam = optional(object({
      role = optional(object({
        arn = string
      }), null)
      managed_policy_arns          = optional(map(string), {})
      additional_trust_policy_json = optional(string, null)
      path                         = optional(string, null)
      permissions_boundary         = optional(string, null)
    }), {})
  })
| n/a | yes | | [ssm](#input\_ssm) | Parameter Store paths, encryption, tag scopes, and housekeeper configuration.

- `paths.root`: Root Parameter Store path for this runner configuration.
- `paths.tokens`: Path segment under `paths.root` used for registration tokens and just-in-time configuration.
- `paths.config`: Path segment under `paths.root` used for persistent runner configuration.
- `kms_key_id`: Optional customer-managed KMS key ARN used by control-plane IAM policies to decrypt shared GitHub App parameters. The ARN may be unknown until apply; null omits the provider-owned KMS statements. It does not select encryption for runtime-created runner parameters.
- `tags`: Shared tags for SSM-related resources. These override module-level `tags` and are inherited by parameter and housekeeper resources.
- `parameters.tags`: Tags for Terraform-managed runner configuration parameters and temporary parameters created by the scale-up and pool Lambdas. These override module-level and `ssm.tags` values with the same key.
- `housekeeper.schedule_expression`: EventBridge schedule expression that invokes the SSM housekeeper.
- `housekeeper.state`: EventBridge rule state, such as `ENABLED` or `DISABLED`.
- `housekeeper.tags`: Tags for housekeeper resources, including the Lambda function, log group, EventBridge rule, and IAM role. These override module-level, `ssm.tags`, shared Lambda, and shared log tags when keys conflict.
- `housekeeper.lambda.artifact`: Component-owned SSM-housekeeper artifact selection. Set at most one of `zip` or `s3`; when neither is selected, the module uses its packaged runner control-plane archive. This selector does not inherit an orchestration-provider artifact.
- `housekeeper.lambda.artifact.zip`: Optional local path to the SSM-housekeeper Lambda archive.
- `housekeeper.lambda.artifact.s3`: Optional object key and version in the shared `lambda.artifact.s3.bucket`. Selecting S3 requires that common bucket.
- `housekeeper.lambda.artifact.s3.key`: Object key of the SSM-housekeeper Lambda archive.
- `housekeeper.lambda.artifact.s3.object_version`: Optional object version of the SSM-housekeeper Lambda archive.
- `housekeeper.lambda.memory_size`: Memory allocated to the SSM housekeeper Lambda in MB.
- `housekeeper.lambda.timeout`: SSM housekeeper Lambda timeout in seconds.
- `housekeeper.config.tokenPath`: Parameter Store token path cleaned by the housekeeper. When omitted, the configured runner token path is used.
- `housekeeper.config.minimumDaysOld`: Minimum parameter age in days before deletion is allowed.
- `housekeeper.config.dryRun`: Reports eligible parameters without deleting them when true. | 
object({
    paths = object({
      root   = string
      tokens = string
      config = string
    })
    kms_key_id = optional(string, null)
    tags       = optional(map(string), {})
    parameters = optional(object({
      tags = optional(map(string), {})
    }), {})
    housekeeper = optional(object({
      schedule_expression = optional(string, "rate(1 day)")
      state               = optional(string, "ENABLED")
      tags                = optional(map(string), {})
      lambda = optional(object({
        artifact = optional(object({
          zip = optional(string, null)
          s3 = optional(object({
            key            = string
            object_version = optional(string, null)
          }), null)
        }), {})
        memory_size = optional(number, 512)
        timeout     = optional(number, 60)
      }), {})
      config = optional(object({
        tokenPath      = optional(string)
        minimumDaysOld = optional(number, 1)
        dryRun         = optional(bool, false)
      }), {})
    }), {})
  })
| n/a | yes | @@ -124,6 +124,7 @@ yarn run dist | Name | Description | |------|-------------| +| [compute\_provider\_contract](#output\_compute\_provider\_contract) | Provider-neutral compute-provider capabilities consumed by topology-level orchestration. | | [orchestration\_provider](#output\_orchestration\_provider) | Resources grouped under the selected runner orchestration provider. | | [pool](#output\_pool) | Scheduled pool resources. Null when no pool configuration is supplied. | | [provider](#output\_provider) | Provider-specific resources grouped under the selected provider namespace and type. | diff --git a/modules/runner-config/orchestration-provider.tf b/modules/runner-config/orchestration-provider.tf index 16238fa6c6..bd804ad846 100644 --- a/modules/runner-config/orchestration-provider.tf +++ b/modules/runner-config/orchestration-provider.tf @@ -7,11 +7,16 @@ locals { orchestration_provider_type = one(keys(local.orchestration_providers)) orchestration_provider_enabled = { - webhook = local.orchestration_provider_type == "webhook" + webhook = local.orchestration_provider_type == "webhook" + scale_set = local.orchestration_provider_type == "scale_set" } orchestration_provider_runner_lifecycle = { webhook = one(module.orchestration_webhook[*].runner_lifecycle) + scale_set = { + ephemeral = true + jit_config_enabled = true + } }[local.orchestration_provider_type] } diff --git a/modules/runner-config/outputs.tf b/modules/runner-config/outputs.tf index 486e3261eb..d52230ba5f 100644 --- a/modules/runner-config/outputs.tf +++ b/modules/runner-config/outputs.tf @@ -22,13 +22,26 @@ output "pool" { output "orchestration_provider" { description = "Resources grouped under the selected runner orchestration provider." + value = merge( + { + webhook = local.orchestration_provider_enabled.webhook ? { + scale_up = one(module.orchestration_webhook[*].scale_up) + scale_down = one(module.orchestration_webhook[*].scale_down) + pool = one(module.orchestration_webhook[*].pool) + job_retry = one(module.orchestration_webhook[*].job_retry) + } : null + }, + local.orchestration_provider_enabled.scale_set ? { + scale_set = {} + } : {}, + ) +} + +output "compute_provider_contract" { + description = "Provider-neutral compute-provider capabilities consumed by topology-level orchestration." value = { - webhook = local.orchestration_provider_enabled.webhook ? { - scale_up = one(module.orchestration_webhook[*].scale_up) - scale_down = one(module.orchestration_webhook[*].scale_down) - pool = one(module.orchestration_webhook[*].pool) - job_retry = one(module.orchestration_webhook[*].job_retry) - } : null + type = local.provider_contract.type + capabilities = local.provider_contract.capabilities } } diff --git a/modules/runner-config/validations.tf b/modules/runner-config/validations.tf index f510bf0422..0c4ff5bcf0 100644 --- a/modules/runner-config/validations.tf +++ b/modules/runner-config/validations.tf @@ -90,7 +90,12 @@ resource "terraform_data" "validate_config" { for provider_name, provider_config in var.orchestration_provider : provider_name if provider_config != null ]) == 1 - error_message = "Exactly one orchestration provider must be configured. Supported providers: webhook." + error_message = "Exactly one orchestration provider must be configured. Supported providers: webhook and scale_set." + } + + precondition { + condition = var.orchestration_provider.scale_set == null ? true : local.provider_contract.capabilities.scale_set != null + error_message = "The selected compute provider must expose a scale_set capability when scale_set orchestration is selected." } precondition { diff --git a/modules/runner-config/variables.orchestration-provider.tf b/modules/runner-config/variables.orchestration-provider.tf index 44fa3525fd..d51136c85f 100644 --- a/modules/runner-config/variables.orchestration-provider.tf +++ b/modules/runner-config/variables.orchestration-provider.tf @@ -3,7 +3,8 @@ variable "orchestration_provider" { description = <<-EOT Runner demand-orchestration provider configuration. Exactly one provider block must be non-null. Wrapper presence selects the provider and must therefore be known during planning; values inside the selected provider may remain unknown until apply. - - `webhook`: Selects the workflow-job webhook control plane. It owns runner lifecycle and capacity, the build queue reference, the runner-control artifact, scale-up, scale-down, scheduled pool, and optional job-retry controls. Future providers can be added as sibling blocks without moving this contract. + - `webhook`: Selects the workflow-job webhook control plane. It owns runner lifecycle and capacity, the build queue reference, the runner-control artifact, scale-up, scale-down, scheduled pool, and optional job-retry controls. + - `scale_set`: Selects scale-set orchestration for this runner config. The multi-runner topology owns the shared controller service and passes this plan-known selection marker to runner-config. Scale-set runners always use ephemeral JIT registration. - `webhook.runner`: Runner lifecycle, boot timeout, and capacity settings owned by webhook orchestration. - `webhook.runner.boot_time_in_minutes`: Expected runner boot duration used by scale-down and pool controls. The default is `5`. - `webhook.runner.ephemeral`: Registers runners in ephemeral mode. The default is `false`. @@ -137,6 +138,14 @@ variable "orchestration_provider" { }), {}) }), {}) }), null) + scale_set = optional(object({ + name = string + runner = optional(object({ + min_runners = optional(number, 0) + max_runners = optional(number, 10) + boot_time_in_minutes = optional(number, 10) + }), {}) + }), null) }) nullable = false diff --git a/modules/ssm/README.md b/modules/ssm/README.md index e35c340071..9a3f98a379 100644 --- a/modules/ssm/README.md +++ b/modules/ssm/README.md @@ -31,6 +31,7 @@ No modules. | [aws_ssm_parameter.additional_github_app_key_base64](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) | resource | | [aws_ssm_parameter.additional_github_apps_manifest](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) | resource | | [aws_ssm_parameter.github_app_id](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) | resource | +| [aws_ssm_parameter.github_app_installation_id](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) | resource | | [aws_ssm_parameter.github_app_key_base64](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) | resource | | [aws_ssm_parameter.github_app_webhook_secret](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) | resource | @@ -39,7 +40,7 @@ No modules. | Name | Description | Type | Default | Required | |------|-------------|------|---------|:--------:| | [additional\_github\_apps](#input\_additional\_github\_apps) | Additional GitHub Apps for distributing API rate limit usage. | 
list(object({
    key_base64          = optional(string)
    key_base64_ssm      = optional(object({ arn = string, name = string }))
    id                  = optional(string)
    id_ssm              = optional(object({ arn = string, name = string }))
    installation_id     = optional(string)
    installation_id_ssm = optional(object({ arn = string, name = string }))
  }))
| `[]` | no | -| [github\_app](#input\_github\_app) | GitHub app parameters, see your github app.
You can optionally create the SSM parameters yourself and provide the ARN and name here, through the `*_ssm` attributes.
If you chose to provide the configuration values directly here,
please ensure the key is the base64-encoded `.pem` file (the output of `base64 app.private-key.pem`, not the content of `private-key.pem`).
Note: the provided SSM parameters arn and name have a precedence over the actual value (i.e `key_base64_ssm` has a precedence over `key_base64` etc). | 
object({
    key_base64 = optional(string)
    key_base64_ssm = optional(object({
      arn  = string
      name = string
    }))
    id = optional(string)
    id_ssm = optional(object({
      arn  = string
      name = string
    }))
    webhook_secret = optional(string)
    webhook_secret_ssm = optional(object({
      arn  = string
      name = string
    }))
  })
| n/a | yes | +| [github\_app](#input\_github\_app) | GitHub app parameters, see your github app.
You can optionally create the SSM parameters yourself and provide the ARN and name here, through the `*_ssm` attributes.
If you chose to provide the configuration values directly here,
please ensure the key is the base64-encoded `.pem` file (the output of `base64 app.private-key.pem`, not the content of `private-key.pem`).
Note: the provided SSM parameters arn and name have a precedence over the actual value (i.e `key_base64_ssm` has a precedence over `key_base64` etc). | 
object({
    key_base64 = optional(string)
    key_base64_ssm = optional(object({
      arn  = string
      name = string
    }))
    id = optional(string)
    id_ssm = optional(object({
      arn  = string
      name = string
    }))
    installation_id = optional(string)
    installation_id_ssm = optional(object({
      arn  = string
      name = string
    }))
    webhook_secret = optional(string)
    webhook_secret_ssm = optional(object({
      arn  = string
      name = string
    }))
  })
| n/a | yes | | [kms\_key\_arn](#input\_kms\_key\_arn) | Optional CMK Key ARN to be used for Parameter Store. | `string` | `null` | no | | [path\_prefix](#input\_path\_prefix) | The path prefix used for naming resources | `string` | n/a | yes | | [tags](#input\_tags) | Map of tags that will be added to created resources. By default resources will be tagged with name and environment. | `map(string)` | `{}` | no | diff --git a/modules/ssm/outputs.tf b/modules/ssm/outputs.tf index e1afaf990e..f37b28d939 100644 --- a/modules/ssm/outputs.tf +++ b/modules/ssm/outputs.tf @@ -8,6 +8,10 @@ output "parameters" { name = var.github_app.key_base64_ssm != null ? var.github_app.key_base64_ssm.name : aws_ssm_parameter.github_app_key_base64[0].name arn = var.github_app.key_base64_ssm != null ? var.github_app.key_base64_ssm.arn : aws_ssm_parameter.github_app_key_base64[0].arn } + github_app_installation_id = var.github_app.installation_id_ssm != null || var.github_app.installation_id != null ? { + name = var.github_app.installation_id_ssm != null ? var.github_app.installation_id_ssm.name : aws_ssm_parameter.github_app_installation_id[0].name + arn = var.github_app.installation_id_ssm != null ? var.github_app.installation_id_ssm.arn : aws_ssm_parameter.github_app_installation_id[0].arn + } : null github_app_webhook_secret = { name = var.github_app.webhook_secret_ssm != null ? var.github_app.webhook_secret_ssm.name : aws_ssm_parameter.github_app_webhook_secret[0].name arn = var.github_app.webhook_secret_ssm != null ? var.github_app.webhook_secret_ssm.arn : aws_ssm_parameter.github_app_webhook_secret[0].arn diff --git a/modules/ssm/ssm.tf b/modules/ssm/ssm.tf index 9467a136e5..ab8a406f2d 100644 --- a/modules/ssm/ssm.tf +++ b/modules/ssm/ssm.tf @@ -16,6 +16,15 @@ resource "aws_ssm_parameter" "github_app_key_base64" { tags = var.tags } +resource "aws_ssm_parameter" "github_app_installation_id" { + count = var.github_app.installation_id_ssm != null || var.github_app.installation_id == null ? 0 : 1 + name = "${var.path_prefix}/github_app_installation_id" + type = "SecureString" + value = var.github_app.installation_id + key_id = local.kms_key_arn + tags = var.tags +} + resource "aws_ssm_parameter" "github_app_webhook_secret" { count = var.github_app.webhook_secret_ssm != null ? 0 : 1 name = "${var.path_prefix}/github_app_webhook_secret" diff --git a/modules/ssm/variables.tf b/modules/ssm/variables.tf index d7387ecc30..d1c0f41f36 100644 --- a/modules/ssm/variables.tf +++ b/modules/ssm/variables.tf @@ -17,6 +17,11 @@ variable "github_app" { arn = string name = string })) + installation_id = optional(string) + installation_id_ssm = optional(object({ + arn = string + name = string + })) webhook_secret = optional(string) webhook_secret_ssm = optional(object({ arn = string From 49db3b581092cb8c9b386338dd5b1a66dcd9fada Mon Sep 17 00:00:00 2001 From: Ederson Brilhante Date: Thu, 17 Sep 2026 23:44:15 +0200 Subject: [PATCH 07/20] test(examples): add scale-set orchestration example (#5405) ## Description Restore the multi-runner scale-set orchestration example removed by revert PR #5403, replacing merged PR #5378. The example provides the Terraform configuration, provider locks, outputs, and documentation needed to deploy an ECS scale-set controller with the EC2 runner compute provider. It is intentionally limited to the example and its generated/provider metadata; the reusable MiniStack fixture support and ECS/MockServer lifecycle smoke test are provided by the follow-up PRs. ## Test Plan - Terraform formatting passed through the repository hooks. - Merge-conflict checks passed. - Parent-branch Terraform checks passed after the idle-configuration correction. - No live AWS deployment was performed for this example-only PR; CI validation remains the authoritative deployment check. ## Related Issues Depends on #5350. Replaces #5378. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- .github/workflows/ministack.yml | 1 + .github/workflows/terraform.yml | 1 + .../.terraform.lock.hcl | 93 ++++++++ .../.terraform.lock.hcl.tofu | 150 +++++++++++++ examples/multi-runner-scale-set/README.md | 84 +++++++ examples/multi-runner-scale-set/main.tf | 208 ++++++++++++++++++ examples/multi-runner-scale-set/outputs.tf | 8 + examples/multi-runner-scale-set/providers.tf | 9 + examples/multi-runner-scale-set/variables.tf | 58 +++++ examples/multi-runner-scale-set/versions.tf | 17 ++ tests/ministack/README.md | 13 +- tests/ministack/multi-runner-scale-set.tfvars | 50 +++++ tests/ministack/multi-runner-v2.tfvars | 2 + tests/ministack/run-example.sh | 11 +- 14 files changed, 697 insertions(+), 8 deletions(-) create mode 100644 examples/multi-runner-scale-set/.terraform.lock.hcl create mode 100644 examples/multi-runner-scale-set/.terraform.lock.hcl.tofu create mode 100644 examples/multi-runner-scale-set/README.md create mode 100644 examples/multi-runner-scale-set/main.tf create mode 100644 examples/multi-runner-scale-set/outputs.tf create mode 100644 examples/multi-runner-scale-set/providers.tf create mode 100644 examples/multi-runner-scale-set/variables.tf create mode 100644 examples/multi-runner-scale-set/versions.tf create mode 100644 tests/ministack/multi-runner-scale-set.tfvars diff --git a/.github/workflows/ministack.yml b/.github/workflows/ministack.yml index 80af872e42..aa6e2b505d 100644 --- a/.github/workflows/ministack.yml +++ b/.github/workflows/ministack.yml @@ -72,6 +72,7 @@ jobs: - ephemeral - multi-runner - multi-runner-v2 + - multi-runner-scale-set - migration-test - termination-watcher services: diff --git a/.github/workflows/terraform.yml b/.github/workflows/terraform.yml index a2baab51dd..22af6b7a90 100644 --- a/.github/workflows/terraform.yml +++ b/.github/workflows/terraform.yml @@ -23,6 +23,7 @@ env: modules/download-lambda modules/lambda modules/multi-runner + modules/orchestration-providers/scale-set modules/orchestration-providers/webhook modules/orchestration-providers/webhook/job-retry modules/orchestration-providers/webhook/pool diff --git a/examples/multi-runner-scale-set/.terraform.lock.hcl b/examples/multi-runner-scale-set/.terraform.lock.hcl new file mode 100644 index 0000000000..c96d2b19bf --- /dev/null +++ b/examples/multi-runner-scale-set/.terraform.lock.hcl @@ -0,0 +1,93 @@ +# This file is maintained automatically by "terraform init". +# Manual edits may be lost in future updates. + +provider "registry.terraform.io/hashicorp/aws" { + version = "6.63.0" + constraints = ">= 5.0.0, >= 6.21.0, >= 6.33.0" + hashes = [ + "h1:9cre7jh1lSs/9igpgAcENMUAUlYW3HCtkav3up4oit0=", + "h1:dRlYHkc+r6fgzF57WC7Zjcmb6sF/6TTGDEgwGK+LAZY=", + "zh:005d56736afd17d963998c405cee6f434dbc23a415109f9435ff1542879ae611", + "zh:026ef126321a86ad7080b5d858e2527f96f5289678cbcd8856296e229c43339d", + "zh:06e0b58b2d1eddb5137fc86bee7ad2d07953c0bc3f57cccfc5ae0d2456068a3a", + "zh:07221735d61ababed84734e5ffcfc5bd59d01f29f029166ba5f2175895dceed1", + "zh:1a72db00583112bdb8c19b213a78a3f5de754fffc08f07e061f4e326289fab7d", + "zh:32968e74a53b03e97a084dc7050c22ef661fb5b3ea8a44f5a63e47bc45ad0e7c", + "zh:4b357dfe4b820e3e4acd2881cff8288b2186491e63416751f0d12692ba478ceb", + "zh:81e30884d7de686265e7d87bb92527e802878c65a378470ede2a1e9f4e40ccc9", + "zh:82e137297f6a5a08b9ce2138f7aabea245ad99495d9d9eff502f752d6ca90dbd", + "zh:8eb83b67099f0ea9df238a979dff933ff50ce06a2e3ff05a48556a10f10dd204", + "zh:9b12af85486a96aedd8d7984b0ff811a4b42e3d88dad1a3fb4c0b580d04fa425", + "zh:d0ba30886cbe41850fee689f51ef9088578f323cfd21817bb409951d43c465eb", + "zh:dd48e7089784454bc03d713e9057f5ca0ea1613bd402125054a51894957b7925", + "zh:f250fa81e54cf60fcb0e9c0fc4ac043f1ecc2ac24967f628b3609364fcab3d04", + "zh:f38fc09fc25a8d2cf89a4d4cd6a5ef7cb1aad72798dbdcad58b8876b6a551a54", + "zh:f7c7380fdf126e1901f2084588dbfd724c76cb131ccfa795a541219111103c06", + ] +} + +provider "registry.terraform.io/hashicorp/local" { + version = "2.9.0" + constraints = "~> 2.0" + hashes = [ + "h1:9rBZCMNpxKwMlRbWH2QpwD3kqUCAejdOZQ/aiiDObXQ=", + "h1:m24fjcInWvTVZ1XSo2MaNuKPe+X/gfG8SIi09rA7a7M=", + "zh:0baa4566cf77f1ff52f4293d1c8536202dd23edc197c3196413a28343c3ac3a0", + "zh:16b5559c3c07088ddad11a9bb9e9c0799999363c2958e9a5be2bcbbf2cd9ca64", + "zh:197c79015a10d1cce904a8ea722cbc750c42aeae2da53f44a6a0751d9fd1aa90", + "zh:29d0b03e5343a80677ebfeb2e2c31cbe4b1f65e736e53417454a4277fec2544c", + "zh:4896bfa6cf1d2fd562b47ef2e87f47862ae92a04f8ad5d764380f0c6653473b8", + "zh:531f8529cbca49f681883e57761a05a8398afaef6d1ab0d205d26bf12f4428e8", + "zh:6aaf5011d83161c86d2bfb80c0923ec934e578288758da2f37acb7aec129004b", + "zh:7430275253d3d3c40aa6179e0ec0d63212874dbbc06c5a51b9d07ec590f9756c", + "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", + "zh:be17dc611e95e26cdf6cad79dfccf1064f0e32032a2efeb939a9bbe7fb1cbfe9", + "zh:f0e3b0aa644202e1d79d2000dca91f6019425da71e9800fa23f27e51c034f195", + "zh:f62bae4519e4ead49182ddc8afe8cf61e2a4c3ba3973b0fbba967736a2696aa3", + "zh:fcafa360a5b0b96244f26f4e3a6d642b716a376557142c2442ff2fb12d11da18", + ] +} + +provider "registry.terraform.io/hashicorp/null" { + version = "3.3.1" + constraints = "~> 3.0, ~> 3.2" + hashes = [ + "h1:TuxJq10DVnRP7c5HBZPyyvQGcckNVfijyU1eXEu5e4M=", + "h1:m5FqidbIgh+E9OigiZh8/xbkvpUQFSj3hZo/jqNLCLQ=", + "zh:08c59776542ea16e5a8545752787b17ff412922182b4cfabe16139197be8ac44", + "zh:123109cc7e5ed6d515787fbc212f2a3fd5e75647bb24ab7c801ccd4d4ed42451", + "zh:14b3fa4372754b54844b41d5dbd4671a292d8d6828b90169061feb4d7b15dd05", + "zh:56a4daaa3212f57b764bf3d1f333141c6610c5f21abb240e0111221f7c7fa4d4", + "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", + "zh:7e888a026dbacd2474a42264227ae35f639780f0f0c613529d10a95cd61988b3", + "zh:85a53646267e87d600df7124e4767ffde9bba3b6356d45d961618bdd68131cc7", + "zh:8ffa0e9c7c39b2ab0905b472465d6e35ef0b776b3f6273bb34c150340b61bff1", + "zh:9846510a1841530d4403f4818e233f91e3b3bade7441047599fbf800742f65be", + "zh:afa98d44860875f037c6def0a7e6ff208e042712ba771f620482b143cd336891", + "zh:bdca130d9ef27488ae0b13bc8fd8019e8bbdd4f2ceff29da066bd333165d68c5", + "zh:cb3b94cbca88210dd0d1f11e2b8a89333f48c3857faf8f70f589072ce7c28610", + "zh:f0c0ba87925fe32f84b80f7513b1efb1b0866f51f899ba825e95ad59ff09b018", + ] +} + +provider "registry.terraform.io/hashicorp/random" { + version = "3.9.0" + constraints = "~> 3.0" + hashes = [ + "h1:OO+IuvQJSPmWdN8AyyIEvPJbLvDQpgX/zbktoa9KsJE=", + "h1:UlBuNVuCGJ39tTv2c5gz2NRZnQbXfbIWbTzWcth5o74=", + "zh:161ad0bd9a75768c82f53fb6e7172a9d8be2d4889b012645a34795031aaf1bf1", + "zh:19dc9a5b17729725ccfc4f45b0500af0ee5bc6b6b160c7adb8f2bf617d2c80ea", + "zh:269eda8fe42daa7974d5a34d166c3ba9defe80cde86c01e4dadcfdf2e1f05e5f", + "zh:373f7c65566f8f2cc7f45d698654feb9d988996957e1266a69ca00c52d6d16d0", + "zh:5599d16804c41c83009ec621b6d6b6f74e102f5827678a4750f8809055546b61", + "zh:583be0440469a22bff70dcfa56593b01566860b29607437264adb51060cf46fc", + "zh:5f211d8ec3f2e1f414870d9584bfe26e6995560ef81c748f8447a48164767398", + "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", + "zh:7b547fd16216761ef86efc3ed516ac5ac0c5c42b7c7eb24a08cef2d93f69ed5e", + "zh:7e7c0679daf2a382151d05068c8c3f0dae6b7b7dccf818827b73dd08638df2ef", + "zh:8089dec888a8038b9b4fb23b3df7e1057293dbc5b60b42cc47ff690d69d4b61b", + "zh:c51f15a031edfd6f23ce8ced3446ca7f8d8d647e2499890d7d5d10d5016d7257", + "zh:c94784f005708890dc6895afd53636ec00ec1e430b15d41e5aebfb1d4b39bd04", + ] +} diff --git a/examples/multi-runner-scale-set/.terraform.lock.hcl.tofu b/examples/multi-runner-scale-set/.terraform.lock.hcl.tofu new file mode 100644 index 0000000000..577519c08d --- /dev/null +++ b/examples/multi-runner-scale-set/.terraform.lock.hcl.tofu @@ -0,0 +1,150 @@ +# This file is maintained automatically by "tofu init". +# Manual edits may be lost in future updates. + +provider "registry.opentofu.org/hashicorp/aws" { + version = "6.63.0" + constraints = ">= 5.0.0, >= 6.21.0, >= 6.33.0" + hashes = [ + "h1:1jhQJPHOPu2mzDG/ke3tK8PNcEqQHA4vhF05WWlM/yg=", + "h1:3+pvT0KN/bkJ6TBuExj+gxptEozhnpo80Ztblwq85eo=", + "h1:5aTequ87wZS7Mh4dEIayDGKcFdaFgHtw74NtqY5Idi0=", + "h1:AMRlrrM3z1SmrslOtotqKq02zapxLKtXaSN9Jbs0Oho=", + "h1:OTjECFWTDxsjcUfOKCNBp75Z5lGrW/KplRDsjTZYT2g=", + "h1:b8LORLOKMOOl+nK1M2UhCjELSjjziClJuAv6hYuySHs=", + "h1:bUfTX1giRLOyfDbBvsDbwR3tJmsTFRWcOTQdj2npDWA=", + "h1:dzs4kwx+itVGAH7yEOyeoWcE3LNRMnWtlt4ROgyAa0M=", + "h1:lnjou+SiwpYJ+j9PXWozXPHSPlhxIZb0RqpsSEBzfGw=", + "h1:pqzUeHAQj9NctgkwaynaF2aB+3QiZXcoslzMGjT743w=", + "h1:qTXEWOWxA6sfUpC29UXrsbHnNzWH7+j1RTUVG4YCm+U=", + "h1:qdHKOKt/ISn9RLjUe22OZBpN3F7H2DFeHJL/CSc2x8E=", + "h1:tpNzIZBzzUW7/kLU3BhYf3jhdO5uNwYfNmgC9B8kvMM=", + "h1:uVVlFgjg6GyxJLbCsTO1+R5fTNbZ73mLpVpSd0mMrFk=", + "h1:xGJsV5IFf7c11cXzJrsY40hiJCghp4odT0eJyTyAUYY=", + "zh:039a03e920e55f14a691feb67216a2d142bfee603128e15f9c5138f9ecd85016", + "zh:14e060b7f46ca7b0fa009b91aef419c58cbdff854de96e9a1d853166f8d902fd", + "zh:18803e8fe2c291c8db5526c71b3287ff7c81453f10ca6d8e69cdf9c535b00783", + "zh:1b83fce6e31a6095e932d80a7c3f47ac04252653a2de2b98ec6204563310fcba", + "zh:2add7bc976ceebb1a94d84598762c9b9cf281ca52ec83deeb4e95e90aa200a12", + "zh:2f22cd5372408f11937fa5513a7b960d3cebc334c5ec65fc5322c3bac1c1f664", + "zh:41c5e857dacfd83b7ca12a435204957ff6ca8830b9efefd0d381ad4d63b19779", + "zh:4eace6246e46999782d219bc4f50f83d19ef9156bacf5ca1528da12da4918015", + "zh:5e1c1281c3f929399e2ed3dbdce03426fd57a9ec55cd36e04acf1712aa5954ba", + "zh:608272b1f5d75ead123c9d933aa1fed7dc832cedd1506019046b4c8fdcc91dce", + "zh:6b3680f8a2f7be2c171953aba89d639fb2624b9cf52ec304e16434874566601d", + "zh:99aa1006f2141f3341a02020e1c91abfb02280e57c77e0415c98b8d900353d88", + "zh:9ad235bef34a89a8dd9943f9fa9f05cc729bb52a4e0dc926a31bb13cb0ae2418", + "zh:e0e3ac361e04748a4ca0c1cdbb6abab2aa817f4ad67e1692817d16e370161d59", + "zh:f60962c982a41fde956e796425e7194b4311741c179c060c1c8b5e16a557d635", + ] +} + +provider "registry.opentofu.org/hashicorp/local" { + version = "2.9.0" + constraints = "~> 2.0" + hashes = [ + "h1:1dtKYW/5a1qob3yneL6WzOlnSGfYtJ6a2XeejCk9yb4=", + "h1:5NseXq5wU8O20ersTtV4ocrLYFFtgFr7n0pRLO1W2Rw=", + "h1:5d22ZPPK4iiygPbwRz/PJF5Es/0axVpMlPRpCR0Padw=", + "h1:AnwyolirmIlBMjH6+tV8bKkvT+5axJNYxi2y2IguiX4=", + "h1:PBp+HeseY021Fw3sLznCG27idgwPoff4cBuNmKgPL2w=", + "h1:VDxIhe4GbzdOCdmt7mQaqdwERQW6GSI7Roonts42Gr0=", + "h1:ZO6eWWnf8LjjV1q/JNeL9WLtZ6fwIttOnyN5LjCNSEo=", + "h1:dPIAf8oUAz+vW2E0iZunMvpuPddRZIztRsPSY1u+VnY=", + "h1:fwTDVG9AhFVKQZIb1EXkHv4FqzsZNlLWgkyPGDmZZEE=", + "h1:kDc465XPC7/6XFCjrMC4mTqhA9ef0FHKuJ3ZgfGNfeg=", + "h1:kGbjxrI2P8MHeyVtE1U3Q1TbyF71ExnHxtkrE+Aj6UU=", + "h1:kcoK6Afbsj54u9zaEqpecWAFKytqjBijtguCNwV3d4M=", + "h1:rxomJjDwOo+YZ+WIPc25FqEgsz9orh/2MCyUcZmFjvw=", + "h1:t0CMn/Rkwquw8l2yQ+O4ApzbMZfY2UazbsDnZygzACA=", + "h1:tJwgm2BS4xCGlElCDQEFXQoefY9Y4t0JdSKTtsPBbBo=", + "zh:13ef7ecd1e397ec5b20ea588508dd3e3b8d6c50d809ae76b079abf9dd8d02e4b", + "zh:2190c9325980076489ce02b0f5dd2c0b91fc8711cefa99e714d8619a32827ad1", + "zh:2a0cfc5600730093705071707e4a4e4e953e7d9091859e0f66b46daa1060dd5d", + "zh:2ff53eac1af43ab9a2248a0e53c963d46e19cf04bc4c3f323591cfcebb218252", + "zh:4ebc3dee700f60af9da29970052fd02fa947813162b224716862dc9d7f1f7542", + "zh:5fe6dab84ceeaa8eb3f1567c5f05578333370c472240ca5c5bfc25e92d4d5586", + "zh:66bbec16367bbf440045502c9779b11f4ac5b022c8d8d17afe12d431950838b5", + "zh:7641e5c2e4b529e869cde29ab5b1de2fd1091489eb745b19ac2709bd7f4dfd84", + "zh:855bfba0756d17ce07595ff57d7cf664443d1495127cb88fb063362734b8b22a", + "zh:aaec10f237921d60c581d1b7a66f0a8a8019d9802dc04af11b5b981f6682e01d", + "zh:e460835a38ffa1e74f6929904bfd14ef473d217fd537b7ce834abe5ce5e2ce07", + "zh:ecc4295215db0e4aea3c9329611c31e09a853e1ae207d56742403bd4f5516703", + "zh:ee6d9fae63a612072e00402894e14826af7a3351c235b9c5b423b7629a77ca29", + "zh:f2b5c8db74aa7ebcf7cd423672358437d42401675069ef67b01ff910054e49d5", + "zh:f5aff74d3eb96d4592c7bca5cd3ea89b469e84efbf382944bd0f844a57059c09", + ] +} + +provider "registry.opentofu.org/hashicorp/null" { + version = "3.3.1" + constraints = "~> 3.0, ~> 3.2" + hashes = [ + "h1:2wld81FnmHW0WVgy081sIfokCr2+NuatS8yjeLEet7Y=", + "h1:AClQjJ6X22V4qcRgcYSxiXCMmp2pz0G8WVQC7wAx66o=", + "h1:AY3XQbuviNd2X5VhHYEbhNta1m/CG3JD2BKFKhCt1Y4=", + "h1:CUOZUd7H11lsU+4tISlnYIiP5BqnX8IDwFCVqfLJyAg=", + "h1:JIfV0nA/pLWnIFGscvTfuavQCn2NeHxJBeb6UUg/joA=", + "h1:RejAh+nyCwqDGExGln2Kb4Ro5LyHak0eJe0P9g8CHPc=", + "h1:SHOuTZjYymsmy4asuRq6NC3yW+zdVZOOt4f5nrb+EPM=", + "h1:WwPat/gT4gO8GvvKNdSkkXWVD65JppLJfqKOt9HhOqQ=", + "h1:Z3hXVLrOyaRiiLmmL5UCOdcRMguwjN1x5TYNdmBDgls=", + "h1:dd78Ad5HdfPzPts7A9qIxfitXhAriV/qza38fr2ukjk=", + "h1:dyVb++KwDdybzLTE6bf7GZiVQ31iWsgKPWmhTQ8G42k=", + "h1:gD8ZH6WWe+5gg5+y8SpLWGPUDzSxcQ3HKP8IDM/wW3I=", + "h1:juXCww0zRQKFTDZoKqYR0+Sn1lu99oeL6pr0Jh6LWx0=", + "h1:kFAySmtsshyNV7IhIrEdASzVcvwy68eeZCVC66P7yNk=", + "h1:nS5azDopRisB2NInwDx3Hrfg2FdVt8Gw0gTQzC0rd70=", + "zh:164eb061d84e01759f391265865fb31828083d0a06b25f7af7e094cbdb18c799", + "zh:1bb9b669a82b52c0cba2860c71e9ee6699ef302f28cb8ed06f572d39bc6c7c4f", + "zh:1ea9b31a8f29302122c1e8d673693f3ac270336dae560af803cd1117265a469a", + "zh:238bd463cb0154fb935dc331da40c0a9cbe5db9cee615ae5f35ccad5eed7dc41", + "zh:30ef2b7384cf7e20f33fe75754b54cf669d59816f3ad4fc73bfb2b26fb6735e9", + "zh:35b5cded16e4b57c207d03ee0979b14baf486fa520e6edb7a2eecf18f1b85471", + "zh:3dc840d13a50cd215c7540573f27e2b61f739ba90aee5b7c3846079aa0ab5534", + "zh:3f9309a18db608f975d5691fcb47a6e14d77199156a52e9c39dcafe3737f2b07", + "zh:44263a219f7dbd1848b545d080110b4f7d0495e77b71cd3c7a0b5ec52a09accb", + "zh:4dec54aa5f445eeea035bbd4839bcded5e47ecd07cba0e70c5a09e9272cb592f", + "zh:5e8fb319d7c6d6c4566a18b9d0c91580b4901a96acd7fdc476bfc79f074368e2", + "zh:b0e8b6d41834b57fcfbb5ca00da52ccb757e1a95b6a2d546c0dae8bfbeca1cdf", + "zh:bbde4c3a1dcc1718027a61a4cdf661619d17af1b58df1038fe27bcf43c3dc29b", + "zh:c4140fff9f692baf29236557f706f9515f93229413438527d764023a82301da3", + "zh:f8e9d83184e4bbeb97c6f0d569833007c48ba5a7ff334def201df4991d03a962", + ] +} + +provider "registry.opentofu.org/hashicorp/random" { + version = "3.9.0" + constraints = "~> 3.0" + hashes = [ + "h1:8EQU5KSxezcjo/phRSe69rDOI0lk4pSaggj7FsskYp8=", + "h1:Lw9im2VBBJQ3RyAbHPQ0rcvcmmcZWm3x+kIOpN+Tv9s=", + "h1:U8KXqGCoNI9/guYbTvzgdtVk3fRthoG0UXwm1JoEpIs=", + "h1:YXaVd4p6qXPPVaxIBaIDNXmBwT02ZqDn0qD+tYpw8sA=", + "h1:cOpc03fphEt/G9Rfc4jLL/fW0D7tgvlXqiDKPF4vuww=", + "h1:g09RR7T1xWkeGrZwWvWMT9ncJrFGr1k3CBD585UmO7w=", + "h1:gGDdPPibmw2EWROx+sh1RGLjR5+nPwZyrf6/N9jXfeM=", + "h1:haE7/nXCOhXKP4oXeEnER3t5CaVQWqujz4nBnpeTUv4=", + "h1:ieSVpfZS2lKuMr05ph0QsOVpCzg7uk3cgKBaXR+Ikug=", + "h1:ig2s1IS9IzehorRjvVAnKIsUUj8fkgyxct1L/kswcc4=", + "h1:j3lS+ZEERFnoab8t1ppDrScGVP/cgWbzlCrEYKTCXYw=", + "h1:lxezrKmOiQIySHAM+os8qLVq7hqufDr8h3Hpzvsk+78=", + "h1:lzRqBJAG+NETxHbEZUJ/YP3RMEjZBinTX7VmgH3lw60=", + "h1:tdSNWK5ApqUsgbdYieyeYLTu6nIZUV3hR1oFqUfAuGo=", + "h1:xedet8yH/zI2CfdxsGlK0nlFWc/Bp61yrWsEa3fHB8g=", + "zh:03f1114cc20b8913523735ab76e0f0a2b16ce13c92923a53304bf85f07fc0dbc", + "zh:105b678ee72322a3067f105d7e05e940f6143238f377f6e87ff4ec909246ac2a", + "zh:55f3bbf13ea18cbace61a706566a80f25f33fe2b1780b6f3d7b582af2a05b6d2", + "zh:63adf996db48f082f7a6351eb485e219cd88795fc71e6ec60a837263ab0d2cb1", + "zh:7e99550738a4e3cc68b8a467714b0d69371025fe95e3326d5323d026d55653e9", + "zh:8342b54af3a18a37e075eeae61be57f4de2ba71b35d95c5075d402dd2c1f289d", + "zh:83ee18e32ac9dd5fc91298554b7c4cfa4c3a1db50f4c797945637cc93c0844ae", + "zh:993ecc0adbf6bd535a59fbc9b735d8c33950e6f6eb5e621d750da9b71d65d80a", + "zh:ad722bc59d4edbf1415e827fc007c0efe6e0e9462d5568bae20b34be1058a261", + "zh:ae9448e1f87b2f9a6c5197a0e9862162ec6b137cb3a3835e11522995d8939e7c", + "zh:bc9cdd3aac784f759125c6627f6f6416e8726a1c184eb9cf3e55b9edbc94c627", + "zh:c8e35b89572ba1c40a9b20022e033a3395fb8d42e7604d50c900f193ba10382e", + "zh:e2deaa8a9975ef81d9f62baed12c41286918b0a10908e0e031f13f69a3b730a1", + "zh:ee39707557210a0ab1098aa357d2cdfe502e5a312d0dbdffb09d08facc4d3fc5", + "zh:f81afe4eb63e8aa9e0ea71be6c990f0dc69cb360e7191c0742a991f4a5081b64", + ] +} diff --git a/examples/multi-runner-scale-set/README.md b/examples/multi-runner-scale-set/README.md new file mode 100644 index 0000000000..6c0a4e686c --- /dev/null +++ b/examples/multi-runner-scale-set/README.md @@ -0,0 +1,84 @@ +# Multi-runner scale-set example + +This example demonstrates the experimental multi-runner v2 interface. Shared +defaults are configured with `global_config*` variables, while +each runner lane uses `multi_runner_config` for its matcher, +runner lifecycle, and compute-provider settings. + +The example creates four lanes from one deployment: + +- Linux ARM64 Amazon Linux runners. +- Ephemeral Linux x64 Amazon Linux runners with job retry enabled. +- Linux x64 runners managed by a GitHub Actions scale set. +- Windows x64 Server Core 2022 runners. + +The v2 interface keeps provider-owned settings inside the selected provider +configuration. For example, VPC and subnet settings are under +`global_config_compute_provider.aws.ec2`, while the per-lane +instance types and AMI filter are under each lane's compute provider block. + +The scale-set lane uses `orchestration_provider.scale_set`. Its controller +network is configured under the global scale-set block and its GitHub +installation ID is provided by `var.github_app`. + +Configure the GitHub App variables before applying: + +```bash +terraform init +terraform apply \ + -var='github_app={id="123456",key_base64="...",installation_id="123456789"}' \ + -var='scale_set={name="linux-scale-set",container={image="ghcr.io/github-aws-runners/terraform-aws-github-runner-scale-set-service@sha256:"}}' +``` + +The `github_app` value is sensitive and should be supplied through a secure +variable source in real deployments rather than committed to configuration. +The GitHub App must be installed for the configured GitHub account. + + +## Requirements + +| Name | Version | +|------|---------| +| [terraform](#requirement\_terraform) | >= 1.5.6 | +| [aws](#requirement\_aws) | >= 6.33 | +| [local](#requirement\_local) | ~> 2.0 | +| [random](#requirement\_random) | ~> 3.0 | + +## Providers + +| Name | Version | +|------|---------| +| [random](#provider\_random) | 3.9.0 | + +## Modules + +| Name | Source | Version | +|------|--------|---------| +| [base](#module\_base) | ../base | n/a | +| [runners](#module\_runners) | ../../modules/multi-runner | n/a | +| [webhook\_github\_app](#module\_webhook\_github\_app) | ../../modules/webhook-github-app | n/a | + +## Resources + +| Name | Type | +|------|------| +| [random_id.random](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/id) | resource | + +## Inputs + +| Name | Description | Type | Default | Required | +|------|-------------|------|---------|:--------:| +| [ami](#input\_ami) | Optional AMI configuration keyed by runner lane. | 
map(object({
    filter = optional(map(list(string)), { state = ["available"] })
    owners = optional(list(string), ["amazon"])
    id_ssm_parameter = optional(object({
      arn = string
    }), null)
    kms_key = optional(object({
      arn = string
    }), null)
  }))
| `{}` | no | +| [aws\_region](#input\_aws\_region) | AWS region to deploy to. | `string` | `"eu-west-1"` | no | +| [environment](#input\_environment) | Environment name, used as prefix. | `string` | `null` | no | +| [github\_app](#input\_github\_app) | GitHub App ID, base64-encoded private key, and installation ID. | 
object({
    id              = string
    key_base64      = string
    installation_id = optional(string, null)
  })
| n/a | yes | +| [runner\_binaries\_enabled](#input\_runner\_binaries\_enabled) | Whether runner binary synchronization is enabled. | `bool` | `true` | no | +| [scale\_set](#input\_scale\_set) | GitHub Actions scale-set configuration. | 
object({
    name = string
    container = optional(object({
      image = optional(string, null)
    }), {})
  })
| n/a | yes | + +## Outputs + +| Name | Description | +|------|-------------| +| [webhook\_endpoint](#output\_webhook\_endpoint) | n/a | +| [webhook\_secret](#output\_webhook\_secret) | n/a | + diff --git a/examples/multi-runner-scale-set/main.tf b/examples/multi-runner-scale-set/main.tf new file mode 100644 index 0000000000..3bf1fd02a4 --- /dev/null +++ b/examples/multi-runner-scale-set/main.tf @@ -0,0 +1,208 @@ +locals { + environment = var.environment != null ? var.environment : "multi-runner-v2" + aws_region = var.aws_region +} + +resource "random_id" "random" { + byte_length = 20 +} + +module "base" { + source = "../base" + + prefix = local.environment + aws_region = local.aws_region +} + +module "runners" { + source = "../../modules/multi-runner" + + prefix = local.environment + aws_region = local.aws_region + + experimental_features = ["multi-runner-v2"] + + global_config = { + tags = { + Example = local.environment + Project = "ProjectX" + } + runner = { + os = "linux" + architecture = "x64" + extra_labels = ["v2"] + } + } + + global_config_github = { + app = { + key_base64 = var.github_app.key_base64 + id = var.github_app.id + installation_id = var.github_app.installation_id + webhook_secret = random_id.random.hex + } + } + + global_config_lambda = { + architecture = "arm64" + } + + global_config_orchestration_provider = { + webhook = { + eventbridge = { + enabled = true + accept_events = ["workflow_job"] + } + } + scale_set = { + grouping = { + strategy = "runner_config" + } + container = var.scale_set.container + network = { + vpc_id = module.base.vpc.vpc_id + subnet_ids = module.base.vpc.private_subnets + } + } + } + + global_config_compute_provider = { + aws = { + ec2 = { + vpc_id = module.base.vpc.vpc_id + subnet_ids = module.base.vpc.private_subnets + ssm_enabled = true + runner_binaries = { + enabled = var.runner_binaries_enabled + } + } + } + } + + multi_runner_config = { + linux-arm64 = { + runner = { + architecture = "arm64" + name_prefix = "amazon-arm64-" + extra_labels = ["amazon"] + } + orchestration_provider = { + webhook = { + runner = { + maximum_count = 1 + } + matcherConfig = { + exactMatch = true + labelMatchers = [["self-hosted", "linux", "arm64", "amazon"]] + } + } + } + compute_provider = { + aws = { + ec2 = { + instance_types = ["t4g.large", "c6g.large"] + ami = lookup(var.ami, "linux-arm64", null) + } + } + } + } + + linux-x64 = { + runner = { + name_prefix = "amazon-x64-" + extra_labels = ["amazon"] + } + orchestration_provider = { + webhook = { + runner = { + ephemeral = true + maximum_count = 1 + } + matcherConfig = { + labelMatchers = [["self-hosted", "linux", "x64", "amazon"]] + exactMatch = false + priority = 1 + } + queue = { + delay_webhook_event = 0 + } + job_retry = { + enabled = true + } + } + } + compute_provider = { + aws = { + ec2 = { + instance_types = ["m5a.large", "m5ad.large"] + ami = lookup(var.ami, "linux-x64", null) + } + } + } + } + + linux-scale-set = { + runner = { + name_prefix = "scale-set-" + extra_labels = ["scale-set"] + } + orchestration_provider = { + scale_set = { + name = var.scale_set.name + runner = { + min_runners = 0 + max_runners = 10 + boot_time_in_minutes = 10 + } + } + } + compute_provider = { + aws = { + ec2 = { + instance_types = ["m5.large"] + ami = lookup(var.ami, "linux-scale-set", null) + } + } + } + } + + windows-x64 = { + runner = { + os = "windows" + name_prefix = "windows-x64-" + } + orchestration_provider = { + webhook = { + runner = { + boot_time_in_minutes = 20 + maximum_count = 1 + } + matcherConfig = { + exactMatch = true + labelMatchers = [["self-hosted", "windows", "x64", "servercore-2022"]] + } + } + } + compute_provider = { + aws = { + ec2 = { + instance_types = ["m5.large", "c5.large"] + ami = lookup(var.ami, "windows-x64", null) + } + } + } + } + } +} + +module "webhook_github_app" { + source = "../../modules/webhook-github-app" + depends_on = [module.runners] + + github_app = { + key_base64 = var.github_app.key_base64 + id = var.github_app.id + webhook_secret = random_id.random.hex + } + webhook_endpoint = module.runners.webhook.endpoint +} diff --git a/examples/multi-runner-scale-set/outputs.tf b/examples/multi-runner-scale-set/outputs.tf new file mode 100644 index 0000000000..1feaf2e671 --- /dev/null +++ b/examples/multi-runner-scale-set/outputs.tf @@ -0,0 +1,8 @@ +output "webhook_endpoint" { + value = module.runners.webhook.endpoint +} + +output "webhook_secret" { + sensitive = true + value = random_id.random.hex +} diff --git a/examples/multi-runner-scale-set/providers.tf b/examples/multi-runner-scale-set/providers.tf new file mode 100644 index 0000000000..eca2fe96a7 --- /dev/null +++ b/examples/multi-runner-scale-set/providers.tf @@ -0,0 +1,9 @@ +provider "aws" { + region = local.aws_region + + default_tags { + tags = { + Example = local.environment + } + } +} diff --git a/examples/multi-runner-scale-set/variables.tf b/examples/multi-runner-scale-set/variables.tf new file mode 100644 index 0000000000..c41fea1186 --- /dev/null +++ b/examples/multi-runner-scale-set/variables.tf @@ -0,0 +1,58 @@ +variable "github_app" { + description = "GitHub App ID, base64-encoded private key, and installation ID." + + type = object({ + id = string + key_base64 = string + installation_id = optional(string, null) + }) + sensitive = true +} + +variable "scale_set" { + description = "GitHub Actions scale-set configuration." + + type = object({ + name = string + container = optional(object({ + image = optional(string, null) + }), {}) + }) +} + +variable "environment" { + description = "Environment name, used as prefix." + + type = string + default = null +} + +variable "aws_region" { + description = "AWS region to deploy to." + + type = string + default = "eu-west-1" +} + +variable "runner_binaries_enabled" { + description = "Whether runner binary synchronization is enabled." + + type = bool + default = true +} + +variable "ami" { + description = "Optional AMI configuration keyed by runner lane." + + type = map(object({ + filter = optional(map(list(string)), { state = ["available"] }) + owners = optional(list(string), ["amazon"]) + id_ssm_parameter = optional(object({ + arn = string + }), null) + kms_key = optional(object({ + arn = string + }), null) + })) + default = {} +} diff --git a/examples/multi-runner-scale-set/versions.tf b/examples/multi-runner-scale-set/versions.tf new file mode 100644 index 0000000000..6af69ab915 --- /dev/null +++ b/examples/multi-runner-scale-set/versions.tf @@ -0,0 +1,17 @@ +terraform { + required_providers { + aws = { + source = "hashicorp/aws" + version = ">= 6.33" + } + local = { + source = "hashicorp/local" + version = "~> 2.0" + } + random = { + source = "hashicorp/random" + version = "~> 3.0" + } + } + required_version = ">= 1.5.6" +} diff --git a/tests/ministack/README.md b/tests/ministack/README.md index 3e6cdc82fa..706b5157a2 100644 --- a/tests/ministack/README.md +++ b/tests/ministack/README.md @@ -1,7 +1,8 @@ # MiniStack example tests The MiniStack workflow runs the `base`, `prebuilt`, `default`, `ephemeral`, -`multi-runner`, `multi-runner-v2`, and `termination-watcher` examples directly +`multi-runner`, `multi-runner-v2`, `multi-runner-scale-set`, and +`termination-watcher` examples directly with Terraform 1.5.6 and the latest Terraform release, and with OpenTofu 1.11 and the latest OpenTofu release. The examples with input variables get their inputs from their own tfvars files @@ -10,8 +11,8 @@ and uses the configuration checked into the example itself. No override files, setup module, or Terraform fixture configuration is checked in. The helper creates and removes a temporary AMI override for `default` and `ephemeral`, temporary SSM parameters for `multi-runner`, and temporary AMI -fixtures for `multi-runner-v2`. The migration test uses its dedicated -`run-migration-test.sh` lifecycle script. +fixtures for `multi-runner-v2` and `multi-runner-scale-set`. The migration test +uses its dedicated `run-migration-test.sh` lifecycle script. Start MiniStack, set the AWS endpoint and test credentials, then run: @@ -28,6 +29,8 @@ tests/ministack/run-example.sh apply multi-runner # or tests/ministack/run-example.sh apply multi-runner-v2 # or +tests/ministack/run-example.sh apply multi-runner-scale-set +# or tests/ministack/run-example.sh apply termination-watcher ``` @@ -36,8 +39,8 @@ ZIP fixtures in the paths expected by the modules when they are absent, and removes only the files it created. For `prebuilt`, it seeds AMI metadata through MiniStack's AWS-compatible EC2 API, then removes only the resources it created during cleanup. MiniStack v1.5.11 provides the EC2 image behavior needed by the -`default`, `ephemeral`, and `multi-runner` examples, so they are included in -the same lifecycle matrix. +`default`, `ephemeral`, `multi-runner`, and `multi-runner-scale-set` examples, +so they are included in the same lifecycle matrix. ## Webhook and runner lifecycle smoke test diff --git a/tests/ministack/multi-runner-scale-set.tfvars b/tests/ministack/multi-runner-scale-set.tfvars new file mode 100644 index 0000000000..67bc39ea7a --- /dev/null +++ b/tests/ministack/multi-runner-scale-set.tfvars @@ -0,0 +1,50 @@ +environment = "ministack-scale-set" +aws_region = "eu-west-1" + +github_app = { + id = "0" + key_base64 = "ministack-invalid-key" + installation_id = "1" +} + +runner_binaries_enabled = false + +ami = { + "linux-arm64" = { + filter = { + name = ["ministack-scale-set-linux-arm64"] + state = ["available"] + } + owners = ["self"] + } + "linux-x64" = { + filter = { + name = ["ministack-scale-set-linux-x64"] + state = ["available"] + } + owners = ["self"] + } + "linux-scale-set" = { + filter = { + name = ["ministack-scale-set-linux-x64"] + state = ["available"] + } + owners = ["self"] + } + "windows-x64" = { + filter = { + name = ["ministack-scale-set-windows-x64"] + state = ["available"] + } + owners = ["self"] + } +} + +scale_set = { + config_url = "https://github.com/example" + name = "ministack-scale-set" + id = 1 + runner_group_id = 1 + runner_owner = "example" + runner_registration_level = "organization" +} diff --git a/tests/ministack/multi-runner-v2.tfvars b/tests/ministack/multi-runner-v2.tfvars index 0f9c6073fc..de54e291cf 100644 --- a/tests/ministack/multi-runner-v2.tfvars +++ b/tests/ministack/multi-runner-v2.tfvars @@ -6,6 +6,8 @@ github_app = { key_base64 = "ministack-invalid-key" } +runner_binaries_enabled = false + ami = { "linux-arm64" = { filter = { diff --git a/tests/ministack/run-example.sh b/tests/ministack/run-example.sh index 961b3a8cab..ab16343d95 100755 --- a/tests/ministack/run-example.sh +++ b/tests/ministack/run-example.sh @@ -23,7 +23,7 @@ case "$iac_binary" in esac case "$example" in - base | prebuilt | default | ephemeral | multi-runner | multi-runner-v2) + base | prebuilt | default | ephemeral | multi-runner | multi-runner-v2 | multi-runner-scale-set) use_tfvars=true ;; migration-test) @@ -33,7 +33,7 @@ case "$example" in use_tfvars=false ;; *) - echo "Supported examples for the runner are: base, prebuilt, default, ephemeral, multi-runner, multi-runner-v2, migration-test, termination-watcher" >&2 + echo "Supported examples for the runner are: base, prebuilt, default, ephemeral, multi-runner, multi-runner-v2, multi-runner-scale-set, migration-test, termination-watcher" >&2 exit 64 ;; esac @@ -41,7 +41,7 @@ esac case "$action" in init | plan | apply | destroy) ;; *) - echo "Usage: $0 {init|plan|apply|destroy} {base|prebuilt|default|ephemeral|multi-runner|multi-runner-v2|migration-test|termination-watcher} [TFVARS_FILE]" >&2 + echo "Usage: $0 {init|plan|apply|destroy} {base|prebuilt|default|ephemeral|multi-runner|multi-runner-v2|multi-runner-scale-set|migration-test|termination-watcher} [TFVARS_FILE]" >&2 exit 64 ;; esac @@ -325,6 +325,11 @@ $lambda_zip" create_ami_fixture "ministack-v2-linux-x64" x86_64 >/dev/null create_ami_fixture "ministack-v2-windows-x64" x86_64 >/dev/null ;; + multi-runner-scale-set) + create_ami_fixture "ministack-scale-set-linux-x64" x86_64 >/dev/null + create_ami_fixture "ministack-scale-set-linux-arm64" arm64 >/dev/null + create_ami_fixture "ministack-scale-set-windows-x64" x86_64 >/dev/null + ;; esac } From c91d8cafb2e6713ccd10e23f5b160047fa598800 Mon Sep 17 00:00:00 2001 From: edersonbrilhante Date: Fri, 18 Sep 2026 00:09:33 +0200 Subject: [PATCH 08/20] fix(scale-set): remove invalid policy validation reference --- .../orchestration-providers/scale-set/README.md | 1 - .../scale-set/tests/scale-set.tftest.hcl | 2 +- .../scale-set/validations.tf | 14 -------------- 3 files changed, 1 insertion(+), 16 deletions(-) diff --git a/modules/orchestration-providers/scale-set/README.md b/modules/orchestration-providers/scale-set/README.md index cc7c90eb23..495a8cc314 100644 --- a/modules/orchestration-providers/scale-set/README.md +++ b/modules/orchestration-providers/scale-set/README.md @@ -218,7 +218,6 @@ No modules. | [terraform_data.validate_compute_role_policy](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | | [terraform_data.validate_config_store](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | | [terraform_data.validate_contract](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | -| [terraform_data.validate_group_compute_policy](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | | [terraform_data.validate_group_task_policy](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | | [terraform_data.validate_grouping](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | | [terraform_data.validate_runtime](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | diff --git a/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl b/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl index 870a73b9b2..eb61c8a6e8 100644 --- a/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl +++ b/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl @@ -891,7 +891,7 @@ run "rejects_controller_group_policy_above_inline_quota" { } override_data { - target = data.aws_iam_policy_document.task["ec2"] + target = data.aws_iam_policy_document.task values = { json = <<-JSON {"payload":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"} diff --git a/modules/orchestration-providers/scale-set/validations.tf b/modules/orchestration-providers/scale-set/validations.tf index d956e21f2f..9553581597 100644 --- a/modules/orchestration-providers/scale-set/validations.tf +++ b/modules/orchestration-providers/scale-set/validations.tf @@ -423,17 +423,3 @@ resource "terraform_data" "validate_compute_role_policy" { } } } - -resource "terraform_data" "validate_group_compute_policy" { - for_each = local.controller_groups - - lifecycle { - precondition { - condition = ( - floor(length(base64encode(data.aws_iam_policy_document.task_compute[each.key].json)) * 3 / 4) - - (endswith(base64encode(data.aws_iam_policy_document.task_compute[each.key].json), "==") ? 2 : endswith(base64encode(data.aws_iam_policy_document.task_compute[each.key].json), "=") ? 1 : 0) - ) <= 10240 - error_message = "Controller group ${each.key} produces a compute-provider task-role policy exceeding AWS's 10240-byte inline role-policy quota. Split the group or reduce provider IAM statements." - } - } -} From 5b52346df523346c37f564ad302e5370fc01e31f Mon Sep 17 00:00:00 2001 From: edersonbrilhante Date: Fri, 18 Sep 2026 00:49:51 +0200 Subject: [PATCH 09/20] ci: fix multi runner scaleset test --- .github/workflows/terraform.yml | 80 ++++++++++++++++--- examples/multi-runner-scale-set/main.tf | 2 + examples/multi-runner-scale-set/variables.tf | 9 +++ modules/multi-runner/main.tf | 2 +- tests/ministack/multi-runner-scale-set.tfvars | 5 ++ 5 files changed, 85 insertions(+), 13 deletions(-) diff --git a/.github/workflows/terraform.yml b/.github/workflows/terraform.yml index 22af6b7a90..e84a47873a 100644 --- a/.github/workflows/terraform.yml +++ b/.github/workflows/terraform.yml @@ -1,4 +1,5 @@ name: "Terraform checks" + on: push: branches: @@ -54,11 +55,13 @@ env: TEST_MODULES: | modules/runners modules/multi-runner + jobs: verify_modules: name: Verify modules (${{ matrix.iac.name }} ${{ matrix.iac.version }}) strategy: - fail-fast: false + fail-fast: true + max-parallel: 1 matrix: iac: - name: terraform @@ -73,7 +76,9 @@ jobs: - name: tofu-latest version: latest command: tofu + runs-on: ubuntu-latest + steps: - name: Harden the runner (Audit all outbound calls) uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 @@ -98,7 +103,7 @@ jobs: mkdir -p "$HOME/.terraform.d/plugin" echo "TF_PLUGIN_CACHE_DIR=$HOME/.terraform.d/plugin" >> "$GITHUB_ENV" - - name: "Fake zip files" # Validate will fail if it cannot find the zip files + - name: Fake zip files run: | touch lambdas/functions/webhook/webhook.zip touch lambdas/functions/control-plane/runners.zip @@ -126,7 +131,10 @@ jobs: run: | printf '%s\n' "${MODULES}" | while IFS= read -r module; do [ -z "${module}" ] && continue + + echo "::group::Running $IAC_COMMAND init for module: ${module}" $IAC_COMMAND -chdir="${module}" init -get -backend=false -input=false + echo "::endgroup::" done - name: Check ${{ matrix.iac.name }} formatting @@ -135,7 +143,10 @@ jobs: run: | printf '%s\n' "${MODULES}" | while IFS= read -r module; do [ -z "${module}" ] && continue + + echo "::group::Checking $IAC_COMMAND formatting for module: ${module}" $IAC_COMMAND -chdir="${module}" fmt -recursive -check=true -write=false + echo "::endgroup::" done continue-on-error: ${{ matrix.iac.version == 'latest' }} @@ -145,7 +156,10 @@ jobs: run: | printf '%s\n' "${MODULES}" | while IFS= read -r module; do [ -z "${module}" ] && continue + + echo "::group::Validating module: ${module}" $IAC_COMMAND -chdir="${module}" validate + echo "::endgroup::" done - uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 @@ -162,15 +176,23 @@ jobs: - name: Run TFLint run: | tflint --init -c ${GITHUB_WORKSPACE}/.tflint.hcl + printf '%s\n' "${MODULES}" | while IFS= read -r module; do [ -z "${module}" ] && continue - tflint -f compact -c ${GITHUB_WORKSPACE}/.tflint.hcl --var-file ${GITHUB_WORKSPACE}/.github/lint/tflint.tfvars --chdir "${module}" + + echo "::group::Running TFLint for module: ${module}" + tflint -f compact \ + -c ${GITHUB_WORKSPACE}/.tflint.hcl \ + --var-file ${GITHUB_WORKSPACE}/.github/lint/tflint.tfvars \ + --chdir "${module}" + echo "::endgroup::" done verify_examples: name: Verify examples (${{ matrix.iac.name }} ${{ matrix.iac.version }}) strategy: - fail-fast: false + fail-fast: true + max-parallel: 1 matrix: iac: - name: terraform @@ -189,7 +211,9 @@ jobs: version: latest command: tofu lockfile: .terraform.lock.hcl.tofu + runs-on: ubuntu-latest + steps: - name: Harden the runner (Audit all outbound calls) uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 @@ -235,7 +259,11 @@ jobs: run: | printf '%s\n' "${EXAMPLES}" | while IFS= read -r example; do [ -z "${example}" ] && continue - cp "examples/${example}/${IAC_LOCK_FILE}" "examples/${example}/.terraform.lock.hcl" + + echo "::group::Selecting lockfile for example: ${example}" + cp "examples/${example}/${IAC_LOCK_FILE}" \ + "examples/${example}/.terraform.lock.hcl" + echo "::endgroup::" done - name: ${{ matrix.iac.name }} init @@ -244,7 +272,11 @@ jobs: run: | printf '%s\n' "${EXAMPLES}" | while IFS= read -r example; do [ -z "${example}" ] && continue - $IAC_COMMAND -chdir="examples/${example}" init -get -backend=false -input=false -lockfile=readonly + + echo "::group::Running $IAC_COMMAND init for example: ${example}" + $IAC_COMMAND -chdir="examples/${example}" init \ + -get -backend=false -input=false -lockfile=readonly + echo "::endgroup::" done - name: Check ${{ matrix.iac.name }} formatting @@ -253,7 +285,11 @@ jobs: run: | printf '%s\n' "${EXAMPLES}" | while IFS= read -r example; do [ -z "${example}" ] && continue - $IAC_COMMAND -chdir="examples/${example}" fmt -recursive -check=true -write=false + + echo "::group::Checking $IAC_COMMAND formatting for example: ${example}" + $IAC_COMMAND -chdir="examples/${example}" fmt \ + -recursive -check=true -write=false + echo "::endgroup::" done continue-on-error: ${{ matrix.iac.version == 'latest' }} @@ -263,7 +299,10 @@ jobs: run: | printf '%s\n' "${EXAMPLES}" | while IFS= read -r example; do [ -z "${example}" ] && continue + + echo "::group::Validating example: ${example}" $IAC_COMMAND -chdir="examples/${example}" validate + echo "::endgroup::" done - uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 @@ -280,15 +319,23 @@ jobs: - name: Run TFLint run: | tflint --init -c ${GITHUB_WORKSPACE}/.tflint.hcl + printf '%s\n' "${EXAMPLES}" | while IFS= read -r example; do [ -z "${example}" ] && continue - tflint -f compact -c ${GITHUB_WORKSPACE}/.tflint.hcl --var-file ${GITHUB_WORKSPACE}/.github/lint/tflint.tfvars --chdir "examples/${example}" + + echo "::group::Running TFLint for example: ${example}" + tflint -f compact \ + -c ${GITHUB_WORKSPACE}/.tflint.hcl \ + --var-file ${GITHUB_WORKSPACE}/.github/lint/tflint.tfvars \ + --chdir "examples/${example}" + echo "::endgroup::" done terraform_test: name: ${{ matrix.iac.name }} test strategy: - fail-fast: false + fail-fast: true + max-parallel: 1 matrix: iac: - name: terraform @@ -297,7 +344,9 @@ jobs: - name: tofu version: latest command: tofu + runs-on: ubuntu-latest + steps: - name: Harden the runner (Audit all outbound calls) uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 @@ -342,7 +391,11 @@ jobs: run: | printf '%s\n' "${TEST_MODULES}" | while IFS= read -r module; do [ -z "${module}" ] && continue - $IAC_COMMAND -chdir="${module}" init -backend=false -input=false + + echo "::group::Running $IAC_COMMAND init for test module: ${module}" + $IAC_COMMAND -chdir="${module}" init \ + -backend=false -input=false + echo "::endgroup::" done - name: ${{ matrix.iac.name }} test @@ -351,5 +404,8 @@ jobs: run: | printf '%s\n' "${TEST_MODULES}" | while IFS= read -r module; do [ -z "${module}" ] && continue - $IAC_COMMAND -chdir="${module}" test -test-directory=tests - done + + echo "::group::Running $IAC_COMMAND test for module: ${module}" + $IAC_COMMAND -chdir="${module}" test -test-directory=tests -compact-warnings + echo "::endgroup::" + done \ No newline at end of file diff --git a/examples/multi-runner-scale-set/main.tf b/examples/multi-runner-scale-set/main.tf index 3bf1fd02a4..66c6274469 100644 --- a/examples/multi-runner-scale-set/main.tf +++ b/examples/multi-runner-scale-set/main.tf @@ -41,6 +41,8 @@ module "runners" { installation_id = var.github_app.installation_id webhook_secret = random_id.random.hex } + runner_registration_level = var.github_config.runner_registration_level + runner_owner = var.github_config.runner_owner } global_config_lambda = { diff --git a/examples/multi-runner-scale-set/variables.tf b/examples/multi-runner-scale-set/variables.tf index c41fea1186..14a5895d17 100644 --- a/examples/multi-runner-scale-set/variables.tf +++ b/examples/multi-runner-scale-set/variables.tf @@ -9,6 +9,15 @@ variable "github_app" { sensitive = true } +variable "github_config" { + description = "GitHub configuration for runner registration." + + type = object({ + runner_registration_level = optional(string, null) + runner_owner = optional(string, null) + }) +} + variable "scale_set" { description = "GitHub Actions scale-set configuration." diff --git a/modules/multi-runner/main.tf b/modules/multi-runner/main.tf index 99a41d5207..611774f7fb 100644 --- a/modules/multi-runner/main.tf +++ b/modules/multi-runner/main.tf @@ -5,7 +5,7 @@ locals { primary_app_id = coalesce(local.effective_config.github.app.id_ssm, module.ssm.parameters.github_app_id) primary_app_key_base64 = coalesce(local.effective_config.github.app.key_base64_ssm, module.ssm.parameters.github_app_key_base64) - primary_app_installation_id = local.effective_config.github.app.installation_id_ssm != null ? local.effective_config.github.app.installation_id_ssm : module.ssm.parameters.github_app_installation_id + primary_app_installation_id = coalesce(local.effective_config.github.app.installation_id_ssm, module.ssm.parameters.github_app_installation_id) github_app_parameters = { id = local.primary_app_id diff --git a/tests/ministack/multi-runner-scale-set.tfvars b/tests/ministack/multi-runner-scale-set.tfvars index 67bc39ea7a..88c5d9bf48 100644 --- a/tests/ministack/multi-runner-scale-set.tfvars +++ b/tests/ministack/multi-runner-scale-set.tfvars @@ -7,6 +7,11 @@ github_app = { installation_id = "1" } +github_config = { + runner_registration_level = "organization" + runner_owner = "example" +} + runner_binaries_enabled = false ami = { From 0710bc377ef5e405bc34f480f4db715ba85b8dbc Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Thu, 17 Sep 2026 22:56:37 +0000 Subject: [PATCH 10/20] docs: auto update terraform docs --- examples/multi-runner-scale-set/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/examples/multi-runner-scale-set/README.md b/examples/multi-runner-scale-set/README.md index 6c0a4e686c..232c761418 100644 --- a/examples/multi-runner-scale-set/README.md +++ b/examples/multi-runner-scale-set/README.md @@ -72,6 +72,7 @@ The GitHub App must be installed for the configured GitHub account. | [aws\_region](#input\_aws\_region) | AWS region to deploy to. | `string` | `"eu-west-1"` | no | | [environment](#input\_environment) | Environment name, used as prefix. | `string` | `null` | no | | [github\_app](#input\_github\_app) | GitHub App ID, base64-encoded private key, and installation ID. | 
object({
    id              = string
    key_base64      = string
    installation_id = optional(string, null)
  })
| n/a | yes | +| [github\_config](#input\_github\_config) | GitHub configuration for runner registration. | 
object({
    runner_registration_level = optional(string, null)
    runner_owner              = optional(string, null)
  })
| n/a | yes | | [runner\_binaries\_enabled](#input\_runner\_binaries\_enabled) | Whether runner binary synchronization is enabled. | `bool` | `true` | no | | [scale\_set](#input\_scale\_set) | GitHub Actions scale-set configuration. | 
object({
    name = string
    container = optional(object({
      image = optional(string, null)
    }), {})
  })
| n/a | yes | From ec123d2df861106c616f0a3c6d057a8fb8a6c989 Mon Sep 17 00:00:00 2001 From: edersonbrilhante Date: Fri, 18 Sep 2026 01:03:07 +0200 Subject: [PATCH 11/20] fix: fix tests --- .github/workflows/terraform.yml | 4 +++- examples/multi-runner-scale-set/main.tf | 2 +- examples/multi-runner-scale-set/variables.tf | 3 +-- modules/multi-runner/main.tf | 2 +- modules/multi-runner/orchestration-provider.scale-set.tf | 2 +- modules/runner-config/tests/pool.tftest.hcl | 1 - tests/ministack/run-example.sh | 6 +++--- 7 files changed, 10 insertions(+), 10 deletions(-) diff --git a/.github/workflows/terraform.yml b/.github/workflows/terraform.yml index e84a47873a..4243fa074e 100644 --- a/.github/workflows/terraform.yml +++ b/.github/workflows/terraform.yml @@ -51,10 +51,12 @@ env: termination-watcher multi-runner multi-runner-v2 + multi-runner-scale-set external-managed-ssm-secrets TEST_MODULES: | modules/runners modules/multi-runner + modules/orchestration-providers/scale-set jobs: verify_modules: @@ -408,4 +410,4 @@ jobs: echo "::group::Running $IAC_COMMAND test for module: ${module}" $IAC_COMMAND -chdir="${module}" test -test-directory=tests -compact-warnings echo "::endgroup::" - done \ No newline at end of file + done diff --git a/examples/multi-runner-scale-set/main.tf b/examples/multi-runner-scale-set/main.tf index 66c6274469..e6e6ac339a 100644 --- a/examples/multi-runner-scale-set/main.tf +++ b/examples/multi-runner-scale-set/main.tf @@ -1,5 +1,5 @@ locals { - environment = var.environment != null ? var.environment : "multi-runner-v2" + environment = var.environment aws_region = var.aws_region } diff --git a/examples/multi-runner-scale-set/variables.tf b/examples/multi-runner-scale-set/variables.tf index 14a5895d17..62a905fcd6 100644 --- a/examples/multi-runner-scale-set/variables.tf +++ b/examples/multi-runner-scale-set/variables.tf @@ -32,8 +32,7 @@ variable "scale_set" { variable "environment" { description = "Environment name, used as prefix." - type = string - default = null + type = string } variable "aws_region" { diff --git a/modules/multi-runner/main.tf b/modules/multi-runner/main.tf index 611774f7fb..0fe79b9f2b 100644 --- a/modules/multi-runner/main.tf +++ b/modules/multi-runner/main.tf @@ -5,7 +5,7 @@ locals { primary_app_id = coalesce(local.effective_config.github.app.id_ssm, module.ssm.parameters.github_app_id) primary_app_key_base64 = coalesce(local.effective_config.github.app.key_base64_ssm, module.ssm.parameters.github_app_key_base64) - primary_app_installation_id = coalesce(local.effective_config.github.app.installation_id_ssm, module.ssm.parameters.github_app_installation_id) + primary_app_installation_id = try(coalesce(local.effective_config.github.app.installation_id_ssm, module.ssm.parameters.github_app_installation_id), null) github_app_parameters = { id = local.primary_app_id diff --git a/modules/multi-runner/orchestration-provider.scale-set.tf b/modules/multi-runner/orchestration-provider.scale-set.tf index f18a7c1230..f0459e1d30 100644 --- a/modules/multi-runner/orchestration-provider.scale-set.tf +++ b/modules/multi-runner/orchestration-provider.scale-set.tf @@ -42,7 +42,7 @@ locals { module "orchestration_scale_set" { source = "../orchestration-providers/scale-set" - count = length(local.scale_set_runner_configs) > 0 && local.primary_app_installation_id != null ? 1 : 0 + count = length(local.scale_set_runner_configs) > 0 ? 1 : 0 prefix = var.prefix log_level = var.global_config_observability.logs.level diff --git a/modules/runner-config/tests/pool.tftest.hcl b/modules/runner-config/tests/pool.tftest.hcl index 5639c0d217..7e9ed2a19e 100644 --- a/modules/runner-config/tests/pool.tftest.hcl +++ b/modules/runner-config/tests/pool.tftest.hcl @@ -48,7 +48,6 @@ variables { key = "runners/linux/actions-runner.tar.gz" } } - ssm_enabled = true } } } diff --git a/tests/ministack/run-example.sh b/tests/ministack/run-example.sh index ab16343d95..17b516961b 100755 --- a/tests/ministack/run-example.sh +++ b/tests/ministack/run-example.sh @@ -357,14 +357,14 @@ case "$action" in ;; plan) iac_init - iac_example plan -input=false -parallelism=1 + iac_example plan -input=false -parallelism=1 -compact-warnings ;; apply) iac_init - iac_example apply -auto-approve -input=false -parallelism=1 + iac_example apply -auto-approve -input=false -parallelism=1 -compact-warnings ;; destroy) iac_init - iac_example destroy -auto-approve -input=false -parallelism=1 + iac_example destroy -auto-approve -input=false -parallelism=1 -compact-warnings ;; esac From 283484bd022b3bff91ad4f33c34c16eb5ec0a092 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Thu, 17 Sep 2026 23:15:09 +0000 Subject: [PATCH 12/20] docs: auto update terraform docs --- examples/multi-runner-scale-set/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/multi-runner-scale-set/README.md b/examples/multi-runner-scale-set/README.md index 232c761418..6a32ada69d 100644 --- a/examples/multi-runner-scale-set/README.md +++ b/examples/multi-runner-scale-set/README.md @@ -70,7 +70,7 @@ The GitHub App must be installed for the configured GitHub account. |------|-------------|------|---------|:--------:| | [ami](#input\_ami) | Optional AMI configuration keyed by runner lane. | 
map(object({
    filter = optional(map(list(string)), { state = ["available"] })
    owners = optional(list(string), ["amazon"])
    id_ssm_parameter = optional(object({
      arn = string
    }), null)
    kms_key = optional(object({
      arn = string
    }), null)
  }))
| `{}` | no | | [aws\_region](#input\_aws\_region) | AWS region to deploy to. | `string` | `"eu-west-1"` | no | -| [environment](#input\_environment) | Environment name, used as prefix. | `string` | `null` | no | +| [environment](#input\_environment) | Environment name, used as prefix. | `string` | n/a | yes | | [github\_app](#input\_github\_app) | GitHub App ID, base64-encoded private key, and installation ID. | 
object({
    id              = string
    key_base64      = string
    installation_id = optional(string, null)
  })
| n/a | yes | | [github\_config](#input\_github\_config) | GitHub configuration for runner registration. | 
object({
    runner_registration_level = optional(string, null)
    runner_owner              = optional(string, null)
  })
| n/a | yes | | [runner\_binaries\_enabled](#input\_runner\_binaries\_enabled) | Whether runner binary synchronization is enabled. | `bool` | `true` | no | From 99ae1bc6ed525f08be25698f79a26c5db239ba69 Mon Sep 17 00:00:00 2001 From: edersonbrilhante Date: Fri, 18 Sep 2026 01:28:33 +0200 Subject: [PATCH 13/20] fix: fix validations --- modules/multi-runner/validations.tf | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/modules/multi-runner/validations.tf b/modules/multi-runner/validations.tf index dafb757a9f..750ac1177e 100644 --- a/modules/multi-runner/validations.tf +++ b/modules/multi-runner/validations.tf @@ -115,15 +115,11 @@ resource "terraform_data" "validate_v2" { } precondition { - condition = alltrue([ - for config in local.resolved_config.multi_runner_config : ( - try(config.orchestration_provider.scale_set, null) == null ? true : ( - try(var.global_config_github.app.installation_id, null) != null || - try(var.global_config_github.app.installation_id_ssm, null) != null - ) - ) - ]) - error_message = "Scale-set lanes require global_config_github.app.installation_id or global_config_github.app.installation_id_ssm." + condition = ( + length(local.scale_set_runner_configs) == 0 || + local.primary_app_installation_id != null + ) + error_message = "Scale-set runner configurations require a resolved primary GitHub App installation ID." } } From 4d8cffad6d8317b44094251f0b3fb5da0035cbcb Mon Sep 17 00:00:00 2001 From: edersonbrilhante Date: Fri, 18 Sep 2026 12:13:48 +0200 Subject: [PATCH 14/20] fix(multi-runner): validate scale-set installation inputs --- .../tests/config-resolution.tftest.hcl | 4 ++++ modules/multi-runner/validations.tf | 14 +++++++++----- 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/modules/multi-runner/tests/config-resolution.tftest.hcl b/modules/multi-runner/tests/config-resolution.tftest.hcl index ff61de79f9..e2ea9cebe1 100644 --- a/modules/multi-runner/tests/config-resolution.tftest.hcl +++ b/modules/multi-runner/tests/config-resolution.tftest.hcl @@ -751,6 +751,10 @@ run "scale_set_lane_requires_owner_for_non_enterprise_registration" { run "scale_set_lane_requires_installation_id" { command = plan + plan_options { + target = [terraform_data.validate_v2] + } + variables { experimental_features = ["multi-runner-v2"] diff --git a/modules/multi-runner/validations.tf b/modules/multi-runner/validations.tf index 750ac1177e..dafb757a9f 100644 --- a/modules/multi-runner/validations.tf +++ b/modules/multi-runner/validations.tf @@ -115,11 +115,15 @@ resource "terraform_data" "validate_v2" { } precondition { - condition = ( - length(local.scale_set_runner_configs) == 0 || - local.primary_app_installation_id != null - ) - error_message = "Scale-set runner configurations require a resolved primary GitHub App installation ID." + condition = alltrue([ + for config in local.resolved_config.multi_runner_config : ( + try(config.orchestration_provider.scale_set, null) == null ? true : ( + try(var.global_config_github.app.installation_id, null) != null || + try(var.global_config_github.app.installation_id_ssm, null) != null + ) + ) + ]) + error_message = "Scale-set lanes require global_config_github.app.installation_id or global_config_github.app.installation_id_ssm." } } From ea91c63d934c9bd7f77885892a5fe3aeb62d28b0 Mon Sep 17 00:00:00 2001 From: edersonbrilhante Date: Fri, 18 Sep 2026 12:18:00 +0200 Subject: [PATCH 15/20] test: fix example and test --- examples/multi-runner-scale-set/README.md | 9 +++--- examples/multi-runner-scale-set/main.tf | 13 +++++--- examples/multi-runner-scale-set/variables.tf | 18 +++++++---- tests/ministack/multi-runner-scale-set.tfvars | 30 ++++++++++--------- 4 files changed, 42 insertions(+), 28 deletions(-) diff --git a/examples/multi-runner-scale-set/README.md b/examples/multi-runner-scale-set/README.md index 6a32ada69d..167ae7f275 100644 --- a/examples/multi-runner-scale-set/README.md +++ b/examples/multi-runner-scale-set/README.md @@ -27,6 +27,7 @@ Configure the GitHub App variables before applying: terraform init terraform apply \ -var='github_app={id="123456",key_base64="...",installation_id="123456789"}' \ + -var='github={runner_owner="example",registration_level="organization"}' \ -var='scale_set={name="linux-scale-set",container={image="ghcr.io/github-aws-runners/terraform-aws-github-runner-scale-set-service@sha256:"}}' ``` @@ -70,11 +71,11 @@ The GitHub App must be installed for the configured GitHub account. |------|-------------|------|---------|:--------:| | [ami](#input\_ami) | Optional AMI configuration keyed by runner lane. | 
map(object({
    filter = optional(map(list(string)), { state = ["available"] })
    owners = optional(list(string), ["amazon"])
    id_ssm_parameter = optional(object({
      arn = string
    }), null)
    kms_key = optional(object({
      arn = string
    }), null)
  }))
| `{}` | no | | [aws\_region](#input\_aws\_region) | AWS region to deploy to. | `string` | `"eu-west-1"` | no | -| [environment](#input\_environment) | Environment name, used as prefix. | `string` | n/a | yes | +| [environment](#input\_environment) | Environment name, used as prefix. | `string` | `null` | no | +| [github](#input\_github) | Optional GitHub endpoint and scale-set ownership settings. | 
object({
    config_url         = optional(string, null)
    ssl_verify         = optional(bool, true)
    runner_owner       = optional(string, null)
    registration_level = optional(string, "organization")
  })
| `{}` | no | | [github\_app](#input\_github\_app) | GitHub App ID, base64-encoded private key, and installation ID. | 
object({
    id              = string
    key_base64      = string
    installation_id = optional(string, null)
  })
| n/a | yes | -| [github\_config](#input\_github\_config) | GitHub configuration for runner registration. | 
object({
    runner_registration_level = optional(string, null)
    runner_owner              = optional(string, null)
  })
| n/a | yes | | [runner\_binaries\_enabled](#input\_runner\_binaries\_enabled) | Whether runner binary synchronization is enabled. | `bool` | `true` | no | -| [scale\_set](#input\_scale\_set) | GitHub Actions scale-set configuration. | 
object({
    name = string
    container = optional(object({
      image = optional(string, null)
    }), {})
  })
| n/a | yes | +| [scale\_set](#input\_scale\_set) | GitHub Actions scale-set configuration. | 
object({
    name              = string
    runner_group_name = optional(string, "Default")
    min_runners       = optional(number, 0)
    container = optional(object({
      image = optional(string, null)
    }), {})
  })
| n/a | yes | ## Outputs @@ -82,4 +83,4 @@ The GitHub App must be installed for the configured GitHub account. |------|-------------| | [webhook\_endpoint](#output\_webhook\_endpoint) | n/a | | [webhook\_secret](#output\_webhook\_secret) | n/a | - + \ No newline at end of file diff --git a/examples/multi-runner-scale-set/main.tf b/examples/multi-runner-scale-set/main.tf index e6e6ac339a..30d70a3a9b 100644 --- a/examples/multi-runner-scale-set/main.tf +++ b/examples/multi-runner-scale-set/main.tf @@ -41,8 +41,12 @@ module "runners" { installation_id = var.github_app.installation_id webhook_secret = random_id.random.hex } - runner_registration_level = var.github_config.runner_registration_level - runner_owner = var.github_config.runner_owner + enterprise_server = { + url = var.github.url + ssl_verify = var.github.ssl_verify + } + runner_owner = var.github.runner_owner + runner_registration_level = var.github.registration_level } global_config_lambda = { @@ -147,12 +151,13 @@ module "runners" { runner = { name_prefix = "scale-set-" extra_labels = ["scale-set"] + group_name = var.scale_set.runner_group_name } orchestration_provider = { scale_set = { name = var.scale_set.name runner = { - min_runners = 0 + min_runners = var.scale_set.min_runners max_runners = 10 boot_time_in_minutes = 10 } @@ -207,4 +212,4 @@ module "webhook_github_app" { webhook_secret = random_id.random.hex } webhook_endpoint = module.runners.webhook.endpoint -} +} \ No newline at end of file diff --git a/examples/multi-runner-scale-set/variables.tf b/examples/multi-runner-scale-set/variables.tf index 62a905fcd6..f0f9f66c4d 100644 --- a/examples/multi-runner-scale-set/variables.tf +++ b/examples/multi-runner-scale-set/variables.tf @@ -9,20 +9,26 @@ variable "github_app" { sensitive = true } -variable "github_config" { - description = "GitHub configuration for runner registration." +variable "github" { + description = "Optional GitHub endpoint and scale-set ownership settings." type = object({ - runner_registration_level = optional(string, null) - runner_owner = optional(string, null) + url = optional(string, null) + ssl_verify = optional(bool, true) + runner_owner = optional(string, null) + registration_level = optional(string, "organization") }) + + default = {} } variable "scale_set" { description = "GitHub Actions scale-set configuration." type = object({ - name = string + name = string + runner_group_name = optional(string, "Default") + min_runners = optional(number, 0) container = optional(object({ image = optional(string, null) }), {}) @@ -63,4 +69,4 @@ variable "ami" { }), null) })) default = {} -} +} \ No newline at end of file diff --git a/tests/ministack/multi-runner-scale-set.tfvars b/tests/ministack/multi-runner-scale-set.tfvars index 88c5d9bf48..6a47126ad6 100644 --- a/tests/ministack/multi-runner-scale-set.tfvars +++ b/tests/ministack/multi-runner-scale-set.tfvars @@ -1,15 +1,17 @@ environment = "ministack-scale-set" aws_region = "eu-west-1" -github_app = { - id = "0" - key_base64 = "ministack-invalid-key" - installation_id = "1" +github = { + url = "https://mockserver:1080" + ssl_verify = false + runner_owner = "example" + registration_level = "organization" } -github_config = { - runner_registration_level = "organization" - runner_owner = "example" +github_app = { + id = "123" + key_base64 = "ministack-invalid-key" + installation_id = "456" } runner_binaries_enabled = false @@ -46,10 +48,10 @@ ami = { } scale_set = { - config_url = "https://github.com/example" - name = "ministack-scale-set" - id = 1 - runner_group_id = 1 - runner_owner = "example" - runner_registration_level = "organization" -} + name = "medium" + runner_group_name = "experimental-euw1-sl-cicd-forge-emu" + min_runners = 1 + container = { + image = "localhost:4566/scale-set-controller:smoke" + } +} \ No newline at end of file From 3310bd8dfad8ab9d4600d56259ef21af2e374037 Mon Sep 17 00:00:00 2001 From: Ederson Brilhante Date: Fri, 18 Sep 2026 14:19:13 +0200 Subject: [PATCH 16/20] test: fix tofu test --- modules/multi-runner/.terraform.lock.hcl | 71 ------------------- .../tests/config-resolution.tftest.hcl | 12 ++++ 2 files changed, 12 insertions(+), 71 deletions(-) delete mode 100644 modules/multi-runner/.terraform.lock.hcl diff --git a/modules/multi-runner/.terraform.lock.hcl b/modules/multi-runner/.terraform.lock.hcl deleted file mode 100644 index 9559f5fbd8..0000000000 --- a/modules/multi-runner/.terraform.lock.hcl +++ /dev/null @@ -1,71 +0,0 @@ -# This file is maintained automatically by "terraform init". -# Manual edits may be lost in future updates. - -provider "registry.terraform.io/hashicorp/aws" { - version = "6.63.0" - constraints = ">= 6.21.0, >= 6.33.0" - hashes = [ - "h1:9cre7jh1lSs/9igpgAcENMUAUlYW3HCtkav3up4oit0=", - "h1:dRlYHkc+r6fgzF57WC7Zjcmb6sF/6TTGDEgwGK+LAZY=", - "zh:005d56736afd17d963998c405cee6f434dbc23a415109f9435ff1542879ae611", - "zh:026ef126321a86ad7080b5d858e2527f96f5289678cbcd8856296e229c43339d", - "zh:06e0b58b2d1eddb5137fc86bee7ad2d07953c0bc3f57cccfc5ae0d2456068a3a", - "zh:07221735d61ababed84734e5ffcfc5bd59d01f29f029166ba5f2175895dceed1", - "zh:1a72db00583112bdb8c19b213a78a3f5de754fffc08f07e061f4e326289fab7d", - "zh:32968e74a53b03e97a084dc7050c22ef661fb5b3ea8a44f5a63e47bc45ad0e7c", - "zh:4b357dfe4b820e3e4acd2881cff8288b2186491e63416751f0d12692ba478ceb", - "zh:81e30884d7de686265e7d87bb92527e802878c65a378470ede2a1e9f4e40ccc9", - "zh:82e137297f6a5a08b9ce2138f7aabea245ad99495d9d9eff502f752d6ca90dbd", - "zh:8eb83b67099f0ea9df238a979dff933ff50ce06a2e3ff05a48556a10f10dd204", - "zh:9b12af85486a96aedd8d7984b0ff811a4b42e3d88dad1a3fb4c0b580d04fa425", - "zh:d0ba30886cbe41850fee689f51ef9088578f323cfd21817bb409951d43c465eb", - "zh:dd48e7089784454bc03d713e9057f5ca0ea1613bd402125054a51894957b7925", - "zh:f250fa81e54cf60fcb0e9c0fc4ac043f1ecc2ac24967f628b3609364fcab3d04", - "zh:f38fc09fc25a8d2cf89a4d4cd6a5ef7cb1aad72798dbdcad58b8876b6a551a54", - "zh:f7c7380fdf126e1901f2084588dbfd724c76cb131ccfa795a541219111103c06", - ] -} - -provider "registry.terraform.io/hashicorp/null" { - version = "3.3.1" - constraints = "~> 3.0, ~> 3.2" - hashes = [ - "h1:TuxJq10DVnRP7c5HBZPyyvQGcckNVfijyU1eXEu5e4M=", - "h1:m5FqidbIgh+E9OigiZh8/xbkvpUQFSj3hZo/jqNLCLQ=", - "zh:08c59776542ea16e5a8545752787b17ff412922182b4cfabe16139197be8ac44", - "zh:123109cc7e5ed6d515787fbc212f2a3fd5e75647bb24ab7c801ccd4d4ed42451", - "zh:14b3fa4372754b54844b41d5dbd4671a292d8d6828b90169061feb4d7b15dd05", - "zh:56a4daaa3212f57b764bf3d1f333141c6610c5f21abb240e0111221f7c7fa4d4", - "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", - "zh:7e888a026dbacd2474a42264227ae35f639780f0f0c613529d10a95cd61988b3", - "zh:85a53646267e87d600df7124e4767ffde9bba3b6356d45d961618bdd68131cc7", - "zh:8ffa0e9c7c39b2ab0905b472465d6e35ef0b776b3f6273bb34c150340b61bff1", - "zh:9846510a1841530d4403f4818e233f91e3b3bade7441047599fbf800742f65be", - "zh:afa98d44860875f037c6def0a7e6ff208e042712ba771f620482b143cd336891", - "zh:bdca130d9ef27488ae0b13bc8fd8019e8bbdd4f2ceff29da066bd333165d68c5", - "zh:cb3b94cbca88210dd0d1f11e2b8a89333f48c3857faf8f70f589072ce7c28610", - "zh:f0c0ba87925fe32f84b80f7513b1efb1b0866f51f899ba825e95ad59ff09b018", - ] -} - -provider "registry.terraform.io/hashicorp/random" { - version = "3.9.0" - constraints = "~> 3.0" - hashes = [ - "h1:OO+IuvQJSPmWdN8AyyIEvPJbLvDQpgX/zbktoa9KsJE=", - "h1:UlBuNVuCGJ39tTv2c5gz2NRZnQbXfbIWbTzWcth5o74=", - "zh:161ad0bd9a75768c82f53fb6e7172a9d8be2d4889b012645a34795031aaf1bf1", - "zh:19dc9a5b17729725ccfc4f45b0500af0ee5bc6b6b160c7adb8f2bf617d2c80ea", - "zh:269eda8fe42daa7974d5a34d166c3ba9defe80cde86c01e4dadcfdf2e1f05e5f", - "zh:373f7c65566f8f2cc7f45d698654feb9d988996957e1266a69ca00c52d6d16d0", - "zh:5599d16804c41c83009ec621b6d6b6f74e102f5827678a4750f8809055546b61", - "zh:583be0440469a22bff70dcfa56593b01566860b29607437264adb51060cf46fc", - "zh:5f211d8ec3f2e1f414870d9584bfe26e6995560ef81c748f8447a48164767398", - "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", - "zh:7b547fd16216761ef86efc3ed516ac5ac0c5c42b7c7eb24a08cef2d93f69ed5e", - "zh:7e7c0679daf2a382151d05068c8c3f0dae6b7b7dccf818827b73dd08638df2ef", - "zh:8089dec888a8038b9b4fb23b3df7e1057293dbc5b60b42cc47ff690d69d4b61b", - "zh:c51f15a031edfd6f23ce8ced3446ca7f8d8d647e2499890d7d5d10d5016d7257", - "zh:c94784f005708890dc6895afd53636ec00ec1e430b15d41e5aebfb1d4b39bd04", - ] -} diff --git a/modules/multi-runner/tests/config-resolution.tftest.hcl b/modules/multi-runner/tests/config-resolution.tftest.hcl index e2ea9cebe1..c5bb783664 100644 --- a/modules/multi-runner/tests/config-resolution.tftest.hcl +++ b/modules/multi-runner/tests/config-resolution.tftest.hcl @@ -5,6 +5,18 @@ mock_provider "aws" { } } + mock_data "aws_partition" { + defaults = { + partition = "aws" + } + } + + mock_data "aws_region" { + defaults = { + region = "eu-west-1" + } + } + mock_data "aws_iam_policy_document" { defaults = { json = "{\"Version\":\"2012-10-17\",\"Statement\":[]}" From 4256d6f415db8e16bf61b2ee36e80616547e2c79 Mon Sep 17 00:00:00 2001 From: Ederson Brilhante Date: Fri, 18 Sep 2026 14:19:14 +0200 Subject: [PATCH 17/20] test: add fixture to support tofu tests --- examples/multi-runner-scale-set/README.md | 18 +- .../tests/config-resolution.tftest.hcl | 42 ++++ .../fixtures/base-runner-configs/README.md | 32 +++ .../fixtures/base-runner-configs/main.tf | 9 + .../fixtures/base-runner-configs/versions.tf | 10 + .../scale-set/tests/scale-set.tftest.hcl | 204 ++++++++++-------- 6 files changed, 218 insertions(+), 97 deletions(-) create mode 100644 modules/orchestration-providers/scale-set/tests/fixtures/base-runner-configs/README.md create mode 100644 modules/orchestration-providers/scale-set/tests/fixtures/base-runner-configs/main.tf create mode 100644 modules/orchestration-providers/scale-set/tests/fixtures/base-runner-configs/versions.tf diff --git a/examples/multi-runner-scale-set/README.md b/examples/multi-runner-scale-set/README.md index 167ae7f275..1076f23fe9 100644 --- a/examples/multi-runner-scale-set/README.md +++ b/examples/multi-runner-scale-set/README.md @@ -39,7 +39,7 @@ The GitHub App must be installed for the configured GitHub account. ## Requirements | Name | Version | -|------|---------| +| ---- | ------- | | [terraform](#requirement\_terraform) | >= 1.5.6 | | [aws](#requirement\_aws) | >= 6.33 | | [local](#requirement\_local) | ~> 2.0 | @@ -48,13 +48,13 @@ The GitHub App must be installed for the configured GitHub account. ## Providers | Name | Version | -|------|---------| +| ---- | ------- | | [random](#provider\_random) | 3.9.0 | ## Modules | Name | Source | Version | -|------|--------|---------| +| ---- | ------ | ------- | | [base](#module\_base) | ../base | n/a | | [runners](#module\_runners) | ../../modules/multi-runner | n/a | | [webhook\_github\_app](#module\_webhook\_github\_app) | ../../modules/webhook-github-app | n/a | @@ -62,17 +62,17 @@ The GitHub App must be installed for the configured GitHub account. ## Resources | Name | Type | -|------|------| +| ---- | ---- | | [random_id.random](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/id) | resource | ## Inputs | Name | Description | Type | Default | Required | -|------|-------------|------|---------|:--------:| +| ---- | ----------- | ---- | ------- | :------: | | [ami](#input\_ami) | Optional AMI configuration keyed by runner lane. | 
map(object({
    filter = optional(map(list(string)), { state = ["available"] })
    owners = optional(list(string), ["amazon"])
    id_ssm_parameter = optional(object({
      arn = string
    }), null)
    kms_key = optional(object({
      arn = string
    }), null)
  }))
| `{}` | no | | [aws\_region](#input\_aws\_region) | AWS region to deploy to. | `string` | `"eu-west-1"` | no | -| [environment](#input\_environment) | Environment name, used as prefix. | `string` | `null` | no | -| [github](#input\_github) | Optional GitHub endpoint and scale-set ownership settings. | 
object({
    config_url         = optional(string, null)
    ssl_verify         = optional(bool, true)
    runner_owner       = optional(string, null)
    registration_level = optional(string, "organization")
  })
| `{}` | no | +| [environment](#input\_environment) | Environment name, used as prefix. | `string` | n/a | yes | +| [github](#input\_github) | Optional GitHub endpoint and scale-set ownership settings. | 
object({
    url                = optional(string, null)
    ssl_verify         = optional(bool, true)
    runner_owner       = optional(string, null)
    registration_level = optional(string, "organization")
  })
| `{}` | no | | [github\_app](#input\_github\_app) | GitHub App ID, base64-encoded private key, and installation ID. | 
object({
    id              = string
    key_base64      = string
    installation_id = optional(string, null)
  })
| n/a | yes | | [runner\_binaries\_enabled](#input\_runner\_binaries\_enabled) | Whether runner binary synchronization is enabled. | `bool` | `true` | no | | [scale\_set](#input\_scale\_set) | GitHub Actions scale-set configuration. | 
object({
    name              = string
    runner_group_name = optional(string, "Default")
    min_runners       = optional(number, 0)
    container = optional(object({
      image = optional(string, null)
    }), {})
  })
| n/a | yes | @@ -80,7 +80,7 @@ The GitHub App must be installed for the configured GitHub account. ## Outputs | Name | Description | -|------|-------------| +| ---- | ----------- | | [webhook\_endpoint](#output\_webhook\_endpoint) | n/a | | [webhook\_secret](#output\_webhook\_secret) | n/a | - \ No newline at end of file + diff --git a/modules/multi-runner/tests/config-resolution.tftest.hcl b/modules/multi-runner/tests/config-resolution.tftest.hcl index c5bb783664..3ef4f859a7 100644 --- a/modules/multi-runner/tests/config-resolution.tftest.hcl +++ b/modules/multi-runner/tests/config-resolution.tftest.hcl @@ -532,6 +532,27 @@ run "v2_inputs_reject_legacy_runner_config" { run "scale_set_only_lane_omits_webhook_queues" { command = plan + override_resource { + target = module.ssm.aws_ssm_parameter.github_app_id + values = { + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github-action-runners/test/app/github_app_id" + } + } + + override_resource { + target = module.ssm.aws_ssm_parameter.github_app_key_base64 + values = { + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github-action-runners/test/app/github_app_key_base64" + } + } + + override_resource { + target = module.ssm.aws_ssm_parameter.github_app_installation_id + values = { + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github-action-runners/test/app/github_app_installation_id" + } + } + variables { experimental_features = ["multi-runner-v2"] @@ -619,6 +640,27 @@ run "scale_set_only_lane_omits_webhook_queues" { run "mixed_webhook_and_scale_set_lanes_create_webhook_queues_only_for_webhook" { command = plan + override_resource { + target = module.ssm.aws_ssm_parameter.github_app_id + values = { + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github-action-runners/test/app/github_app_id" + } + } + + override_resource { + target = module.ssm.aws_ssm_parameter.github_app_key_base64 + values = { + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github-action-runners/test/app/github_app_key_base64" + } + } + + override_resource { + target = module.ssm.aws_ssm_parameter.github_app_installation_id + values = { + arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github-action-runners/test/app/github_app_installation_id" + } + } + variables { experimental_features = ["multi-runner-v2"] diff --git a/modules/orchestration-providers/scale-set/tests/fixtures/base-runner-configs/README.md b/modules/orchestration-providers/scale-set/tests/fixtures/base-runner-configs/README.md new file mode 100644 index 0000000000..36bd50828b --- /dev/null +++ b/modules/orchestration-providers/scale-set/tests/fixtures/base-runner-configs/README.md @@ -0,0 +1,32 @@ + +## Requirements + +| Name | Version | +| ---- | ------- | +| [terraform](#requirement\_terraform) | >= 1.5.6 | +| [aws](#requirement\_aws) | >= 6.33 | + +## Providers + +No providers. + +## Modules + +No modules. + +## Resources + +No resources. + +## Inputs + +| Name | Description | Type | Default | Required | +| ---- | ----------- | ---- | ------- | :------: | +| [runner\_configs](#input\_runner\_configs) | Base runner configuration fixture forwarded to the scale-set tests. | `any` | n/a | yes | + +## Outputs + +| Name | Description | +| ---- | ----------- | +| [runner\_configs](#output\_runner\_configs) | Base runner configuration fixture for later test runs. | + diff --git a/modules/orchestration-providers/scale-set/tests/fixtures/base-runner-configs/main.tf b/modules/orchestration-providers/scale-set/tests/fixtures/base-runner-configs/main.tf new file mode 100644 index 0000000000..fb3b7e509a --- /dev/null +++ b/modules/orchestration-providers/scale-set/tests/fixtures/base-runner-configs/main.tf @@ -0,0 +1,9 @@ +variable "runner_configs" { + description = "Base runner configuration fixture forwarded to the scale-set tests." + type = any +} + +output "runner_configs" { + description = "Base runner configuration fixture for later test runs." + value = var.runner_configs +} diff --git a/modules/orchestration-providers/scale-set/tests/fixtures/base-runner-configs/versions.tf b/modules/orchestration-providers/scale-set/tests/fixtures/base-runner-configs/versions.tf new file mode 100644 index 0000000000..0bedc91fd5 --- /dev/null +++ b/modules/orchestration-providers/scale-set/tests/fixtures/base-runner-configs/versions.tf @@ -0,0 +1,10 @@ +terraform { + required_version = ">= 1.5.6" + + required_providers { + aws = { + source = "hashicorp/aws" + version = ">= 6.33" + } + } +} diff --git a/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl b/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl index eb61c8a6e8..b939c0310b 100644 --- a/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl +++ b/modules/orchestration-providers/scale-set/tests/scale-set.tftest.hcl @@ -232,6 +232,14 @@ variables { } } +run "base_runner_configs" { + command = apply + + module { + source = "./tests/fixtures/base-runner-configs" + } +} + run "groups_by_compute_provider_and_hardens_each_task" { command = plan @@ -549,15 +557,23 @@ run "rejects_oversized_standard_parameter" { } variables { - runner_configs = merge(var.runner_configs, { - linux-small = merge(var.runner_configs.linux-small, { - compute_provider = merge(var.runner_configs.linux-small.compute_provider, { - capabilities = { - scale_set = merge(var.runner_configs.linux-small.compute_provider.capabilities.scale_set, { - configuration_json = jsonencode({ payload = join("", [for index in range(1000) : "xxxxxx"]) }) - }) + runner_configs = merge(run.base_runner_configs.runner_configs, { + linux-small = merge(run.base_runner_configs.runner_configs.linux-small, { + compute_provider = merge( + run.base_runner_configs.runner_configs.linux-small.compute_provider, + { + capabilities = { + scale_set = merge( + run.base_runner_configs.runner_configs.linux-small.compute_provider.capabilities.scale_set, + { + configuration_json = jsonencode({ + payload = join("", [for index in range(1000) : "xxxxxx"]) + }) + } + ) + } } - }) + ) }) }) } @@ -576,21 +592,33 @@ run "accepts_advanced_parameter_within_eight_kib" { config_store = { tier = "Advanced" } - runner_configs = merge(var.runner_configs, { - linux-small = merge(var.runner_configs.linux-small, { - compute_provider = merge(var.runner_configs.linux-small.compute_provider, { - capabilities = { - scale_set = merge(var.runner_configs.linux-small.compute_provider.capabilities.scale_set, { - configuration_json = jsonencode({ payload = join("", [for index in range(800) : "xxxxxx"]) }) - }) + + runner_configs = merge(run.base_runner_configs.runner_configs, { + linux-small = merge(run.base_runner_configs.runner_configs.linux-small, { + compute_provider = merge( + run.base_runner_configs.runner_configs.linux-small.compute_provider, + { + capabilities = { + scale_set = merge( + run.base_runner_configs.runner_configs.linux-small.compute_provider.capabilities.scale_set, + { + configuration_json = jsonencode({ + payload = join("", [for index in range(800) : "xxxxxx"]) + }) + } + ) + } } - }) + ) }) }) } assert { - condition = local.reconciler_config_bytes["ec2/linux-small"] > 4096 && local.reconciler_config_bytes["ec2/linux-small"] <= 8192 + condition = ( + local.reconciler_config_bytes["ec2/linux-small"] > 4096 && + local.reconciler_config_bytes["ec2/linux-small"] <= 8192 + ) error_message = "Advanced Parameter Store tier must accept reconciler JSON between four and eight KiB." } } @@ -599,21 +627,21 @@ run "assembles_github_config_url_from_registration_scope_and_owner" { command = apply variables { - runner_configs = merge(var.runner_configs, { - linux-small = merge(var.runner_configs.linux-small, { - github = merge(var.runner_configs.linux-small.github, { + runner_configs = merge(run.base_runner_configs.runner_configs, { + linux-small = merge(run.base_runner_configs.runner_configs.linux-small, { + github = merge(run.base_runner_configs.runner_configs.linux-small.github, { runner_registration_level = "organization" runner_owner = "example" }) }) - linux-large = merge(var.runner_configs.linux-large, { - github = merge(var.runner_configs.linux-large.github, { + linux-large = merge(run.base_runner_configs.runner_configs.linux-large, { + github = merge(run.base_runner_configs.runner_configs.linux-large.github, { runner_registration_level = "organization" runner_owner = "example" }) }) - microvm = merge(var.runner_configs.microvm, { - github = merge(var.runner_configs.microvm.github, { + microvm = merge(run.base_runner_configs.runner_configs.microvm, { + github = merge(run.base_runner_configs.runner_configs.microvm.github, { runner_registration_level = "repository" runner_owner = "example/repository" }) @@ -639,19 +667,19 @@ run "rejects_duplicate_scale_set_ownership_across_groups" { } variables { - runner_configs = merge(var.runner_configs, { - linux-small = merge(var.runner_configs.linux-small, { - github = merge(var.runner_configs.linux-small.github, { + runner_configs = merge(run.base_runner_configs.runner_configs, { + linux-small = merge(run.base_runner_configs.runner_configs.linux-small, { + github = merge(run.base_runner_configs.runner_configs.linux-small.github, { enterprise_server = { url = "https://mygithub.com" } }) }) - microvm = merge(var.runner_configs.microvm, { - github = merge(var.runner_configs.microvm.github, { + microvm = merge(run.base_runner_configs.runner_configs.microvm, { + github = merge(run.base_runner_configs.runner_configs.microvm.github, { enterprise_server = { url = "https://mygithub.com:443/" } runner_registration_level = "organization" runner_owner = "example" }) - scale_set = merge(var.runner_configs.microvm.scale_set, { + scale_set = merge(run.base_runner_configs.runner_configs.microvm.scale_set, { name = "linux-small" }) }) @@ -669,12 +697,12 @@ run "rejects_leading_zero_default_port_spelling" { } variables { - runner_configs = merge(var.runner_configs, { - microvm = merge(var.runner_configs.microvm, { - github = merge(var.runner_configs.microvm.github, { + runner_configs = merge(run.base_runner_configs.runner_configs, { + microvm = merge(run.base_runner_configs.runner_configs.microvm, { + github = merge(run.base_runner_configs.runner_configs.microvm.github, { enterprise_server = { url = "https://github.com:0443/" } }) - scale_set = merge(var.runner_configs.microvm.scale_set, { + scale_set = merge(run.base_runner_configs.runner_configs.microvm.scale_set, { name = "linux-small" }) }) @@ -692,9 +720,9 @@ run "rejects_port_above_url_maximum" { } variables { - runner_configs = merge(var.runner_configs, { - microvm = merge(var.runner_configs.microvm, { - github = merge(var.runner_configs.microvm.github, { + runner_configs = merge(run.base_runner_configs.runner_configs, { + microvm = merge(run.base_runner_configs.runner_configs.microvm, { + github = merge(run.base_runner_configs.runner_configs.microvm.github, { enterprise_server = { url = "https://github.com:65536/" } }) }) @@ -712,9 +740,9 @@ run "rejects_non_ascii_scale_set_name" { } variables { - runner_configs = merge(var.runner_configs, { - microvm = merge(var.runner_configs.microvm, { - scale_set = merge(var.runner_configs.microvm.scale_set, { + runner_configs = merge(run.base_runner_configs.runner_configs, { + microvm = merge(run.base_runner_configs.runner_configs.microvm, { + scale_set = merge(run.base_runner_configs.runner_configs.microvm.scale_set, { name = "microvm-☃" }) }) @@ -732,9 +760,9 @@ run "rejects_invalid_compute_provider_type_identifier" { } variables { - runner_configs = merge(var.runner_configs, { - microvm = merge(var.runner_configs.microvm, { - compute_provider = merge(var.runner_configs.microvm.compute_provider, { + runner_configs = merge(run.base_runner_configs.runner_configs, { + microvm = merge(run.base_runner_configs.runner_configs.microvm, { + compute_provider = merge(run.base_runner_configs.runner_configs.microvm.compute_provider, { type = "AWS.MicroVM" }) }) @@ -752,11 +780,11 @@ run "rejects_credential_arn_name_mismatch" { } variables { - runner_configs = merge(var.runner_configs, { - microvm = merge(var.runner_configs.microvm, { - github = merge(var.runner_configs.microvm.github, { - app = merge(var.runner_configs.microvm.github.app, { - app_id = merge(var.runner_configs.microvm.github.app.app_id, { + runner_configs = merge(run.base_runner_configs.runner_configs, { + microvm = merge(run.base_runner_configs.runner_configs.microvm, { + github = merge(run.base_runner_configs.runner_configs.microvm.github, { + app = merge(run.base_runner_configs.runner_configs.microvm.github.app, { + app_id = merge(run.base_runner_configs.runner_configs.microvm.github.app.app_id, { arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/another/app-id" }) }) @@ -776,11 +804,11 @@ run "rejects_cross_account_credential_parameter" { } variables { - runner_configs = merge(var.runner_configs, { - microvm = merge(var.runner_configs.microvm, { - github = merge(var.runner_configs.microvm.github, { - app = merge(var.runner_configs.microvm.github.app, { - app_id = merge(var.runner_configs.microvm.github.app.app_id, { + runner_configs = merge(run.base_runner_configs.runner_configs, { + microvm = merge(run.base_runner_configs.runner_configs.microvm, { + github = merge(run.base_runner_configs.runner_configs.microvm.github, { + app = merge(run.base_runner_configs.runner_configs.microvm.github.app, { + app_id = merge(run.base_runner_configs.runner_configs.microvm.github.app.app_id, { arn = "arn:aws:ssm:eu-west-1:210987654321:parameter/github/microvm/app-id" }) }) @@ -800,12 +828,12 @@ run "allows_same_scale_set_name_in_another_github_scope" { } variables { - runner_configs = merge(var.runner_configs, { - microvm = merge(var.runner_configs.microvm, { - github = merge(var.runner_configs.microvm.github, { + runner_configs = merge(run.base_runner_configs.runner_configs, { + microvm = merge(run.base_runner_configs.runner_configs.microvm, { + github = merge(run.base_runner_configs.runner_configs.microvm.github, { enterprise_server = { url = "https://github.example.test" } }) - scale_set = merge(var.runner_configs.microvm.scale_set, { + scale_set = merge(run.base_runner_configs.runner_configs.microvm.scale_set, { name = "linux-small" }) }) @@ -910,11 +938,11 @@ run "rejects_conflicting_group_environment_variables" { } variables { - runner_configs = merge(var.runner_configs, { - linux-large = merge(var.runner_configs.linux-large, { - compute_provider = merge(var.runner_configs.linux-large.compute_provider, { + runner_configs = merge(run.base_runner_configs.runner_configs, { + linux-large = merge(run.base_runner_configs.runner_configs.linux-large, { + compute_provider = merge(run.base_runner_configs.runner_configs.linux-large.compute_provider, { capabilities = { - scale_set = merge(var.runner_configs.linux-large.compute_provider.capabilities.scale_set, { + scale_set = merge(run.base_runner_configs.runner_configs.linux-large.compute_provider.capabilities.scale_set, { environment_variables = { EC2_CONTROLLER_MODE = "isolated" } @@ -936,13 +964,13 @@ run "rejects_controller_group_environment_above_task_definition_budget" { } variables { - runner_configs = merge(var.runner_configs, { - linux-small = merge(var.runner_configs.linux-small, { - compute_provider = merge(var.runner_configs.linux-small.compute_provider, { + runner_configs = merge(run.base_runner_configs.runner_configs, { + linux-small = merge(run.base_runner_configs.runner_configs.linux-small, { + compute_provider = merge(run.base_runner_configs.runner_configs.linux-small.compute_provider, { capabilities = { - scale_set = merge(var.runner_configs.linux-small.compute_provider.capabilities.scale_set, { + scale_set = merge(run.base_runner_configs.runner_configs.linux-small.compute_provider.capabilities.scale_set, { environment_variables = merge( - var.runner_configs.linux-small.compute_provider.capabilities.scale_set.environment_variables, + run.base_runner_configs.runner_configs.linux-small.compute_provider.capabilities.scale_set.environment_variables, { for index in range(16) : format("EC2_QUOTA_%02d", index) => join("", [for part in range(1024) : "xxxx"]) }, @@ -965,11 +993,11 @@ run "rejects_reserved_provider_environment_variables" { } variables { - runner_configs = merge(var.runner_configs, { - linux-small = merge(var.runner_configs.linux-small, { - compute_provider = merge(var.runner_configs.linux-small.compute_provider, { + runner_configs = merge(run.base_runner_configs.runner_configs, { + linux-small = merge(run.base_runner_configs.runner_configs.linux-small, { + compute_provider = merge(run.base_runner_configs.runner_configs.linux-small.compute_provider, { capabilities = { - scale_set = merge(var.runner_configs.linux-small.compute_provider.capabilities.scale_set, { + scale_set = merge(run.base_runner_configs.runner_configs.linux-small.compute_provider.capabilities.scale_set, { environment_variables = { SCALE_SET_OVERRIDE = "unsafe" } @@ -991,10 +1019,10 @@ run "rejects_invalid_boot_timeout" { } variables { - runner_configs = merge(var.runner_configs, { - linux-small = merge(var.runner_configs.linux-small, { - scale_set = merge(var.runner_configs.linux-small.scale_set, { - runner = merge(var.runner_configs.linux-small.scale_set.runner, { + runner_configs = merge(run.base_runner_configs.runner_configs, { + linux-small = merge(run.base_runner_configs.runner_configs.linux-small, { + scale_set = merge(run.base_runner_configs.runner_configs.linux-small.scale_set, { + runner = merge(run.base_runner_configs.runner_configs.linux-small.scale_set.runner, { boot_time_in_minutes = 0 }) }) @@ -1013,9 +1041,9 @@ run "rejects_enterprise_runner_registration_level" { } variables { - runner_configs = merge(var.runner_configs, { - linux-small = merge(var.runner_configs.linux-small, { - github = merge(var.runner_configs.linux-small.github, { + runner_configs = merge(run.base_runner_configs.runner_configs, { + linux-small = merge(run.base_runner_configs.runner_configs.linux-small, { + github = merge(run.base_runner_configs.runner_configs.linux-small.github, { runner_registration_level = "enterprise" runner_owner = null }) @@ -1034,9 +1062,9 @@ run "rejects_invalid_runner_registration_level" { } variables { - runner_configs = merge(var.runner_configs, { - linux-small = merge(var.runner_configs.linux-small, { - github = merge(var.runner_configs.linux-small.github, { + runner_configs = merge(run.base_runner_configs.runner_configs, { + linux-small = merge(run.base_runner_configs.runner_configs.linux-small, { + github = merge(run.base_runner_configs.runner_configs.linux-small.github, { runner_registration_level = "invalid" }) }) @@ -1055,7 +1083,7 @@ run "rejects_controller_group_above_runtime_reconciler_limit" { variables { runner_configs = { - for index in range(1001) : format("runner-%04d", index) => var.runner_configs.linux-small + for index in range(1001) : format("runner-%04d", index) => run.base_runner_configs.runner_configs.linux-small } grouping = { strategy = "custom" @@ -1084,10 +1112,10 @@ run "rejects_controller_group_above_runtime_config_bytes" { tier = "Advanced" } runner_configs = { - for index in range(900) : format("runner-%04d", index) => merge(var.runner_configs.linux-small, { - compute_provider = merge(var.runner_configs.linux-small.compute_provider, { + for index in range(900) : format("runner-%04d", index) => merge(run.base_runner_configs.runner_configs.linux-small, { + compute_provider = merge(run.base_runner_configs.runner_configs.linux-small.compute_provider, { capabilities = { - scale_set = merge(var.runner_configs.linux-small.compute_provider.capabilities.scale_set, { + scale_set = merge(run.base_runner_configs.runner_configs.linux-small.compute_provider.capabilities.scale_set, { configuration_json = jsonencode({ payload = join("", [for part in range(1000) : "xxxxx"]) }) @@ -1119,10 +1147,10 @@ run "rejects_runtime_invalid_credential_parameter_name" { } variables { - runner_configs = merge(var.runner_configs, { - microvm = merge(var.runner_configs.microvm, { - github = merge(var.runner_configs.microvm.github, { - app = merge(var.runner_configs.microvm.github.app, { + runner_configs = merge(run.base_runner_configs.runner_configs, { + microvm = merge(run.base_runner_configs.runner_configs.microvm, { + github = merge(run.base_runner_configs.runner_configs.microvm.github, { + app = merge(run.base_runner_configs.runner_configs.microvm.github.app, { app_id = { name = "/github/microvm/bad app-id" arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/github/microvm/bad app-id" From f0fa1e8be48a6b6552fd634d28e9177d7f571263 Mon Sep 17 00:00:00 2001 From: Ederson Brilhante Date: Fri, 18 Sep 2026 14:19:33 +0200 Subject: [PATCH 18/20] docs: auto update terraform docs --- examples/multi-runner-scale-set/README.md | 12 ++++++------ modules/multi-runner/README.md | 4 ++-- .../tests/fixtures/base-runner-configs/README.md | 6 +++--- 3 files changed, 11 insertions(+), 11 deletions(-) diff --git a/examples/multi-runner-scale-set/README.md b/examples/multi-runner-scale-set/README.md index 1076f23fe9..a2ca69cff4 100644 --- a/examples/multi-runner-scale-set/README.md +++ b/examples/multi-runner-scale-set/README.md @@ -39,7 +39,7 @@ The GitHub App must be installed for the configured GitHub account. ## Requirements | Name | Version | -| ---- | ------- | +|------|---------| | [terraform](#requirement\_terraform) | >= 1.5.6 | | [aws](#requirement\_aws) | >= 6.33 | | [local](#requirement\_local) | ~> 2.0 | @@ -48,13 +48,13 @@ The GitHub App must be installed for the configured GitHub account. ## Providers | Name | Version | -| ---- | ------- | +|------|---------| | [random](#provider\_random) | 3.9.0 | ## Modules | Name | Source | Version | -| ---- | ------ | ------- | +|------|--------|---------| | [base](#module\_base) | ../base | n/a | | [runners](#module\_runners) | ../../modules/multi-runner | n/a | | [webhook\_github\_app](#module\_webhook\_github\_app) | ../../modules/webhook-github-app | n/a | @@ -62,13 +62,13 @@ The GitHub App must be installed for the configured GitHub account. ## Resources | Name | Type | -| ---- | ---- | +|------|------| | [random_id.random](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/id) | resource | ## Inputs | Name | Description | Type | Default | Required | -| ---- | ----------- | ---- | ------- | :------: | +|------|-------------|------|---------|:--------:| | [ami](#input\_ami) | Optional AMI configuration keyed by runner lane. | 
map(object({
    filter = optional(map(list(string)), { state = ["available"] })
    owners = optional(list(string), ["amazon"])
    id_ssm_parameter = optional(object({
      arn = string
    }), null)
    kms_key = optional(object({
      arn = string
    }), null)
  }))
| `{}` | no | | [aws\_region](#input\_aws\_region) | AWS region to deploy to. | `string` | `"eu-west-1"` | no | | [environment](#input\_environment) | Environment name, used as prefix. | `string` | n/a | yes | @@ -80,7 +80,7 @@ The GitHub App must be installed for the configured GitHub account. ## Outputs | Name | Description | -| ---- | ----------- | +|------|-------------| | [webhook\_endpoint](#output\_webhook\_endpoint) | n/a | | [webhook\_secret](#output\_webhook\_secret) | n/a | diff --git a/modules/multi-runner/README.md b/modules/multi-runner/README.md index 404a281580..57f3e84b0d 100644 --- a/modules/multi-runner/README.md +++ b/modules/multi-runner/README.md @@ -109,8 +109,8 @@ module "multi-runner" { | Name | Version | |------|---------| -| [aws](#provider\_aws) | 6.63.0 | -| [random](#provider\_random) | 3.9.0 | +| [aws](#provider\_aws) | >= 6.33 | +| [random](#provider\_random) | ~> 3.0 | | [terraform](#provider\_terraform) | n/a | ## Modules diff --git a/modules/orchestration-providers/scale-set/tests/fixtures/base-runner-configs/README.md b/modules/orchestration-providers/scale-set/tests/fixtures/base-runner-configs/README.md index 36bd50828b..d01dc76415 100644 --- a/modules/orchestration-providers/scale-set/tests/fixtures/base-runner-configs/README.md +++ b/modules/orchestration-providers/scale-set/tests/fixtures/base-runner-configs/README.md @@ -2,7 +2,7 @@ ## Requirements | Name | Version | -| ---- | ------- | +|------|---------| | [terraform](#requirement\_terraform) | >= 1.5.6 | | [aws](#requirement\_aws) | >= 6.33 | @@ -21,12 +21,12 @@ No resources. ## Inputs | Name | Description | Type | Default | Required | -| ---- | ----------- | ---- | ------- | :------: | +|------|-------------|------|---------|:--------:| | [runner\_configs](#input\_runner\_configs) | Base runner configuration fixture forwarded to the scale-set tests. | `any` | n/a | yes | ## Outputs | Name | Description | -| ---- | ----------- | +|------|-------------| | [runner\_configs](#output\_runner\_configs) | Base runner configuration fixture for later test runs. | From 852b92e1b75e4b85ec63fe45a36a2ae948826fa1 Mon Sep 17 00:00:00 2001 From: Ederson Brilhante Date: Mon, 21 Sep 2026 14:44:51 +0200 Subject: [PATCH 19/20] feat(scale-set): add service controller (#5300) ## Description Adds the TypeScript implementation of the GitHub Actions runner scale-set control plane on top of the Terraform orchestration introduced by #5299. This PR includes: - A reusable GitHub Actions scale-set client that supports GitHub.com, GHES, and data-residency endpoints; GitHub App access tokens; runner-group and scale-set discovery/registration; JIT runner configuration; runner removal; and message-session polling, acknowledgement, job acquisition, refresh, and close. - A long-running scale-set controller service for ECS. It loads versioned runner configuration from SSM Parameter Store, runs independent reconcilers for each runner configuration, exposes liveness/readiness endpoints, and performs bounded shutdown and session recovery. - A scale-set compute-provider contract and EC2 implementation. The provider reconciles capacity from `totalAssignedJobs`, generates JIT configurations for launches, tracks provider-owned instances with tags, preserves busy or unknown runners during scale-down, and supports task-role or assumed-role credentials. - Configuration, authentication, transport, retry, cancellation, ownership, lifecycle, health, and reconciliation safeguards. Sensitive tokens, message bodies, and JIT configurations are excluded from manifests and logs; disabling TLS verification is scoped to the relevant client instead of changing global Node.js state. - Unit-test coverage for the scale-set client and HTTP/session behavior, service configuration and credentials, controller lifecycle and health, and EC2 provider configuration, inventory, reconciliation, scale-up, and scale-down. The Terraform documentation and workflow changes are tracked separately in #5347. ## Test Plan - Added focused TypeScript tests alongside the client, service, controller, and compute-provider implementations. - The scale-set service package provides `typecheck`, `build`, `format-check`, `lint`, and `test` targets for CI validation. - End-to-end ECS deployment is not part of this PR; the service consumes the scale-set configuration and IAM contracts supplied by the Terraform stack. ## Related Issues - Stacked on #5299. - Builds on the EC2 orchestration boundary from #5312. - Follow-up documentation and workflow changes: #5347. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Guilherme Caulada --- .github/dependabot.yml | 16 + .../scripts/scale-set-container-smoke-test.sh | 45 ++ .github/workflows/lambda.yml | 49 ++ .github/workflows/ministack.yml | 75 ++ .github/workflows/release.yml | 74 +- docs/security.md | 10 +- .../compute-providers/aws/ec2/scale-set.ts | 33 + .../aws/ec2/src/runners.test.ts | 7 + .../compute-providers/aws/ec2/src/runners.ts | 23 +- .../ec2/src/scale-set/configuration.test.ts | 28 + .../aws/ec2/src/scale-set/configuration.ts | 348 ++++++++ .../aws/ec2/src/scale-set/inventory.test.ts | 113 +++ .../aws/ec2/src/scale-set/inventory.ts | 241 ++++++ .../aws/ec2/src/scale-set/provider.test.ts | 132 ++++ .../aws/ec2/src/scale-set/provider.ts | 126 +++ .../aws/ec2/src/scale-set/reconcile.test.ts | 32 + .../aws/ec2/src/scale-set/reconcile.ts | 131 +++ .../aws/ec2/src/scale-set/scale-down.test.ts | 195 +++++ .../aws/ec2/src/scale-set/scale-down.ts | 118 +++ .../aws/ec2/src/scale-set/scale-up.test.ts | 200 +++++ .../aws/ec2/src/scale-set/scale-up.ts | 310 ++++++++ .../aws/ec2/src/scale-set/test/fixtures.ts | 103 +++ .../src/scale-set/test/provider-harness.ts | 40 + lambdas/libs/compute-providers/core/index.ts | 2 +- lambdas/libs/compute-providers/package.json | 3 + .../providers.config.scale-set.ts | 5 + .../libs/compute-providers/scale-set.test.ts | 124 +++ lambdas/libs/compute-providers/scale-set.ts | 216 +++++ .../libs/compute-providers/vitest.config.ts | 2 +- .../LICENSE.actions-scaleset | 21 + .../libs/github-actions-scale-set/README.md | 69 ++ .../github-actions-scale-set/package.json | 35 + .../src/client.test.ts | 434 ++++++++++ .../github-actions-scale-set/src/client.ts | 554 +++++++++++++ .../src/config.test.ts | 43 + .../github-actions-scale-set/src/config.ts | 119 +++ .../github-actions-scale-set/src/endpoints.ts | 4 + .../github-actions-scale-set/src/errors.ts | 171 ++++ .../github-actions-scale-set/src/http.test.ts | 299 +++++++ .../libs/github-actions-scale-set/src/http.ts | 409 ++++++++++ .../github-actions-scale-set/src/index.ts | 29 + .../src/message-session-client.test.ts | 319 ++++++++ .../src/message-session-client.ts | 488 ++++++++++++ .../github-actions-scale-set/src/types.ts | 184 +++++ .../libs/github-actions-scale-set/src/url.ts | 24 + .../github-actions-scale-set/tsconfig.json | 8 + .../github-actions-scale-set/vitest.config.ts | 22 + lambdas/package.json | 8 +- lambdas/services/scale-set/Dockerfile | 25 + .../scale-set/Dockerfile.dockerignore | 7 + lambdas/services/scale-set/README.md | 107 +++ lambdas/services/scale-set/healthcheck.cjs | 13 + lambdas/services/scale-set/package.json | 43 + lambdas/services/scale-set/src/config.test.ts | 222 ++++++ lambdas/services/scale-set/src/config.ts | 545 +++++++++++++ lambdas/services/scale-set/src/controller.ts | 51 ++ .../scale-set/src/credentials.test.ts | 200 +++++ lambdas/services/scale-set/src/credentials.ts | 209 +++++ .../scale-set/src/github-http.test.ts | 58 ++ lambdas/services/scale-set/src/github-http.ts | 34 + .../scale-set/src/health-server.test.ts | 15 + .../services/scale-set/src/health-server.ts | 53 ++ lambdas/services/scale-set/src/health.test.ts | 46 ++ lambdas/services/scale-set/src/health.ts | 137 ++++ lambdas/services/scale-set/src/index.ts | 9 + .../services/scale-set/src/lifecycle.test.ts | 45 ++ lambdas/services/scale-set/src/lifecycle.ts | 51 ++ lambdas/services/scale-set/src/logger.test.ts | 50 ++ lambdas/services/scale-set/src/logger.ts | 91 +++ lambdas/services/scale-set/src/main.ts | 144 ++++ .../scale-set/src/parameter-store.test.ts | 78 ++ .../services/scale-set/src/parameter-store.ts | 141 ++++ .../services/scale-set/src/reconciler.test.ts | 539 +++++++++++++ lambdas/services/scale-set/src/reconciler.ts | 743 ++++++++++++++++++ lambdas/services/scale-set/tsconfig.json | 8 + lambdas/services/scale-set/vitest.config.ts | 18 + lambdas/yarn.lock | 625 +++++++++++---- tests/ministack/README.md | 6 + tests/ministack/initializerJson.json | 138 ++++ tests/ministack/multi-runner-scale-set.tfvars | 2 +- tests/ministack/run-scale-set-integration.sh | 600 ++++++++++++++ 81 files changed, 10913 insertions(+), 181 deletions(-) create mode 100755 .github/scripts/scale-set-container-smoke-test.sh create mode 100644 lambdas/libs/compute-providers/aws/ec2/scale-set.ts create mode 100644 lambdas/libs/compute-providers/aws/ec2/src/scale-set/configuration.test.ts create mode 100644 lambdas/libs/compute-providers/aws/ec2/src/scale-set/configuration.ts create mode 100644 lambdas/libs/compute-providers/aws/ec2/src/scale-set/inventory.test.ts create mode 100644 lambdas/libs/compute-providers/aws/ec2/src/scale-set/inventory.ts create mode 100644 lambdas/libs/compute-providers/aws/ec2/src/scale-set/provider.test.ts create mode 100644 lambdas/libs/compute-providers/aws/ec2/src/scale-set/provider.ts create mode 100644 lambdas/libs/compute-providers/aws/ec2/src/scale-set/reconcile.test.ts create mode 100644 lambdas/libs/compute-providers/aws/ec2/src/scale-set/reconcile.ts create mode 100644 lambdas/libs/compute-providers/aws/ec2/src/scale-set/scale-down.test.ts create mode 100644 lambdas/libs/compute-providers/aws/ec2/src/scale-set/scale-down.ts create mode 100644 lambdas/libs/compute-providers/aws/ec2/src/scale-set/scale-up.test.ts create mode 100644 lambdas/libs/compute-providers/aws/ec2/src/scale-set/scale-up.ts create mode 100644 lambdas/libs/compute-providers/aws/ec2/src/scale-set/test/fixtures.ts create mode 100644 lambdas/libs/compute-providers/aws/ec2/src/scale-set/test/provider-harness.ts create mode 100644 lambdas/libs/compute-providers/providers.config.scale-set.ts create mode 100644 lambdas/libs/compute-providers/scale-set.test.ts create mode 100644 lambdas/libs/compute-providers/scale-set.ts create mode 100644 lambdas/libs/github-actions-scale-set/LICENSE.actions-scaleset create mode 100644 lambdas/libs/github-actions-scale-set/README.md create mode 100644 lambdas/libs/github-actions-scale-set/package.json create mode 100644 lambdas/libs/github-actions-scale-set/src/client.test.ts create mode 100644 lambdas/libs/github-actions-scale-set/src/client.ts create mode 100644 lambdas/libs/github-actions-scale-set/src/config.test.ts create mode 100644 lambdas/libs/github-actions-scale-set/src/config.ts create mode 100644 lambdas/libs/github-actions-scale-set/src/endpoints.ts create mode 100644 lambdas/libs/github-actions-scale-set/src/errors.ts create mode 100644 lambdas/libs/github-actions-scale-set/src/http.test.ts create mode 100644 lambdas/libs/github-actions-scale-set/src/http.ts create mode 100644 lambdas/libs/github-actions-scale-set/src/index.ts create mode 100644 lambdas/libs/github-actions-scale-set/src/message-session-client.test.ts create mode 100644 lambdas/libs/github-actions-scale-set/src/message-session-client.ts create mode 100644 lambdas/libs/github-actions-scale-set/src/types.ts create mode 100644 lambdas/libs/github-actions-scale-set/src/url.ts create mode 100644 lambdas/libs/github-actions-scale-set/tsconfig.json create mode 100644 lambdas/libs/github-actions-scale-set/vitest.config.ts create mode 100644 lambdas/services/scale-set/Dockerfile create mode 100644 lambdas/services/scale-set/Dockerfile.dockerignore create mode 100644 lambdas/services/scale-set/README.md create mode 100644 lambdas/services/scale-set/healthcheck.cjs create mode 100644 lambdas/services/scale-set/package.json create mode 100644 lambdas/services/scale-set/src/config.test.ts create mode 100644 lambdas/services/scale-set/src/config.ts create mode 100644 lambdas/services/scale-set/src/controller.ts create mode 100644 lambdas/services/scale-set/src/credentials.test.ts create mode 100644 lambdas/services/scale-set/src/credentials.ts create mode 100644 lambdas/services/scale-set/src/github-http.test.ts create mode 100644 lambdas/services/scale-set/src/github-http.ts create mode 100644 lambdas/services/scale-set/src/health-server.test.ts create mode 100644 lambdas/services/scale-set/src/health-server.ts create mode 100644 lambdas/services/scale-set/src/health.test.ts create mode 100644 lambdas/services/scale-set/src/health.ts create mode 100644 lambdas/services/scale-set/src/index.ts create mode 100644 lambdas/services/scale-set/src/lifecycle.test.ts create mode 100644 lambdas/services/scale-set/src/lifecycle.ts create mode 100644 lambdas/services/scale-set/src/logger.test.ts create mode 100644 lambdas/services/scale-set/src/logger.ts create mode 100644 lambdas/services/scale-set/src/main.ts create mode 100644 lambdas/services/scale-set/src/parameter-store.test.ts create mode 100644 lambdas/services/scale-set/src/parameter-store.ts create mode 100644 lambdas/services/scale-set/src/reconciler.test.ts create mode 100644 lambdas/services/scale-set/src/reconciler.ts create mode 100644 lambdas/services/scale-set/tsconfig.json create mode 100644 lambdas/services/scale-set/vitest.config.ts create mode 100644 tests/ministack/initializerJson.json create mode 100755 tests/ministack/run-scale-set-integration.sh diff --git a/.github/dependabot.yml b/.github/dependabot.yml index b70462a9e1..0280daa315 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -100,6 +100,22 @@ updates: allow: - dependency-name: "ghcr.io/ministackorg/ministack" + - package-ecosystem: "docker" + directory: "/lambdas/services/scale-set" + schedule: + interval: "weekly" + cooldown: + default-days: 7 + labels: + - "dependencies" + - "docker" + commit-message: + prefix: "chore(scale-set)" + # Keep the service runtime aligned with the supported Node.js major. + ignore: + - dependency-name: "node" + update-types: ["version-update:semver-major"] + - package-ecosystem: "pip" directory: "/.github/workflows/mkdocs" schedule: diff --git a/.github/scripts/scale-set-container-smoke-test.sh b/.github/scripts/scale-set-container-smoke-test.sh new file mode 100755 index 0000000000..eea46eea72 --- /dev/null +++ b/.github/scripts/scale-set-container-smoke-test.sh @@ -0,0 +1,45 @@ +#!/usr/bin/env bash +set -euo pipefail + +container_name="scale-set-service-smoke-${GITHUB_RUN_ID:-$$}" +response_file="$(mktemp)" +# shellcheck disable=SC2329 # cleanup is invoked indirectly by the EXIT trap. +cleanup() { + docker rm -f "$container_name" >/dev/null 2>&1 || true + rm -f "$response_file" +} +trap cleanup EXIT + +scale_set_controller_manifest='{"version":1,"groupName":"ci-smoke","revision":"image-test","reconcilers":[{"schemaVersion":1,"runnerConfigName":"smoke","scaleSetId":1,"scaleSetName":"ci-smoke","githubConfigUrl":"https://github.com/example-org","githubApp":{"appIdParameterName":"/ci/app-id","privateKeyParameterName":"/ci/private-key"},"computeProvider":{"type":"ec2","configuration":{"region":"us-east-1","environment":"ci","runnerNamePrefix":"ci","jitConfigParameterPath":"/ci/jit","subnets":["subnet-00000000"],"launchTemplateName":"ci","ec2instanceCriteria":{"instanceTypes":["t3.micro"],"targetCapacityType":"on-demand","instanceAllocationStrategy":"lowest-price"}}},"minRunners":0,"maxRunners":0}]}' + +docker run --detach \ + --name "$container_name" \ + --network none \ + --read-only \ + --cap-drop ALL \ + --security-opt no-new-privileges \ + --env AWS_REGION=us-east-1 \ + --env AWS_EC2_METADATA_DISABLED=true \ + --env SCALE_SET_HEALTH_PORT=8080 \ + --env "SCALE_SET_CONTROLLER_MANIFEST=$scale_set_controller_manifest" \ + scale-set-service:smoke-test + +attempt=0 +while (( attempt < 30 )); do + ((attempt += 1)) + docker exec "$container_name" node --input-type=module -e \ + 'const response = await fetch("http://127.0.0.1:8080/healthz", { signal: AbortSignal.timeout(1000) }); process.stdout.write(JSON.stringify({ status: response.status, body: await response.json() }));' \ + >"$response_file" 2>/dev/null || true + if jq --exit-status \ + --arg group_name ci-smoke \ + '(.status == 200 or .status == 503) and .body.groupName == $group_name and (.body.live | type == "boolean") and (.body.ready | type == "boolean") and .body.reconcilers.smoke != null' \ + "$response_file" >/dev/null 2>&1; then + jq . "$response_file" + exit 0 + fi + sleep 1 +done + +docker logs "$container_name" +echo "scale-set service image did not return the expected health response" >&2 +exit 1 diff --git a/.github/workflows/lambda.yml b/.github/workflows/lambda.yml index 09d96892a1..9de8ab0a12 100644 --- a/.github/workflows/lambda.yml +++ b/.github/workflows/lambda.yml @@ -4,6 +4,7 @@ on: pull_request: paths: - 'lambdas/**' + - '.github/scripts/**' - '.github/workflows/lambda.yml' concurrency: @@ -32,17 +33,23 @@ jobs: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false + - name: Install dependencies run: yarn install --frozen-lockfile + - name: Run prettier run: yarn format-check + - name: Run linter run: yarn lint + - name: Run tests id: test run: yarn test + - name: Build distribution run: yarn build + - name: Upload coverage report uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: ${{ failure() }} @@ -50,3 +57,45 @@ jobs: name: coverage-reports path: ./**/coverage retention-days: 5 + + scale-set-container: + name: Build scale-set service container + runs-on: ubuntu-latest + steps: + - name: Harden the runner (Audit all outbound calls) + uses: step-security/harden-runner@9af89fc71515a100421586dfdb3dc9c984fbf411 # v2.19.4 + with: + egress-policy: audit + + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - name: Set up QEMU + uses: docker/setup-qemu-action@96fe6ef7f33517b61c61be40b68a1882f3264fb8 # v4.2.0 + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0 + + - name: Build scale-set service image + uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0 + with: + context: . + file: ./lambdas/services/scale-set/Dockerfile + platforms: linux/amd64,linux/arm64 + push: false + cache-from: type=gha,scope=scale-set-service + cache-to: type=gha,mode=max,scope=scale-set-service + + - name: Build scale-set service image for smoke test + uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0 + with: + context: . + file: ./lambdas/services/scale-set/Dockerfile + platforms: linux/amd64 + load: true + tags: scale-set-service:smoke-test + cache-from: type=gha,scope=scale-set-service + + - name: Run scale-set service image smoke test + run: ./.github/scripts/scale-set-container-smoke-test.sh diff --git a/.github/workflows/ministack.yml b/.github/workflows/ministack.yml index aa6e2b505d..fb8e14b4a5 100644 --- a/.github/workflows/ministack.yml +++ b/.github/workflows/ministack.yml @@ -196,3 +196,78 @@ jobs: MINISTACK_GITHUB_MOCK_PORT: "1080" MINISTACK_GITHUB_MOCK_URL: ${{ steps.mockserver.outputs.url }} run: sh tests/ministack/run-smoke.sh + + integration_scaleset_smoke: + name: Run scale-set ECS smoke test against MiniStack and MockServer + runs-on: ubuntu-latest + timeout-minutes: 30 + services: + ministack: + image: ghcr.io/ministackorg/ministack:1.5.12@sha256:41fe1ce2e666c6cc410c6047a9db8bf1df69cd0028ebc0a6c6e5517c3a83d6e0 + ports: + - 4566:4566 + options: >- + --add-host=host.docker.internal:host-gateway + --volume /var/run/docker.sock:/var/run/docker.sock + env: + MINISTACK_ACCOUNT_ID: "000000000000" + MINISTACK_REGION: eu-west-1 + steps: + - name: Harden the runner (Audit all outbound calls) + uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1 + with: + egress-policy: audit + + - name: Checkout + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - name: Setup Node.js + uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 + with: + node-version-file: lambdas/.nvmrc + package-manager-cache: false + + - name: Setup Terraform + uses: hashicorp/setup-terraform@dfe3c3f87815947d99a8997f908cb6525fc44e9e # v4.0.1 + with: + terraform_version: latest + terraform_wrapper: false + + - name: Install Lambda dependencies + working-directory: lambdas + run: yarn install --frozen-lockfile + + - name: Build smoke-test Lambda distributions + working-directory: lambdas + run: | + yarn workspace @aws-github-runner/webhook dist + yarn workspace @aws-github-runner/control-plane dist + + - name: Start MockServer + id: mockserver + uses: mock-server/setup-mockserver@24612c2ccef1f83d587f331ed77cc5cef441e0b1 # v1.0.0 + with: + version: '7.6.0@sha256:80b3b1a26f3553d0c81a3f3896b5b7274c17b2a2e52f0fd2b28e246bc9efa290' + port: '1080' + startup-timeout: '60' + + - name: Connect MockServer to MiniStack network + shell: bash + run: | + set -euo pipefail + ministack_container="$(docker ps --format '{{.ID}} {{.Image}}' | awk '$2 ~ /ministack/ {print $1; exit}')" + network="$(docker inspect --format '{{range $name, $_ := .NetworkSettings.Networks}}{{println $name}}{{end}}' "$ministack_container" | sed -n '1p')" + docker network connect --alias mockserver "$network" mockserver + + - name: Mark repository as safe + shell: sh + run: git config --global --add safe.directory "$GITHUB_WORKSPACE" + + - name: Run scale-set ECS/MockServer smoke test + env: + MINISTACK_GITHUB_MOCK_HOST: mockserver + MINISTACK_GITHUB_MOCK_PORT: "1080" + MINISTACK_GITHUB_MOCK_URL: ${{ steps.mockserver.outputs.url }} + run: sh tests/ministack/run-scale-set-integration.sh diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 68f5a38341..59d52d08d8 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -1,4 +1,7 @@ name: Release build +env: + SCALE_SET_IMAGE: ghcr.io/${{ github.repository_owner }}/terraform-aws-github-runner-scale-set-service + on: push: branches: @@ -21,6 +24,8 @@ jobs: actions: write # for release-please-action to trigger other workflows id-token: write # for actions/attest-build-provenance to generate attestations attestations: write # for actions/attest-build-provenance to write attestations + artifact-metadata: write # for publishing linked container attestations + packages: write # for publishing the scale-set service image to GHCR environment: release steps: - name: Harden the runner (Audit all outbound calls) @@ -32,22 +37,27 @@ jobs: with: node-version: 24 package-manager-cache: false + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false + - name: Build dist working-directory: lambdas run: yarn install --frozen-lockfile && yarn run test && yarn dist + - name: Get installation token uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 id: token with: app-id: ${{ vars.RELEASER_APP_ID }} private-key: ${{ secrets.RELEASER_APP_PRIVATE_KEY }} + - name: Extract branch name id: branch shell: bash run: echo "name=${GITHUB_REF#refs/heads/}" >> $GITHUB_OUTPUT + - name: Release id: release uses: googleapis/release-please-action@45996ed1f6d02564a971a2fa1b5860e934307cf7 # v5.0.0 @@ -55,30 +65,85 @@ jobs: target-branch: ${{ steps.branch.outputs.name }} release-type: terraform-module token: ${{ steps.token.outputs.token }} + + - name: Set up QEMU + if: ${{ steps.release.outputs.releases_created == 'true' }} + uses: docker/setup-qemu-action@96fe6ef7f33517b61c61be40b68a1882f3264fb8 # v4.2.0 + + - name: Set up Docker Buildx + if: ${{ steps.release.outputs.releases_created == 'true' }} + uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0 + + - name: Log in to the GitHub Container Registry + if: ${{ steps.release.outputs.releases_created == 'true' }} + uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Build and publish scale-set service image + if: ${{ steps.release.outputs.releases_created == 'true' }} + id: scale-set-image + uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0 + with: + context: . + file: ./lambdas/services/scale-set/Dockerfile + platforms: linux/amd64,linux/arm64 + push: true + tags: | + ${{ env.SCALE_SET_IMAGE }}:${{ steps.release.outputs.tag_name }} + ${{ env.SCALE_SET_IMAGE }}:latest + labels: | + org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }} + org.opencontainers.image.revision=${{ github.sha }} + org.opencontainers.image.version=${{ steps.release.outputs.tag_name }} + sbom: true + provenance: mode=max + cache-from: type=gha,scope=scale-set-service + cache-to: type=gha,mode=max,scope=scale-set-service + + - name: Attest scale-set service image + if: ${{ steps.release.outputs.releases_created == 'true' }} + id: scale-set-image-attest + uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2 + with: + subject-name: ${{ env.SCALE_SET_IMAGE }} + subject-digest: ${{ steps.scale-set-image.outputs.digest }} + push-to-registry: true + - name: Attest if: ${{ steps.release.outputs.releases_created == 'true' }} id: attest uses: actions/attest-build-provenance@4d101475d8b20a2381f78447822ac1eab6504dd8 # v4.2.2 with: subject-path: '${{ github.workspace }}/lambdas/functions/**/*.zip' + - name: Update release notes with attestation if: ${{ steps.release.outputs.releases_created == 'true' }} env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - VERSION: ${{ github.event.inputs.version }} TAG_NAME: ${{ steps.release.outputs.tag_name }} ATTESTATION_URL: ${{ steps.attest.outputs.attestation-url }} + CONTAINER_ATTESTATION_URL: ${{ steps.scale-set-image-attest.outputs.attestation-url }} + CONTAINER_IMAGE: ${{ env.SCALE_SET_IMAGE }} + CONTAINER_DIGEST: ${{ steps.scale-set-image.outputs.digest }} REPOSITORY: ${{ github.repository }} run: | - version="${VERSION}" tag_name="${TAG_NAME}" attestation_url="${ATTESTATION_URL}" + container_attestation_url="${CONTAINER_ATTESTATION_URL}" + container_image="${CONTAINER_IMAGE}" + container_digest="${CONTAINER_DIGEST}" repository="${REPOSITORY}" - gh release view $version --json body -q '.body' > new-release-notes.md + gh release view "$tag_name" --json body -q '.body' > new-release-notes.md echo "## Attestation" >> new-release-notes.md echo "Attestation url: $attestation_url" >> new-release-notes.md echo "Verify the artifacts by running \`gh attestation verify --repo ${repository}\`" >> new-release-notes.md - gh release edit $tag_name -F new-release-notes.md -t $tag_name + echo "Scale-set service image: \`${container_image}@${container_digest}\`" >> new-release-notes.md + echo "Container attestation url: $container_attestation_url" >> new-release-notes.md + gh release edit "$tag_name" -F new-release-notes.md -t "$tag_name" + - name: Upload release assets if: ${{ steps.release.outputs.releases_created == 'true' }} env: @@ -89,6 +154,7 @@ jobs: for f in $(find . -name '*.zip'); do gh release upload $tag_name $f done + - name: Attach attestation if: ${{ steps.release.outputs.releases_created == 'true' }} env: diff --git a/docs/security.md b/docs/security.md index a94688b234..4ef4d17b94 100644 --- a/docs/security.md +++ b/docs/security.md @@ -14,6 +14,14 @@ The examples are using standard AMI's for different operating systems. Instances ## Attestation -The module is released using GitHub actions and the lambda artifacts are attached to the release as attachment. During the release attestations are created. The attestations are created by the release pipeline. You find a link to the attestation in the GitHub release. The attestation only provides provenance information about the release. The attestations are not a security guarantee. We recommend you to verify the attestation after downloading the lambda artifacts. +The module is released using GitHub Actions and the Lambda artifacts are attached to the release. The release pipeline creates provenance attestations for those artifacts. You can find a link to the attestation in the GitHub release. The attestation only provides provenance information about the release; it is not a security guarantee. We recommend verifying the attestation after downloading the Lambda artifacts. + +Releases also publish the multi-architecture scale-set service image to the GitHub Container Registry with an SBOM, build provenance, and a registry attestation. The convenience image default follows the latest module release. Production deployments should override it with the immutable image digest printed in the release notes, then verify that image with: + +```bash +gh attestation verify \ + oci://ghcr.io/github-aws-runners/terraform-aws-github-runner-scale-set-service@sha256: \ + --repo github-aws-runners/terraform-aws-github-runner +``` --8<-- "SECURITY.md:mkdocsrunners" diff --git a/lambdas/libs/compute-providers/aws/ec2/scale-set.ts b/lambdas/libs/compute-providers/aws/ec2/scale-set.ts new file mode 100644 index 0000000000..c1eff12475 --- /dev/null +++ b/lambdas/libs/compute-providers/aws/ec2/scale-set.ts @@ -0,0 +1,33 @@ +import type { ScaleSetComputeProviderModule, ScaleSetComputeProviderPlugin } from '../../scale-set'; + +import { createEc2ScaleSetProvider, type Ec2ScaleSetProviderDependencies } from './src/scale-set/provider'; + +export type { Ec2ScaleSetProviderConfig, Ec2ScaleSetProviderDependencies } from './src/scale-set/provider'; +export { createEc2ScaleSetProvider, parseEc2ScaleSetProviderConfig } from './src/scale-set/provider'; + +export function createEc2ScaleSetPlugin( + dependencies: Ec2ScaleSetProviderDependencies = {}, +): ScaleSetComputeProviderPlugin<'ec2'> { + return { + type: 'ec2', + capabilities: { + environmentVariables: {}, + create: ({ runnerConfigName, scaleSetId, githubScope, credentials, configuration }) => + createEc2ScaleSetProvider( + { + runnerConfigName, + scaleSetId, + githubScope, + credentials, + configuration: configuration as Parameters[0]['configuration'], + }, + dependencies, + ), + }, + }; +} + +export const provider = { + type: 'ec2', + createPlugin: createEc2ScaleSetPlugin, +} satisfies ScaleSetComputeProviderModule<'ec2'>; diff --git a/lambdas/libs/compute-providers/aws/ec2/src/runners.test.ts b/lambdas/libs/compute-providers/aws/ec2/src/runners.test.ts index 9e5c116ef4..1abea33b70 100644 --- a/lambdas/libs/compute-providers/aws/ec2/src/runners.test.ts +++ b/lambdas/libs/compute-providers/aws/ec2/src/runners.test.ts @@ -1344,6 +1344,7 @@ function expectedCreateFleetRequest(expectedValues: ExpectedFleetRequestValues): Key: 'ghr:created_by', Value: expectedValues.source, }, + { Key: 'ghr:environment', Value: ENVIRONMENT }, { Key: 'ghr:Type', Value: expectedValues.type }, { Key: 'ghr:Owner', Value: REPO_NAME }, ]; @@ -1519,6 +1520,7 @@ describe('create runner with useDedicatedHost', () => { Tags: [ { Key: 'ghr:Application', Value: 'github-action-runner' }, { Key: 'ghr:created_by', Value: 'scale-up-lambda' }, + { Key: 'ghr:environment', Value: ENVIRONMENT }, { Key: 'ghr:Type', Value: 'Org' }, { Key: 'ghr:Owner', Value: REPO_NAME }, ], @@ -1528,6 +1530,7 @@ describe('create runner with useDedicatedHost', () => { Tags: [ { Key: 'ghr:Application', Value: 'github-action-runner' }, { Key: 'ghr:created_by', Value: 'scale-up-lambda' }, + { Key: 'ghr:environment', Value: ENVIRONMENT }, { Key: 'ghr:Type', Value: 'Org' }, { Key: 'ghr:Owner', Value: REPO_NAME }, ], @@ -1567,6 +1570,7 @@ describe('create runner with useDedicatedHost', () => { Tags: [ { Key: 'ghr:Application', Value: 'github-action-runner' }, { Key: 'ghr:created_by', Value: 'scale-up-lambda' }, + { Key: 'ghr:environment', Value: ENVIRONMENT }, { Key: 'ghr:Type', Value: 'Org' }, { Key: 'ghr:Owner', Value: REPO_NAME }, ], @@ -1576,6 +1580,7 @@ describe('create runner with useDedicatedHost', () => { Tags: [ { Key: 'ghr:Application', Value: 'github-action-runner' }, { Key: 'ghr:created_by', Value: 'scale-up-lambda' }, + { Key: 'ghr:environment', Value: ENVIRONMENT }, { Key: 'ghr:Type', Value: 'Org' }, { Key: 'ghr:Owner', Value: REPO_NAME }, ], @@ -1702,6 +1707,7 @@ describe('create runner with useDedicatedHost', () => { Tags: [ { Key: 'ghr:Application', Value: 'github-action-runner' }, { Key: 'ghr:created_by', Value: 'scale-up-lambda' }, + { Key: 'ghr:environment', Value: ENVIRONMENT }, { Key: 'ghr:Type', Value: 'Org' }, { Key: 'ghr:Owner', Value: REPO_NAME }, ], @@ -1711,6 +1717,7 @@ describe('create runner with useDedicatedHost', () => { Tags: [ { Key: 'ghr:Application', Value: 'github-action-runner' }, { Key: 'ghr:created_by', Value: 'scale-up-lambda' }, + { Key: 'ghr:environment', Value: ENVIRONMENT }, { Key: 'ghr:Type', Value: 'Org' }, { Key: 'ghr:Owner', Value: REPO_NAME }, ], diff --git a/lambdas/libs/compute-providers/aws/ec2/src/runners.ts b/lambdas/libs/compute-providers/aws/ec2/src/runners.ts index 6269790adb..b03eb9fec3 100644 --- a/lambdas/libs/compute-providers/aws/ec2/src/runners.ts +++ b/lambdas/libs/compute-providers/aws/ec2/src/runners.ts @@ -209,6 +209,7 @@ interface AwsErrorLike extends Error { $fault?: 'client' | 'server'; $metadata?: { httpStatusCode?: number; + requestId?: string; }; } @@ -219,8 +220,18 @@ function safeFailureIdentifier(value: unknown): string | undefined { return typeof value === 'string' && SAFE_FAILURE_IDENTIFIER.test(value) ? value : undefined; } -function failureMessage(error: unknown): string { - return error instanceof Error ? error.message : String(error); +function failureDetails(error: unknown): Record { + if (!(error instanceof Error)) return { errorMessage: String(error) }; + + const awsError = error as AwsErrorLike; + return { + errorName: error.name, + errorMessage: error.message, + ...(awsError.code === undefined ? {} : { errorCode: awsError.code }), + ...(awsError.$fault === undefined ? {} : { errorFault: awsError.$fault }), + ...(awsError.$metadata?.httpStatusCode === undefined ? {} : { httpStatusCode: awsError.$metadata.httpStatusCode }), + ...(awsError.$metadata?.requestId === undefined ? {} : { requestId: awsError.$metadata.requestId }), + }; } function requestFailureCodes(error: unknown): Ec2RunnerFailureCode[] { @@ -394,7 +405,7 @@ async function createEc2Runner( const failureCodes = requestFailureCodes(error); logger.warn('Runner creation failed before an EC2 request could be made.', { failedInstanceCount: runnerParameters.numberOfRunners, - error: failureMessage(error), + ...failureDetails(error), failureCodes, }); return failedCreateRunnerResult(runnerParameters.numberOfRunners, failureCodes); @@ -417,7 +428,7 @@ async function createEc2Runner( const failureCodes = requestFailureCodes(error); logger.warn('Create fleet request failed.', { failedInstanceCount: runnerParameters.numberOfRunners, - error: failureMessage(error), + ...failureDetails(error), failureCodes, }); return failedCreateRunnerResult(runnerParameters.numberOfRunners, failureCodes); @@ -566,6 +577,7 @@ async function createInstances( const tags = [ { Key: 'ghr:Application', Value: 'github-action-runner' }, { Key: 'ghr:created_by', Value: runnerParameters.source }, + { Key: 'ghr:environment', Value: runnerParameters.environment }, { Key: 'ghr:Type', Value: runnerParameters.runnerType }, { Key: 'ghr:Owner', Value: runnerParameters.runnerOwner }, ]; @@ -644,6 +656,7 @@ async function createInstancesWithRunInstances( const tags = [ { Key: 'ghr:Application', Value: 'github-action-runner' }, { Key: 'ghr:created_by', Value: runnerParameters.source }, + { Key: 'ghr:environment', Value: runnerParameters.environment }, { Key: 'ghr:Type', Value: runnerParameters.runnerType }, { Key: 'ghr:Owner', Value: runnerParameters.runnerOwner }, ]; @@ -692,7 +705,7 @@ async function createInstancesWithRunInstances( const failureCodes = requestFailureCodes(error); logger.warn('RunInstances request failed for dedicated host.', { failedInstanceCount: runnerParameters.numberOfRunners, - error: failureMessage(error), + ...failureDetails(error), failureCodes, }); return failedCreateRunnerResult(runnerParameters.numberOfRunners, failureCodes); diff --git a/lambdas/libs/compute-providers/aws/ec2/src/scale-set/configuration.test.ts b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/configuration.test.ts new file mode 100644 index 0000000000..5fe55d59aa --- /dev/null +++ b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/configuration.test.ts @@ -0,0 +1,28 @@ +import { describe, expect, it } from 'vitest'; + +import { parseEc2ScaleSetProviderConfig } from './configuration'; +import { config } from './test/fixtures'; + +describe('EC2 scale-set provider configuration', () => { + it('strictly parses the supported provider-owned configuration', () => { + expect(parseEc2ScaleSetProviderConfig(config)).toMatchObject(config); + expect(parseEc2ScaleSetProviderConfig({ ...config, runnerNamePrefix: '' })).toMatchObject({ + runnerNamePrefix: '', + }); + expect(parseEc2ScaleSetProviderConfig({ ...config, runnerNamePrefix: 'r'.repeat(45) })).toMatchObject({ + runnerNamePrefix: 'r'.repeat(45), + }); + }); + + it.each([ + [{ ...config, region: 'eu-west-one' }], + [{ ...config, subnets: ['subnet-12345678', 'subnet-12345678'] }], + [{ ...config, ec2instanceCriteria: { ...config.ec2instanceCriteria, instanceAllocationStrategy: 'diversified' } }], + [{ ...config, ec2OverrideConfig: { UserData: 'untrusted' } }], + [{ ...config, ssmParameterTags: [{ Key: 'aws:owner', Value: 'untrusted' }] }], + [{ ...config, runnerNamePrefix: 'r'.repeat(46) }], + [{ ...config, bootTimeoutMinutes: 10 }], + ])('rejects invalid or unsupported values instead of forwarding them to AWS', (invalid) => { + expect(() => parseEc2ScaleSetProviderConfig(invalid)).toThrow(); + }); +}); diff --git a/lambdas/libs/compute-providers/aws/ec2/src/scale-set/configuration.ts b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/configuration.ts new file mode 100644 index 0000000000..91ae93d555 --- /dev/null +++ b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/configuration.ts @@ -0,0 +1,348 @@ +import type { Tag as SsmTag } from '@aws-sdk/client-ssm'; + +import type { Ec2OverrideConfig, RunnerInputParameters } from '../runners.d'; +import type { ScaleSetComputeProviderCredentialProvider } from '../../../../scale-set'; +import { isRecord, Ec2ScaleSetValidationError } from './reconcile'; + +const SPOT_ALLOCATION_STRATEGIES = new Set([ + 'lowest-price', + 'diversified', + 'capacity-optimized', + 'capacity-optimized-prioritized', + 'price-capacity-optimized', +]); +const ON_DEMAND_ALLOCATION_STRATEGIES = new Set(['lowest-price', 'prioritized']); + +export interface Ec2ScaleSetProviderConfig { + region: string; + environment: string; + runnerNamePrefix: string; + jitConfigParameterPath: string; + subnets: string[]; + launchTemplateName: string; + ec2instanceCriteria: RunnerInputParameters['ec2instanceCriteria']; + ec2OverrideConfig?: Ec2OverrideConfig; + amiIdSsmParameterName?: string; + tracingEnabled?: boolean; + onDemandFailoverOnError?: string[]; + useDedicatedHost?: boolean; + ssmKmsKeyId?: string; + ssmParameterTags?: SsmTag[]; +} + +export interface CreateEc2ScaleSetProviderInput { + runnerConfigName: string; + scaleSetId: number; + githubScope: string; + credentials?: ScaleSetComputeProviderCredentialProvider; + configuration: Ec2ScaleSetProviderConfig; +} + +function rejectUnknownKeys(value: Record, allowedKeys: ReadonlySet, name: string): void { + const unknownKey = Object.keys(value).find((key) => !allowedKeys.has(key)); + if (unknownKey !== undefined) { + throw new Ec2ScaleSetValidationError(`Unsupported EC2 scale-set configuration field '${name}.${unknownKey}'`); + } +} + +function requireString(value: unknown, name: string, pattern: RegExp, maximumLength: number): string { + if (typeof value !== 'string' || value.length === 0 || value.length > maximumLength || !pattern.test(value)) { + throw new Ec2ScaleSetValidationError(`Invalid EC2 scale-set configuration field '${name}'`); + } + return value; +} + +function requirePossiblyEmptyString(value: unknown, name: string, pattern: RegExp, maximumLength: number): string { + if (typeof value !== 'string' || value.length > maximumLength || !pattern.test(value)) { + throw new Ec2ScaleSetValidationError(`Invalid EC2 scale-set configuration field '${name}'`); + } + return value; +} + +function optionalString(value: unknown, name: string, pattern: RegExp, maximumLength: number): string | undefined { + if (value === undefined) return undefined; + return requireString(value, name, pattern, maximumLength); +} + +function optionalBoolean(value: unknown, name: string): boolean | undefined { + if (value === undefined) return undefined; + if (typeof value !== 'boolean') { + throw new Ec2ScaleSetValidationError(`Invalid EC2 scale-set configuration field '${name}'`); + } + return value; +} + +function requireStringArray( + value: unknown, + name: string, + pattern: RegExp, + maximumItemLength: number, + allowEmpty = false, +): string[] { + if (!Array.isArray(value) || (!allowEmpty && value.length === 0) || value.length > 100) { + throw new Ec2ScaleSetValidationError(`Invalid EC2 scale-set configuration field '${name}'`); + } + const parsed = value.map((item, index) => requireString(item, `${name}[${index}]`, pattern, maximumItemLength)); + if (new Set(parsed).size !== parsed.length) { + throw new Ec2ScaleSetValidationError(`EC2 scale-set configuration field '${name}' contains duplicate values`); + } + return parsed; +} + +function parseInstanceTypePriorities(value: unknown): Record | undefined { + if (value === undefined) return undefined; + if (!isRecord(value)) { + throw new Ec2ScaleSetValidationError("Invalid EC2 scale-set configuration field 'instanceTypePriorities'"); + } + + const result = Object.create(null) as Record; + for (const [instanceType, priority] of Object.entries(value)) { + requireString(instanceType, 'instanceTypePriorities key', /^[a-z0-9][a-z0-9.-]*$/, 64); + if (typeof priority !== 'number' || !Number.isSafeInteger(priority) || priority < 0 || priority > 1000) { + throw new Ec2ScaleSetValidationError( + `Invalid EC2 scale-set configuration priority for instance type '${instanceType}'`, + ); + } + result[instanceType] = priority; + } + return result; +} + +function requireSsmTagValue(value: unknown): string { + if (typeof value !== 'string' || value.length > 256) { + throw new Ec2ScaleSetValidationError("Invalid EC2 scale-set configuration field 'ssmParameterTags.Value'"); + } + for (const character of value) { + const codePoint = character.codePointAt(0)!; + if (codePoint < 32 || codePoint === 127) { + throw new Ec2ScaleSetValidationError("Invalid EC2 scale-set configuration field 'ssmParameterTags.Value'"); + } + } + return value; +} + +function parseEc2OverrideConfig(value: unknown): Ec2OverrideConfig | undefined { + if (value === undefined) return undefined; + if (!isRecord(value)) { + throw new Ec2ScaleSetValidationError("Invalid EC2 scale-set configuration field 'ec2OverrideConfig'"); + } + + const supportedKeys = new Set([ + 'InstanceType', + 'MaxPrice', + 'SubnetId', + 'AvailabilityZone', + 'AvailabilityZoneId', + 'WeightedCapacity', + 'Priority', + 'ImageId', + ]); + if (Object.keys(value).some((key) => !supportedKeys.has(key))) { + throw new Ec2ScaleSetValidationError('EC2 scale-set configuration contains an unsupported launch override'); + } + + const weightedCapacity = value.WeightedCapacity; + const priority = value.Priority; + for (const [name, number] of [ + ['WeightedCapacity', weightedCapacity], + ['Priority', priority], + ] as const) { + if (number !== undefined && (typeof number !== 'number' || !Number.isFinite(number) || number < 0)) { + throw new Ec2ScaleSetValidationError(`Invalid EC2 scale-set configuration field '${name}'`); + } + } + + return { + InstanceType: optionalString( + value.InstanceType, + 'ec2OverrideConfig.InstanceType', + /^[a-z0-9][a-z0-9.-]*$/, + 64, + ) as Ec2OverrideConfig['InstanceType'], + MaxPrice: optionalString(value.MaxPrice, 'ec2OverrideConfig.MaxPrice', /^(?:0|[1-9]\d*)(?:\.\d{1,6})?$/, 32), + SubnetId: optionalString(value.SubnetId, 'ec2OverrideConfig.SubnetId', /^subnet-[0-9a-f]+$/, 32), + AvailabilityZone: optionalString( + value.AvailabilityZone, + 'ec2OverrideConfig.AvailabilityZone', + /^[a-z]{2}(?:-[a-z0-9]+)+-\d[a-z]$/, + 64, + ), + AvailabilityZoneId: optionalString( + value.AvailabilityZoneId, + 'ec2OverrideConfig.AvailabilityZoneId', + /^[a-z0-9-]+$/, + 64, + ), + WeightedCapacity: weightedCapacity as number | undefined, + Priority: priority as number | undefined, + ImageId: optionalString(value.ImageId, 'ec2OverrideConfig.ImageId', /^ami-[0-9a-f]+$/, 32), + }; +} + +function parseSsmTags(value: unknown): SsmTag[] | undefined { + if (value === undefined) return undefined; + if (!Array.isArray(value) || value.length > 45) { + throw new Ec2ScaleSetValidationError("Invalid EC2 scale-set configuration field 'ssmParameterTags'"); + } + + const tags: SsmTag[] = []; + const keys = new Set(); + for (const item of value) { + if (!isRecord(item)) { + throw new Ec2ScaleSetValidationError("Invalid EC2 scale-set configuration field 'ssmParameterTags'"); + } + const key = requireString(item.Key, 'ssmParameterTags.Key', /^[A-Za-z0-9_.:/=+@-]+$/, 128); + const tagValue = requireSsmTagValue(item.Value); + if (key.toLowerCase().startsWith('aws:') || keys.has(key)) { + throw new Ec2ScaleSetValidationError(`Invalid or duplicate SSM tag key '${key}'`); + } + keys.add(key); + tags.push({ Key: key, Value: tagValue }); + } + return tags; +} + +export function parseEc2ScaleSetProviderConfig(value: unknown): Ec2ScaleSetProviderConfig { + if (!isRecord(value)) { + throw new Ec2ScaleSetValidationError('EC2 scale-set provider configuration must be an object'); + } + rejectUnknownKeys( + value, + new Set([ + 'region', + 'environment', + 'runnerNamePrefix', + 'jitConfigParameterPath', + 'subnets', + 'launchTemplateName', + 'ec2instanceCriteria', + 'ec2OverrideConfig', + 'amiIdSsmParameterName', + 'tracingEnabled', + 'onDemandFailoverOnError', + 'useDedicatedHost', + 'ssmKmsKeyId', + 'ssmParameterTags', + ]), + 'configuration', + ); + if (!isRecord(value.ec2instanceCriteria)) { + throw new Ec2ScaleSetValidationError("Invalid EC2 scale-set configuration field 'ec2instanceCriteria'"); + } + rejectUnknownKeys( + value.ec2instanceCriteria, + new Set([ + 'instanceTypes', + 'instanceTypePriorities', + 'targetCapacityType', + 'maxSpotPrice', + 'instanceAllocationStrategy', + ]), + 'ec2instanceCriteria', + ); + + const targetCapacityType = value.ec2instanceCriteria.targetCapacityType; + if (targetCapacityType !== 'on-demand' && targetCapacityType !== 'spot') { + throw new Ec2ScaleSetValidationError("Invalid EC2 scale-set configuration field 'targetCapacityType'"); + } + const instanceAllocationStrategy = requireString( + value.ec2instanceCriteria.instanceAllocationStrategy, + 'instanceAllocationStrategy', + /^[a-z-]+$/, + 64, + ) as RunnerInputParameters['ec2instanceCriteria']['instanceAllocationStrategy']; + const allowedAllocationStrategies = + targetCapacityType === 'spot' ? SPOT_ALLOCATION_STRATEGIES : ON_DEMAND_ALLOCATION_STRATEGIES; + if (!allowedAllocationStrategies.has(instanceAllocationStrategy)) { + throw new Ec2ScaleSetValidationError( + `Invalid allocation strategy '${instanceAllocationStrategy}' for '${targetCapacityType}' capacity`, + ); + } + + const jitConfigParameterPath = requireString( + value.jitConfigParameterPath, + 'jitConfigParameterPath', + /^\/(?!.*\/\/)[A-Za-z0-9_.\-/]+$/, + 900, + ).replace(/\/$/, ''); + + return { + region: requireString(value.region, 'region', /^[a-z]{2}(?:-[a-z0-9]+)+-\d$/, 32), + environment: requireString(value.environment, 'environment', /^[A-Za-z0-9][A-Za-z0-9._-]*$/, 128), + runnerNamePrefix: requirePossiblyEmptyString(value.runnerNamePrefix, 'runnerNamePrefix', /^[A-Za-z0-9._-]*$/, 45), + jitConfigParameterPath, + subnets: requireStringArray(value.subnets, 'subnets', /^subnet-[0-9a-f]+$/, 32), + launchTemplateName: requireString(value.launchTemplateName, 'launchTemplateName', /^[A-Za-z0-9()./_-]+$/, 128), + ec2instanceCriteria: { + instanceTypes: requireStringArray( + value.ec2instanceCriteria.instanceTypes, + 'instanceTypes', + /^[a-z0-9][a-z0-9.-]*$/, + 64, + ), + instanceTypePriorities: parseInstanceTypePriorities(value.ec2instanceCriteria.instanceTypePriorities), + targetCapacityType, + maxSpotPrice: optionalString( + value.ec2instanceCriteria.maxSpotPrice, + 'maxSpotPrice', + /^(?:0|[1-9]\d*)(?:\.\d{1,6})?$/, + 32, + ), + instanceAllocationStrategy, + }, + ec2OverrideConfig: parseEc2OverrideConfig(value.ec2OverrideConfig), + amiIdSsmParameterName: optionalString( + value.amiIdSsmParameterName, + 'amiIdSsmParameterName', + /^\/(?!.*\/\/)[A-Za-z0-9_.\-/]+$/, + 900, + ), + tracingEnabled: optionalBoolean(value.tracingEnabled, 'tracingEnabled'), + onDemandFailoverOnError: requireStringArray( + value.onDemandFailoverOnError ?? [], + 'onDemandFailoverOnError', + /^[A-Za-z0-9._-]+$/, + 128, + true, + ), + useDedicatedHost: optionalBoolean(value.useDedicatedHost, 'useDedicatedHost'), + ssmKmsKeyId: optionalString(value.ssmKmsKeyId, 'ssmKmsKeyId', /^[A-Za-z0-9_:/+=,.@-]+$/, 2048), + ssmParameterTags: parseSsmTags(value.ssmParameterTags), + }; +} + +export function validateFactoryInput(input: CreateEc2ScaleSetProviderInput): void { + requireString(input.runnerConfigName, 'runnerConfigName', /^[A-Za-z0-9][A-Za-z0-9._-]*$/, 128); + if (!Number.isSafeInteger(input.scaleSetId) || input.scaleSetId <= 0) { + throw new Ec2ScaleSetValidationError('scaleSetId must be a positive safe integer'); + } + validateCanonicalGitHubScope(input.githubScope); +} + +function validateCanonicalGitHubScope(value: unknown): string { + if (typeof value !== 'string' || value.length === 0 || value.length > 2048) { + throw new Ec2ScaleSetValidationError('githubScope must be a canonical HTTPS GitHub configuration URL'); + } + let url: URL; + try { + url = new URL(value); + } catch { + throw new Ec2ScaleSetValidationError('githubScope must be a canonical HTTPS GitHub configuration URL'); + } + if (url.protocol !== 'https:' || url.username || url.password || url.search || url.hash) { + throw new Ec2ScaleSetValidationError('githubScope must be a canonical HTTPS GitHub configuration URL'); + } + const parts = url.pathname + .replace(/^\/+|\/+$/g, '') + .split('/') + .filter(Boolean); + if (parts.length < 1 || parts.length > 2 || (parts[0].toLowerCase() === 'enterprises' && parts.length !== 2)) { + throw new Ec2ScaleSetValidationError('githubScope must be a canonical HTTPS GitHub configuration URL'); + } + url.pathname = `/${parts.join('/')}`; + const canonical = url.toString().replace(/\/$/, ''); + if (canonical !== value) { + throw new Ec2ScaleSetValidationError('githubScope must be a canonical HTTPS GitHub configuration URL'); + } + return value; +} diff --git a/lambdas/libs/compute-providers/aws/ec2/src/scale-set/inventory.test.ts b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/inventory.test.ts new file mode 100644 index 0000000000..0823b3b7ea --- /dev/null +++ b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/inventory.test.ts @@ -0,0 +1,113 @@ +import { CreateFleetCommand, DescribeInstancesCommand } from '@aws-sdk/client-ec2'; +import { beforeEach, describe, expect, it } from 'vitest'; + +import { createRequest, githubState, ownedInstance } from './test/fixtures'; +import { createTestProvider, ec2Mock, resetAwsMocks } from './test/provider-harness'; + +beforeEach(resetAwsMocks); + +describe('EC2 scale-set inventory', () => { + it('lists only the exact environment, owner, and type ownership boundary', async () => { + ec2Mock.on(DescribeInstancesCommand).resolves({ + Reservations: [ + { + Instances: [ + ownedInstance('i-owned', { runnerId: 101, runnerName: 'runner-i-owned' }), + ownedInstance('i-other-environment', undefined, { environment: 'other' }), + ownedInstance('i-other-owner', undefined, { runnerOwner: 'another' }), + ], + }, + ], + }); + + const result = await createTestProvider().reconcile(createRequest()); + + expect(result).toMatchObject({ status: 'converged', desiredRunners: 1, currentRunners: 1 }); + expect(ec2Mock).toHaveReceivedCommandWith(DescribeInstancesCommand, { + Filters: expect.arrayContaining([ + { Name: 'tag:ghr:Application', Values: ['github-action-runner'] }, + { Name: 'tag:ghr:created_by', Values: ['scale-set-service'] }, + { Name: 'tag:ghr:environment', Values: ['unit-test'] }, + { Name: 'tag:ghr:Type', Values: ['Org'] }, + { Name: 'tag:ghr:Owner', Values: ['example'] }, + ]), + }); + expect(ec2Mock).not.toHaveReceivedCommand(CreateFleetCommand); + }); + + it('counts a young handed-off instance as serving during its bounded boot window', async () => { + ec2Mock.on(DescribeInstancesCommand).resolves({ + Reservations: [ + { + Instances: [ownedInstance('i-booting', { runnerId: 101, runnerName: 'runner-i-booting' })], + }, + ], + }); + + const result = await createTestProvider({ now: () => new Date('2026-08-24T10:09:59Z').getTime() }).reconcile( + createRequest(), + ); + + expect(result).toMatchObject({ + status: 'converged', + currentRunners: 1, + actions: { launched: 0, retainedUnknown: 0 }, + }); + expect(ec2Mock).not.toHaveReceivedCommand(CreateFleetCommand); + }); + + it('uses the orchestration request boot window instead of provider configuration', async () => { + ec2Mock.on(DescribeInstancesCommand).resolves({ + Reservations: [ + { + Instances: [ownedInstance('i-at-timeout', { runnerId: 101, runnerName: 'runner-i-at-timeout' })], + }, + ], + }); + + const result = await createTestProvider({ now: () => new Date('2026-08-24T10:05:00Z').getTime() }).reconcile( + createRequest({ bootTimeoutMinutes: 5 }), + ); + + expect(result).toMatchObject({ + status: 'converged', + currentRunners: 1, + actions: { launched: 0, retainedUnknown: 0 }, + }); + expect(ec2Mock).not.toHaveReceivedCommand(CreateFleetCommand); + }); + + it('counts tagged capacity after the boot window without public runner inventory', async () => { + const instance = ownedInstance('i-old', { runnerId: 101, runnerName: 'runner-i-old' }); + ec2Mock.on(DescribeInstancesCommand).resolves({ Reservations: [{ Instances: [instance] }] }); + const result = await createTestProvider({ now: () => new Date('2026-08-24T10:10:00Z').getTime() }).reconcile( + createRequest({ busyRunners: 1 }), + ); + + expect(result).toMatchObject({ + status: 'converged', + currentRunners: 1, + actions: { launched: 0, retainedUnknown: 0 }, + }); + expect(ec2Mock).not.toHaveReceivedCommand(CreateFleetCommand); + }); + + it('counts an exact JobStarted identity as serving without waiting for public inventory', async () => { + const instance = ownedInstance('i-started', { runnerId: 101, runnerName: 'runner-i-started' }); + ec2Mock.on(DescribeInstancesCommand).resolves({ Reservations: [{ Instances: [instance] }] }); + + const result = await createTestProvider({ now: () => new Date('2026-08-24T12:00:00Z').getTime() }).reconcile( + createRequest({ + runnerStates: [ + githubState(101, 'runner-i-started', { status: 'unknown', busy: undefined, lifecycle: 'started' }), + ], + }), + ); + + expect(result).toMatchObject({ + status: 'converged', + currentRunners: 1, + actions: { launched: 0, retainedUnknown: 0 }, + }); + }); +}); diff --git a/lambdas/libs/compute-providers/aws/ec2/src/scale-set/inventory.ts b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/inventory.ts new file mode 100644 index 0000000000..8abbd0a6ad --- /dev/null +++ b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/inventory.ts @@ -0,0 +1,241 @@ +import { createHash } from 'node:crypto'; + +import { DescribeInstancesCommand, type EC2Client, type Instance } from '@aws-sdk/client-ec2'; + +import type { ScaleSetReconcileRequest, ScaleSetRunnerState } from '../../../../scale-set'; +import type { CreateEc2ScaleSetProviderInput } from './configuration'; +import { retainUnknown, type MutableReconcileState } from './reconcile'; + +export const EC2_RUNNER_CONFIG_TAG = 'ghr:runner_config'; +export const EC2_SCALE_SET_ID_TAG = 'ghr:scale_set_id'; +export const EC2_GITHUB_SCOPE_HASH_TAG = 'ghr:github_scope_hash'; +export const EC2_SCALE_SET_STATE_TAG = 'ghr:scale_set_state'; +export const EC2_RUNNER_NAME_TAG = 'ghr:runner_name'; +export const EC2_GITHUB_RUNNER_ID_TAG = 'ghr:github_runner_id'; + +const APPLICATION_TAG = 'ghr:Application'; +const APPLICATION_VALUE = 'github-action-runner'; +const CREATED_BY_TAG = 'ghr:created_by'; +export const SCALE_SET_RUNNER_SOURCE = 'scale-set-service'; +const ENVIRONMENT_TAG = 'ghr:environment'; +const RUNNER_TYPE_TAG = 'ghr:Type'; +const RUNNER_OWNER_TAG = 'ghr:Owner'; +export const GITHUB_RUNNER_NAME_MAX_LENGTH = 64; + +type Ec2ScaleSetState = 'provisioning' | 'publishing' | 'config-published' | 'retiring'; + +export interface OwnedEc2Runner { + instanceId: string; + launchTime?: Date; + githubRunnerId?: number; + runnerName?: string; + scaleSetState?: Ec2ScaleSetState; +} + +export function githubScopeHash(githubScope: string): string { + return createHash('sha256').update(githubScope, 'utf8').digest('hex'); +} + +export function runnerIdentityFromGitHubScope(githubScope: string): { + runnerOwner: string; + runnerType: 'Org' | 'Repo'; +} { + const pathParts = new URL(githubScope).pathname.replace(/^\/+|\/+$/g, '').split('/'); + if (pathParts.length === 2 && pathParts[0].toLowerCase() !== 'enterprises') { + return { runnerOwner: pathParts.join('/'), runnerType: 'Repo' }; + } + + // The legacy EC2 tags do not have an enterprise discriminator. The runner + // type and owner tags remain the ownership boundary for scale-set inventory. + return { + runnerOwner: pathParts[0].toLowerCase() === 'enterprises' ? pathParts[1] : pathParts[0], + runnerType: 'Org', + }; +} + +export async function listOwnedRunners( + input: CreateEc2ScaleSetProviderInput, + ec2Client: EC2Client, + signal: AbortSignal, +): Promise { + const runners: OwnedEc2Runner[] = []; + const runnerIdentity = runnerIdentityFromGitHubScope(input.githubScope); + let nextToken: string | undefined; + do { + const response = await ec2Client.send( + new DescribeInstancesCommand({ + Filters: [ + { Name: 'instance-state-name', Values: ['pending', 'running'] }, + { Name: `tag:${APPLICATION_TAG}`, Values: [APPLICATION_VALUE] }, + { Name: `tag:${CREATED_BY_TAG}`, Values: [SCALE_SET_RUNNER_SOURCE] }, + { Name: `tag:${ENVIRONMENT_TAG}`, Values: [input.configuration.environment] }, + { Name: `tag:${RUNNER_TYPE_TAG}`, Values: [runnerIdentity.runnerType] }, + { Name: `tag:${RUNNER_OWNER_TAG}`, Values: [runnerIdentity.runnerOwner] }, + ], + NextToken: nextToken, + }), + { abortSignal: signal }, + ); + nextToken = response.NextToken; + + for (const instance of response.Reservations?.flatMap((reservation) => reservation.Instances ?? []) ?? []) { + const runner = parseOwnedRunner(instance, input); + if (runner) runners.push(runner); + } + } while (nextToken); + + return runners; +} + +function parseOwnedRunner(instance: Instance, input: CreateEc2ScaleSetProviderInput): OwnedEc2Runner | undefined { + if (!instance.InstanceId) return undefined; + const tags = new Map((instance.Tags ?? []).flatMap((tag) => (tag.Key ? [[tag.Key, tag.Value]] : []))); + const runnerIdentity = runnerIdentityFromGitHubScope(input.githubScope); + + if ( + tags.get(APPLICATION_TAG) !== APPLICATION_VALUE || + tags.get(CREATED_BY_TAG) !== SCALE_SET_RUNNER_SOURCE || + tags.get(ENVIRONMENT_TAG) !== input.configuration.environment || + tags.get(RUNNER_TYPE_TAG) !== runnerIdentity.runnerType || + tags.get(RUNNER_OWNER_TAG) !== runnerIdentity.runnerOwner + ) { + return undefined; + } + + const taggedRunnerId = tags.get(EC2_GITHUB_RUNNER_ID_TAG); + const githubRunnerId = taggedRunnerId === undefined ? undefined : Number(taggedRunnerId); + const rawScaleSetState = tags.get(EC2_SCALE_SET_STATE_TAG); + const scaleSetState = ['provisioning', 'publishing', 'config-published', 'retiring'].includes(rawScaleSetState ?? '') + ? (rawScaleSetState as Ec2ScaleSetState) + : undefined; + + return { + instanceId: instance.InstanceId, + launchTime: instance.LaunchTime, + githubRunnerId: Number.isSafeInteger(githubRunnerId) && githubRunnerId! > 0 ? githubRunnerId : undefined, + runnerName: tags.get(EC2_RUNNER_NAME_TAG), + scaleSetState, + }; +} + +function validRunnerState(value: ScaleSetRunnerState): boolean { + return ( + Number.isSafeInteger(value.runnerId) && + value.runnerId > 0 && + Number.isSafeInteger(value.scaleSetId) && + value.scaleSetId > 0 && + typeof value.runnerName === 'string' && + value.runnerName.length > 0 && + value.runnerName.length <= GITHUB_RUNNER_NAME_MAX_LENGTH && + ['online', 'offline', 'unknown'].includes(value.status) && + (typeof value.busy === 'boolean' || value.busy === undefined) && + ['started', 'completed', 'unknown'].includes(value.lifecycle) + ); +} + +export function indexRunnerStates( + runnerStates: readonly ScaleSetRunnerState[], + scaleSetId: number, +): { byName: Map; ambiguousNames: Set; ambiguousIds: Set } { + const byName = new Map(); + const byId = new Map(); + const ambiguousNames = new Set(); + const ambiguousIds = new Set(); + + for (const state of runnerStates) { + if (!validRunnerState(state) || state.scaleSetId !== scaleSetId) continue; + if (byName.has(state.runnerName)) ambiguousNames.add(state.runnerName); + const existingName = byId.get(state.runnerId); + if (existingName !== undefined && existingName !== state.runnerName) { + ambiguousIds.add(state.runnerId); + ambiguousNames.add(existingName); + ambiguousNames.add(state.runnerName); + } + byName.set(state.runnerName, state); + byId.set(state.runnerId, state.runnerName); + } + return { byName, ambiguousNames, ambiguousIds }; +} + +export function matchingRunnerState( + runner: OwnedEc2Runner, + index: ReturnType, + scaleSetId: number, +): ScaleSetRunnerState | undefined { + if (!runner.runnerName || !runner.githubRunnerId) return undefined; + if (index.ambiguousNames.has(runner.runnerName) || index.ambiguousIds.has(runner.githubRunnerId)) return undefined; + const state = index.byName.get(runner.runnerName); + if ( + !state || + state.runnerId !== runner.githubRunnerId || + state.runnerName !== runner.runnerName || + state.scaleSetId !== scaleSetId + ) { + return undefined; + } + return state; +} + +function isWithinBootTimeout(runner: OwnedEc2Runner, bootTimeoutMinutes: number, now: number): boolean { + const launchTime = runner.launchTime?.getTime(); + if (launchTime === undefined || !Number.isFinite(launchTime)) return false; + const ageMilliseconds = now - launchTime; + return ageMilliseconds >= 0 && ageMilliseconds < bootTimeoutMinutes * 60_000; +} + +function isConfirmedServingState(state: ScaleSetRunnerState): boolean { + return state.lifecycle === 'started' || state.status === 'online'; +} + +export function servingCapacity( + input: CreateEc2ScaleSetProviderInput, + runners: readonly OwnedEc2Runner[], + request: ScaleSetReconcileRequest, + state: MutableReconcileState, + now: number, +): OwnedEc2Runner[] { + const runnerStateIndex = indexRunnerStates(request.runnerStates, input.scaleSetId); + const serving: OwnedEc2Runner[] = []; + + for (const runner of runners) { + if (runner.scaleSetState !== 'config-published') { + // An interrupted publication may already have been consumed. Preserve it, + // but do not let it suppress replacement capacity indefinitely. + retainUnknown(state, runner.instanceId); + continue; + } + + const githubState = matchingRunnerState(runner, runnerStateIndex, input.scaleSetId); + if (githubState !== undefined && isConfirmedServingState(githubState)) { + serving.push(runner); + continue; + } + if (isWithinBootTimeout(runner, request.bootTimeoutMinutes, now)) { + serving.push(runner); + continue; + } + + // A config-published EC2 instance is provider-owned capacity. The exact + // Actions-service identity is used only when removing it; no public + // GitHub runner inventory is needed to count capacity. + if (runner.githubRunnerId !== undefined && runner.runnerName !== undefined) { + serving.push(runner); + continue; + } + + retainUnknown(state, runner.instanceId); + // Unknown lifecycle state is not allowed to suppress replacement capacity. + // Aggregate busy state still protects scale-down, while tagged EC2 + // inventory remains the source of current provider-owned capacity. + } + + return serving; +} + +export function isSafeScaleDownState(state: ScaleSetRunnerState): boolean { + return state.busy === false || (state.lifecycle === 'completed' && state.busy !== true); +} + +export function isBusyState(state: ScaleSetRunnerState): boolean { + return state.lifecycle === 'started' || state.busy === true; +} diff --git a/lambdas/libs/compute-providers/aws/ec2/src/scale-set/provider.test.ts b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/provider.test.ts new file mode 100644 index 0000000000..704283ac64 --- /dev/null +++ b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/provider.test.ts @@ -0,0 +1,132 @@ +import { CreateFleetCommand, DescribeInstancesCommand, TerminateInstancesCommand } from '@aws-sdk/client-ec2'; +import { PutParameterCommand } from '@aws-sdk/client-ssm'; +import { beforeEach, describe, expect, it } from 'vitest'; + +import { config, createRequest, githubState, ownedInstance } from './test/fixtures'; +import { createTestProvider, ec2Mock, resetAwsMocks, ssmMock } from './test/provider-harness'; + +beforeEach(resetAwsMocks); + +describe('EC2 scale-set provider orchestration', () => { + it('rejects non-canonical GitHub ownership scopes before creating clients', () => { + expect(() => createTestProvider({ githubScope: 'https://GITHUB.com/example/' })).toThrow( + 'githubScope must be a canonical HTTPS GitHub configuration URL', + ); + }); + + it('counts an old tagged handoff as provider capacity without public inventory', async () => { + const old = ownedInstance('i-old-offline', { runnerId: 100, runnerName: 'runner-i-old-offline' }); + const replacementId = 'i-1234567890abcdef0'; + const replacement = ownedInstance( + replacementId, + { runnerId: 101, runnerName: `runner-${replacementId}` }, + { + launchTime: new Date('2026-08-24T10:10:30Z'), + }, + ); + ec2Mock + .on(DescribeInstancesCommand) + .resolvesOnce({ Reservations: [{ Instances: [old] }] }) + .resolves({ Reservations: [{ Instances: [old, replacement] }] }); + ec2Mock.on(CreateFleetCommand).resolves({ Instances: [{ InstanceIds: [replacementId] }] }); + const computeProvider = createTestProvider({ now: () => new Date('2026-08-24T10:11:00Z').getTime() }); + const completeInventory = createRequest({ + runnerStates: [ + githubState(100, 'runner-i-old-offline', { + status: 'offline', + busy: false, + lifecycle: 'completed', + }), + ], + }); + + const result = await computeProvider.reconcile(completeInventory); + const nextResult = await computeProvider.reconcile(completeInventory); + + expect(result).toMatchObject({ + status: 'converged', + currentRunners: 1, + actions: { launched: 0, retainedUnknown: 0 }, + }); + expect(nextResult).toMatchObject({ + status: 'converged', + currentRunners: 1, + actions: { launched: 0, retainedUnknown: 0 }, + }); + expect(ec2Mock).not.toHaveReceivedCommand(CreateFleetCommand); + }); + + it.each(['provisioning', 'publishing'])( + 'retains interrupted %s capacity but provisions a replacement', + async (scaleSetState) => { + const stuck = ownedInstance( + 'i-stuck', + scaleSetState === 'publishing' ? { runnerId: 100, runnerName: 'runner-i-stuck' } : undefined, + { scaleSetState }, + ); + const replacement = 'i-1234567890abcdef0'; + ec2Mock + .on(DescribeInstancesCommand) + .resolvesOnce({ Reservations: [{ Instances: [stuck] }] }) + .resolves({ + Reservations: [ + { + Instances: [stuck, ownedInstance(replacement, { runnerId: 101, runnerName: `runner-${replacement}` })], + }, + ], + }); + ec2Mock.on(CreateFleetCommand).resolves({ Instances: [{ InstanceIds: [replacement] }] }); + const computeProvider = createTestProvider(); + + const result = await computeProvider.reconcile(createRequest()); + + expect(result).toMatchObject({ + status: 'retained', + currentRunners: 2, + actions: { launched: 1, terminated: 0, retainedUnknown: 1 }, + errors: [], + }); + expect(ec2Mock).not.toHaveReceivedCommandWith(TerminateInstancesCommand, { InstanceIds: ['i-stuck'] }); + expect(ssmMock).toHaveReceivedCommandWith(PutParameterCommand, { + Name: `${config.jitConfigParameterPath}/${replacement}`, + }); + + const nextResult = await computeProvider.reconcile(createRequest()); + expect(nextResult).toMatchObject({ + status: 'retained', + currentRunners: 2, + actions: { launched: 0, terminated: 0, retainedUnknown: 1 }, + errors: [], + }); + expect(ec2Mock).toHaveReceivedCommandTimes(CreateFleetCommand, 1); + }, + ); + + it('caps retained-capacity replacement surge when every replacement remains ambiguous', async () => { + const ambiguous = [ + ownedInstance('i-stuck-1', undefined, { scaleSetState: 'provisioning' }), + ownedInstance('i-stuck-2', { runnerId: 102, runnerName: 'runner-i-stuck-2' }, { scaleSetState: 'publishing' }), + ]; + ec2Mock.on(DescribeInstancesCommand).resolves({ Reservations: [{ Instances: ambiguous }] }); + + const result = await createTestProvider().reconcile(createRequest()); + + expect(result).toMatchObject({ + status: 'retained', + currentRunners: 2, + actions: { launched: 0, terminated: 0, retainedUnknown: 2 }, + errors: [], + }); + expect(ec2Mock).not.toHaveReceivedCommand(CreateFleetCommand); + }); + + it('propagates cancellation instead of converting shutdown into a retry result', async () => { + const abort = new AbortController(); + abort.abort(new Error('service stopping')); + + await expect(createTestProvider().reconcile(createRequest({ signal: abort.signal }))).rejects.toThrow( + 'service stopping', + ); + expect(ec2Mock).not.toHaveReceivedCommand(DescribeInstancesCommand); + }); +}); diff --git a/lambdas/libs/compute-providers/aws/ec2/src/scale-set/provider.ts b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/provider.ts new file mode 100644 index 0000000000..3001734891 --- /dev/null +++ b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/provider.ts @@ -0,0 +1,126 @@ +import { EC2Client } from '@aws-sdk/client-ec2'; +import { SSMClient } from '@aws-sdk/client-ssm'; + +import type { ScaleSetComputeProvider, ScaleSetReconcileResult } from '../../../../scale-set'; +import { createEc2RunnerClient } from '../runners'; +import { + parseEc2ScaleSetProviderConfig, + validateFactoryInput, + type CreateEc2ScaleSetProviderInput, + type Ec2ScaleSetProviderConfig, +} from './configuration'; +import { listOwnedRunners, servingCapacity, type OwnedEc2Runner } from './inventory'; +import { + emptyState, + finish, + safeError, + throwIfAborted, + validateBootTimeout, + validateBusyRunners, + validateDesiredRunners, +} from './reconcile'; +import { scaleDown } from './scale-down'; +import { scaleUp } from './scale-up'; + +export type { CreateEc2ScaleSetProviderInput, Ec2ScaleSetProviderConfig } from './configuration'; +export { parseEc2ScaleSetProviderConfig } from './configuration'; +export { + EC2_GITHUB_RUNNER_ID_TAG, + EC2_GITHUB_SCOPE_HASH_TAG, + EC2_RUNNER_CONFIG_TAG, + EC2_RUNNER_NAME_TAG, + EC2_SCALE_SET_ID_TAG, + EC2_SCALE_SET_STATE_TAG, +} from './inventory'; + +const RETAINED_CAPACITY_REPLACEMENT_SURGE = 1; + +export interface Ec2ScaleSetProviderDependencies { + ec2Client?: EC2Client; + ssmClient?: SSMClient; + now?: () => number; +} + +function createClients( + config: Ec2ScaleSetProviderConfig, + dependencies: Ec2ScaleSetProviderDependencies, + credentials?: CreateEc2ScaleSetProviderInput['credentials'], +) { + return { + ec2Client: dependencies.ec2Client ?? new EC2Client({ region: config.region, credentials }), + ssmClient: + dependencies.ssmClient ?? + new SSMClient({ + region: config.region, + maxAttempts: 10, + retryMode: 'adaptive', + credentials, + }), + }; +} + +export function createEc2ScaleSetProvider( + input: CreateEc2ScaleSetProviderInput, + dependencies: Ec2ScaleSetProviderDependencies = {}, +): ScaleSetComputeProvider { + const normalizedInput = { + ...input, + configuration: parseEc2ScaleSetProviderConfig(input.configuration), + }; + validateFactoryInput(normalizedInput); + const clients = createClients(normalizedInput.configuration, dependencies, normalizedInput.credentials); + const runnerClient = createEc2RunnerClient(clients.ec2Client); + const now = dependencies.now ?? Date.now; + + return { + async reconcile(request): Promise { + request.signal.throwIfAborted(); + const validationError = + validateDesiredRunners(request.desiredRunners) ?? + validateBusyRunners(request.busyRunners) ?? + validateBootTimeout(request.bootTimeoutMinutes); + if (validationError) { + const state = emptyState(0); + state.errors.push(validationError); + return finish(state, request.desiredRunners); + } + + const runnerOperations = runnerClient.forRequest({ signal: request.signal }); + let ownedRunners: OwnedEc2Runner[]; + try { + ownedRunners = await listOwnedRunners(normalizedInput, clients.ec2Client, request.signal); + } catch (error) { + throwIfAborted(request.signal, error); + const state = emptyState(0); + state.errors.push(safeError('list', error)); + return finish(state, request.desiredRunners); + } + + const state = emptyState(ownedRunners.length); + const servingRunners = servingCapacity(normalizedInput, ownedRunners, request, state, now()); + + if (servingRunners.length < request.desiredRunners) { + const capacityDeficit = request.desiredRunners - servingRunners.length; + const availableReplacementSlots = Math.max( + 0, + request.desiredRunners + RETAINED_CAPACITY_REPLACEMENT_SURGE - ownedRunners.length, + ); + const launchCount = Math.min(capacityDeficit, availableReplacementSlots); + if (launchCount > 0) { + await scaleUp(normalizedInput, launchCount, request, state, runnerOperations, clients.ssmClient); + } + } else if (servingRunners.length > request.desiredRunners) { + await scaleDown( + normalizedInput, + servingRunners, + servingRunners.length - request.desiredRunners, + request, + state, + runnerOperations, + ); + } + + return finish(state, request.desiredRunners); + }, + }; +} diff --git a/lambdas/libs/compute-providers/aws/ec2/src/scale-set/reconcile.test.ts b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/reconcile.test.ts new file mode 100644 index 0000000000..a462b935b8 --- /dev/null +++ b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/reconcile.test.ts @@ -0,0 +1,32 @@ +import { DescribeInstancesCommand } from '@aws-sdk/client-ec2'; +import { beforeEach, describe, expect, it } from 'vitest'; + +import { createRequest } from './test/fixtures'; +import { createTestProvider, ec2Mock, resetAwsMocks } from './test/provider-harness'; + +beforeEach(resetAwsMocks); + +describe('EC2 scale-set reconciliation validation', () => { + it('rejects an invalid desired count without touching AWS', async () => { + const result = await createTestProvider().reconcile(createRequest({ desiredRunners: -1 })); + + expect(result).toMatchObject({ + status: 'error', + desiredRunners: -1, + currentRunners: 0, + }); + expect(result.errors).toEqual([{ operation: 'validate', code: 'INVALID_DESIRED_RUNNER_COUNT' }]); + expect(ec2Mock).not.toHaveReceivedCommand(DescribeInstancesCommand); + }); + + it.each([0, 121, 1.5])('rejects invalid orchestration boot timeout %s without touching AWS', async (value) => { + const result = await createTestProvider().reconcile(createRequest({ bootTimeoutMinutes: value })); + + expect(result).toMatchObject({ + status: 'error', + currentRunners: 0, + }); + expect(result.errors).toEqual([{ operation: 'validate', code: 'INVALID_BOOT_TIMEOUT' }]); + expect(ec2Mock).not.toHaveReceivedCommand(DescribeInstancesCommand); + }); +}); diff --git a/lambdas/libs/compute-providers/aws/ec2/src/scale-set/reconcile.ts b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/reconcile.ts new file mode 100644 index 0000000000..a49e8c0293 --- /dev/null +++ b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/reconcile.ts @@ -0,0 +1,131 @@ +import type { + ScaleSetReconcileActions, + ScaleSetReconcileError, + ScaleSetReconcileOperation, + ScaleSetReconcileResult, +} from '../../../../scale-set'; + +const MAX_BOOT_TIMEOUT_MINUTES = 120; + +export interface MutableReconcileState { + currentRunners: number; + retainedUnknownResourceIds: Set; + actions: ScaleSetReconcileActions; + errors: ScaleSetReconcileError[]; +} + +export class Ec2ScaleSetValidationError extends Error { + constructor(message: string) { + super(message); + this.name = 'Ec2ScaleSetValidationError'; + } +} + +export function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value); +} + +export function safeError( + operation: ScaleSetReconcileOperation, + error: unknown, + details: Pick = {}, +): ScaleSetReconcileError { + return { + operation, + code: safeErrorCode(error), + ...details, + }; +} + +function safeErrorCode(error: unknown): string { + if (error instanceof Ec2ScaleSetValidationError) return 'INVALID_CONFIGURATION'; + if (!isRecord(error)) return 'UNEXPECTED_ERROR'; + for (const candidate of [error.name, error.code]) { + if (typeof candidate === 'string' && /^[A-Za-z][A-Za-z0-9._-]{0,127}$/.test(candidate)) { + return candidate; + } + } + return 'UNEXPECTED_ERROR'; +} + +export function throwIfAborted(signal: AbortSignal, error?: unknown): void { + if (signal.aborted || (isRecord(error) && error.name === 'AbortError')) { + signal.throwIfAborted(); + throw error; + } +} + +function resultStatus(errors: readonly ScaleSetReconcileError[], current: number, desired: number) { + if (errors.length > 0 || current < desired) return 'error' as const; + if (current > desired) return 'retained' as const; + return 'converged' as const; +} + +export function finish(state: MutableReconcileState, desiredRunners: number): ScaleSetReconcileResult { + if (desiredRunners >= 0 && state.currentRunners < desiredRunners && state.errors.length === 0) { + state.errors.push({ + operation: 'reconcile', + code: 'CAPACITY_NOT_PROVISIONED', + }); + } + return { + status: resultStatus(state.errors, state.currentRunners, desiredRunners), + desiredRunners, + currentRunners: state.currentRunners, + actions: state.actions, + errors: state.errors, + }; +} + +export function emptyState(currentRunners: number): MutableReconcileState { + return { + currentRunners, + retainedUnknownResourceIds: new Set(), + actions: { launched: 0, terminated: 0, retainedBusy: 0, retainedUnknown: 0 }, + errors: [], + }; +} + +export function retainUnknown(state: MutableReconcileState, resourceId?: string): void { + if (resourceId === undefined) { + state.actions.retainedUnknown++; + return; + } + if (state.retainedUnknownResourceIds.has(resourceId)) return; + state.retainedUnknownResourceIds.add(resourceId); + state.actions.retainedUnknown++; +} + +export function validateDesiredRunners(desiredRunners: number): ScaleSetReconcileError | undefined { + if (!Number.isSafeInteger(desiredRunners) || desiredRunners < 0 || desiredRunners > 10000) { + return { + operation: 'validate', + code: 'INVALID_DESIRED_RUNNER_COUNT', + }; + } + return undefined; +} + +export function validateBootTimeout(bootTimeoutMinutes: number): ScaleSetReconcileError | undefined { + if ( + !Number.isSafeInteger(bootTimeoutMinutes) || + bootTimeoutMinutes < 1 || + bootTimeoutMinutes > MAX_BOOT_TIMEOUT_MINUTES + ) { + return { + operation: 'validate', + code: 'INVALID_BOOT_TIMEOUT', + }; + } + return undefined; +} + +export function validateBusyRunners(busyRunners: number): ScaleSetReconcileError | undefined { + if (!Number.isSafeInteger(busyRunners) || busyRunners < 0 || busyRunners > 2_147_483_647) { + return { + operation: 'validate', + code: 'INVALID_BUSY_RUNNER_COUNT', + }; + } + return undefined; +} diff --git a/lambdas/libs/compute-providers/aws/ec2/src/scale-set/scale-down.test.ts b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/scale-down.test.ts new file mode 100644 index 0000000000..40b64e1c2b --- /dev/null +++ b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/scale-down.test.ts @@ -0,0 +1,195 @@ +import { DescribeInstancesCommand, TerminateInstancesCommand } from '@aws-sdk/client-ec2'; +import { beforeEach, describe, expect, it, vi } from 'vitest'; + +import { createRequest, githubState, ownedInstance, signal } from './test/fixtures'; +import { createTestProvider, ec2Mock, resetAwsMocks } from './test/provider-harness'; + +beforeEach(resetAwsMocks); + +describe('EC2 scale-set scale down', () => { + it('terminates only exact known-idle or completed runners and retains busy or unknown runners', async () => { + const completed = ownedInstance('i-completed', { runnerId: 101, runnerName: 'runner-completed' }); + const busy = ownedInstance('i-busy', { runnerId: 102, runnerName: 'runner-busy' }); + const unknown = ownedInstance('i-unknown', { runnerId: 103, runnerName: 'runner-unknown' }); + ec2Mock.on(DescribeInstancesCommand).resolves({ Reservations: [{ Instances: [completed, busy, unknown] }] }); + const removeRunner = vi.fn().mockResolvedValue({ status: 'removed' }); + + const result = await createTestProvider().reconcile( + createRequest({ + desiredRunners: 2, + runnerStates: [ + githubState(101, 'runner-completed', { status: 'offline', busy: undefined, lifecycle: 'completed' }), + githubState(102, 'runner-busy', { busy: true, lifecycle: 'started' }), + ], + removeRunner, + }), + ); + + expect(result).toEqual({ + status: 'converged', + desiredRunners: 2, + currentRunners: 2, + actions: { launched: 0, terminated: 1, retainedBusy: 1, retainedUnknown: 0 }, + errors: [], + }); + expect(removeRunner).toHaveBeenCalledTimes(1); + expect(removeRunner).toHaveBeenCalledWith({ + runnerId: 101, + runnerName: 'runner-completed', + scaleSetId: 42, + signal, + }); + expect(ec2Mock).toHaveReceivedCommandWith(TerminateInstancesCommand, { InstanceIds: ['i-completed'] }); + expect(ec2Mock).not.toHaveReceivedCommandWith(TerminateInstancesCommand, { InstanceIds: ['i-busy'] }); + expect(ec2Mock).not.toHaveReceivedCommandWith(TerminateInstancesCommand, { InstanceIds: ['i-unknown'] }); + }); + + it('uses aggregate idle state to remove a tagged runner after a restart', async () => { + const instance = ownedInstance('i-completed', { runnerId: 101, runnerName: 'runner-completed' }); + ec2Mock.on(DescribeInstancesCommand).resolves({ Reservations: [{ Instances: [instance] }] }); + const removeRunner = vi.fn().mockResolvedValue({ status: 'removed' }); + const result = await createTestProvider().reconcile( + createRequest({ desiredRunners: 0, busyRunners: 0, runnerStates: [], removeRunner }), + ); + + expect(result).toMatchObject({ + status: 'converged', + currentRunners: 0, + actions: { terminated: 1 }, + }); + expect(removeRunner).toHaveBeenCalledTimes(1); + }); + + it('never lets a completed lifecycle marker override a current busy signal', async () => { + const instance = ownedInstance('i-completed-busy', { runnerId: 101, runnerName: 'runner-completed-busy' }); + ec2Mock.on(DescribeInstancesCommand).resolves({ Reservations: [{ Instances: [instance] }] }); + const removeRunner = vi.fn(); + + const result = await createTestProvider().reconcile( + createRequest({ + desiredRunners: 0, + busyRunners: 1, + runnerStates: [ + githubState(101, 'runner-completed-busy', { + lifecycle: 'completed', + status: 'online', + busy: true, + }), + ], + removeRunner, + }), + ); + + expect(result).toMatchObject({ + status: 'retained', + currentRunners: 1, + actions: { terminated: 0, retainedBusy: 1 }, + errors: [], + }); + expect(removeRunner).not.toHaveBeenCalled(); + expect(ec2Mock).not.toHaveReceivedCommand(TerminateInstancesCommand); + }); + + it('retains a runner without an error when the exact removal check observes that it became busy', async () => { + const instance = ownedInstance('i-raced-busy', { runnerId: 101, runnerName: 'runner-raced-busy' }); + ec2Mock.on(DescribeInstancesCommand).resolves({ Reservations: [{ Instances: [instance] }] }); + const removeRunner = vi.fn().mockResolvedValue({ status: 'retained_busy' }); + + const result = await createTestProvider().reconcile( + createRequest({ + desiredRunners: 0, + busyRunners: 0, + runnerStates: [githubState(101, 'runner-raced-busy')], + removeRunner, + }), + ); + + expect(result).toMatchObject({ + status: 'retained', + currentRunners: 1, + actions: { terminated: 0, retainedBusy: 1, retainedUnknown: 0 }, + errors: [], + }); + expect(ec2Mock).not.toHaveReceivedCommand(TerminateInstancesCommand); + }); + + it('retains a runner and requests inventory when exact removal observes identity drift', async () => { + const instance = ownedInstance('i-raced-unknown', { runnerId: 101, runnerName: 'runner-raced-unknown' }); + ec2Mock.on(DescribeInstancesCommand).resolves({ Reservations: [{ Instances: [instance] }] }); + const removeRunner = vi.fn().mockResolvedValue({ status: 'retained_unknown' }); + + const result = await createTestProvider().reconcile( + createRequest({ + desiredRunners: 0, + busyRunners: 0, + runnerStates: [githubState(101, 'runner-raced-unknown')], + removeRunner, + }), + ); + + expect(result).toMatchObject({ + status: 'retained', + currentRunners: 1, + actions: { terminated: 0, retainedBusy: 0, retainedUnknown: 1 }, + errors: [], + }); + expect(ec2Mock).not.toHaveReceivedCommand(TerminateInstancesCommand); + }); + + it('does not trust a mutable EC2 GitHub-runner-id tag when controller identity disagrees', async () => { + const instance = ownedInstance('i-mismatch', { runnerId: 999, runnerName: 'runner-exact' }); + ec2Mock.on(DescribeInstancesCommand).resolves({ Reservations: [{ Instances: [instance] }] }); + const removeRunner = vi.fn(); + + const result = await createTestProvider().reconcile( + createRequest({ + desiredRunners: 0, + busyRunners: 0, + runnerStates: [githubState(101, 'runner-exact')], + removeRunner, + }), + ); + + expect(result).toMatchObject({ + status: 'retained', + currentRunners: 1, + actions: { terminated: 0, retainedUnknown: 1 }, + errors: [], + }); + expect(removeRunner).not.toHaveBeenCalled(); + expect(ec2Mock).not.toHaveReceivedCommand(TerminateInstancesCommand); + }); + + it('does not terminate compute when exact GitHub removal fails', async () => { + const instance = ownedInstance('i-idle', { runnerId: 101, runnerName: 'runner-idle' }); + ec2Mock.on(DescribeInstancesCommand).resolves({ Reservations: [{ Instances: [instance] }] }); + const removeRunner = vi + .fn() + .mockRejectedValue(Object.assign(new Error('must not leak'), { name: 'ServiceUnavailable' })); + + const result = await createTestProvider().reconcile( + createRequest({ + desiredRunners: 0, + busyRunners: 0, + runnerStates: [githubState(101, 'runner-idle')], + removeRunner, + }), + ); + + expect(result).toMatchObject({ + status: 'error', + currentRunners: 1, + actions: { terminated: 0, retainedUnknown: 1 }, + }); + expect(result.errors).toEqual([ + { + operation: 'remove_runner', + code: 'ServiceUnavailable', + runnerName: 'runner-idle', + resourceId: 'i-idle', + }, + ]); + expect(JSON.stringify(result)).not.toContain('must not leak'); + expect(ec2Mock).not.toHaveReceivedCommand(TerminateInstancesCommand); + }); +}); diff --git a/lambdas/libs/compute-providers/aws/ec2/src/scale-set/scale-down.ts b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/scale-down.ts new file mode 100644 index 0000000000..78fddce9cd --- /dev/null +++ b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/scale-down.ts @@ -0,0 +1,118 @@ +import type { ScaleSetReconcileRequest, ScaleSetRunnerState } from '../../../../scale-set'; +import type { Ec2RunnerResourceOperations } from '../runners'; +import type { CreateEc2ScaleSetProviderInput } from './configuration'; +import { + indexRunnerStates, + isBusyState, + isSafeScaleDownState, + matchingRunnerState, + type OwnedEc2Runner, +} from './inventory'; +import { retainUnknown, safeError, throwIfAborted, type MutableReconcileState } from './reconcile'; + +async function terminateKnownIdleRunner( + runner: OwnedEc2Runner, + githubState: ScaleSetRunnerState, + request: ScaleSetReconcileRequest, + state: MutableReconcileState, + runnerOperations: Ec2RunnerResourceOperations, +): Promise { + let removalResult; + try { + removalResult = await request.removeRunner({ + runnerId: githubState.runnerId, + runnerName: githubState.runnerName, + scaleSetId: githubState.scaleSetId, + signal: request.signal, + }); + } catch (error) { + throwIfAborted(request.signal, error); + state.errors.push( + safeError('remove_runner', error, { runnerName: githubState.runnerName, resourceId: runner.instanceId }), + ); + retainUnknown(state, runner.instanceId); + return false; + } + + if (removalResult.status === 'retained_busy') { + state.actions.retainedBusy++; + return false; + } + if (removalResult.status !== 'removed') { + retainUnknown(state, runner.instanceId); + return false; + } + + try { + await runnerOperations.terminate(runner.instanceId); + state.currentRunners--; + state.actions.terminated++; + return true; + } catch (error) { + throwIfAborted(request.signal, error); + state.errors.push( + safeError('terminate', error, { runnerName: githubState.runnerName, resourceId: runner.instanceId }), + ); + retainUnknown(state, runner.instanceId); + return false; + } +} + +export async function scaleDown( + input: CreateEc2ScaleSetProviderInput, + runners: readonly OwnedEc2Runner[], + count: number, + request: ScaleSetReconcileRequest, + state: MutableReconcileState, + runnerOperations: Ec2RunnerResourceOperations, +): Promise { + const runnerStateIndex = indexRunnerStates(request.runnerStates, input.scaleSetId); + const candidates: { runner: OwnedEc2Runner; githubState: ScaleSetRunnerState }[] = []; + + for (const runner of runners) { + const githubState = matchingRunnerState(runner, runnerStateIndex, input.scaleSetId); + const hasContradictoryState = runner.runnerName !== undefined && runnerStateIndex.byName.has(runner.runnerName); + if (!githubState) { + if ( + !hasContradictoryState && + request.busyRunners === 0 && + runner.githubRunnerId !== undefined && + runner.runnerName !== undefined + ) { + candidates.push({ + runner, + githubState: { + runnerId: runner.githubRunnerId, + runnerName: runner.runnerName, + scaleSetId: input.scaleSetId, + status: 'unknown', + busy: false, + lifecycle: 'unknown', + }, + }); + } else { + retainUnknown(state, runner.instanceId); + } + } else if (isBusyState(githubState)) { + state.actions.retainedBusy++; + } else if (isSafeScaleDownState(githubState)) { + candidates.push({ runner, githubState }); + } else { + retainUnknown(state, runner.instanceId); + } + } + + candidates.sort((left, right) => { + const launchOrder = (right.runner.launchTime?.getTime() ?? 0) - (left.runner.launchTime?.getTime() ?? 0); + return launchOrder || left.runner.instanceId.localeCompare(right.runner.instanceId); + }); + + let remaining = count; + for (const candidate of candidates) { + if (remaining === 0) break; + request.signal.throwIfAborted(); + if (await terminateKnownIdleRunner(candidate.runner, candidate.githubState, request, state, runnerOperations)) { + remaining--; + } + } +} diff --git a/lambdas/libs/compute-providers/aws/ec2/src/scale-set/scale-up.test.ts b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/scale-up.test.ts new file mode 100644 index 0000000000..d72c8a50e8 --- /dev/null +++ b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/scale-up.test.ts @@ -0,0 +1,200 @@ +import { + CreateFleetCommand, + CreateTagsCommand, + DescribeInstancesCommand, + TerminateInstancesCommand, +} from '@aws-sdk/client-ec2'; +import { DeleteParameterCommand, GetParameterCommand, PutParameterCommand } from '@aws-sdk/client-ssm'; +import { beforeEach, describe, expect, it, vi } from 'vitest'; + +import { + EC2_GITHUB_RUNNER_ID_TAG, + EC2_GITHUB_SCOPE_HASH_TAG, + EC2_RUNNER_CONFIG_TAG, + EC2_RUNNER_NAME_TAG, + EC2_SCALE_SET_ID_TAG, + EC2_SCALE_SET_STATE_TAG, +} from './inventory'; +import { config, createRequest, githubScopeHash, jitResult, signal } from './test/fixtures'; +import { createTestProvider, ec2Mock, resetAwsMocks, ssmMock } from './test/provider-harness'; + +beforeEach(resetAwsMocks); + +describe('EC2 scale-set scale up', () => { + it('launches owned compute, verifies JIT identity, and publishes only a SecureString', async () => { + const instanceId = 'i-1234567890abcdef0'; + ec2Mock.on(DescribeInstancesCommand).resolves({}); + ec2Mock.on(CreateFleetCommand).resolves({ Instances: [{ InstanceIds: [instanceId] }] }); + const generateJitConfiguration = vi.fn().mockResolvedValue(jitResult(instanceId)); + + const result = await createTestProvider().reconcile(createRequest({ generateJitConfiguration })); + + expect(result).toEqual({ + status: 'converged', + desiredRunners: 1, + currentRunners: 1, + actions: { launched: 1, terminated: 0, retainedBusy: 0, retainedUnknown: 0 }, + errors: [], + }); + expect(generateJitConfiguration).toHaveBeenCalledWith({ runnerName: `runner-${instanceId}`, signal }); + expect(ec2Mock).toHaveReceivedCommandWith(CreateFleetCommand, { + TagSpecifications: expect.arrayContaining([ + expect.objectContaining({ + ResourceType: 'instance', + Tags: expect.arrayContaining([ + { Key: 'ghr:created_by', Value: 'scale-set-service' }, + { Key: 'ghr:Owner', Value: 'example' }, + { Key: 'ghr:Type', Value: 'Org' }, + ]), + }), + ]), + }); + expect(ec2Mock).toHaveReceivedCommandWith(CreateTagsCommand, { + Resources: [instanceId], + Tags: [ + { Key: EC2_RUNNER_NAME_TAG, Value: `runner-${instanceId}` }, + { Key: EC2_GITHUB_RUNNER_ID_TAG, Value: '101' }, + { Key: EC2_SCALE_SET_STATE_TAG, Value: 'publishing' }, + ], + }); + expect(ssmMock).toHaveReceivedCommandWith(PutParameterCommand, { + Name: `${config.jitConfigParameterPath}/${instanceId}`, + Value: 'sensitive-encoded-jit-configuration', + Type: 'SecureString', + Overwrite: false, + Tags: expect.arrayContaining([ + { Key: 'InstanceId', Value: instanceId }, + { Key: EC2_RUNNER_CONFIG_TAG, Value: 'linux' }, + { Key: EC2_SCALE_SET_ID_TAG, Value: '42' }, + { Key: EC2_GITHUB_SCOPE_HASH_TAG, Value: githubScopeHash }, + ]), + }); + }); + + it('resolves an AMI parameter through the provider-owned SSM client', async () => { + const instanceId = 'i-1234567890abcdef0'; + const amiIdSsmParameterName = '/github-action-runners/unit-test/ami'; + ec2Mock.on(DescribeInstancesCommand).resolves({}); + ec2Mock.on(CreateFleetCommand).resolves({ Instances: [{ InstanceIds: [instanceId] }] }); + ssmMock.on(GetParameterCommand).resolves({ Parameter: { Value: 'ami-0123456789abcdef0' } }); + + const result = await createTestProvider({ + configuration: { ...config, amiIdSsmParameterName }, + }).reconcile(createRequest()); + + expect(result.actions.launched).toBe(1); + expect(ssmMock).toHaveReceivedCommandWith(GetParameterCommand, { + Name: amiIdSsmParameterName, + WithDecryption: true, + }); + }); + + it('does not remove an unrelated GitHub runner when JIT identity validation fails', async () => { + const instanceId = 'i-1234567890abcdef0'; + ec2Mock.on(DescribeInstancesCommand).resolves({}); + ec2Mock.on(CreateFleetCommand).resolves({ Instances: [{ InstanceIds: [instanceId] }] }); + const removeRunner = vi.fn(); + + const result = await createTestProvider().reconcile( + createRequest({ + generateJitConfiguration: vi.fn().mockResolvedValue({ + ...jitResult(instanceId), + runnerName: 'runner-owned-by-another-config', + }), + removeRunner, + }), + ); + + expect(result).toMatchObject({ + status: 'error', + currentRunners: 0, + actions: { launched: 0, terminated: 1 }, + }); + expect(result.errors).toEqual([ + { + operation: 'generate_jit_configuration', + code: 'INVALID_CONFIGURATION', + runnerName: `runner-${instanceId}`, + resourceId: instanceId, + }, + ]); + expect(removeRunner).not.toHaveBeenCalled(); + expect(ssmMock).not.toHaveReceivedCommand(PutParameterCommand); + expect(ec2Mock).toHaveReceivedCommandWith(TerminateInstancesCommand, { InstanceIds: [instanceId] }); + }); + + it('retains compute when failed JIT publication cannot be safely cancelled', async () => { + const instanceId = 'i-1234567890abcdef0'; + ec2Mock.on(DescribeInstancesCommand).resolves({}); + ec2Mock.on(CreateFleetCommand).resolves({ Instances: [{ InstanceIds: [instanceId] }] }); + ssmMock.on(PutParameterCommand).rejects(Object.assign(new Error('redacted secret'), { name: 'TimeoutError' })); + ssmMock.on(DeleteParameterCommand).rejects(Object.assign(new Error('missing'), { name: 'ParameterNotFound' })); + const removeRunner = vi.fn(); + + const result = await createTestProvider().reconcile(createRequest({ removeRunner })); + + expect(result).toMatchObject({ + status: 'error', + currentRunners: 1, + actions: { launched: 0, terminated: 0, retainedUnknown: 1 }, + }); + expect(result.errors).toEqual([ + { + operation: 'publish_jit_configuration', + code: 'TimeoutError', + runnerName: `runner-${instanceId}`, + resourceId: instanceId, + }, + ]); + expect(JSON.stringify(result)).not.toContain('redacted secret'); + expect(removeRunner).not.toHaveBeenCalled(); + expect(ec2Mock).not.toHaveReceivedCommand(TerminateInstancesCommand); + }); + + it('does not treat a successful DeleteParameter as proof that bootstrap did not read JIT first', async () => { + const instanceId = 'i-1234567890abcdef0'; + ec2Mock.on(DescribeInstancesCommand).resolves({}); + ec2Mock.on(CreateFleetCommand).resolves({ Instances: [{ InstanceIds: [instanceId] }] }); + ssmMock.on(PutParameterCommand).rejects(Object.assign(new Error('throttled'), { name: 'ThrottlingException' })); + ssmMock.on(DeleteParameterCommand).resolves({}); + const removeRunner = vi.fn().mockResolvedValue({ status: 'removed' }); + + const result = await createTestProvider().reconcile(createRequest({ removeRunner })); + + expect(result).toMatchObject({ + status: 'error', + currentRunners: 1, + actions: { launched: 0, terminated: 0, retainedUnknown: 1 }, + }); + expect(result.errors).toEqual([ + { + operation: 'publish_jit_configuration', + code: 'ThrottlingException', + runnerName: `runner-${instanceId}`, + resourceId: instanceId, + }, + ]); + expect(ssmMock).toHaveReceivedCommandWith(DeleteParameterCommand, { + Name: `${config.jitConfigParameterPath}/${instanceId}`, + }); + expect(removeRunner).not.toHaveBeenCalled(); + expect(ec2Mock).not.toHaveReceivedCommand(TerminateInstancesCommand); + }); + + it.each(['ThrottlingException', 'InvalidParameterValue'])( + 'collapses EC2 launch failure %s into one scale-set error', + async (errorCode) => { + ec2Mock.on(DescribeInstancesCommand).resolves({}); + ec2Mock.on(CreateFleetCommand).resolves({ Errors: [{ ErrorCode: errorCode }] }); + + const result = await createTestProvider().reconcile(createRequest()); + + expect(result).toMatchObject({ + status: 'error', + currentRunners: 0, + actions: { launched: 0, terminated: 0 }, + }); + expect(result.errors).toEqual([{ operation: 'launch', code: 'EC2_LAUNCH_FAILED' }]); + }, + ); +}); diff --git a/lambdas/libs/compute-providers/aws/ec2/src/scale-set/scale-up.ts b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/scale-up.ts new file mode 100644 index 0000000000..ef6cfc3a88 --- /dev/null +++ b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/scale-up.ts @@ -0,0 +1,310 @@ +import { DeleteParameterCommand, PutParameterCommand, type SSMClient, type Tag as SsmTag } from '@aws-sdk/client-ssm'; +import { createChildLogger } from '@aws-github-runner/aws-powertools-util'; + +import type { GenerateScaleSetJitConfigurationResult, ScaleSetReconcileRequest } from '../../../../scale-set'; +import type { Ec2RunnerResourceOperations } from '../runners'; +import type { CreateEc2ScaleSetProviderInput, Ec2ScaleSetProviderConfig } from './configuration'; +import { + EC2_GITHUB_RUNNER_ID_TAG, + EC2_GITHUB_SCOPE_HASH_TAG, + EC2_RUNNER_CONFIG_TAG, + EC2_RUNNER_NAME_TAG, + EC2_SCALE_SET_ID_TAG, + EC2_SCALE_SET_STATE_TAG, + GITHUB_RUNNER_NAME_MAX_LENGTH, + githubScopeHash, + runnerIdentityFromGitHubScope, + SCALE_SET_RUNNER_SOURCE, +} from './inventory'; +import { + Ec2ScaleSetValidationError, + retainUnknown, + safeError, + throwIfAborted, + type MutableReconcileState, +} from './reconcile'; + +const SSM_STANDARD_TIER_THRESHOLD = 4000; +const SSM_ADVANCED_TIER_MAX_BYTES = 8192; +const logger = createChildLogger('ec2-scale-set'); + +interface AwsErrorLike extends Error { + code?: string; + $fault?: 'client' | 'server'; + $metadata?: { + httpStatusCode?: number; + requestId?: string; + }; +} + +function errorDetails(error: unknown): Record { + if (!(error instanceof Error)) return { errorMessage: String(error) }; + const awsError = error as AwsErrorLike; + return { + errorName: error.name, + errorMessage: error.message + .replace(/encoded authorization failure message:\s*\S+/gi, 'encoded authorization failure message: [REDACTED]') + .replace(/[\r\n\u2028\u2029]/g, ' '), + ...(awsError.code === undefined ? {} : { errorCode: awsError.code }), + ...(awsError.$fault === undefined ? {} : { errorFault: awsError.$fault }), + ...(awsError.$metadata?.httpStatusCode === undefined ? {} : { httpStatusCode: awsError.$metadata.httpStatusCode }), + ...(awsError.$metadata?.requestId === undefined ? {} : { requestId: awsError.$metadata.requestId }), + }; +} + +function jitParameterName(config: Ec2ScaleSetProviderConfig, instanceId: string): string { + return `${config.jitConfigParameterPath}/${instanceId}`; +} + +function jitParameterTags(input: CreateEc2ScaleSetProviderInput, instanceId: string): SsmTag[] { + const reserved = new Set(['InstanceId', EC2_RUNNER_CONFIG_TAG, EC2_SCALE_SET_ID_TAG, EC2_GITHUB_SCOPE_HASH_TAG]); + return [ + ...(input.configuration.ssmParameterTags ?? []).filter((tag) => tag.Key && !reserved.has(tag.Key)), + { Key: 'InstanceId', Value: instanceId }, + { Key: EC2_RUNNER_CONFIG_TAG, Value: input.runnerConfigName }, + { Key: EC2_SCALE_SET_ID_TAG, Value: String(input.scaleSetId) }, + { Key: EC2_GITHUB_SCOPE_HASH_TAG, Value: githubScopeHash(input.githubScope) }, + ]; +} + +async function publishJitConfiguration( + input: CreateEc2ScaleSetProviderInput, + instanceId: string, + encodedJitConfiguration: string, + ssmClient: SSMClient, + signal: AbortSignal, +): Promise { + const valueSize = Buffer.byteLength(encodedJitConfiguration, 'utf8'); + if (valueSize === 0 || valueSize > SSM_ADVANCED_TIER_MAX_BYTES) { + throw new Ec2ScaleSetValidationError('JIT configuration must be between 1 and 8192 bytes'); + } + + await ssmClient.send( + new PutParameterCommand({ + Name: jitParameterName(input.configuration, instanceId), + Value: encodedJitConfiguration, + Type: 'SecureString', + KeyId: input.configuration.ssmKmsKeyId, + Overwrite: false, + Tier: valueSize >= SSM_STANDARD_TIER_THRESHOLD ? 'Advanced' : 'Standard', + Tags: jitParameterTags(input, instanceId), + }), + { abortSignal: signal }, + ); +} + +async function bestEffortCancelJitPublication( + input: CreateEc2ScaleSetProviderInput, + instanceId: string, + ssmClient: SSMClient, + signal: AbortSignal, +): Promise { + try { + await ssmClient.send(new DeleteParameterCommand({ Name: jitParameterName(input.configuration, instanceId) }), { + abortSignal: signal, + }); + } catch (error) { + throwIfAborted(signal, error); + } +} + +function validateJitResult( + result: GenerateScaleSetJitConfigurationResult, + expectedRunnerName: string, + scaleSetId: number, +): void { + if ( + !Number.isSafeInteger(result.runnerId) || + result.runnerId <= 0 || + result.runnerName !== expectedRunnerName || + result.scaleSetId !== scaleSetId + ) { + throw new Ec2ScaleSetValidationError('JIT configuration returned an unexpected runner identity'); + } + const valueSize = Buffer.byteLength(result.encodedJitConfiguration, 'utf8'); + if (valueSize === 0 || valueSize > SSM_ADVANCED_TIER_MAX_BYTES) { + throw new Ec2ScaleSetValidationError('JIT configuration has an invalid size'); + } +} + +async function terminateUnpublishedRunner( + instanceId: string, + state: MutableReconcileState, + runners: Ec2RunnerResourceOperations, + signal: AbortSignal, +): Promise { + try { + await runners.terminate(instanceId); + state.currentRunners--; + state.actions.terminated++; + } catch (error) { + throwIfAborted(signal, error); + retainUnknown(state, instanceId); + state.errors.push(safeError('terminate', error, { resourceId: instanceId })); + logger.error('scale_set_ec2_unpublished_runner_termination_failed', { + instanceId, + ...errorDetails(error), + }); + } +} + +async function cleanGitHubRunner( + jit: GenerateScaleSetJitConfigurationResult, + request: ScaleSetReconcileRequest, + state: MutableReconcileState, +): Promise { + try { + await request.removeRunner({ + runnerId: jit.runnerId, + runnerName: jit.runnerName, + scaleSetId: jit.scaleSetId, + signal: request.signal, + }); + } catch (error) { + throwIfAborted(request.signal, error); + state.errors.push(safeError('remove_runner', error, { runnerName: jit.runnerName })); + logger.error('scale_set_ec2_github_runner_cleanup_failed', { + runnerName: jit.runnerName, + runnerId: jit.runnerId, + scaleSetId: jit.scaleSetId, + ...errorDetails(error), + }); + } +} + +async function configureLaunchedRunner( + input: CreateEc2ScaleSetProviderInput, + instanceId: string, + request: ScaleSetReconcileRequest, + state: MutableReconcileState, + runners: Ec2RunnerResourceOperations, + ssmClient: SSMClient, +): Promise { + const runnerName = `${input.configuration.runnerNamePrefix}${instanceId}`; + if (runnerName.length > GITHUB_RUNNER_NAME_MAX_LENGTH) { + state.errors.push({ + operation: 'generate_jit_configuration', + code: 'RUNNER_NAME_TOO_LONG', + resourceId: instanceId, + }); + await terminateUnpublishedRunner(instanceId, state, runners, request.signal); + return; + } + + let jit: GenerateScaleSetJitConfigurationResult; + try { + jit = await request.generateJitConfiguration({ runnerName, signal: request.signal }); + validateJitResult(jit, runnerName, input.scaleSetId); + } catch (error) { + throwIfAborted(request.signal, error); + state.errors.push(safeError('generate_jit_configuration', error, { runnerName, resourceId: instanceId })); + logger.error('scale_set_ec2_jit_configuration_generation_failed', { + instanceId, + runnerName, + ...errorDetails(error), + }); + await terminateUnpublishedRunner(instanceId, state, runners, request.signal); + return; + } + + try { + // CreateFleet/RunInstances already applies the ownership tags. This call only + // adds the runner identity and lifecycle tags allowed by the compute policy. + await runners.tag(instanceId, [ + { Key: EC2_RUNNER_NAME_TAG, Value: jit.runnerName }, + { Key: EC2_GITHUB_RUNNER_ID_TAG, Value: String(jit.runnerId) }, + { Key: EC2_SCALE_SET_STATE_TAG, Value: 'publishing' }, + ]); + } catch (error) { + throwIfAborted(request.signal, error); + state.errors.push(safeError('launch', error, { runnerName, resourceId: instanceId })); + logger.error('scale_set_ec2_runner_tagging_failed', { + instanceId, + runnerName, + tagPhase: 'publishing', + tagKeys: [EC2_RUNNER_NAME_TAG, EC2_GITHUB_RUNNER_ID_TAG, EC2_SCALE_SET_STATE_TAG], + ...errorDetails(error), + }); + await cleanGitHubRunner(jit, request, state); + await terminateUnpublishedRunner(instanceId, state, runners, request.signal); + return; + } + + try { + await publishJitConfiguration(input, instanceId, jit.encodedJitConfiguration, ssmClient, request.signal); + } catch (error) { + throwIfAborted(request.signal, error); + state.errors.push(safeError('publish_jit_configuration', error, { runnerName, resourceId: instanceId })); + logger.error('scale_set_ec2_jit_configuration_publication_failed', { + instanceId, + runnerName, + ...errorDetails(error), + }); + await bestEffortCancelJitPublication(input, instanceId, ssmClient, request.signal); + // Main's bootstrap reads before deleting. Even a successful controller-side + // DeleteParameter can race after that read and cannot prove non-consumption. + // Preserve both GitHub and compute state until an exact lifecycle signal is observed. + retainUnknown(state, instanceId); + return; + } + + state.actions.launched++; + try { + await runners.tag(instanceId, [{ Key: EC2_SCALE_SET_STATE_TAG, Value: 'config-published' }]); + } catch (error) { + throwIfAborted(request.signal, error); + // Publication may already have been consumed. Preserve the instance and exact GitHub identity. + retainUnknown(state, instanceId); + state.errors.push(safeError('launch', error, { runnerName, resourceId: instanceId })); + logger.error('scale_set_ec2_runner_state_tagging_failed', { + instanceId, + runnerName, + tagPhase: 'config-published', + tagKeys: [EC2_SCALE_SET_STATE_TAG], + ...errorDetails(error), + }); + } +} + +export async function scaleUp( + input: CreateEc2ScaleSetProviderInput, + count: number, + request: ScaleSetReconcileRequest, + state: MutableReconcileState, + runners: Ec2RunnerResourceOperations, + ssmClient: SSMClient, +): Promise { + const runnerIdentity = runnerIdentityFromGitHubScope(input.githubScope); + let createResult; + try { + createResult = await runners.create({ + environment: input.configuration.environment, + runnerOwner: runnerIdentity.runnerOwner, + runnerType: runnerIdentity.runnerType, + subnets: input.configuration.subnets, + launchTemplateName: input.configuration.launchTemplateName, + ec2instanceCriteria: input.configuration.ec2instanceCriteria, + ec2OverrideConfig: input.configuration.ec2OverrideConfig, + numberOfRunners: count, + source: SCALE_SET_RUNNER_SOURCE, + amiIdSsmParameterName: input.configuration.amiIdSsmParameterName, + tracingEnabled: input.configuration.tracingEnabled, + onDemandFailoverOnError: input.configuration.onDemandFailoverOnError, + useDedicatedHost: input.configuration.useDedicatedHost, + }); + } catch (error) { + throwIfAborted(request.signal, error); + state.errors.push(safeError('launch', error)); + return; + } + + state.currentRunners += createResult.instances.length; + if (createResult.failedInstanceCount > 0) { + state.errors.push({ operation: 'launch', code: 'EC2_LAUNCH_FAILED' }); + } + + for (const instanceId of createResult.instances) { + request.signal.throwIfAborted(); + await configureLaunchedRunner(input, instanceId, request, state, runners, ssmClient); + } +} diff --git a/lambdas/libs/compute-providers/aws/ec2/src/scale-set/test/fixtures.ts b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/test/fixtures.ts new file mode 100644 index 0000000000..46f7b77708 --- /dev/null +++ b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/test/fixtures.ts @@ -0,0 +1,103 @@ +import { createHash } from 'node:crypto'; + +import type { Instance } from '@aws-sdk/client-ec2'; +import { vi } from 'vitest'; + +import type { + GenerateScaleSetJitConfigurationResult, + ScaleSetReconcileRequest, + ScaleSetRunnerState, +} from '../../../../../scale-set'; +import type { Ec2ScaleSetProviderConfig } from '../configuration'; +import { EC2_GITHUB_RUNNER_ID_TAG, EC2_RUNNER_NAME_TAG, EC2_SCALE_SET_STATE_TAG } from '../inventory'; + +export const signal = new AbortController().signal; +export const githubScope = 'https://github.com/example'; +export const githubScopeHash = createHash('sha256').update(githubScope, 'utf8').digest('hex'); + +export const config: Ec2ScaleSetProviderConfig = { + region: 'eu-west-1', + environment: 'unit-test', + runnerNamePrefix: 'runner-', + jitConfigParameterPath: '/github-action-runners/unit-test/runners/tokens', + subnets: ['subnet-12345678'], + launchTemplateName: 'unit-test-runners', + ec2instanceCriteria: { + instanceTypes: ['m7i.large'], + targetCapacityType: 'on-demand', + instanceAllocationStrategy: 'lowest-price', + }, + ssmParameterTags: [{ Key: 'Project', Value: 'runner-tests' }], +}; + +export function ownedInstance( + instanceId: string, + identity?: { runnerId: number; runnerName: string }, + overrides: { + scaleSetState?: string; + launchTime?: Date; + environment?: string; + runnerOwner?: string; + runnerType?: string; + } = {}, +): Instance { + return { + InstanceId: instanceId, + LaunchTime: overrides.launchTime ?? new Date('2026-08-24T10:00:00Z'), + Tags: [ + { Key: 'ghr:Application', Value: 'github-action-runner' }, + { Key: 'ghr:created_by', Value: 'scale-set-service' }, + { Key: 'ghr:environment', Value: overrides.environment ?? 'unit-test' }, + { Key: 'ghr:Type', Value: overrides.runnerType ?? 'Org' }, + { Key: 'ghr:Owner', Value: overrides.runnerOwner ?? 'example' }, + { + Key: EC2_SCALE_SET_STATE_TAG, + Value: overrides.scaleSetState ?? (identity ? 'config-published' : 'provisioning'), + }, + ...(identity + ? [ + { Key: EC2_RUNNER_NAME_TAG, Value: identity.runnerName }, + { Key: EC2_GITHUB_RUNNER_ID_TAG, Value: String(identity.runnerId) }, + ] + : []), + ], + }; +} + +export function githubState( + runnerId: number, + runnerName: string, + overrides: Partial = {}, +): ScaleSetRunnerState { + return { + runnerId, + runnerName, + scaleSetId: 42, + status: 'online', + busy: false, + lifecycle: 'unknown', + ...overrides, + }; +} + +export function jitResult(instanceId = 'i-1234567890abcdef0'): GenerateScaleSetJitConfigurationResult { + return { + encodedJitConfiguration: 'sensitive-encoded-jit-configuration', + runnerId: 101, + runnerName: `runner-${instanceId}`, + scaleSetId: 42, + }; +} + +export function createRequest(overrides: Partial = {}): ScaleSetReconcileRequest { + return { + desiredRunners: 1, + busyRunners: 0, + bootTimeoutMinutes: 10, + runnerStates: [], + signal, + generateJitConfiguration: vi.fn().mockResolvedValue(jitResult()), + removeRunner: vi.fn().mockResolvedValue({ status: 'removed' }), + ...overrides, + }; +} diff --git a/lambdas/libs/compute-providers/aws/ec2/src/scale-set/test/provider-harness.ts b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/test/provider-harness.ts new file mode 100644 index 0000000000..510009001a --- /dev/null +++ b/lambdas/libs/compute-providers/aws/ec2/src/scale-set/test/provider-harness.ts @@ -0,0 +1,40 @@ +import { CreateTagsCommand, EC2Client, TerminateInstancesCommand } from '@aws-sdk/client-ec2'; +import { DeleteParameterCommand, PutParameterCommand, SSMClient } from '@aws-sdk/client-ssm'; +import { mockClient } from 'aws-sdk-client-mock'; +import 'aws-sdk-client-mock-jest/vitest'; + +import type { Ec2ScaleSetProviderConfig } from '../configuration'; +import { createEc2ScaleSetProvider } from '../provider'; +import { config, githubScope } from './fixtures'; + +export const ec2Mock = mockClient(EC2Client); +export const ssmMock = mockClient(SSMClient); +const ec2Client = new EC2Client({ region: 'eu-west-1' }); +const ssmClient = new SSMClient({ region: 'eu-west-1' }); + +export function createTestProvider( + options: { githubScope?: string; now?: () => number; configuration?: Ec2ScaleSetProviderConfig } = {}, +) { + return createEc2ScaleSetProvider( + { + runnerConfigName: 'linux', + scaleSetId: 42, + githubScope: options.githubScope ?? githubScope, + configuration: options.configuration ?? config, + }, + { + ec2Client, + ssmClient, + now: options.now ?? (() => new Date('2026-08-24T10:05:00Z').getTime()), + }, + ); +} + +export function resetAwsMocks(): void { + ec2Mock.reset(); + ssmMock.reset(); + ec2Mock.on(CreateTagsCommand).resolves({}); + ec2Mock.on(TerminateInstancesCommand).resolves({}); + ssmMock.on(PutParameterCommand).resolves({}); + ssmMock.on(DeleteParameterCommand).resolves({}); +} diff --git a/lambdas/libs/compute-providers/core/index.ts b/lambdas/libs/compute-providers/core/index.ts index 2457719a42..0ad066c6ea 100644 --- a/lambdas/libs/compute-providers/core/index.ts +++ b/lambdas/libs/compute-providers/core/index.ts @@ -6,7 +6,7 @@ export interface ComputeProvider { type: ComputeProviderType; } -export type RunnerSource = 'scale-up-lambda' | 'pool-lambda'; +export type RunnerSource = 'scale-up-lambda' | 'pool-lambda' | 'scale-set-service'; export type RunnerType = 'Org' | 'Repo'; export interface CreateGitHubRunnerConfig { diff --git a/lambdas/libs/compute-providers/package.json b/lambdas/libs/compute-providers/package.json index a6fecab50c..b62e261495 100644 --- a/lambdas/libs/compute-providers/package.json +++ b/lambdas/libs/compute-providers/package.json @@ -8,8 +8,10 @@ "./provider-types": "./provider-types.ts", "./webhook": "./webhook.ts", "./control-plane": "./control-plane.ts", + "./scale-set": "./scale-set.ts", "./aws/ec2/webhook": "./aws/ec2/webhook.ts", "./aws/ec2/control-plane": "./aws/ec2/control-plane.ts", + "./aws/ec2/scale-set": "./aws/ec2/scale-set.ts", "./aws/ec2/runners": "./aws/ec2/src/runners.ts", "./aws/ec2/control-plane/runner-creation": "./aws/ec2/src/control-plane/runner-creation.ts" }, @@ -28,6 +30,7 @@ "@aws-github-runner/aws-ssm-util": "*", "@aws-github-runner/storage-providers": "*", "@aws-sdk/client-ec2": "^3.1009.0", + "@aws-sdk/client-ssm": "^3.1009.0", "@octokit/rest": "22.0.1", "moment": "2.29.4", "yn": "3.1.1" diff --git a/lambdas/libs/compute-providers/providers.config.scale-set.ts b/lambdas/libs/compute-providers/providers.config.scale-set.ts new file mode 100644 index 0000000000..7f8f0ecc64 --- /dev/null +++ b/lambdas/libs/compute-providers/providers.config.scale-set.ts @@ -0,0 +1,5 @@ +import { provider as ec2 } from './aws/ec2/scale-set'; +import type { ScaleSetComputeProviderModule } from './scale-set'; + +/** Provider plugins included in the scale-set service bundle. */ +export const enabledScaleSetProviders = [ec2] as const satisfies readonly ScaleSetComputeProviderModule[]; diff --git a/lambdas/libs/compute-providers/scale-set.test.ts b/lambdas/libs/compute-providers/scale-set.test.ts new file mode 100644 index 0000000000..f56e044d86 --- /dev/null +++ b/lambdas/libs/compute-providers/scale-set.test.ts @@ -0,0 +1,124 @@ +import { describe, expect, it, vi } from 'vitest'; + +import { createEc2ScaleSetPlugin } from './aws/ec2/scale-set'; +import { + createScaleSetComputeProviderRegistry, + type ScaleSetComputeProviderPlugin, + validateScaleSetProviderEnvironmentVariables, +} from './scale-set'; + +const configuration = { + region: 'eu-west-1', + environment: 'unit-test', + runnerNamePrefix: 'runner-', + jitConfigParameterPath: '/github-action-runners/unit-test/runners/tokens', + subnets: ['subnet-12345678'], + launchTemplateName: 'unit-test-runners', + ec2instanceCriteria: { + instanceTypes: ['m7i.large'], + targetCapacityType: 'on-demand', + instanceAllocationStrategy: 'lowest-price', + }, +}; + +describe('scale-set compute-provider registry', () => { + it('creates a separate provider instance for every runner config', () => { + const registry = createScaleSetComputeProviderRegistry([createEc2ScaleSetPlugin()]); + const first = registry.create('ec2', { + runnerConfigName: 'shared', + scaleSetId: 1, + githubScope: 'https://github.com/first', + configuration, + }); + const second = registry.create('ec2', { + runnerConfigName: 'shared', + scaleSetId: 1, + githubScope: 'https://github.com/second', + configuration, + }); + + expect(first).not.toBe(second); + expect(first.reconcile).toEqual(expect.any(Function)); + expect(second.reconcile).toEqual(expect.any(Function)); + expect(registry.environmentVariables('ec2')).toEqual({}); + expect(Object.isFrozen(registry.environmentVariables('ec2'))).toBe(true); + }); + + it('rejects duplicate and missing plugins explicitly', () => { + const plugin: ScaleSetComputeProviderPlugin = { + type: 'test', + capabilities: { + environmentVariables: {}, + create: vi.fn(() => ({ reconcile: vi.fn() })), + }, + }; + + expect(() => createScaleSetComputeProviderRegistry([plugin, plugin])).toThrow( + "Duplicate scale-set compute provider plugin 'test'", + ); + expect(() => + createScaleSetComputeProviderRegistry([]).create('missing', { + runnerConfigName: 'runner', + scaleSetId: 1, + githubScope: 'https://github.com/example', + configuration: {}, + }), + ).toThrow("No scale-set compute provider plugin registered for 'missing'"); + expect(() => createScaleSetComputeProviderRegistry([]).environmentVariables('missing')).toThrow( + "No scale-set compute provider plugin registered for 'missing'", + ); + }); + + it('returns a validated immutable provider environment', () => { + const source = { EC2_ENDPOINT_MODE: 'regional' }; + const registry = createScaleSetComputeProviderRegistry([ + { + type: 'test', + capabilities: { + environmentVariables: source, + create: vi.fn(() => ({ reconcile: vi.fn() })), + }, + }, + ]); + + const environment = registry.environmentVariables('test'); + source.EC2_ENDPOINT_MODE = 'changed-after-registration'; + expect(environment).toEqual({ EC2_ENDPOINT_MODE: 'regional' }); + expect(Object.isFrozen(environment)).toBe(true); + }); + + it.each>>([ + { AWS_REGION: 'eu-west-1' }, + { SCALE_SET_OVERRIDE: 'unsafe' }, + { NODE_OPTIONS: '--import=untrusted' }, + { PATH: '/untrusted' }, + { lower_case: 'value' }, + { VALID_NAME: 'line\nbreak' }, + { VALID_NAME: 'x'.repeat(4097) }, + ])('rejects reserved or unsafe provider environment variables: %o', (environmentVariables) => { + expect(() => + createScaleSetComputeProviderRegistry([ + { + type: 'test', + capabilities: { + environmentVariables, + create: vi.fn(() => ({ reconcile: vi.fn() })), + }, + }, + ]), + ).toThrow(/reserved or invalid|invalid value/); + }); + + it('rejects malformed or oversized provider environments', () => { + expect(() => validateScaleSetProviderEnvironmentVariables(null as never)).toThrow('must be an object'); + expect(() => validateScaleSetProviderEnvironmentVariables([] as never)).toThrow('must be an object'); + expect(() => validateScaleSetProviderEnvironmentVariables({ VALID_NAME: 42 } as never)).toThrow( + 'has an invalid value', + ); + expect(() => + validateScaleSetProviderEnvironmentVariables( + Object.fromEntries(Array.from({ length: 65 }, (_, index) => [`PROVIDER_${index}`, 'value'])), + ), + ).toThrow('must contain at most 64 entries'); + }); +}); diff --git a/lambdas/libs/compute-providers/scale-set.ts b/lambdas/libs/compute-providers/scale-set.ts new file mode 100644 index 0000000000..8d99125a98 --- /dev/null +++ b/lambdas/libs/compute-providers/scale-set.ts @@ -0,0 +1,216 @@ +import { enabledScaleSetProviders } from './providers.config.scale-set'; + +export type ScaleSetRunnerStatus = 'online' | 'offline' | 'unknown'; +export type ScaleSetRunnerLifecycle = 'started' | 'completed' | 'unknown'; + +/** + * Controller-observed GitHub state for one runner. + * + * The compute provider treats missing, duplicate, or unrecognized state as + * unknown. Callers must not infer `busy: false` when GitHub did not provide a + * busy state. + */ +export interface ScaleSetRunnerState { + runnerId: number; + runnerName: string; + scaleSetId: number; + status: ScaleSetRunnerStatus; + busy: boolean | undefined; + lifecycle: ScaleSetRunnerLifecycle; +} + +export interface GenerateScaleSetJitConfigurationInput { + runnerName: string; + signal?: AbortSignal; +} + +export interface GenerateScaleSetJitConfigurationResult { + encodedJitConfiguration: string; + runnerId: number; + runnerName: string; + scaleSetId: number; +} + +export type GenerateScaleSetJitConfiguration = ( + input: GenerateScaleSetJitConfigurationInput, +) => Promise; + +export interface RemoveScaleSetRunnerInput { + runnerId: number; + runnerName: string; + scaleSetId: number; + signal?: AbortSignal; +} + +export type ScaleSetRemoveRunnerStatus = 'removed' | 'retained_busy' | 'retained_unknown'; + +export interface ScaleSetRemoveRunnerResult { + status: ScaleSetRemoveRunnerStatus; +} + +export type RemoveScaleSetRunner = (input: RemoveScaleSetRunnerInput) => Promise; + +export interface ScaleSetReconcileRequest { + desiredRunners: number; + /** Aggregate busy-runner count reported by the GitHub Actions scale-set session. */ + busyRunners: number; + bootTimeoutMinutes: number; + runnerStates: readonly ScaleSetRunnerState[]; + signal: AbortSignal; + generateJitConfiguration: GenerateScaleSetJitConfiguration; + removeRunner: RemoveScaleSetRunner; +} + +export type ScaleSetReconcileStatus = 'converged' | 'retained' | 'error'; + +export type ScaleSetReconcileOperation = + | 'validate' + | 'reconcile' + | 'list' + | 'launch' + | 'generate_jit_configuration' + | 'publish_jit_configuration' + | 'remove_runner' + | 'terminate'; + +/** Error metadata is deliberately bounded and never contains a JIT configuration or raw upstream error message. */ +export interface ScaleSetReconcileError { + operation: ScaleSetReconcileOperation; + code: string; + runnerName?: string; + resourceId?: string; +} + +export interface ScaleSetReconcileActions { + launched: number; + terminated: number; + retainedBusy: number; + retainedUnknown: number; +} + +export interface ScaleSetReconcileResult { + status: ScaleSetReconcileStatus; + desiredRunners: number; + /** Best-known owned capacity after actions completed; the next reconciliation re-observes AWS. */ + currentRunners: number; + actions: ScaleSetReconcileActions; + errors: readonly ScaleSetReconcileError[]; +} + +export interface ScaleSetComputeProvider { + reconcile(request: ScaleSetReconcileRequest): Promise; +} + +export interface ScaleSetComputeProviderCredentials { + accessKeyId: string; + secretAccessKey: string; + sessionToken?: string; + expiration?: Date; +} + +export type ScaleSetComputeProviderCredentialProvider = () => Promise; + +export interface ScaleSetComputeProviderFactoryInput { + runnerConfigName: string; + scaleSetId: number; + /** Canonical GitHub configuration URL used as an immutable provider ownership scope. */ + githubScope: string; + /** Optional credentials selected by the orchestrator for provider API calls. */ + credentials?: ScaleSetComputeProviderCredentialProvider; + /** Provider-owned configuration. The selected provider validates it before use. */ + configuration: unknown; +} + +export interface ScaleSetComputeProviderCapabilities { + /** Provider-owned, non-secret task environment. Values are validated and immutable after registration. */ + environmentVariables: Readonly>; + create(input: ScaleSetComputeProviderFactoryInput): ScaleSetComputeProvider; +} + +export interface ScaleSetComputeProviderPlugin { + type: TType; + capabilities: ScaleSetComputeProviderCapabilities; +} + +export interface ScaleSetComputeProviderModule { + type: TType; + createPlugin(): ScaleSetComputeProviderPlugin; +} + +const SCALE_SET_ENVIRONMENT_KEY = /^[A-Z][A-Z0-9_]{0,127}$/; +const RESERVED_SCALE_SET_ENVIRONMENT_PREFIXES = ['AWS_', 'ECS_', 'GITHUB_', 'SCALE_SET_', 'NODE_']; +const RESERVED_SCALE_SET_ENVIRONMENT_KEYS = new Set(['PATH', 'HOME', 'HOSTNAME', 'PWD', 'SHLVL']); +const MAX_SCALE_SET_ENVIRONMENT_VARIABLES = 64; +const MAX_SCALE_SET_ENVIRONMENT_VALUE_BYTES = 4096; + +export function validateScaleSetProviderEnvironmentVariables( + value: Readonly>, +): Readonly> { + if (typeof value !== 'object' || value === null || Array.isArray(value)) { + throw new Error('Scale-set provider environmentVariables must be an object'); + } + const entries = Object.entries(value); + if (entries.length > MAX_SCALE_SET_ENVIRONMENT_VARIABLES) { + throw new Error( + `Scale-set provider environmentVariables must contain at most ${MAX_SCALE_SET_ENVIRONMENT_VARIABLES} entries`, + ); + } + + const normalized = Object.create(null) as Record; + for (const [key, environmentValue] of entries) { + if ( + !SCALE_SET_ENVIRONMENT_KEY.test(key) || + RESERVED_SCALE_SET_ENVIRONMENT_KEYS.has(key) || + RESERVED_SCALE_SET_ENVIRONMENT_PREFIXES.some((prefix) => key.startsWith(prefix)) + ) { + throw new Error(`Scale-set provider environment variable '${key}' is reserved or invalid`); + } + if ( + typeof environmentValue !== 'string' || + Buffer.byteLength(environmentValue, 'utf8') > MAX_SCALE_SET_ENVIRONMENT_VALUE_BYTES || + [...environmentValue].some((character) => { + const code = character.charCodeAt(0); + return code <= 31 || code === 127; + }) + ) { + throw new Error(`Scale-set provider environment variable '${key}' has an invalid value`); + } + normalized[key] = environmentValue; + } + return Object.freeze(normalized); +} + +export function createScaleSetComputeProviderRegistry( + plugins: readonly ScaleSetComputeProviderPlugin[] = enabledScaleSetProviders.map((provider) => + provider.createPlugin(), + ), +) { + const pluginsByType = new Map(); + const environmentVariablesByType = new Map>>(); + for (const plugin of plugins) { + if (pluginsByType.has(plugin.type)) { + throw new Error(`Duplicate scale-set compute provider plugin '${plugin.type}'`); + } + pluginsByType.set(plugin.type, plugin); + environmentVariablesByType.set( + plugin.type, + validateScaleSetProviderEnvironmentVariables(plugin.capabilities.environmentVariables), + ); + } + + function get(type: string): ScaleSetComputeProviderPlugin { + const plugin = pluginsByType.get(type); + if (!plugin) throw new Error(`No scale-set compute provider plugin registered for '${type}'`); + return plugin; + } + + return { + create(type: string, input: ScaleSetComputeProviderFactoryInput): ScaleSetComputeProvider { + return get(type).capabilities.create(input); + }, + environmentVariables(type: string): Readonly> { + get(type); + return environmentVariablesByType.get(type)!; + }, + }; +} diff --git a/lambdas/libs/compute-providers/vitest.config.ts b/lambdas/libs/compute-providers/vitest.config.ts index fd62b358c0..24c4554995 100644 --- a/lambdas/libs/compute-providers/vitest.config.ts +++ b/lambdas/libs/compute-providers/vitest.config.ts @@ -16,7 +16,7 @@ export default mergeConfig(defaultConfig, { 'core/**/*.ts', 'aws/**/*.ts', ], - exclude: ['**/*.test.ts', '**/*.d.ts', 'templates/**/*'], + exclude: ['**/*.test.ts', '**/test/**/*.ts', '**/*.d.ts', 'templates/**/*'], thresholds: { statements: 96.16, branches: 95.32, diff --git a/lambdas/libs/github-actions-scale-set/LICENSE.actions-scaleset b/lambdas/libs/github-actions-scale-set/LICENSE.actions-scaleset new file mode 100644 index 0000000000..28a50fa226 --- /dev/null +++ b/lambdas/libs/github-actions-scale-set/LICENSE.actions-scaleset @@ -0,0 +1,21 @@ +MIT License + +Copyright GitHub, Inc. + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/lambdas/libs/github-actions-scale-set/README.md b/lambdas/libs/github-actions-scale-set/README.md new file mode 100644 index 0000000000..6696627a11 --- /dev/null +++ b/lambdas/libs/github-actions-scale-set/README.md @@ -0,0 +1,69 @@ +# GitHub Actions runner scale-set client for TypeScript + +This workspace package implements the GitHub Actions runner scale-set protocol with native `fetch`. It is intended for the `terraform-aws-github-runner` control plane and can also be reused by other Node.js scale-set listeners. + +The protocol is currently a GitHub Public Preview. This implementation follows the public [`actions/scaleset`](https://github.com/actions/scaleset) Go client at commit [`cb0405b`](https://github.com/actions/scaleset/tree/cb0405b2d874500e75ae34eff8d582ab75956b45). + +The upstream copyright and MIT permission notice are retained in [`LICENSE.actions-scaleset`](./LICENSE.actions-scaleset). + +## What it provides + +- HTTPS organization, repository, enterprise, GitHub.com, and GHES registration URLs; +- PAT authentication or an asynchronous access-token provider for existing GitHub App authentication; +- runner scale-set CRUD and runner-group lookup; +- just-in-time runner configuration generation; +- runner lookup and removal; +- message-session creation, refresh, long polling, acknowledgement, and job acquisition; +- bounded retries for idempotent requests that encounter network failures, HTTP 429, or HTTP 5xx responses. + +There is no scale-up or scale-down REST operation. A listener polls the message queue and reports its maximum capacity. GitHub returns `statistics.totalAssignedJobs`; the caller reconciles its compute capacity to that value and terminates ephemeral compute after `JobCompleted` messages. + +## Basic usage + +```ts +import { GitHubActionsScaleSetClient } from '@aws-github-runner/github-actions-scale-set'; + +const client = new GitHubActionsScaleSetClient({ + gitHubConfigUrl: 'https://github.com/example', + accessTokenProvider: async () => installationAccessToken, + systemInfo: { + system: 'terraform-aws-github-runner', + subsystem: 'scale-set-listener', + }, + retry: { + maxRetries: 4, + initialBackoffMs: 1_000, + maxBackoffMs: 30_000, + requestTimeoutMs: 5 * 60_000, + }, +}); + +const scaleSet = await client.createRunnerScaleSet({ + name: 'linux-x64', + runnerGroupId: 1, + runnerSetting: { disableUpdate: true }, +}); + +const session = await client.createMessageSessionClient(scaleSet.id!, 'listener-01'); + +try { + const message = await session.getMessage(0, 20); + if (message) { + await session.deleteMessage(message.messageId); + + const availableIds = message.jobAvailableMessages.map((job) => job.runnerRequestId); + await session.acquireJobs(availableIds); + + // Reconcile compute from message.statistics.totalAssignedJobs and use + // client.generateJitRunnerConfig(...) for every runner being created. + } +} finally { + await session.close(); +} +``` + +Treat encoded JIT configurations and all access tokens as secrets. The upstream Go listener acknowledges a message before job acquisition and scaling callbacks; a later callback failure is returned from the listener and does not cause message redelivery. + +The retry values shown above are the defaults. Automatic transport retries apply only to idempotent methods (`GET`, `HEAD`, `OPTIONS`, `PUT`, and `DELETE`). Non-idempotent `POST` and `PATCH` operations, including JIT generation, session creation, and job acquisition, are attempted once so a lost response cannot cause the operation to be replayed. `Retry-After` is honored for eligible 429/5xx responses and capped by `maxBackoffMs`. Caller cancellation interrupts both an active request and retry backoff. + +The `/actions/runner-registration` admin bootstrap is the sole narrowly scoped exception: its `POST` retries transient transport failures and 429/5xx responses, plus 401/403 while RemoteAuth propagates. Queue 401 responses are not transport-retried; they trigger the message-session token refresh flow once. diff --git a/lambdas/libs/github-actions-scale-set/package.json b/lambdas/libs/github-actions-scale-set/package.json new file mode 100644 index 0000000000..8e0d87576a --- /dev/null +++ b/lambdas/libs/github-actions-scale-set/package.json @@ -0,0 +1,35 @@ +{ + "name": "@aws-github-runner/github-actions-scale-set", + "version": "1.0.0", + "main": "src/index.ts", + "type": "module", + "license": "MIT", + "exports": { + ".": "./src/index.ts", + "./config": "./src/config.ts", + "./errors": "./src/errors.ts", + "./types": "./src/types.ts" + }, + "scripts": { + "test": "NODE_ENV=test nx test", + "test:watch": "NODE_ENV=test nx test --watch", + "lint": "eslint src", + "format": "prettier --write \"**/*.ts\"", + "format-check": "prettier --check \"**/*.ts\"", + "typecheck": "tsc --noEmit", + "all": "yarn format && yarn lint && yarn typecheck && yarn test" + }, + "devDependencies": { + "@types/node": "^22.19.3", + "typescript": "^5.9.3" + }, + "nx": { + "includedScripts": [ + "format", + "format-check", + "lint", + "typecheck", + "all" + ] + } +} diff --git a/lambdas/libs/github-actions-scale-set/src/client.test.ts b/lambdas/libs/github-actions-scale-set/src/client.test.ts new file mode 100644 index 0000000000..246ee8cbb8 --- /dev/null +++ b/lambdas/libs/github-actions-scale-set/src/client.test.ts @@ -0,0 +1,434 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest'; + +import { actionsServiceUrl, GitHubActionsScaleSetClient } from './client'; +import { SCALE_SET_ERROR_CODES, ScaleSetHttpError, ScaleSetProtocolError } from './errors'; +import { ScaleSetFetch } from './types'; + +type RequestInput = Parameters[0]; +type ServiceHandler = (url: URL, init: RequestInit) => Response | Promise; + +function requestUrl(input: RequestInput): URL { + if (input instanceof Request) { + return new URL(input.url); + } + return new URL(input.toString()); +} + +function jsonResponse(value: unknown, status = 200): Response { + return new Response(JSON.stringify(value), { + status, + headers: { 'Content-Type': 'application/json' }, + }); +} + +function actionsAdminToken(expiresAt = Math.floor(Date.now() / 1000) + 60 * 60): string { + const header = Buffer.from(JSON.stringify({ alg: 'RS256', typ: 'JWT' })).toString('base64url'); + const payload = Buffer.from(JSON.stringify({ exp: expiresAt })).toString('base64url'); + return `${header}.${payload}.signature`; +} + +function clientFixture(serviceHandler: ServiceHandler, options: { adminToken?: () => string; now?: () => Date } = {}) { + const accessTokenProvider = vi.fn(async () => 'github-access-token'); + const registrationRequests: Array<{ url: URL; init: RequestInit }> = []; + const serviceRequests: Array<{ url: URL; init: RequestInit }> = []; + const fetchImplementation = vi.fn(async (input, init = {}) => { + const url = requestUrl(input); + + if (url.pathname === '/orgs/example/actions/runners/registration-token') { + registrationRequests.push({ url, init }); + return jsonResponse({ token: 'runner-registration-token' }, 201); + } + if (url.pathname === '/actions/runner-registration') { + registrationRequests.push({ url, init }); + return jsonResponse({ + url: 'https://actions.example/tenant/123/', + token: options.adminToken?.() ?? actionsAdminToken(), + }); + } + + serviceRequests.push({ url, init }); + return serviceHandler(url, init); + }); + const client = new GitHubActionsScaleSetClient({ + gitHubConfigUrl: 'https://github.com/example', + accessTokenProvider, + fetch: fetchImplementation, + systemInfo: { system: 'unit-test', subsystem: 'sdk' }, + now: options.now, + }); + + return { + accessTokenProvider, + client, + fetchImplementation, + registrationRequests, + serviceRequests, + }; +} + +describe('actionsServiceUrl', () => { + it('normalizes a long trailing slash sequence before joining the request path', () => { + const base = `https://actions.example/tenant/123${'/'.repeat(10_000)}`; + + expect(actionsServiceUrl(base, '').pathname).toBe('/tenant/123'); + expect(actionsServiceUrl(base, '_apis/runtime/runnerscalesets').pathname).toBe( + '/tenant/123/_apis/runtime/runnerscalesets', + ); + const url = actionsServiceUrl(base, '/_apis/runtime/runnerscalesets'); + expect(url.pathname).toBe('/tenant/123/_apis/runtime/runnerscalesets'); + expect(url.searchParams.get('api-version')).toBe('6.0-preview'); + }); +}); + +describe('GitHubActionsScaleSetClient', () => { + beforeEach(() => { + vi.restoreAllMocks(); + }); + + it('bootstraps Actions authentication once and reuses the unexpired admin token', async () => { + const fixture = clientFixture(() => jsonResponse({ count: 0, value: [] })); + + await expect(fixture.client.getRunnerScaleSet(4, 'linux')).resolves.toBeNull(); + await expect(fixture.client.getRunnerScaleSet(4, 'linux')).resolves.toBeNull(); + + expect(fixture.accessTokenProvider).toHaveBeenCalledOnce(); + expect(fixture.registrationRequests).toHaveLength(2); + expect(fixture.serviceRequests).toHaveLength(2); + + const registrationTokenRequest = fixture.registrationRequests[0]; + expect(registrationTokenRequest.url.toString()).toBe( + 'https://api.github.com/orgs/example/actions/runners/registration-token', + ); + expect(new Headers(registrationTokenRequest.init.headers).get('Authorization')).toBe('Bearer github-access-token'); + expect(new Headers(registrationTokenRequest.init.headers).get('Content-Type')).toBe( + 'application/vnd.github.v3+json', + ); + expect(JSON.parse(new Headers(registrationTokenRequest.init.headers).get('User-Agent') as string)).toMatchObject({ + build_commit_sha: '', + kind: 'scaleset', + system: 'unit-test', + }); + + const adminConnectionRequest = fixture.registrationRequests[1]; + expect(adminConnectionRequest.url.toString()).toBe('https://api.github.com/actions/runner-registration'); + expect(new Headers(adminConnectionRequest.init.headers).get('Authorization')).toBe( + 'RemoteAuth runner-registration-token', + ); + expect(JSON.parse(adminConnectionRequest.init.body as string)).toEqual({ + url: 'https://github.com/example', + runner_event: 'register', + }); + + for (const { url, init } of fixture.serviceRequests) { + expect(url.pathname).toBe('/tenant/123/_apis/runtime/runnerscalesets'); + expect(url.searchParams.get('api-version')).toBe('6.0-preview'); + expect(url.searchParams.get('runnerGroupId')).toBe('4'); + expect(url.searchParams.get('name')).toBe('linux'); + expect(new Headers(init.headers).get('Authorization')).toMatch(/^Bearer /); + } + }); + + it.each([ + 'http://actions.example/tenant/123', + 'https://user:password@actions.example/tenant/123', + 'https://actions.example/tenant/123?signature=secret', + 'https://actions.example/tenant/123#fragment', + ])('rejects an unsafe Actions service admin URL: %s', async (unsafeUrl) => { + const fetchImplementation = vi.fn(async (input) => { + const url = requestUrl(input); + if (url.pathname === '/orgs/example/actions/runners/registration-token') { + return jsonResponse({ token: 'runner-registration-token' }, 201); + } + if (url.pathname === '/actions/runner-registration') { + return jsonResponse({ url: unsafeUrl, token: actionsAdminToken() }); + } + return new Response(null, { status: 500 }); + }); + const client = new GitHubActionsScaleSetClient({ + gitHubConfigUrl: 'https://github.com/example', + personalAccessToken: 'github-access-token', + fetch: fetchImplementation, + }); + + await expect(client.getRunnerScaleSetById(42)).rejects.toBeInstanceOf(ScaleSetProtocolError); + }); + + it('refreshes the Actions admin token when it enters the 60-second expiry window', async () => { + let nowMs = Date.UTC(2026, 7, 14, 12, 0, 0); + let tokenIssue = 0; + const fixture = clientFixture(() => jsonResponse({ count: 0, value: [] }), { + now: () => new Date(nowMs), + adminToken: () => { + tokenIssue += 1; + const lifetimeSeconds = tokenIssue === 1 ? 120 : 3_600; + return actionsAdminToken(Math.floor(nowMs / 1000) + lifetimeSeconds); + }, + }); + + await fixture.client.getRunnerScaleSet(4, 'linux'); + nowMs += 70_000; + await fixture.client.getRunnerScaleSet(4, 'linux'); + + expect(fixture.accessTokenProvider).toHaveBeenCalledTimes(2); + expect(fixture.registrationRequests).toHaveLength(4); + expect(tokenIssue).toBe(2); + }); + + it('retries transient and propagation failures only while bootstrapping the admin connection', async () => { + let registrationTokenRequests = 0; + let adminConnectionRequests = 0; + let actionsRequests = 0; + const fetchImplementation = vi.fn(async (input) => { + const url = requestUrl(input); + + if (url.pathname === '/orgs/example/actions/runners/registration-token') { + registrationTokenRequests += 1; + return jsonResponse({ token: 'runner-registration-token' }, 201); + } + if (url.pathname === '/actions/runner-registration') { + adminConnectionRequests += 1; + if (adminConnectionRequests === 1) { + return jsonResponse({ message: 'temporarily unavailable' }, 503); + } + if (adminConnectionRequests === 2) { + return jsonResponse({ message: 'not propagated' }, 401); + } + if (adminConnectionRequests === 3) { + return jsonResponse({ message: 'not propagated' }, 403); + } + return jsonResponse({ + url: 'https://actions.example/tenant/123/', + token: actionsAdminToken(), + }); + } + + actionsRequests += 1; + return jsonResponse({ count: 0, value: [] }); + }); + const client = new GitHubActionsScaleSetClient({ + gitHubConfigUrl: 'https://github.com/example', + personalAccessToken: 'github-access-token', + fetch: fetchImplementation, + retry: { + maxRetries: 3, + initialBackoffMs: 0, + maxBackoffMs: 0, + requestTimeoutMs: 1_000, + }, + }); + + await expect(client.getRunnerScaleSet(4, 'linux')).resolves.toBeNull(); + + expect(registrationTokenRequests).toBe(1); + expect(adminConnectionRequests).toBe(4); + expect(actionsRequests).toBe(1); + }); + + it('does not replay JIT generation when its POST receives a transient failure', async () => { + let jitRequests = 0; + const fetchImplementation = vi.fn(async (input) => { + const url = requestUrl(input); + + if (url.pathname === '/orgs/example/actions/runners/registration-token') { + return jsonResponse({ token: 'runner-registration-token' }, 201); + } + if (url.pathname === '/actions/runner-registration') { + return jsonResponse({ + url: 'https://actions.example/tenant/123/', + token: actionsAdminToken(), + }); + } + if (url.pathname.endsWith('/generatejitconfig')) { + jitRequests += 1; + return jsonResponse({ message: 'temporarily unavailable' }, 503); + } + return new Response(null, { status: 500 }); + }); + const client = new GitHubActionsScaleSetClient({ + gitHubConfigUrl: 'https://github.com/example', + personalAccessToken: 'github-access-token', + fetch: fetchImplementation, + retry: { + maxRetries: 4, + initialBackoffMs: 0, + maxBackoffMs: 0, + requestTimeoutMs: 1_000, + }, + }); + + await expect(client.generateJitRunnerConfig({ name: 'runner-71', workFolder: '_work' }, 42)).rejects.toMatchObject({ + status: 503, + }); + expect(jitRequests).toBe(1); + }); + + it('raises a typed HTTP error with Actions exception and request metadata', async () => { + const fixture = clientFixture( + () => + new Response( + JSON.stringify({ + typeName: 'Microsoft.TeamFoundation.DistributedTask.WebApi.AgentExistsException', + message: 'runner already exists', + }), + { + status: 409, + headers: { + ActivityId: 'activity-123', + 'Content-Type': 'application/json', + 'X-GitHub-Request-Id': 'github-456', + }, + }, + ), + ); + + const request = fixture.client.getRunner(71); + await expect(request).rejects.toBeInstanceOf(ScaleSetHttpError); + await expect(request).rejects.toMatchObject({ + code: SCALE_SET_ERROR_CODES.runnerExists, + status: 409, + activityId: 'activity-123', + githubRequestId: 'github-456', + exceptionName: 'Microsoft.TeamFoundation.DistributedTask.WebApi.AgentExistsException', + }); + }); + + it('uses the exact scale-set CRUD endpoints and legacy request body casing', async () => { + const fixture = clientFixture((url, init) => { + const method = init.method; + if (method === 'GET' && url.pathname.endsWith('/runnerscalesets')) { + return jsonResponse({ + count: url.searchParams.has('name') ? 1 : 2, + value: [ + { id: 11, name: 'linux', RunnerSetting: { disableUpdate: true } }, + { id: 12, name: 'windows', RunnerSetting: {} }, + ], + }); + } + if (method === 'GET' && url.pathname.endsWith('/runnerscalesets/11')) { + return jsonResponse({ id: 11, name: 'linux', RunnerSetting: { disableUpdate: true } }); + } + if (method === 'POST') { + return jsonResponse({ id: 11, name: 'linux', RunnerSetting: { disableUpdate: true } }); + } + if (method === 'PATCH') { + return jsonResponse({ id: 11, name: 'linux', RunnerSetting: { disableUpdate: false } }); + } + if (method === 'DELETE') { + return new Response(null, { status: 204 }); + } + return new Response(null, { status: 500 }); + }); + const createInput = { + name: 'linux', + runnerGroupId: 4, + runnerSetting: { disableUpdate: true }, + }; + const updateInput = { + labels: [{ name: 'arm64' }], + runnerSetting: { disableUpdate: false }, + }; + + await expect(fixture.client.getRunnerScaleSet(4, 'linux')).resolves.toMatchObject({ + id: 11, + runnerSetting: { disableUpdate: true }, + }); + await expect(fixture.client.listRunnerScaleSets(4)).resolves.toHaveLength(2); + await expect(fixture.client.getRunnerScaleSetById(11)).resolves.toMatchObject({ id: 11 }); + await expect(fixture.client.createRunnerScaleSet(createInput)).resolves.toMatchObject({ id: 11 }); + await expect(fixture.client.updateRunnerScaleSet(11, updateInput)).resolves.toMatchObject({ + id: 11, + }); + await expect(fixture.client.deleteRunnerScaleSet(11)).resolves.toBeUndefined(); + + expect(createInput).toMatchObject({ labels: [{ name: 'linux', type: 'System' }] }); + expect(updateInput).toMatchObject({ labels: [{ name: 'arm64', type: 'System' }] }); + + const createRequest = fixture.serviceRequests.find(({ init }) => init.method === 'POST'); + const updateRequest = fixture.serviceRequests.find(({ init }) => init.method === 'PATCH'); + expect(createRequest).toBeDefined(); + expect(updateRequest).toBeDefined(); + + const createBody = JSON.parse(createRequest?.init.body as string) as Record; + expect(createBody).toMatchObject({ + name: 'linux', + runnerGroupId: 4, + labels: [{ name: 'linux', type: 'System' }], + RunnerSetting: { disableUpdate: true }, + }); + expect(createBody).not.toHaveProperty('runnerSetting'); + + const updateBody = JSON.parse(updateRequest?.init.body as string) as Record; + expect(updateBody).toMatchObject({ + labels: [{ name: 'arm64', type: 'System' }], + RunnerSetting: { disableUpdate: false }, + }); + expect(updateBody).not.toHaveProperty('runnerSetting'); + + expect(fixture.serviceRequests.map(({ url, init }) => [init.method, url.pathname])).toEqual([ + ['GET', '/tenant/123/_apis/runtime/runnerscalesets'], + ['GET', '/tenant/123/_apis/runtime/runnerscalesets'], + ['GET', '/tenant/123/_apis/runtime/runnerscalesets/11'], + ['POST', '/tenant/123/_apis/runtime/runnerscalesets'], + ['PATCH', '/tenant/123/_apis/runtime/runnerscalesets/11'], + ['DELETE', '/tenant/123/_apis/runtime/runnerscalesets/11'], + ]); + for (const { url } of fixture.serviceRequests) { + expect(url.searchParams.get('api-version')).toBe('6.0-preview'); + } + }); + + it('uses the exact runner-group, JIT, and agent endpoints', async () => { + const fixture = clientFixture((url, init) => { + if (url.pathname.endsWith('/runnergroups/')) { + return jsonResponse({ + count: 1, + value: [{ id: 8, name: 'default', size: 0, isDefaultGroup: true }], + }); + } + if (url.pathname.endsWith('/generatejitconfig')) { + return jsonResponse({ + runner: { id: 71, name: 'runner-71', runnerScaleSetId: 42 }, + encodedJITConfig: 'encoded-jit', + }); + } + if (init.method === 'GET' && url.pathname.endsWith('/agents/71')) { + return jsonResponse({ id: 71, name: 'runner-71', runnerScaleSetId: 42 }); + } + if (init.method === 'GET' && url.pathname.endsWith('/agents')) { + return jsonResponse({ + count: 1, + value: [{ id: 71, name: 'runner-71', runnerScaleSetId: 42 }], + }); + } + if (init.method === 'DELETE' && url.pathname.endsWith('/agents/71')) { + return new Response(null, { status: 204 }); + } + return new Response(null, { status: 500 }); + }); + + await expect(fixture.client.getRunnerGroupByName('default')).resolves.toMatchObject({ id: 8 }); + await expect( + fixture.client.generateJitRunnerConfig({ name: 'runner-71', workFolder: '_work' }, 42), + ).resolves.toEqual({ + runner: { id: 71, name: 'runner-71', runnerScaleSetId: 42 }, + encodedJITConfig: 'encoded-jit', + }); + await expect(fixture.client.getRunner(71)).resolves.toMatchObject({ id: 71 }); + await expect(fixture.client.getRunnerByName('runner-71')).resolves.toMatchObject({ id: 71 }); + await expect(fixture.client.removeRunner(71)).resolves.toBeUndefined(); + + expect(fixture.serviceRequests.map(({ url, init }) => [init.method, url.pathname])).toEqual([ + ['GET', '/tenant/123/_apis/runtime/runnergroups/'], + ['POST', '/tenant/123/_apis/runtime/runnerscalesets/42/generatejitconfig'], + ['GET', '/tenant/123/_apis/distributedtask/pools/0/agents/71'], + ['GET', '/tenant/123/_apis/distributedtask/pools/0/agents'], + ['DELETE', '/tenant/123/_apis/distributedtask/pools/0/agents/71'], + ]); + expect(fixture.serviceRequests[0].url.searchParams.get('groupName')).toBe('default'); + expect(fixture.serviceRequests[3].url.searchParams.get('agentName')).toBe('runner-71'); + expect(JSON.parse(fixture.serviceRequests[1].init.body as string)).toEqual({ + name: 'runner-71', + workFolder: '_work', + }); + }); +}); diff --git a/lambdas/libs/github-actions-scale-set/src/client.ts b/lambdas/libs/github-actions-scale-set/src/client.ts new file mode 100644 index 0000000000..36e89cd586 --- /dev/null +++ b/lambdas/libs/github-actions-scale-set/src/client.ts @@ -0,0 +1,554 @@ +import { githubApiUrl, ParsedGitHubConfig, parseGitHubConfigUrl, runnerRegistrationTokenPath } from './config'; +import { ACTIONS_API_VERSION, RUNNER_ENDPOINT, RUNNER_GROUP_ENDPOINT, SCALE_SET_ENDPOINT } from './endpoints'; +import { ScaleSetProtocolError } from './errors'; +import { createRetryingFetch, executeRequest, HttpResult, parseJsonResponse } from './http'; +import { MessageSessionClient } from './message-session-client'; +import { + AccessTokenProvider, + GitHubActionsScaleSetClientOptions, + RunnerGroup, + RunnerReference, + RunnerScaleSet, + RunnerScaleSetJitRunnerConfig, + RunnerScaleSetJitRunnerSetting, + ScaleSetFetch, + ScaleSetRequestOptions, + SystemInfo, +} from './types'; +import { trimTrailingSlashes } from './url'; + +const ADMIN_TOKEN_REFRESH_SKEW_MS = 60_000; +const SUCCESS_STATUSES = Array.from({ length: 100 }, (_, index) => index + 200); + +interface RegistrationTokenResponse { + token?: string; + expires_at?: string; +} + +interface ActionsServiceAdminConnectionResponse { + url?: string; + token?: string; +} + +interface ActionsServiceAdminToken { + token: string; + expiresAt: Date; + url: string; +} + +interface RunnerScaleSetListResponse { + count: number; + value: RunnerScaleSet[]; +} + +interface RunnerGroupListResponse { + count: number; + value: RunnerGroup[]; +} + +interface RunnerReferenceListResponse { + count: number; + value: RunnerReference[]; +} + +interface ActionsRequestOptions extends ScaleSetRequestOptions { + query?: Record; + body?: unknown; + expectedStatuses: readonly number[]; + authorization?: string; +} + +function joinUrlPath(base: string, path: string): string { + const normalizedBase = trimTrailingSlashes(base); + if (normalizedBase === '') { + if (path === '') { + return ''; + } + return path.startsWith('/') ? path : `/${path}`; + } + if (path === '') { + return normalizedBase; + } + return `${normalizedBase}${path.startsWith('/') ? '' : '/'}${path}`; +} + +export function actionsServiceUrl( + base: string, + path: string, + query: Record = {}, +): URL { + const [pathOnly, pathQuery = ''] = path.split('?', 2); + const result = new URL(joinUrlPath(base, pathOnly)); + const mergedQuery = new URLSearchParams(pathQuery); + + for (const [name, value] of Object.entries(query)) { + if (value !== undefined) { + mergedQuery.set(name, String(value)); + } + } + if (!mergedQuery.get('api-version')) { + mergedQuery.set('api-version', ACTIONS_API_VERSION); + } + result.search = mergedQuery.toString(); + return result; +} + +function encodeSystemUserAgent(systemInfo: SystemInfo): string { + return JSON.stringify({ + system: systemInfo.system ?? '', + version: systemInfo.version ?? '', + commit_sha: systemInfo.commitSha ?? '', + scale_set_id: systemInfo.scaleSetId ?? 0, + subsystem: systemInfo.subsystem ?? '', + build_version: '1.0.0', + build_commit_sha: '', + kind: 'scaleset', + }); +} + +function parseJwtExpiration(token: string): Date { + const parts = token.split('.'); + if (parts.length < 2) { + throw new ScaleSetProtocolError('Actions service admin token is not a JWT'); + } + + let claims: unknown; + try { + claims = JSON.parse(Buffer.from(parts[1], 'base64url').toString('utf8')) as unknown; + } catch (error) { + throw new ScaleSetProtocolError('failed to decode Actions service admin token claims', { + cause: error, + }); + } + + if ( + typeof claims !== 'object' || + claims === null || + !('exp' in claims) || + typeof claims.exp !== 'number' || + !Number.isFinite(claims.exp) + ) { + throw new ScaleSetProtocolError('Actions service admin token is missing a numeric exp claim'); + } + + return new Date(claims.exp * 1000); +} + +function applyDefaultLabelTypes(scaleSet: RunnerScaleSet): void { + for (const label of scaleSet.labels ?? []) { + label.type ||= 'System'; + } +} + +function ensureLabels(scaleSet: RunnerScaleSet): void { + if ((scaleSet.labels?.length ?? 0) > 0) { + return; + } + if (!scaleSet.name) { + throw new ScaleSetProtocolError('runner scale set must have a name or at least one label'); + } + scaleSet.labels = [{ name: scaleSet.name, type: 'System' }]; +} + +function runnerScaleSetRequestBody(scaleSet: RunnerScaleSet): Record { + const wire = { ...scaleSet } as Record; + delete wire.runnerSetting; + // The capital R is intentional and matches the Actions scale set wire contract. + wire.RunnerSetting = scaleSet.runnerSetting ?? {}; + return wire; +} + +function normalizeRunnerScaleSet(scaleSet: RunnerScaleSet | null): RunnerScaleSet | null { + if (scaleSet === null) { + return null; + } + + const wire = scaleSet as RunnerScaleSet & { RunnerSetting?: RunnerScaleSet['runnerSetting'] }; + if (wire.runnerSetting === undefined && wire.RunnerSetting !== undefined) { + wire.runnerSetting = wire.RunnerSetting; + } + delete wire.RunnerSetting; + return wire; +} + +/** A native-fetch client for the GitHub Actions runner scale set APIs. */ +export class GitHubActionsScaleSetClient { + private readonly config: ParsedGitHubConfig; + private readonly fetchImplementation: ScaleSetFetch; + private readonly adminConnectionFetchImplementation: ScaleSetFetch; + private readonly accessTokenProvider: AccessTokenProvider; + private readonly now: () => Date; + private readonly customUserAgent?: string; + private currentSystemInfo: SystemInfo; + private currentUserAgent: string; + private adminToken?: ActionsServiceAdminToken; + private adminTokenRefresh?: Promise; + + constructor(options: GitHubActionsScaleSetClientOptions) { + this.config = parseGitHubConfigUrl(options.gitHubConfigUrl, options.forceGhes); + const fetchImplementation = options.fetch ?? globalThis.fetch; + if (typeof fetchImplementation !== 'function') { + throw new TypeError('a fetch implementation is required'); + } + this.fetchImplementation = createRetryingFetch(fetchImplementation, options.retry); + // Upstream retries transient RemoteAuth propagation failures only for this + // bootstrap POST. Queue 401s must remain owned by session-token refresh, + // and other non-idempotent SDK requests must never use this opt-in wrapper. + this.adminConnectionFetchImplementation = createRetryingFetch(fetchImplementation, options.retry, { + additionalRetryStatuses: [401, 403], + additionalRetryMethods: ['POST'], + }); + + const hasPersonalAccessToken = + typeof options.personalAccessToken === 'string' && options.personalAccessToken !== ''; + const hasAccessTokenProvider = typeof options.accessTokenProvider === 'function'; + if (hasPersonalAccessToken === hasAccessTokenProvider) { + throw new TypeError('provide exactly one of personalAccessToken or accessTokenProvider'); + } + + this.accessTokenProvider = hasPersonalAccessToken + ? async () => options.personalAccessToken as string + : (options.accessTokenProvider as AccessTokenProvider); + this.now = options.now ?? (() => new Date()); + this.currentSystemInfo = { ...options.systemInfo }; + this.customUserAgent = options.userAgent; + this.currentUserAgent = options.userAgent ?? encodeSystemUserAgent(this.currentSystemInfo); + } + + get gitHubConfig(): ParsedGitHubConfig { + return { + ...this.config, + configUrl: new URL(this.config.configUrl), + }; + } + + get systemInfo(): SystemInfo { + return { ...this.currentSystemInfo }; + } + + setSystemInfo(systemInfo: SystemInfo): void { + this.currentSystemInfo = { ...systemInfo }; + if (this.customUserAgent === undefined) { + this.currentUserAgent = encodeSystemUserAgent(systemInfo); + } + } + + debugInfo(): string { + return JSON.stringify({ + system_info: this.currentUserAgent, + }); + } + + async getRunnerScaleSet( + runnerGroupId: number, + runnerScaleSetName: string, + options: ScaleSetRequestOptions = {}, + ): Promise { + const { result, url } = await this.actionsRequest('GET', SCALE_SET_ENDPOINT, { + expectedStatuses: [200], + query: { runnerGroupId, name: runnerScaleSetName }, + signal: options.signal, + }); + const list = parseJsonResponse(result, 'GET', url); + + if (list.count === 0) { + return null; + } + if (list.count !== 1) { + throw new ScaleSetProtocolError( + `multiple runner scale sets found with name ${JSON.stringify(runnerScaleSetName)}`, + ); + } + if (list.value.length === 0) { + throw new ScaleSetProtocolError('runner scale set response count was 1 but value was empty'); + } + return normalizeRunnerScaleSet(list.value[0]); + } + + async listRunnerScaleSets(runnerGroupId: number, options: ScaleSetRequestOptions = {}): Promise { + const { result, url } = await this.actionsRequest('GET', SCALE_SET_ENDPOINT, { + expectedStatuses: [200], + query: { runnerGroupId }, + signal: options.signal, + }); + const list = parseJsonResponse(result, 'GET', url); + return list.value.map((scaleSet) => normalizeRunnerScaleSet(scaleSet) as RunnerScaleSet); + } + + async getRunnerScaleSetById( + runnerScaleSetId: number, + options: ScaleSetRequestOptions = {}, + ): Promise { + const path = `/${SCALE_SET_ENDPOINT}/${runnerScaleSetId}`; + const { result, url } = await this.actionsRequest('GET', path, { + expectedStatuses: [200], + signal: options.signal, + }); + return normalizeRunnerScaleSet(parseJsonResponse(result, 'GET', url)); + } + + async getRunnerGroupByName(runnerGroupName: string, options: ScaleSetRequestOptions = {}): Promise { + const { result, url } = await this.actionsRequest('GET', `/${RUNNER_GROUP_ENDPOINT}`, { + expectedStatuses: [200], + query: { groupName: runnerGroupName }, + signal: options.signal, + }); + const list = parseJsonResponse(result, 'GET', url); + + if (list.count === 0) { + throw new ScaleSetProtocolError(`no runner group found with name ${JSON.stringify(runnerGroupName)}`); + } + if (list.count !== 1) { + throw new ScaleSetProtocolError(`multiple runner groups found with name ${JSON.stringify(runnerGroupName)}`); + } + if (list.value.length === 0) { + throw new ScaleSetProtocolError('runner group response count was 1 but value was empty'); + } + return list.value[0]; + } + + async createRunnerScaleSet(scaleSet: RunnerScaleSet, options: ScaleSetRequestOptions = {}): Promise { + ensureLabels(scaleSet); + applyDefaultLabelTypes(scaleSet); + const { result, url } = await this.actionsRequest('POST', SCALE_SET_ENDPOINT, { + body: runnerScaleSetRequestBody(scaleSet), + expectedStatuses: [200], + signal: options.signal, + }); + return normalizeRunnerScaleSet(parseJsonResponse(result, 'POST', url)) as RunnerScaleSet; + } + + async updateRunnerScaleSet( + runnerScaleSetId: number, + scaleSet: RunnerScaleSet, + options: ScaleSetRequestOptions = {}, + ): Promise { + applyDefaultLabelTypes(scaleSet); + const path = `${SCALE_SET_ENDPOINT}/${runnerScaleSetId}`; + const { result, url } = await this.actionsRequest('PATCH', path, { + body: runnerScaleSetRequestBody(scaleSet), + expectedStatuses: [200], + signal: options.signal, + }); + return normalizeRunnerScaleSet(parseJsonResponse(result, 'PATCH', url)) as RunnerScaleSet; + } + + async deleteRunnerScaleSet(runnerScaleSetId: number, options: ScaleSetRequestOptions = {}): Promise { + await this.actionsRequest('DELETE', `/${SCALE_SET_ENDPOINT}/${runnerScaleSetId}`, { + expectedStatuses: [204], + signal: options.signal, + }); + } + + async generateJitRunnerConfig( + setting: RunnerScaleSetJitRunnerSetting, + runnerScaleSetId: number, + options: ScaleSetRequestOptions = {}, + ): Promise { + const path = `/${SCALE_SET_ENDPOINT}/${runnerScaleSetId}/generatejitconfig`; + const { result, url } = await this.actionsRequest('POST', path, { + body: setting, + expectedStatuses: [200], + signal: options.signal, + }); + return parseJsonResponse(result, 'POST', url); + } + + async getRunner(runnerId: number, options: ScaleSetRequestOptions = {}): Promise { + const path = `/${RUNNER_ENDPOINT}/${runnerId}`; + const { result, url } = await this.actionsRequest('GET', path, { + expectedStatuses: [200], + signal: options.signal, + }); + return parseJsonResponse(result, 'GET', url); + } + + async getRunnerByName(runnerName: string, options: ScaleSetRequestOptions = {}): Promise { + const { result, url } = await this.actionsRequest('GET', RUNNER_ENDPOINT, { + expectedStatuses: [200], + query: { agentName: runnerName }, + signal: options.signal, + }); + const list = parseJsonResponse(result, 'GET', url); + + if (list.count === 0) { + return null; + } + if (list.count !== 1) { + throw new ScaleSetProtocolError(`multiple runners found with name ${JSON.stringify(runnerName)}`); + } + if (list.value.length === 0) { + throw new ScaleSetProtocolError('runner response count was 1 but value was empty'); + } + return list.value[0]; + } + + async removeRunner(runnerId: number, options: ScaleSetRequestOptions = {}): Promise { + await this.actionsRequest('DELETE', `/${RUNNER_ENDPOINT}/${runnerId}`, { + expectedStatuses: [204], + signal: options.signal, + }); + } + + async createMessageSessionClient( + runnerScaleSetId: number, + owner: string, + options: ScaleSetRequestOptions = {}, + ): Promise { + return MessageSessionClient.create({ + runnerScaleSetId, + owner, + fetchImplementation: this.fetchImplementation, + userAgent: () => this.currentUserAgent, + actionsRequest: (method, path, requestOptions) => this.actionsRequest(method, path, requestOptions), + signal: options.signal, + }); + } + + /** Alias matching the upstream Go client's factory name. */ + async messageSessionClient( + runnerScaleSetId: number, + owner: string, + options: ScaleSetRequestOptions = {}, + ): Promise { + return this.createMessageSessionClient(runnerScaleSetId, owner, options); + } + + private async actionsRequest( + method: string, + path: string, + options: ActionsRequestOptions, + ): Promise<{ result: HttpResult; url: URL }> { + const adminToken = await this.getAdminToken(options.signal); + const url = actionsServiceUrl(adminToken.url, path, options.query); + const headers: Record = { + 'Content-Type': 'application/json', + Authorization: options.authorization ?? `Bearer ${adminToken.token}`, + 'User-Agent': this.currentUserAgent, + }; + const body = options.body === undefined ? undefined : JSON.stringify(options.body); + const result = await executeRequest( + this.fetchImplementation, + url, + { + method, + headers, + body, + signal: options.signal, + }, + options.expectedStatuses, + ); + return { result, url }; + } + + private async getAdminToken(signal?: AbortSignal): Promise { + if (this.adminTokenIsUsable(this.adminToken)) { + return this.adminToken; + } + + if (this.adminTokenRefresh === undefined) { + this.adminTokenRefresh = this.refreshAdminToken(signal).finally(() => { + this.adminTokenRefresh = undefined; + }); + } + return this.adminTokenRefresh; + } + + private adminTokenIsUsable(token?: ActionsServiceAdminToken): token is ActionsServiceAdminToken { + return token !== undefined && this.now().getTime() + ADMIN_TOKEN_REFRESH_SKEW_MS < token.expiresAt.getTime(); + } + + private async refreshAdminToken(signal?: AbortSignal): Promise { + const registrationToken = await this.getRunnerRegistrationToken(signal); + const adminConnection = await this.getActionsServiceAdminConnection(registrationToken, signal); + const refreshedToken: ActionsServiceAdminToken = { + token: adminConnection.token, + expiresAt: parseJwtExpiration(adminConnection.token), + url: adminConnection.url, + }; + this.adminToken = refreshedToken; + return refreshedToken; + } + + private async getRunnerRegistrationToken(signal?: AbortSignal): Promise { + const accessToken = await this.getAccessToken(); + const url = githubApiUrl(this.config, runnerRegistrationTokenPath(this.config)); + const result = await executeRequest( + this.fetchImplementation, + url, + { + method: 'POST', + headers: { + 'Content-Type': 'application/vnd.github.v3+json', + Authorization: `Bearer ${accessToken}`, + 'User-Agent': this.currentUserAgent, + }, + body: '', + signal, + }, + [201], + ); + const response = parseJsonResponse(result, 'POST', url); + if (!response.token) { + throw new ScaleSetProtocolError('runner registration token response is missing token'); + } + return response.token; + } + + private async getAccessToken(): Promise { + const providedToken = await this.accessTokenProvider(); + const accessToken = typeof providedToken === 'string' ? providedToken : providedToken.token; + if (accessToken === '') throw new ScaleSetProtocolError('access token provider returned an empty token'); + return accessToken; + } + + private async getActionsServiceAdminConnection( + registrationToken: string, + signal?: AbortSignal, + ): Promise<{ url: string; token: string }> { + const url = githubApiUrl(this.config, '/actions/runner-registration'); + const result = await executeRequest( + this.adminConnectionFetchImplementation, + url, + { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + Authorization: `RemoteAuth ${registrationToken}`, + 'User-Agent': this.currentUserAgent, + }, + body: JSON.stringify({ + url: this.config.configUrl.toString(), + runner_event: 'register', + }), + signal, + }, + SUCCESS_STATUSES, + ); + const response = parseJsonResponse(result, 'POST', url); + if (!response.url) { + throw new ScaleSetProtocolError('Actions service admin connection is missing url'); + } + if (!response.token) { + throw new ScaleSetProtocolError('Actions service admin connection is missing token'); + } + let actionsServiceUrl: URL; + try { + actionsServiceUrl = new URL(response.url); + } catch (error) { + throw new ScaleSetProtocolError('Actions service admin connection contains an invalid url', { cause: error }); + } + if ( + actionsServiceUrl.protocol !== 'https:' || + actionsServiceUrl.username || + actionsServiceUrl.password || + actionsServiceUrl.search || + actionsServiceUrl.hash + ) { + throw new ScaleSetProtocolError( + 'Actions service admin connection url must be HTTPS and contain no credentials, query, or fragment', + ); + } + return { url: trimTrailingSlashes(actionsServiceUrl.toString()), token: response.token }; + } +} diff --git a/lambdas/libs/github-actions-scale-set/src/config.test.ts b/lambdas/libs/github-actions-scale-set/src/config.test.ts new file mode 100644 index 0000000000..316fbab015 --- /dev/null +++ b/lambdas/libs/github-actions-scale-set/src/config.test.ts @@ -0,0 +1,43 @@ +import { describe, expect, it } from 'vitest'; + +import { InvalidGitHubConfigUrlError, parseGitHubConfigUrl } from './config'; + +describe('GitHub configuration URL parsing', () => { + it('requires HTTPS for GitHub.com and GHES configuration URLs', () => { + expect(() => parseGitHubConfigUrl('http://github.com/example')).toThrow(InvalidGitHubConfigUrlError); + expect(() => parseGitHubConfigUrl('http://github.example.com/example', true)).toThrow(/should be HTTPS/); + }); + + it('continues to accept an HTTPS GitHub configuration URL', () => { + expect(parseGitHubConfigUrl('https://github.com/example')).toMatchObject({ + scope: 'organization', + organization: 'example', + isHosted: true, + }); + }); + + it('normalizes long slash sequences', () => { + const slashSequence = '/'.repeat(10_000); + + expect(parseGitHubConfigUrl(`https://github.com${slashSequence}example${slashSequence}`)).toMatchObject({ + configUrl: new URL('https://github.com/example'), + scope: 'organization', + organization: 'example', + isHosted: true, + }); + }); + + it.each(['https://github.com////', 'https://github.com/org//repository', 'https://github.com/org/repository/extra'])( + 'continues to reject an invalid path after slash normalization: %s', + (configUrl) => { + expect(() => parseGitHubConfigUrl(configUrl)).toThrow(InvalidGitHubConfigUrlError); + }, + ); + + it.each([ + ['https://github.com/org/repository////', 'repository'], + ['https://github.com/enterprises/example////', 'enterprise'], + ])('retains the scope when normalizing %s', (configUrl, scope) => { + expect(parseGitHubConfigUrl(configUrl)).toMatchObject({ scope }); + }); +}); diff --git a/lambdas/libs/github-actions-scale-set/src/config.ts b/lambdas/libs/github-actions-scale-set/src/config.ts new file mode 100644 index 0000000000..e9b5e1a4a3 --- /dev/null +++ b/lambdas/libs/github-actions-scale-set/src/config.ts @@ -0,0 +1,119 @@ +import { trimSurroundingSlashes, trimTrailingSlashes } from './url'; + +export const GITHUB_SCOPES = { + enterprise: 'enterprise', + organization: 'organization', + repository: 'repository', +} as const; + +export type GitHubScope = (typeof GITHUB_SCOPES)[keyof typeof GITHUB_SCOPES]; + +export interface ParsedGitHubConfig { + configUrl: URL; + scope: GitHubScope; + enterprise?: string; + organization?: string; + repository?: string; + isHosted: boolean; +} + +export class InvalidGitHubConfigUrlError extends Error { + constructor(configUrl: string, options?: ErrorOptions) { + super( + `${JSON.stringify(configUrl)}: invalid config URL, should be HTTPS and point to an enterprise, org, or repository`, + options, + ); + this.name = 'InvalidGitHubConfigUrlError'; + } +} + +function environmentForcesGhes(): boolean { + return ( + typeof process !== 'undefined' && Object.prototype.hasOwnProperty.call(process.env, 'GITHUB_ACTIONS_FORCE_GHES') + ); +} + +function isHostedGitHubUrl(configUrl: URL, forceGhes?: boolean): boolean { + if (forceGhes ?? environmentForcesGhes()) { + return false; + } + + const host = configUrl.host.toLowerCase(); + return host === 'github.com' || host === 'www.github.com' || host === 'github.localhost' || host.endsWith('.ghe.com'); +} + +/** Parse a repository, organization, or enterprise registration URL. */ +export function parseGitHubConfigUrl(configUrl: string, forceGhes?: boolean): ParsedGitHubConfig { + let parsedUrl: URL; + try { + parsedUrl = new URL(trimTrailingSlashes(configUrl.trim())); + } catch (error) { + throw new InvalidGitHubConfigUrlError(configUrl, { cause: error }); + } + + if (parsedUrl.protocol !== 'https:') { + throw new InvalidGitHubConfigUrlError(configUrl); + } + + const pathParts = trimSurroundingSlashes(parsedUrl.pathname).split('/'); + const isHosted = isHostedGitHubUrl(parsedUrl, forceGhes); + + if (pathParts.length === 1 && pathParts[0] !== '') { + parsedUrl.pathname = `/${pathParts[0]}`; + return { + configUrl: parsedUrl, + scope: GITHUB_SCOPES.organization, + organization: pathParts[0], + isHosted, + }; + } + + if (pathParts.length === 2 && pathParts.every((part) => part !== '')) { + parsedUrl.pathname = `/${pathParts.join('/')}`; + if (pathParts[0].toLowerCase() === 'enterprises') { + return { + configUrl: parsedUrl, + scope: GITHUB_SCOPES.enterprise, + enterprise: pathParts[1], + isHosted, + }; + } + + return { + configUrl: parsedUrl, + scope: GITHUB_SCOPES.repository, + organization: pathParts[0], + repository: pathParts[1], + isHosted, + }; + } + + throw new InvalidGitHubConfigUrlError(configUrl); +} + +/** Build a GitHub REST API URL for GitHub.com, ghe.com, or GHES. */ +export function githubApiUrl(config: ParsedGitHubConfig, path: string): URL { + const result = new URL(config.configUrl.origin); + const normalizedPath = path.startsWith('/') ? path : `/${path}`; + + if (config.isHosted) { + result.host = + config.configUrl.host.toLowerCase() === 'www.github.com' ? 'api.github.com' : `api.${config.configUrl.host}`; + result.pathname = normalizedPath; + return result; + } + + result.pathname = `/api/v3${normalizedPath}`; + return result; +} + +export function runnerRegistrationTokenPath(config: ParsedGitHubConfig): string { + switch (config.scope) { + case GITHUB_SCOPES.organization: + return `/orgs/${config.organization}/actions/runners/registration-token`; + case GITHUB_SCOPES.enterprise: + return `/enterprises/${config.enterprise}/actions/runners/registration-token`; + case GITHUB_SCOPES.repository: + return `/repos/${config.organization}/${config.repository}/actions/runners/registration-token`; + } +} diff --git a/lambdas/libs/github-actions-scale-set/src/endpoints.ts b/lambdas/libs/github-actions-scale-set/src/endpoints.ts new file mode 100644 index 0000000000..d409f73704 --- /dev/null +++ b/lambdas/libs/github-actions-scale-set/src/endpoints.ts @@ -0,0 +1,4 @@ +export const RUNNER_ENDPOINT = '_apis/distributedtask/pools/0/agents'; +export const SCALE_SET_ENDPOINT = '_apis/runtime/runnerscalesets'; +export const RUNNER_GROUP_ENDPOINT = '_apis/runtime/runnergroups/'; +export const ACTIONS_API_VERSION = '6.0-preview'; diff --git a/lambdas/libs/github-actions-scale-set/src/errors.ts b/lambdas/libs/github-actions-scale-set/src/errors.ts new file mode 100644 index 0000000000..860e4aaa7d --- /dev/null +++ b/lambdas/libs/github-actions-scale-set/src/errors.ts @@ -0,0 +1,171 @@ +export const SCALE_SET_ERROR_CODES = { + badRequest: 'BAD_REQUEST', + conflict: 'CONFLICT', + jobStillRunning: 'JOB_STILL_RUNNING', + messageQueueTokenExpired: 'MESSAGE_QUEUE_TOKEN_EXPIRED', + notFound: 'NOT_FOUND', + runnerExists: 'RUNNER_EXISTS', + runnerNotFound: 'RUNNER_NOT_FOUND', + unauthorized: 'UNAUTHORIZED', + unexpectedStatus: 'UNEXPECTED_STATUS', +} as const; + +export type ScaleSetErrorCode = (typeof SCALE_SET_ERROR_CODES)[keyof typeof SCALE_SET_ERROR_CODES]; + +export interface ScaleSetHttpErrorDetails { + method: string; + url: string; + status: number; + statusText: string; + headers: Headers; + responseBody: string; + code?: ScaleSetErrorCode; + cause?: unknown; +} + +interface ActionsException { + typeName?: unknown; + message?: unknown; +} + +export function redactUrlForError(value: string): string { + try { + const url = new URL(value); + url.username = ''; + url.password = ''; + url.search = ''; + url.hash = ''; + return url.toString(); + } catch { + return ''; + } +} + +function statusErrorCode(status: number): ScaleSetErrorCode { + switch (status) { + case 400: + return SCALE_SET_ERROR_CODES.badRequest; + case 401: + return SCALE_SET_ERROR_CODES.unauthorized; + case 404: + return SCALE_SET_ERROR_CODES.notFound; + case 409: + return SCALE_SET_ERROR_CODES.conflict; + default: + return SCALE_SET_ERROR_CODES.unexpectedStatus; + } +} + +function exceptionErrorCode(typeName?: string): ScaleSetErrorCode | undefined { + if (typeName?.includes('AgentExistsException')) { + return SCALE_SET_ERROR_CODES.runnerExists; + } + if (typeName?.includes('AgentNotFoundException')) { + return SCALE_SET_ERROR_CODES.runnerNotFound; + } + if (typeName?.includes('JobStillRunningException')) { + return SCALE_SET_ERROR_CODES.jobStillRunning; + } + return undefined; +} + +function parseActionsException(responseBody: string): { typeName?: string; message?: string } { + if (responseBody === '') { + return {}; + } + + try { + const parsed = JSON.parse(responseBody) as ActionsException; + return { + typeName: typeof parsed.typeName === 'string' ? parsed.typeName : undefined, + message: typeof parsed.message === 'string' ? parsed.message : undefined, + }; + } catch { + return {}; + } +} + +/** An unsuccessful HTTP response from either GitHub or the Actions service. */ +export class ScaleSetHttpError extends Error { + readonly code: ScaleSetErrorCode; + readonly status: number; + readonly statusText: string; + readonly method: string; + readonly url: string; + readonly activityId?: string; + readonly githubRequestId?: string; + readonly exceptionName?: string; + readonly responseBody: string; + + constructor(details: ScaleSetHttpErrorDetails) { + const safeUrl = redactUrlForError(details.url); + const exception = parseActionsException(details.responseBody); + const activityId = details.headers.get('ActivityId') ?? undefined; + const githubRequestId = details.headers.get('X-GitHub-Request-Id') ?? undefined; + const responseDescription = [details.status, details.statusText].filter(Boolean).join(' '); + const metadata = [ + `status=${JSON.stringify(responseDescription)}`, + activityId ? `activity_id=${JSON.stringify(activityId)}` : undefined, + githubRequestId ? `github_request_id=${JSON.stringify(githubRequestId)}` : undefined, + ] + .filter((part): part is string => part !== undefined) + .join(', '); + const responseMessage = exception.message ?? (details.responseBody || 'unknown error'); + const exceptionPrefix = exception.typeName ? `${exception.typeName}: ` : ''; + + super(`request ${details.method} ${safeUrl} failed (${metadata}): ${exceptionPrefix}${responseMessage}`, { + cause: details.cause, + }); + this.name = 'ScaleSetHttpError'; + this.code = details.code ?? exceptionErrorCode(exception.typeName) ?? statusErrorCode(details.status); + this.status = details.status; + this.statusText = details.statusText; + this.method = details.method; + this.url = safeUrl; + this.activityId = activityId; + this.githubRequestId = githubRequestId; + this.exceptionName = exception.typeName; + this.responseBody = details.responseBody; + } +} + +export class ScaleSetRequestError extends Error { + readonly method: string; + readonly url: string; + readonly attempts: number; + + constructor(method: string, url: string, cause: unknown, attempts = 1) { + const safeUrl = redactUrlForError(url); + super(`request ${method} ${safeUrl} failed before receiving a response after ${attempts} attempt(s)`, { cause }); + this.name = 'ScaleSetRequestError'; + this.method = method; + this.url = safeUrl; + this.attempts = attempts; + } +} + +export class ScaleSetRequestTimeoutError extends ScaleSetRequestError { + readonly timeoutMs: number; + + constructor(method: string, url: string, timeoutMs: number, attempts: number) { + super(method, url, new Error(`request attempt exceeded ${timeoutMs}ms`), attempts); + this.name = 'ScaleSetRequestTimeoutError'; + this.timeoutMs = timeoutMs; + } +} + +export class ScaleSetProtocolError extends Error { + readonly method?: string; + readonly url?: string; + + constructor(message: string, options: { method?: string; url?: string; cause?: unknown } = {}) { + super(message, { cause: options.cause }); + this.name = 'ScaleSetProtocolError'; + this.method = options.method; + this.url = options.url; + } +} + +export function isScaleSetHttpError(error: unknown): error is ScaleSetHttpError { + return error instanceof ScaleSetHttpError; +} diff --git a/lambdas/libs/github-actions-scale-set/src/http.test.ts b/lambdas/libs/github-actions-scale-set/src/http.test.ts new file mode 100644 index 0000000000..18a36f05ee --- /dev/null +++ b/lambdas/libs/github-actions-scale-set/src/http.test.ts @@ -0,0 +1,299 @@ +import { afterEach, describe, expect, it, vi } from 'vitest'; + +import { ScaleSetRequestError, ScaleSetRequestTimeoutError } from './errors'; +import { + createRetryingFetch, + DEFAULT_SCALE_SET_RETRY_OPTIONS, + executeRequest, + resolveScaleSetRetryOptions, +} from './http'; +import { ScaleSetFetch } from './types'; + +function okResponse(): Response { + return new Response('{"ok":true}', { status: 200 }); +} + +function stalledResponse(onCancel: (reason: unknown) => void): Response { + return new Response( + new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode('{"partial":')); + }, + cancel(reason) { + onCancel(reason); + }, + }), + { status: 200 }, + ); +} + +describe('retrying fetch', () => { + afterEach(() => { + vi.useRealTimers(); + }); + + it('uses bounded defaults close to the upstream client and validates overrides', () => { + expect(resolveScaleSetRetryOptions()).toEqual({ + maxRetries: 4, + initialBackoffMs: 1_000, + maxBackoffMs: 30_000, + requestTimeoutMs: 300_000, + }); + expect(DEFAULT_SCALE_SET_RETRY_OPTIONS).toEqual(resolveScaleSetRetryOptions()); + expect(() => resolveScaleSetRetryOptions({ maxRetries: -1 })).toThrow(/retry\.maxRetries/); + expect(() => resolveScaleSetRetryOptions({ maxRetries: 1.5 })).toThrow(/integer/); + expect(() => resolveScaleSetRetryOptions({ requestTimeoutMs: 0 })).toThrow(/requestTimeoutMs/); + }); + + it('retries a network error after deterministic exponential backoff', async () => { + vi.useFakeTimers(); + const underlyingFetch = vi + .fn() + .mockRejectedValueOnce(new TypeError('socket closed')) + .mockResolvedValueOnce(okResponse()); + const fetchWithRetry = createRetryingFetch(underlyingFetch, { + maxRetries: 1, + initialBackoffMs: 25, + maxBackoffMs: 100, + requestTimeoutMs: 1_000, + }); + + const request = fetchWithRetry('https://actions.example/test'); + await vi.advanceTimersByTimeAsync(24); + expect(underlyingFetch).toHaveBeenCalledOnce(); + await vi.advanceTimersByTimeAsync(1); + + await expect(request).resolves.toMatchObject({ status: 200 }); + expect(underlyingFetch).toHaveBeenCalledTimes(2); + }); + + it('honors Retry-After for 429 responses and caps the wait at maxBackoffMs', async () => { + vi.useFakeTimers(); + const underlyingFetch = vi + .fn() + .mockResolvedValueOnce( + new Response('{"message":"slow down"}', { + status: 429, + headers: { 'Retry-After': '120' }, + }), + ) + .mockResolvedValueOnce(okResponse()); + const fetchWithRetry = createRetryingFetch(underlyingFetch, { + maxRetries: 1, + initialBackoffMs: 10, + maxBackoffMs: 30_000, + requestTimeoutMs: 1_000, + }); + + const request = fetchWithRetry('https://actions.example/test'); + await vi.advanceTimersByTimeAsync(29_999); + expect(underlyingFetch).toHaveBeenCalledOnce(); + await vi.advanceTimersByTimeAsync(1); + + await expect(request).resolves.toMatchObject({ status: 200 }); + expect(underlyingFetch).toHaveBeenCalledTimes(2); + }); + + it('retries 5xx responses only up to maxRetries and returns the final response', async () => { + vi.useFakeTimers(); + const underlyingFetch = vi.fn( + async () => + new Response('{"message":"unavailable"}', { + status: 503, + headers: { 'Retry-After': 'invalid' }, + }), + ); + const fetchWithRetry = createRetryingFetch(underlyingFetch, { + maxRetries: 2, + initialBackoffMs: 10, + maxBackoffMs: 15, + requestTimeoutMs: 1_000, + }); + + const request = fetchWithRetry('https://actions.example/test'); + await vi.runAllTimersAsync(); + + await expect(request).resolves.toMatchObject({ status: 503 }); + expect(underlyingFetch).toHaveBeenCalledTimes(3); + }); + + it('does not retry a queue 401 so message-session refresh remains the owner', async () => { + const underlyingFetch = vi.fn(async () => new Response(null, { status: 401 })); + const fetchWithRetry = createRetryingFetch(underlyingFetch, { + maxRetries: 4, + initialBackoffMs: 1, + maxBackoffMs: 1, + requestTimeoutMs: 100, + }); + + await expect(fetchWithRetry('https://queue.example/messages')).resolves.toMatchObject({ status: 401 }); + expect(underlyingFetch).toHaveBeenCalledOnce(); + }); + + it('does not replay a POST after a retryable HTTP response', async () => { + const underlyingFetch = vi.fn(async () => new Response(null, { status: 503 })); + const fetchWithRetry = createRetryingFetch(underlyingFetch, { + maxRetries: 4, + initialBackoffMs: 1, + maxBackoffMs: 1, + requestTimeoutMs: 100, + }); + + await expect( + fetchWithRetry('https://actions.example/generatejitconfig', { method: 'POST' }), + ).resolves.toMatchObject({ status: 503 }); + expect(underlyingFetch).toHaveBeenCalledOnce(); + }); + + it('does not replay a POST carried by a Request after a network failure', async () => { + const underlyingFetch = vi.fn(async () => { + throw new TypeError('network unavailable'); + }); + const fetchWithRetry = createRetryingFetch(underlyingFetch, { + maxRetries: 4, + initialBackoffMs: 1, + maxBackoffMs: 1, + requestTimeoutMs: 100, + }); + + await expect( + fetchWithRetry(new Request('https://actions.example/sessions', { method: 'POST' })), + ).rejects.toMatchObject({ + name: 'ScaleSetRequestError', + method: 'POST', + attempts: 1, + } satisfies Partial); + expect(underlyingFetch).toHaveBeenCalledOnce(); + }); + + it('times out each attempt and stops after the configured retry bound', async () => { + vi.useFakeTimers(); + const underlyingFetch = vi.fn(() => new Promise(() => undefined)); + const fetchWithRetry = createRetryingFetch(underlyingFetch, { + maxRetries: 1, + initialBackoffMs: 10, + maxBackoffMs: 10, + requestTimeoutMs: 50, + }); + + const request = fetchWithRetry('https://actions.example/hangs'); + const rejection = expect(request).rejects.toMatchObject({ + name: 'ScaleSetRequestTimeoutError', + attempts: 2, + timeoutMs: 50, + } satisfies Partial); + await vi.advanceTimersByTimeAsync(50); + expect(underlyingFetch).toHaveBeenCalledOnce(); + await vi.advanceTimersByTimeAsync(10); + expect(underlyingFetch).toHaveBeenCalledTimes(2); + await vi.advanceTimersByTimeAsync(50); + + await rejection; + }); + + it('cancels a stalled response body when the attempt timeout expires', async () => { + vi.useFakeTimers(); + const cancelled = vi.fn(); + const underlyingFetch = vi.fn().mockResolvedValue(stalledResponse(cancelled)); + const fetchWithRetry = createRetryingFetch(underlyingFetch, { + maxRetries: 0, + requestTimeoutMs: 50, + }); + + const response = await fetchWithRetry('https://actions.example/stalled'); + const reader = response.body!.getReader(); + await expect(reader.read()).resolves.toMatchObject({ done: false }); + const pendingRead = reader.read(); + const rejection = expect(pendingRead).rejects.toMatchObject({ + name: 'ScaleSetRequestTimeoutError', + timeoutMs: 50, + }); + + await vi.advanceTimersByTimeAsync(50); + + await rejection; + expect(cancelled).toHaveBeenCalledWith(expect.objectContaining({ name: 'ScaleSetRequestTimeoutError' })); + }); + + it('forwards caller shutdown cancellation while reading a response body', async () => { + const cancelled = vi.fn(); + const controller = new AbortController(); + const shutdownReason = new Error('shutdown'); + const underlyingFetch = vi.fn().mockResolvedValue(stalledResponse(cancelled)); + const fetchWithRetry = createRetryingFetch(underlyingFetch, { + maxRetries: 0, + requestTimeoutMs: 60_000, + }); + + const response = await fetchWithRetry('https://actions.example/stalled', { signal: controller.signal }); + const reader = response.body!.getReader(); + await expect(reader.read()).resolves.toMatchObject({ done: false }); + const pendingRead = reader.read(); + const rejection = expect(pendingRead).rejects.toBe(shutdownReason); + + controller.abort(shutdownReason); + + await rejection; + expect(cancelled).toHaveBeenCalledWith(shutdownReason); + expect(underlyingFetch.mock.calls[0][1]?.signal?.aborted).toBe(true); + }); + + it('interrupts retry backoff immediately when the caller aborts', async () => { + vi.useFakeTimers(); + const underlyingFetch = vi.fn(async () => new Response(null, { status: 503 })); + const fetchWithRetry = createRetryingFetch(underlyingFetch, { + maxRetries: 4, + initialBackoffMs: 30_000, + maxBackoffMs: 30_000, + requestTimeoutMs: 60_000, + }); + const controller = new AbortController(); + const request = fetchWithRetry('https://actions.example/test', { signal: controller.signal }); + await vi.advanceTimersByTimeAsync(0); + + controller.abort(new Error('caller cancelled')); + + await expect(request).rejects.toThrow('caller cancelled'); + expect(underlyingFetch).toHaveBeenCalledOnce(); + }); + + it('wraps an exhausted idempotent network failure with the final attempt count', async () => { + vi.useFakeTimers(); + const underlyingFetch = vi.fn(async () => { + throw new TypeError('network unavailable'); + }); + const fetchWithRetry = createRetryingFetch(underlyingFetch, { + maxRetries: 1, + initialBackoffMs: 1, + maxBackoffMs: 1, + requestTimeoutMs: 100, + }); + + const request = fetchWithRetry('https://actions.example/test', { method: 'GET' }); + const rejection = expect(request).rejects.toMatchObject({ + name: 'ScaleSetRequestError', + method: 'GET', + attempts: 2, + } satisfies Partial); + await vi.runAllTimersAsync(); + + await rejection; + }); + + it('redacts signed query strings from request errors', async () => { + const fetchImplementation = vi.fn(async () => { + throw new TypeError('network unavailable'); + }); + const request = executeRequest( + fetchImplementation, + 'https://queue.example/messages?signature=do-not-log&token=also-secret', + { method: 'GET' }, + [200], + ); + + await expect(request).rejects.toMatchObject({ + url: 'https://queue.example/messages', + message: expect.not.stringContaining('do-not-log'), + } satisfies Partial); + }); +}); diff --git a/lambdas/libs/github-actions-scale-set/src/http.ts b/lambdas/libs/github-actions-scale-set/src/http.ts new file mode 100644 index 0000000000..aecd95c649 --- /dev/null +++ b/lambdas/libs/github-actions-scale-set/src/http.ts @@ -0,0 +1,409 @@ +import { + ScaleSetErrorCode, + ScaleSetHttpError, + ScaleSetProtocolError, + ScaleSetRequestError, + ScaleSetRequestTimeoutError, + redactUrlForError, +} from './errors'; +import { ScaleSetFetch, ScaleSetRetryOptions } from './types'; + +export interface ResolvedScaleSetRetryOptions { + maxRetries: number; + initialBackoffMs: number; + maxBackoffMs: number; + requestTimeoutMs: number; +} + +export const DEFAULT_SCALE_SET_RETRY_OPTIONS: Readonly = Object.freeze({ + maxRetries: 4, + initialBackoffMs: 1_000, + maxBackoffMs: 30_000, + requestTimeoutMs: 5 * 60_000, +}); + +export interface HttpResult { + response: Response; + body: string; +} + +interface RetryingFetchPolicy { + additionalRetryStatuses?: readonly number[]; + /** + * Escape hatch for a known-safe, operation-scoped wrapper. Non-idempotent + * methods are never retried by the default transport policy. + */ + additionalRetryMethods?: readonly string[]; +} + +const MAX_RESPONSE_BODY_BYTES = 1024 * 1024; +const IDEMPOTENT_RETRY_METHODS = new Set(['GET', 'HEAD', 'OPTIONS', 'PUT', 'DELETE']); + +function trimByteOrderMark(body: string): string { + return body.startsWith('\uFEFF') ? body.slice(1) : body; +} + +function boundedNumber(name: string, value: number, minimum: number, integer: boolean): number { + if (!Number.isFinite(value) || value < minimum || (integer && !Number.isInteger(value))) { + throw new TypeError(`${name} must be ${integer ? 'an integer' : 'a number'} greater than or equal to ${minimum}`); + } + return value; +} + +export function resolveScaleSetRetryOptions(options: ScaleSetRetryOptions = {}): ResolvedScaleSetRetryOptions { + return { + maxRetries: boundedNumber( + 'retry.maxRetries', + options.maxRetries ?? DEFAULT_SCALE_SET_RETRY_OPTIONS.maxRetries, + 0, + true, + ), + initialBackoffMs: boundedNumber( + 'retry.initialBackoffMs', + options.initialBackoffMs ?? DEFAULT_SCALE_SET_RETRY_OPTIONS.initialBackoffMs, + 0, + false, + ), + maxBackoffMs: boundedNumber( + 'retry.maxBackoffMs', + options.maxBackoffMs ?? DEFAULT_SCALE_SET_RETRY_OPTIONS.maxBackoffMs, + 0, + false, + ), + requestTimeoutMs: boundedNumber( + 'retry.requestTimeoutMs', + options.requestTimeoutMs ?? DEFAULT_SCALE_SET_RETRY_OPTIONS.requestTimeoutMs, + 1, + false, + ), + }; +} + +function abortReason(signal: AbortSignal): unknown { + return signal.reason ?? new DOMException('The operation was aborted', 'AbortError'); +} + +function waitForRetry(delayMs: number, signal?: AbortSignal): Promise { + if (signal?.aborted) { + return Promise.reject(abortReason(signal)); + } + if (delayMs === 0) { + return Promise.resolve(); + } + + return new Promise((resolve, reject) => { + const timeout = setTimeout(() => { + signal?.removeEventListener('abort', onAbort); + resolve(); + }, delayMs); + const onAbort = () => { + clearTimeout(timeout); + reject(abortReason(signal as AbortSignal)); + }; + signal?.addEventListener('abort', onAbort, { once: true }); + }); +} + +function retryableStatus(status: number, additionalRetryStatuses: ReadonlySet): boolean { + return status === 429 || (status >= 500 && status <= 599) || additionalRetryStatuses.has(status); +} + +function requestMethod(input: RequestInput, init: RequestInit): string { + return (init.method ?? (input instanceof Request ? input.method : 'GET')).toUpperCase(); +} + +function exponentialBackoffMs(retryIndex: number, options: ResolvedScaleSetRetryOptions): number { + return Math.min(options.initialBackoffMs * 2 ** retryIndex, options.maxBackoffMs); +} + +function retryAfterMs(response: Response, options: ResolvedScaleSetRetryOptions): number | undefined { + const value = response.headers.get('Retry-After')?.trim(); + if (!value) { + return undefined; + } + + let delayMs: number; + if (/^\d+$/.test(value)) { + delayMs = Number(value) * 1_000; + } else { + const retryAt = Date.parse(value); + if (!Number.isFinite(retryAt)) { + return undefined; + } + delayMs = Math.max(0, retryAt - Date.now()); + } + return Math.min(delayMs, options.maxBackoffMs); +} + +function wrapResponseBody(response: Response, signal: AbortSignal, cleanup: () => void): Response { + const source = response.body; + if (source === null) { + cleanup(); + return response; + } + + let reader!: ReadableStreamDefaultReader; + let stopped = false; + let cleaned = false; + let removeAbortListener: () => void = () => undefined; + const release = () => { + if (cleaned) return; + cleaned = true; + removeAbortListener(); + cleanup(); + }; + + const body = new ReadableStream({ + start(controller) { + reader = source.getReader(); + const onAbort = () => { + if (stopped) return; + stopped = true; + const reason = abortReason(signal); + controller.error(reason); + void reader + .cancel(reason) + .catch(() => undefined) + .finally(() => { + release(); + }); + }; + signal.addEventListener('abort', onAbort, { once: true }); + removeAbortListener = () => signal.removeEventListener('abort', onAbort); + if (signal.aborted) onAbort(); + }, + async pull(controller) { + if (stopped) return; + try { + const { done, value } = await reader.read(); + if (done) { + stopped = true; + controller.close(); + release(); + return; + } + controller.enqueue(value); + } catch (error) { + if (!stopped) { + stopped = true; + controller.error(error); + } + release(); + } + }, + cancel(reason) { + stopped = true; + return reader.cancel(reason).finally(release); + }, + }); + + return new Response(body, { + status: response.status, + statusText: response.statusText, + headers: response.headers, + }); +} + +async function fetchAttempt( + fetchImplementation: ScaleSetFetch, + input: RequestInput, + init: RequestInit, + options: ResolvedScaleSetRetryOptions, + attempt: number, +): Promise { + const method = requestMethod(input, init); + const url = input instanceof Request ? input.url : input.toString(); + const callerSignal = init.signal ?? undefined; + if (callerSignal?.aborted) { + throw abortReason(callerSignal); + } + + const attemptController = new AbortController(); + const forwardAbort = () => attemptController.abort(abortReason(callerSignal as AbortSignal)); + callerSignal?.addEventListener('abort', forwardAbort, { once: true }); + const timeoutError = new ScaleSetRequestTimeoutError(method, url, options.requestTimeoutMs, attempt); + const timeout = setTimeout(() => attemptController.abort(timeoutError), options.requestTimeoutMs); + + let onAttemptAbort: (() => void) | undefined; + const aborted = new Promise((_, reject) => { + onAttemptAbort = () => reject(abortReason(attemptController.signal)); + attemptController.signal.addEventListener('abort', onAttemptAbort, { once: true }); + }); + + const cleanup = () => { + clearTimeout(timeout); + callerSignal?.removeEventListener('abort', forwardAbort); + if (onAttemptAbort !== undefined) { + attemptController.signal.removeEventListener('abort', onAttemptAbort); + } + }; + + try { + const response = await Promise.race([ + fetchImplementation(input, { ...init, signal: attemptController.signal }), + aborted, + ]); + void aborted.catch(() => undefined); + return wrapResponseBody(response, attemptController.signal, cleanup); + } catch (error) { + cleanup(); + throw error; + } +} + +type RequestInput = Parameters[0]; + +/** Wrap native fetch with the bounded retry and timeout policy used by every SDK request. */ +export function createRetryingFetch( + fetchImplementation: ScaleSetFetch, + retryOptions: ScaleSetRetryOptions = {}, + policy: RetryingFetchPolicy = {}, +): ScaleSetFetch { + const options = resolveScaleSetRetryOptions(retryOptions); + const additionalRetryStatusSet = new Set(policy.additionalRetryStatuses ?? []); + const additionalRetryMethodSet = new Set((policy.additionalRetryMethods ?? []).map((method) => method.toUpperCase())); + + return async (input, init = {}) => { + const method = requestMethod(input, init); + const url = input instanceof Request ? input.url : input.toString(); + const maxRetries = + IDEMPOTENT_RETRY_METHODS.has(method) || additionalRetryMethodSet.has(method) ? options.maxRetries : 0; + + for (let retryIndex = 0; retryIndex <= maxRetries; retryIndex += 1) { + const attempt = retryIndex + 1; + try { + const response = await fetchAttempt(fetchImplementation, input, init, options, attempt); + if (!retryableStatus(response.status, additionalRetryStatusSet) || retryIndex === maxRetries) { + return response; + } + + const delayMs = retryAfterMs(response, options) ?? exponentialBackoffMs(retryIndex, options); + await response.body?.cancel().catch(() => undefined); + await waitForRetry(delayMs, init.signal ?? undefined); + } catch (error) { + if (init.signal?.aborted) { + throw abortReason(init.signal); + } + if (retryIndex === maxRetries) { + if (error instanceof ScaleSetRequestTimeoutError) { + throw error; + } + throw new ScaleSetRequestError(method, url, error, attempt); + } + await waitForRetry(exponentialBackoffMs(retryIndex, options), init.signal ?? undefined); + } + } + + throw new ScaleSetRequestError(method, url, new Error('retry loop exhausted'), maxRetries + 1); + }; +} + +export async function executeRequest( + fetchImplementation: ScaleSetFetch, + url: string | URL, + init: RequestInit, + expectedStatuses: readonly number[], + errorCode?: ScaleSetErrorCode | ((response: Response) => ScaleSetErrorCode | undefined), +): Promise { + const method = init.method ?? 'GET'; + const urlString = url.toString(); + const displayUrl = redactUrlForError(urlString); + let response: Response; + + try { + response = await fetchImplementation(url, { ...init, redirect: 'error' }); + } catch (error) { + if (init.signal?.aborted || (error instanceof Error && error.name === 'AbortError')) { + throw error; + } + if (error instanceof ScaleSetRequestError) { + throw error; + } + throw new ScaleSetRequestError(method, urlString, error); + } + + let body: string; + try { + const contentLength = response.headers.get('content-length'); + if (contentLength !== null && /^\d+$/.test(contentLength) && Number(contentLength) > MAX_RESPONSE_BODY_BYTES) { + await response.body?.cancel().catch(() => undefined); + throw new ScaleSetProtocolError( + `response body from ${method} ${displayUrl} exceeds ${MAX_RESPONSE_BODY_BYTES} bytes`, + { + method, + url: displayUrl, + }, + ); + } + if (response.body === null) { + body = ''; + } else { + const reader = response.body.getReader(); + const chunks: Uint8Array[] = []; + let total = 0; + while (true) { + const { done, value } = await reader.read(); + if (done) break; + total += value.byteLength; + if (total > MAX_RESPONSE_BODY_BYTES) { + await reader.cancel().catch(() => undefined); + throw new ScaleSetProtocolError( + `response body from ${method} ${displayUrl} exceeds ${MAX_RESPONSE_BODY_BYTES} bytes`, + { method, url: displayUrl }, + ); + } + chunks.push(value); + } + const bytes = new Uint8Array(total); + let offset = 0; + for (const chunk of chunks) { + bytes.set(chunk, offset); + offset += chunk.byteLength; + } + body = trimByteOrderMark(new TextDecoder('utf-8', { fatal: true }).decode(bytes)); + } + } catch (error) { + if (error instanceof ScaleSetProtocolError) throw error; + if (init.signal?.aborted) throw abortReason(init.signal); + if (error instanceof ScaleSetRequestTimeoutError) throw error; + throw new ScaleSetProtocolError(`failed to read the response body from ${method} ${displayUrl}`, { + method, + url: displayUrl, + cause: error, + }); + } + + if (!expectedStatuses.includes(response.status)) { + throw new ScaleSetHttpError({ + method, + url: displayUrl, + status: response.status, + statusText: response.statusText, + headers: response.headers, + responseBody: body, + code: typeof errorCode === 'function' ? errorCode(response) : errorCode, + }); + } + + return { response, body }; +} + +export function parseJsonResponse(result: HttpResult, method: string, url: string | URL): T { + const urlString = redactUrlForError(url.toString()); + if (result.body === '') { + throw new ScaleSetProtocolError(`empty JSON response from ${method} ${urlString}`, { + method, + url: urlString, + }); + } + + try { + return JSON.parse(result.body) as T; + } catch (error) { + throw new ScaleSetProtocolError(`invalid JSON response from ${method} ${urlString}`, { + method, + url: urlString, + cause: error, + }); + } +} diff --git a/lambdas/libs/github-actions-scale-set/src/index.ts b/lambdas/libs/github-actions-scale-set/src/index.ts new file mode 100644 index 0000000000..ab0aad76db --- /dev/null +++ b/lambdas/libs/github-actions-scale-set/src/index.ts @@ -0,0 +1,29 @@ +export { + actionsServiceUrl, + GitHubActionsScaleSetClient as Client, + GitHubActionsScaleSetClient, + GitHubActionsScaleSetClient as ScaleSetClient, +} from './client'; +export { ACTIONS_API_VERSION, RUNNER_ENDPOINT, RUNNER_GROUP_ENDPOINT, SCALE_SET_ENDPOINT } from './endpoints'; +export { + GITHUB_SCOPES, + githubApiUrl, + InvalidGitHubConfigUrlError, + parseGitHubConfigUrl, + runnerRegistrationTokenPath, +} from './config'; +export type { GitHubScope, ParsedGitHubConfig } from './config'; +export { + isScaleSetHttpError, + redactUrlForError, + SCALE_SET_ERROR_CODES, + ScaleSetHttpError, + ScaleSetProtocolError, + ScaleSetRequestError, + ScaleSetRequestTimeoutError, +} from './errors'; +export type { ScaleSetErrorCode, ScaleSetHttpErrorDetails } from './errors'; +export { DEFAULT_SCALE_SET_RETRY_OPTIONS } from './http'; +export type { ResolvedScaleSetRetryOptions } from './http'; +export { HEADER_SCALE_SET_MAX_CAPACITY, MessageSessionClient } from './message-session-client'; +export * from './types'; diff --git a/lambdas/libs/github-actions-scale-set/src/message-session-client.test.ts b/lambdas/libs/github-actions-scale-set/src/message-session-client.test.ts new file mode 100644 index 0000000000..894193e067 --- /dev/null +++ b/lambdas/libs/github-actions-scale-set/src/message-session-client.test.ts @@ -0,0 +1,319 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest'; + +import { GitHubActionsScaleSetClient } from './client'; +import { ScaleSetProtocolError } from './errors'; +import { HEADER_SCALE_SET_MAX_CAPACITY } from './message-session-client'; +import { RunnerScaleSetStatistic, ScaleSetFetch } from './types'; + +type RequestInput = Parameters[0]; +type RequestHandler = (url: URL, init: RequestInit) => Response | Promise; + +const statistics: RunnerScaleSetStatistic = { + totalAvailableJobs: 2, + totalAcquiredJobs: 1, + totalAssignedJobs: 1, + totalRunningJobs: 1, + totalRegisteredRunners: 2, + totalBusyRunners: 1, + totalIdleRunners: 1, +}; + +function requestUrl(input: RequestInput): URL { + if (input instanceof Request) { + return new URL(input.url); + } + return new URL(input.toString()); +} + +function jsonResponse(value: unknown, status = 200): Response { + return new Response(JSON.stringify(value), { + status, + headers: { 'Content-Type': 'application/json' }, + }); +} + +function actionsAdminToken(): string { + const payload = Buffer.from(JSON.stringify({ exp: Math.floor(Date.now() / 1000) + 60 * 60 })).toString('base64url'); + return `header.${payload}.signature`; +} + +function sessionFixture(handler: RequestHandler) { + const requests: Array<{ url: URL; init: RequestInit }> = []; + const fetchImplementation = vi.fn(async (input, init = {}) => { + const url = requestUrl(input); + if (url.pathname === '/orgs/example/actions/runners/registration-token') { + return jsonResponse({ token: 'runner-registration-token' }, 201); + } + if (url.pathname === '/actions/runner-registration') { + return jsonResponse({ + url: 'https://actions.example/tenant/123/', + token: actionsAdminToken(), + }); + } + + requests.push({ url, init }); + return handler(url, init); + }); + const client = new GitHubActionsScaleSetClient({ + gitHubConfigUrl: 'https://github.com/example', + personalAccessToken: 'github-token', + fetch: fetchImplementation, + systemInfo: { system: 'unit-test', subsystem: 'listener' }, + }); + + return { client, fetchImplementation, requests }; +} + +function sessionResponse(token = 'queue-token') { + return { + sessionId: '11111111-1111-1111-1111-111111111111', + ownerName: 'listener-1', + runnerScaleSet: { id: 42, name: 'linux' }, + messageQueueUrl: 'https://queue.example/messages?existing=1', + messageQueueAccessToken: token, + statistics, + }; +} + +describe('MessageSessionClient', () => { + beforeEach(() => { + vi.restoreAllMocks(); + }); + + it('normalizes the capitalized RunnerSetting returned when creating a session', async () => { + const fixture = sessionFixture((url, init) => { + if (url.pathname.endsWith('/runnerscalesets/42/sessions') && init.method === 'POST') { + return jsonResponse({ + ...sessionResponse(), + runnerScaleSet: { id: 42, name: 'linux', RunnerSetting: { disableUpdate: true } }, + }); + } + return new Response(null, { status: 500 }); + }); + + const session = await fixture.client.createMessageSessionClient(42, 'listener-1'); + + expect(session.session.runnerScaleSet?.runnerSetting).toEqual({ disableUpdate: true }); + expect(session.session.runnerScaleSet).not.toHaveProperty('RunnerSetting'); + }); + + it('maps a 202 poll to null and sends the queue capacity contract', async () => { + const fixture = sessionFixture((url, init) => { + if (url.pathname.endsWith('/runnerscalesets/42/sessions') && init.method === 'POST') { + return jsonResponse(sessionResponse()); + } + if (url.hostname === 'queue.example' && init.method === 'GET') { + return new Response(null, { status: 202 }); + } + return new Response(null, { status: 500 }); + }); + const session = await fixture.client.createMessageSessionClient(42, 'listener-1'); + + await expect(session.getMessage(0, 17)).resolves.toBeNull(); + + expect(session.session.statistics).toEqual(statistics); + const queueRequest = fixture.requests.find(({ url }) => url.hostname === 'queue.example'); + expect(queueRequest).toBeDefined(); + expect(queueRequest?.url.toString()).toBe('https://queue.example/messages?existing=1'); + const headers = new Headers(queueRequest?.init.headers); + expect(headers.get('Accept')).toBe('application/json; api-version=6.0-preview'); + expect(headers.get('Authorization')).toBe('Bearer queue-token'); + expect(headers.get(HEADER_SCALE_SET_MAX_CAPACITY)).toBe('17'); + expect(headers.get('User-Agent')).toContain('"kind":"scaleset"'); + }); + + it('decodes known batched messages, ignores unknown types, acknowledges, and acquires jobs', async () => { + const fixture = sessionFixture((url, init) => { + if (url.pathname.endsWith('/runnerscalesets/42/sessions') && init.method === 'POST') { + return jsonResponse(sessionResponse()); + } + if (url.hostname === 'queue.example' && init.method === 'GET') { + return jsonResponse({ + messageId: 19, + messageType: 'RunnerScaleSetJobMessages', + statistics, + body: JSON.stringify([ + { + messageType: 'JobAvailable', + runnerRequestId: 501, + acquireJobUrl: 'https://actions.example/acquire/501', + }, + { + messageType: 'FutureMessageType', + runnerRequestId: 999, + }, + { + messageType: 'JobAssigned', + runnerRequestId: 0, + }, + { + messageType: 'JobStarted', + runnerId: 72, + runnerName: 'runner-72', + }, + { + messageType: 'JobCompleted', + runnerRequestId: 0, + runnerId: 0, + runnerName: 'runner-0', + result: 'Canceled', + }, + { + messageType: 'JobCompleted', + runnerRequestId: 0, + runnerId: 0, + runnerName: '', + result: 'Canceled', + }, + { + messageType: 'JobCompleted', + runnerRequestId: 500, + runnerId: 71, + runnerName: 'runner-71', + result: 'Succeeded', + }, + ]), + }); + } + if (url.hostname === 'queue.example' && init.method === 'DELETE') { + return new Response(null, { status: 204 }); + } + if (url.pathname.endsWith('/runnerscalesets/42/acquirejobs') && init.method === 'POST') { + return jsonResponse({ count: 1, value: [501] }); + } + return new Response(null, { status: 500 }); + }); + const session = await fixture.client.createMessageSessionClient(42, 'listener-1'); + + const message = await session.getMessage(18, 4); + expect(message).toMatchObject({ messageId: 19, statistics }); + expect(message?.jobAvailableMessages).toEqual([ + expect.objectContaining({ messageType: 'JobAvailable', runnerRequestId: 501 }), + ]); + expect(message?.jobCompletedMessages).toEqual([ + expect.objectContaining({ messageType: 'JobCompleted', runnerId: 0, runnerName: 'runner-0' }), + expect.objectContaining({ messageType: 'JobCompleted', runnerId: 0, runnerName: '' }), + expect.objectContaining({ messageType: 'JobCompleted', runnerName: 'runner-71' }), + ]); + expect(message?.jobAssignedMessages).toEqual([ + expect.objectContaining({ messageType: 'JobAssigned', runnerRequestId: 0 }), + ]); + expect(message?.jobStartedMessages).toEqual([ + expect.objectContaining({ messageType: 'JobStarted', runnerId: 72, runnerName: 'runner-72' }), + ]); + + await expect(session.deleteMessage(19)).resolves.toBeUndefined(); + await expect(session.acquireJobs([501, 999])).resolves.toEqual([501]); + + const pollRequest = fixture.requests.find( + ({ url, init }) => url.hostname === 'queue.example' && init.method === 'GET', + ); + expect(pollRequest?.url.searchParams.get('lastMessageId')).toBe('18'); + + const ackRequest = fixture.requests.find( + ({ url, init }) => url.hostname === 'queue.example' && init.method === 'DELETE', + ); + expect(ackRequest?.url.pathname).toBe('/messages/19'); + expect(ackRequest?.url.searchParams.get('existing')).toBe('1'); + expect(new Headers(ackRequest?.init.headers).get('Authorization')).toBe('Bearer queue-token'); + + const acquireRequest = fixture.requests.find(({ url }) => url.pathname.endsWith('/acquirejobs')); + expect(acquireRequest?.url.pathname).toBe('/tenant/123/_apis/runtime/runnerscalesets/42/acquirejobs'); + expect(acquireRequest?.url.searchParams.get('api-version')).toBe('6.0-preview'); + expect(new Headers(acquireRequest?.init.headers).get('Authorization')).toBe('Bearer queue-token'); + expect(JSON.parse(acquireRequest?.init.body as string)).toEqual([501, 999]); + }); + + it.each([ + ['message id', { messageId: 0, messageType: 'RunnerScaleSetJobMessages', statistics, body: '[]' }], + [ + 'statistics', + { + messageId: 1, + messageType: 'RunnerScaleSetJobMessages', + statistics: { ...statistics, totalAssignedJobs: -1 }, + body: '[]', + }, + ], + [ + 'runner request id', + { + messageId: 1, + messageType: 'RunnerScaleSetJobMessages', + statistics, + body: JSON.stringify([{ messageType: 'JobAvailable', runnerRequestId: 0 }]), + }, + ], + [ + 'negative lifecycle request id', + { + messageId: 1, + messageType: 'RunnerScaleSetJobMessages', + statistics, + body: JSON.stringify([{ messageType: 'JobAssigned', runnerRequestId: -1 }]), + }, + ], + [ + 'runner identity', + { + messageId: 1, + messageType: 'RunnerScaleSetJobMessages', + statistics, + body: JSON.stringify([ + { messageType: 'JobCompleted', runnerRequestId: 1, runnerId: 2, runnerName: 'bad\nname' }, + ]), + }, + ], + ])('rejects malformed known message %s fields', async (_name, payload) => { + const fixture = sessionFixture((url, init) => { + if (url.pathname.endsWith('/runnerscalesets/42/sessions') && init.method === 'POST') { + return jsonResponse(sessionResponse()); + } + if (url.hostname === 'queue.example' && init.method === 'GET') return jsonResponse(payload); + return new Response(null, { status: 500 }); + }); + const session = await fixture.client.createMessageSessionClient(42, 'listener-1'); + + await expect(session.getMessage(0, 10)).rejects.toBeInstanceOf(ScaleSetProtocolError); + }); + + it('refreshes the message session once on a queue 401 and retries with the new token', async () => { + let refreshCount = 0; + let oldTokenPolls = 0; + let newTokenPolls = 0; + const fixture = sessionFixture((url, init) => { + if (url.pathname.endsWith('/runnerscalesets/42/sessions') && init.method === 'POST') { + return jsonResponse(sessionResponse('old-queue-token')); + } + if (url.pathname.includes('/runnerscalesets/42/sessions/') && init.method === 'PATCH') { + refreshCount += 1; + return jsonResponse({ + ...sessionResponse('new-queue-token'), + runnerScaleSet: { id: 42, name: 'linux', RunnerSetting: { disableUpdate: false } }, + }); + } + if (url.hostname === 'queue.example' && init.method === 'GET') { + const authorization = new Headers(init.headers).get('Authorization'); + if (authorization === 'Bearer old-queue-token') { + oldTokenPolls += 1; + return jsonResponse({ message: 'expired' }, 401); + } + if (authorization === 'Bearer new-queue-token') { + newTokenPolls += 1; + return new Response(null, { status: 202 }); + } + } + return new Response(null, { status: 500 }); + }); + const session = await fixture.client.createMessageSessionClient(42, 'listener-1'); + + await expect(session.getMessage(0, 10)).resolves.toBeNull(); + + expect(oldTokenPolls).toBe(1); + expect(newTokenPolls).toBe(1); + expect(refreshCount).toBe(1); + expect(session.session.messageQueueAccessToken).toBe('new-queue-token'); + expect(session.session.runnerScaleSet?.runnerSetting).toEqual({ disableUpdate: false }); + expect(session.session.runnerScaleSet).not.toHaveProperty('RunnerSetting'); + }); +}); diff --git a/lambdas/libs/github-actions-scale-set/src/message-session-client.ts b/lambdas/libs/github-actions-scale-set/src/message-session-client.ts new file mode 100644 index 0000000000..7ed05fea96 --- /dev/null +++ b/lambdas/libs/github-actions-scale-set/src/message-session-client.ts @@ -0,0 +1,488 @@ +import { SCALE_SET_ERROR_CODES, ScaleSetHttpError, ScaleSetProtocolError } from './errors'; +import { executeRequest, HttpResult, parseJsonResponse } from './http'; +import { SCALE_SET_ENDPOINT } from './endpoints'; +import { + JobAssigned, + JobAvailable, + JobCompleted, + JobStarted, + MESSAGE_TYPES, + RunnerScaleSetMessage, + RunnerScaleSet, + RunnerScaleSetSession, + RunnerScaleSetStatistic, + ScaleSetFetch, + ScaleSetRequestOptions, +} from './types'; + +export const HEADER_SCALE_SET_MAX_CAPACITY = 'X-ScaleSetMaxCapacity'; + +interface ActionsRequestOptions extends ScaleSetRequestOptions { + body?: unknown; + expectedStatuses: readonly number[]; + authorization?: string; +} + +interface MessageSessionClientCreateOptions extends ScaleSetRequestOptions { + runnerScaleSetId: number; + owner: string; + fetchImplementation: ScaleSetFetch; + userAgent: () => string; + actionsRequest: ( + method: string, + path: string, + options: ActionsRequestOptions, + ) => Promise<{ result: HttpResult; url: URL }>; +} + +interface RunnerScaleSetMessageResponse { + messageId: number; + messageType: string; + body?: string; + statistics?: RunnerScaleSetStatistic | null; +} + +interface AcquireJobsResponse { + count: number; + value: number[]; +} + +interface JobMessageType { + messageType?: unknown; +} + +const STATISTIC_FIELDS = [ + 'totalAvailableJobs', + 'totalAcquiredJobs', + 'totalAssignedJobs', + 'totalRunningJobs', + 'totalRegisteredRunners', + 'totalBusyRunners', + 'totalIdleRunners', +] as const satisfies readonly (keyof RunnerScaleSetStatistic)[]; + +function positiveInteger(value: unknown, field: string): number { + if (!Number.isSafeInteger(value) || (value as number) <= 0) { + throw new ScaleSetProtocolError(`${field} must be a positive integer`); + } + return value as number; +} + +function optionalNonNegativeInteger(value: unknown, field: string): number | undefined { + if (value === undefined) return undefined; + if (!Number.isSafeInteger(value) || (value as number) < 0) { + throw new ScaleSetProtocolError(`${field} must be a non-negative integer when present`); + } + return value as number; +} + +function validateStatistics(value: unknown): RunnerScaleSetStatistic | null { + if (value === null || value === undefined) return null; + if (typeof value !== 'object' || Array.isArray(value)) { + throw new ScaleSetProtocolError('runner scale set statistics must be an object'); + } + for (const field of STATISTIC_FIELDS) { + const statistic = (value as Record)[field]; + if (!Number.isSafeInteger(statistic) || (statistic as number) < 0) { + throw new ScaleSetProtocolError(`statistics.${field} must be a non-negative integer`); + } + } + return value as RunnerScaleSetStatistic; +} + +function validateKnownJobMessage(rawMessage: Record, messageType: string): void { + if (messageType === MESSAGE_TYPES.jobAvailable) { + // JobAvailable is the only message type whose request ID is submitted to + // acquirejobs, so it must identify a real acquisition request. + positiveInteger(rawMessage.runnerRequestId, `${messageType}.runnerRequestId`); + } else { + // GitHub may omit this correlation ID, or send zero, on lifecycle + // notifications. Those messages are still useful for runner state and do + // not participate in job acquisition. + optionalNonNegativeInteger(rawMessage.runnerRequestId, `${messageType}.runnerRequestId`); + } + if (messageType === MESSAGE_TYPES.jobStarted || messageType === MESSAGE_TYPES.jobCompleted) { + // Runner identity is only used for lifecycle correlation. GitHub can + // omit it or send zero in a lifecycle notification; the reconciler will + // safely ignore that observation when it cannot identify a runner. + optionalNonNegativeInteger(rawMessage.runnerId, `${messageType}.runnerId`); + if (rawMessage.runnerName !== undefined) { + if ( + typeof rawMessage.runnerName !== 'string' || + rawMessage.runnerName.length > 256 || + hasAsciiControlCharacter(rawMessage.runnerName) + ) { + throw new ScaleSetProtocolError(`${messageType}.runnerName is invalid`); + } + } + } +} + +function hasAsciiControlCharacter(value: string): boolean { + return [...value].some((character) => { + const code = character.charCodeAt(0); + return code <= 31 || code === 127; + }); +} + +function normalizeRunnerScaleSet(scaleSet: RunnerScaleSet | null | undefined): RunnerScaleSet | null | undefined { + if (scaleSet === null || scaleSet === undefined) { + return scaleSet; + } + + const wire = scaleSet as RunnerScaleSet & { RunnerSetting?: RunnerScaleSet['runnerSetting'] }; + if (wire.runnerSetting === undefined && wire.RunnerSetting !== undefined) { + wire.runnerSetting = wire.RunnerSetting; + } + delete wire.RunnerSetting; + return wire; +} + +function normalizeSession(session: RunnerScaleSetSession): RunnerScaleSetSession { + session.runnerScaleSet = normalizeRunnerScaleSet(session.runnerScaleSet); + return session; +} + +function cloneSession(session: RunnerScaleSetSession): RunnerScaleSetSession { + return { + ...session, + runnerScaleSet: + session.runnerScaleSet === undefined || session.runnerScaleSet === null + ? session.runnerScaleSet + : { + ...session.runnerScaleSet, + labels: session.runnerScaleSet.labels?.map((label) => ({ ...label })), + runnerSetting: + session.runnerScaleSet.runnerSetting === undefined + ? undefined + : { ...session.runnerScaleSet.runnerSetting }, + statistics: + session.runnerScaleSet.statistics === undefined || session.runnerScaleSet.statistics === null + ? session.runnerScaleSet.statistics + : { ...session.runnerScaleSet.statistics }, + }, + statistics: + session.statistics === undefined || session.statistics === null ? session.statistics : { ...session.statistics }, + }; +} + +function validateSession(session: RunnerScaleSetSession): RunnerScaleSetSession { + if (!session.sessionId) { + throw new ScaleSetProtocolError('message session response is missing sessionId'); + } + if (!session.messageQueueUrl) { + throw new ScaleSetProtocolError('message session response is missing messageQueueUrl'); + } + if (!session.messageQueueAccessToken) { + throw new ScaleSetProtocolError('message session response is missing messageQueueAccessToken'); + } + let queueUrl: URL; + try { + queueUrl = new URL(session.messageQueueUrl); + } catch (error) { + throw new ScaleSetProtocolError('message session response contains an invalid messageQueueUrl', { cause: error }); + } + if (queueUrl.protocol !== 'https:' || queueUrl.username || queueUrl.password || queueUrl.hash) { + throw new ScaleSetProtocolError( + 'message session messageQueueUrl must be HTTPS and contain no credentials or fragment', + ); + } + return session; +} + +function parseRunnerScaleSetMessage(result: HttpResult, url: URL): RunnerScaleSetMessage { + const response = parseJsonResponse(result, 'GET', url); + positiveInteger(response.messageId, 'messageId'); + if (response.messageType !== 'RunnerScaleSetJobMessages') { + throw new ScaleSetProtocolError(`unsupported message type: ${response.messageType}`); + } + + let batchedMessages: unknown[] = []; + if (response.body) { + try { + const parsed = JSON.parse(response.body) as unknown; + if (!Array.isArray(parsed)) { + throw new TypeError('message body is not an array'); + } + batchedMessages = parsed; + if (batchedMessages.length > 50) { + throw new TypeError('message body contains more than 50 entries'); + } + } catch (error) { + throw new ScaleSetProtocolError('failed to unmarshal batched runner scale set messages', { + cause: error, + }); + } + } + + const message: RunnerScaleSetMessage = { + messageId: response.messageId, + statistics: validateStatistics(response.statistics), + jobAvailableMessages: [], + jobAssignedMessages: [], + jobStartedMessages: [], + jobCompletedMessages: [], + }; + + for (const rawMessage of batchedMessages) { + if (typeof rawMessage !== 'object' || rawMessage === null) { + throw new ScaleSetProtocolError('runner scale set job message is not an object'); + } + const messageType = (rawMessage as JobMessageType).messageType; + if (typeof messageType !== 'string') { + throw new ScaleSetProtocolError('runner scale set job message is missing messageType'); + } + switch (messageType) { + case MESSAGE_TYPES.jobAvailable: + validateKnownJobMessage(rawMessage as Record, messageType); + message.jobAvailableMessages.push(rawMessage as JobAvailable); + break; + case MESSAGE_TYPES.jobAssigned: + validateKnownJobMessage(rawMessage as Record, messageType); + message.jobAssignedMessages.push(rawMessage as JobAssigned); + break; + case MESSAGE_TYPES.jobStarted: + validateKnownJobMessage(rawMessage as Record, messageType); + message.jobStartedMessages.push(rawMessage as JobStarted); + break; + case MESSAGE_TYPES.jobCompleted: + validateKnownJobMessage(rawMessage as Record, messageType); + message.jobCompletedMessages.push(rawMessage as JobCompleted); + break; + default: + // The upstream client ignores unknown job message types for forward compatibility. + break; + } + } + + return message; +} + +/** A message queue session scoped to one runner scale set. */ +export class MessageSessionClient { + private readonly runnerScaleSetId: number; + private readonly fetchImplementation: ScaleSetFetch; + private readonly userAgent: () => string; + private readonly actionsRequest: MessageSessionClientCreateOptions['actionsRequest']; + private currentSession: RunnerScaleSetSession; + private sessionRefresh?: Promise; + + private constructor(options: MessageSessionClientCreateOptions, session: RunnerScaleSetSession) { + this.runnerScaleSetId = options.runnerScaleSetId; + this.fetchImplementation = options.fetchImplementation; + this.userAgent = options.userAgent; + this.actionsRequest = options.actionsRequest; + this.currentSession = session; + } + + static async create(options: MessageSessionClientCreateOptions): Promise { + const path = `/${SCALE_SET_ENDPOINT}/${options.runnerScaleSetId}/sessions`; + const { result, url } = await options.actionsRequest('POST', path, { + body: { ownerName: options.owner }, + expectedStatuses: [200], + signal: options.signal, + }); + const session = validateSession(normalizeSession(parseJsonResponse(result, 'POST', url))); + return new MessageSessionClient(options, session); + } + + /** A defensive snapshot of the current session and its latest statistics. */ + get session(): RunnerScaleSetSession { + return cloneSession(this.currentSession); + } + + getSession(): RunnerScaleSetSession { + return this.session; + } + + async close(options: ScaleSetRequestOptions = {}): Promise { + const path = `/${SCALE_SET_ENDPOINT}/${this.runnerScaleSetId}/sessions/${this.currentSession.sessionId}`; + await this.actionsRequest('DELETE', path, { + expectedStatuses: [204], + signal: options.signal, + }); + } + + /** + * Long-poll for a batched scale set message. A 202 response means no message + * is currently available and is represented as `null`. + */ + async getMessage( + lastMessageId: number, + maxCapacity: number, + options: ScaleSetRequestOptions = {}, + ): Promise { + return this.withMessageTokenRefresh( + (session) => this.getMessageWithSession(session, lastMessageId, maxCapacity, options), + options, + ); + } + + async pollMessage( + lastMessageId: number, + maxCapacity: number, + options: ScaleSetRequestOptions = {}, + ): Promise { + return this.getMessage(lastMessageId, maxCapacity, options); + } + + /** Delete a queue message after processing it, which acknowledges the batch. */ + async deleteMessage(messageId: number, options: ScaleSetRequestOptions = {}): Promise { + await this.withMessageTokenRefresh( + (session) => this.deleteMessageWithSession(session, messageId, options), + options, + ); + } + + async acknowledgeMessage(messageId: number, options: ScaleSetRequestOptions = {}): Promise { + return this.deleteMessage(messageId, options); + } + + /** Return the authoritative subset of runner request IDs acquired by the service. */ + async acquireJobs(requestIds: number[], options: ScaleSetRequestOptions = {}): Promise { + return this.withMessageTokenRefresh( + (session) => this.acquireJobsWithSession(session, requestIds, options), + options, + ); + } + + private async getMessageWithSession( + session: RunnerScaleSetSession, + lastMessageId: number, + maxCapacity: number, + options: ScaleSetRequestOptions, + ): Promise { + const url = new URL(session.messageQueueUrl); + if (lastMessageId > 0) { + url.searchParams.set('lastMessageId', String(lastMessageId)); + } + const result = await executeRequest( + this.fetchImplementation, + url, + { + method: 'GET', + headers: { + Accept: 'application/json; api-version=6.0-preview', + Authorization: `Bearer ${session.messageQueueAccessToken}`, + 'User-Agent': this.userAgent(), + [HEADER_SCALE_SET_MAX_CAPACITY]: String(maxCapacity), + }, + signal: options.signal, + }, + [200, 202], + (response) => (response.status === 401 ? SCALE_SET_ERROR_CODES.messageQueueTokenExpired : undefined), + ); + + if (result.response.status === 202) { + return null; + } + return parseRunnerScaleSetMessage(result, url); + } + + private async deleteMessageWithSession( + session: RunnerScaleSetSession, + messageId: number, + options: ScaleSetRequestOptions, + ): Promise { + const url = new URL(session.messageQueueUrl); + const valueAfterOrigin = session.messageQueueUrl.slice(url.origin.length); + const originalPath = valueAfterOrigin.startsWith('/') ? url.pathname : ''; + url.pathname = `${originalPath}/${messageId}`; + await executeRequest( + this.fetchImplementation, + url, + { + method: 'DELETE', + headers: { + 'Content-Type': 'application/json', + Authorization: `Bearer ${session.messageQueueAccessToken}`, + 'User-Agent': this.userAgent(), + }, + signal: options.signal, + }, + [204], + (response) => (response.status === 401 ? SCALE_SET_ERROR_CODES.messageQueueTokenExpired : undefined), + ); + } + + private async acquireJobsWithSession( + session: RunnerScaleSetSession, + requestIds: number[], + options: ScaleSetRequestOptions, + ): Promise { + const path = `/${SCALE_SET_ENDPOINT}/${this.runnerScaleSetId}/acquirejobs`; + try { + const { result, url } = await this.actionsRequest('POST', path, { + body: requestIds, + expectedStatuses: [200], + authorization: `Bearer ${session.messageQueueAccessToken}`, + signal: options.signal, + }); + return parseJsonResponse(result, 'POST', url).value; + } catch (error) { + if (error instanceof ScaleSetHttpError && error.status === 401) { + throw new ScaleSetHttpError({ + method: error.method, + url: error.url, + status: error.status, + statusText: error.statusText, + headers: new Headers({ + ...(error.activityId ? { ActivityId: error.activityId } : {}), + ...(error.githubRequestId ? { 'X-GitHub-Request-Id': error.githubRequestId } : {}), + }), + responseBody: error.responseBody, + code: SCALE_SET_ERROR_CODES.messageQueueTokenExpired, + cause: error, + }); + } + throw error; + } + } + + private async withMessageTokenRefresh( + operation: (session: RunnerScaleSetSession) => Promise, + options: ScaleSetRequestOptions, + ): Promise { + const expiredSession = this.currentSession; + try { + return await operation(expiredSession); + } catch (error) { + if (!(error instanceof ScaleSetHttpError) || error.code !== SCALE_SET_ERROR_CODES.messageQueueTokenExpired) { + throw error; + } + } + + await this.refreshMessageSession(expiredSession, options); + return operation(this.currentSession); + } + + private async refreshMessageSession( + expiredSession: RunnerScaleSetSession, + options: ScaleSetRequestOptions, + ): Promise { + if ( + this.currentSession.sessionId !== expiredSession.sessionId || + this.currentSession.messageQueueAccessToken !== expiredSession.messageQueueAccessToken + ) { + return; + } + + if (this.sessionRefresh === undefined) { + this.sessionRefresh = this.doRefreshMessageSession(options).finally(() => { + this.sessionRefresh = undefined; + }); + } + await this.sessionRefresh; + } + + private async doRefreshMessageSession(options: ScaleSetRequestOptions): Promise { + const path = `/${SCALE_SET_ENDPOINT}/${this.runnerScaleSetId}/sessions/${this.currentSession.sessionId}`; + const { result, url } = await this.actionsRequest('PATCH', path, { + expectedStatuses: [200], + signal: options.signal, + }); + this.currentSession = validateSession( + normalizeSession(parseJsonResponse(result, 'PATCH', url)), + ); + } +} diff --git a/lambdas/libs/github-actions-scale-set/src/types.ts b/lambdas/libs/github-actions-scale-set/src/types.ts new file mode 100644 index 0000000000..725d2b04cf --- /dev/null +++ b/lambdas/libs/github-actions-scale-set/src/types.ts @@ -0,0 +1,184 @@ +export const DEFAULT_RUNNER_GROUP = 'default'; + +export const MESSAGE_TYPES = { + jobAvailable: 'JobAvailable', + jobAssigned: 'JobAssigned', + jobStarted: 'JobStarted', + jobCompleted: 'JobCompleted', +} as const; + +export type MessageType = (typeof MESSAGE_TYPES)[keyof typeof MESSAGE_TYPES]; + +export interface JobMessageBase { + messageType: MessageType; + runnerRequestId?: number; + repositoryName: string; + ownerName: string; + jobId: string; + jobWorkflowRef: string; + jobDisplayName: string; + workflowRunId: number; + eventName: string; + requestLabels: string[]; + queueTime: string; + scaleSetAssignTime: string; + runnerAssignTime: string; + finishTime: string; +} + +export interface JobAvailable extends JobMessageBase { + messageType: typeof MESSAGE_TYPES.jobAvailable; + runnerRequestId: number; + acquireJobUrl: string; +} + +export interface JobAssigned extends JobMessageBase { + messageType: typeof MESSAGE_TYPES.jobAssigned; +} + +export interface JobStarted extends JobMessageBase { + messageType: typeof MESSAGE_TYPES.jobStarted; + runnerId?: number; + runnerName?: string; +} + +export interface JobCompleted extends JobMessageBase { + messageType: typeof MESSAGE_TYPES.jobCompleted; + result: string; + runnerId?: number; + runnerName?: string; +} + +export interface Label { + type?: string; + name: string; +} + +export interface RunnerGroup { + id: number; + name: string; + size: number; + isDefaultGroup: boolean; +} + +export interface RunnerSetting { + disableUpdate?: boolean; +} + +/** + * Runner scale set representation used by the Actions service. + * + * The same shape is accepted for create and update operations, so server-owned + * fields are optional. The client translates `runnerSetting` to the upstream + * wire key `RunnerSetting` when it sends a request. + */ +export interface RunnerScaleSet { + id?: number; + name?: string; + runnerGroupId?: number; + runnerGroupName?: string; + labels?: Label[]; + runnerSetting?: RunnerSetting; + createdOn?: string; + runnerJitConfigUrl?: string; + statistics?: RunnerScaleSetStatistic | null; +} + +export interface RunnerScaleSetJitRunnerSetting { + name: string; + workFolder?: string; +} + +export interface RunnerReference { + id: number; + name: string; + runnerScaleSetId: number; +} + +export interface RunnerScaleSetJitRunnerConfig { + runner: RunnerReference | null; + encodedJITConfig: string; +} + +export interface RunnerScaleSetStatistic { + totalAvailableJobs: number; + totalAcquiredJobs: number; + totalAssignedJobs: number; + totalRunningJobs: number; + totalRegisteredRunners: number; + totalBusyRunners: number; + totalIdleRunners: number; +} + +export interface RunnerScaleSetSession { + sessionId: string; + ownerName: string; + runnerScaleSet?: RunnerScaleSet | null; + messageQueueUrl: string; + messageQueueAccessToken: string; + statistics?: RunnerScaleSetStatistic | null; +} + +export interface RunnerScaleSetMessage { + messageId: number; + statistics: RunnerScaleSetStatistic | null; + jobAvailableMessages: JobAvailable[]; + jobAssignedMessages: JobAssigned[]; + jobStartedMessages: JobStarted[]; + jobCompletedMessages: JobCompleted[]; +} + +export interface SystemInfo { + system?: string; + version?: string; + commitSha?: string; + scaleSetId?: number; + subsystem?: string; +} + +export interface AccessToken { + token: string; + expiresAt?: string | Date; +} + +export type AccessTokenProvider = () => Promise; + +export type ScaleSetFetch = typeof globalThis.fetch; + +export interface ScaleSetRequestOptions { + signal?: AbortSignal; +} + +export interface ScaleSetRetryOptions { + /** Number of retries after the initial request for retry-eligible operations. */ + maxRetries?: number; + /** Initial exponential-backoff delay. */ + initialBackoffMs?: number; + /** Upper bound for exponential backoff and Retry-After delays. */ + maxBackoffMs?: number; + /** Timeout applied independently to each fetch attempt. */ + requestTimeoutMs?: number; +} + +interface ScaleSetClientBaseOptions { + gitHubConfigUrl: string; + systemInfo?: SystemInfo; + fetch?: ScaleSetFetch; + forceGhes?: boolean; + userAgent?: string; + /** Intended for deterministic tests. Defaults to `new Date()`. */ + now?: () => Date; + retry?: ScaleSetRetryOptions; +} + +export type GitHubActionsScaleSetClientOptions = ScaleSetClientBaseOptions & + ( + | { + personalAccessToken: string; + accessTokenProvider?: never; + } + | { + personalAccessToken?: never; + accessTokenProvider: AccessTokenProvider; + } + ); diff --git a/lambdas/libs/github-actions-scale-set/src/url.ts b/lambdas/libs/github-actions-scale-set/src/url.ts new file mode 100644 index 0000000000..55d9ecb468 --- /dev/null +++ b/lambdas/libs/github-actions-scale-set/src/url.ts @@ -0,0 +1,24 @@ +const FORWARD_SLASH = '/'.charCodeAt(0); + +export function trimTrailingSlashes(value: string): string { + let end = value.length; + while (end > 0 && value.charCodeAt(end - 1) === FORWARD_SLASH) { + end -= 1; + } + + return end === value.length ? value : value.slice(0, end); +} + +export function trimSurroundingSlashes(value: string): string { + let start = 0; + while (start < value.length && value.charCodeAt(start) === FORWARD_SLASH) { + start += 1; + } + + let end = value.length; + while (end > start && value.charCodeAt(end - 1) === FORWARD_SLASH) { + end -= 1; + } + + return start === 0 && end === value.length ? value : value.slice(start, end); +} diff --git a/lambdas/libs/github-actions-scale-set/tsconfig.json b/lambdas/libs/github-actions-scale-set/tsconfig.json new file mode 100644 index 0000000000..714aa27b6b --- /dev/null +++ b/lambdas/libs/github-actions-scale-set/tsconfig.json @@ -0,0 +1,8 @@ +{ + "extends": "../../tsconfig.json", + "compilerOptions": { + "types": ["node", "vitest/globals"] + }, + "include": ["src/**/*"], + "exclude": ["src/**/*.test.ts"] +} diff --git a/lambdas/libs/github-actions-scale-set/vitest.config.ts b/lambdas/libs/github-actions-scale-set/vitest.config.ts new file mode 100644 index 0000000000..c52b8d7522 --- /dev/null +++ b/lambdas/libs/github-actions-scale-set/vitest.config.ts @@ -0,0 +1,22 @@ +import { mergeConfig } from 'vitest/config'; + +import defaultConfig from '../../vitest.base.config'; + +export default mergeConfig(defaultConfig, { + root: __dirname, + test: { + coverage: { + include: ['src/**/*.ts'], + exclude: ['src/**/*.test.ts', 'src/**/*.d.ts', 'src/index.ts'], + thresholds: { + // Measured by the package-scoped wire-contract suite. These floors keep + // meaningful regression protection without pretending every defensive + // parser/error branch is exercised by production-path tests. + statements: 75, + branches: 60, + functions: 80, + lines: 75, + }, + }, + }, +}); diff --git a/lambdas/package.json b/lambdas/package.json index b223239520..9f20e09e8f 100644 --- a/lambdas/package.json +++ b/lambdas/package.json @@ -3,7 +3,8 @@ "private": true, "workspaces": [ "functions/*", - "libs/*" + "libs/*", + "services/*" ], "scripts": { "build": "nx run-many --target=build --all", @@ -21,7 +22,8 @@ "@octokit/types": "^13.0.0", "brace-expansion": "^2.0.2", "@babel/helpers": "^7.26.10", - "@babel/runtime": "^7.26.10" + "@babel/runtime": "^7.26.10", + "js-yaml": "^3.15.2" }, "devDependencies": { "@eslint/eslintrc": "^3.3.1", @@ -34,7 +36,7 @@ "@trivago/prettier-plugin-sort-imports": "^6.0.0", "@typescript-eslint/eslint-plugin": "^8.47.0", "@typescript-eslint/parser": "^8.46.2", - "@vitest/coverage-v8": "^4.0.5", + "@vitest/coverage-v8": "^4.1.11", "chalk": "^5.6.2", "eslint": "^9.39.2", "eslint-plugin-prettier": "5.5.4", diff --git a/lambdas/services/scale-set/Dockerfile b/lambdas/services/scale-set/Dockerfile new file mode 100644 index 0000000000..d1bfbe429f --- /dev/null +++ b/lambdas/services/scale-set/Dockerfile @@ -0,0 +1,25 @@ +FROM node:24-bookworm-slim@sha256:3638d9a6fe4030bd716be989438248074489337ba3275657f93595428be4fc03 AS build +WORKDIR /workspace/lambdas + +COPY lambdas/package.json lambdas/yarn.lock lambdas/.yarnrc.yml ./ +COPY lambdas/.yarn ./.yarn +COPY lambdas/tsconfig.json lambdas/vitest.base.config.ts ./ +COPY lambdas/functions ./functions +COPY lambdas/libs ./libs +COPY lambdas/services ./services + +RUN corepack enable && yarn install --immutable +RUN yarn workspace @aws-github-runner/scale-set-service build + +FROM node:24-bookworm-slim@sha256:3638d9a6fe4030bd716be989438248074489337ba3275657f93595428be4fc03 AS runtime +ENV NODE_ENV=production \ + SCALE_SET_HEALTH_PORT=8080 +WORKDIR /app + +COPY --from=build --chown=node:node /workspace/lambdas/services/scale-set/dist/ ./ +COPY --from=build --chown=node:node /workspace/lambdas/services/scale-set/healthcheck.cjs ./healthcheck.cjs + +USER node +EXPOSE 8080 +HEALTHCHECK --interval=30s --timeout=5s --start-period=90s --retries=3 CMD ["node", "/app/healthcheck.cjs"] +ENTRYPOINT ["node", "--disable-proto=delete", "/app/index.js"] diff --git a/lambdas/services/scale-set/Dockerfile.dockerignore b/lambdas/services/scale-set/Dockerfile.dockerignore new file mode 100644 index 0000000000..7c72e0e7ab --- /dev/null +++ b/lambdas/services/scale-set/Dockerfile.dockerignore @@ -0,0 +1,7 @@ +**/coverage +**/dist +**/node_modules +.git +.nx +lambdas/.yarn/install-state.gz +*.log diff --git a/lambdas/services/scale-set/README.md b/lambdas/services/scale-set/README.md new file mode 100644 index 0000000000..5cfe30ef9e --- /dev/null +++ b/lambdas/services/scale-set/README.md @@ -0,0 +1,107 @@ +# Scale-set controller service + +This workspace builds the long-running, topology-neutral controller used by the scale-set orchestration provider. + +Each controller group maps to one ECS service, one task definition, and normally one running task. The task contains one application container and one `ScaleSetController`, which supervises one independent reconciler per runner config: + +```text +ECS service (controller group) +└── one ECS task + └── one scale-set container + └── ScaleSetController + ├── reconciler: runner config A → scale set A → session A + └── reconciler: runner config B → scale set B → session B +``` + +A group is only a packing and deployment boundary. Every reconciler retains its own GitHub message session, lifecycle state, retry loop, health, and compute-provider instance. One reconciler failure does not exit the others. + +## Production configuration + +The ECS task receives only these group selectors: + +- `SCALE_SET_CONTROLLER_GROUP_NAME` +- `SCALE_SET_CONTROLLER_GROUP_CONFIG_PATH` +- `SCALE_SET_CONTROLLER_GROUP_CONFIG_REVISION` + +The service reads every direct child under the SSM path with paginated `GetParametersByPath`. Each child name must equal its `runnerConfigName`, and each value uses this flat, versioned schema: + +```json +{ + "schemaVersion": 1, + "runnerConfigName": "linux-x64", + "githubConfigUrl": "https://github.com/example", + "scaleSetName": "linux-x64", + "runnerGroupName": "self-hosted-linux", + "runnerGroupIdParameterName": "/runners/github-app/runner-group/self-hosted-linux", + "minRunners": 0, + "maxRunners": 20, + "bootTimeoutMinutes": 10, + "sslVerify": true, + "githubApp": { + "appIdParameterName": "/runners/github-app/id", + "privateKeyParameterName": "/runners/github-app/key" + }, + "computeProvider": { + "type": "ec2", + "configuration": { + "region": "eu-west-1", + "environment": "example-linux-x64", + "runnerNamePrefix": "", + "jitConfigParameterPath": "/runners/example-linux-x64/tokens", + "subnets": ["subnet-0123456789abcdef0"], + "launchTemplateName": "example-linux-x64-action-runner", + "ec2instanceCriteria": { + "instanceTypes": ["m7i.large"], + "targetCapacityType": "spot", + "instanceAllocationStrategy": "price-capacity-optimized" + }, + "onDemandFailoverOnError": [], + "useDedicatedHost": false, + "ssmParameterTags": [] + } + } +} +``` + +`scaleSetName` and `runnerGroupName` are the GitHub names supplied by the operator. `runnerGroupIdParameterName` is an optional SSM cache path. When present, the service reads the runner-group ID from that parameter; if it is missing, the service resolves the name through the configured GitHub Actions service endpoint and writes the ID back as a non-secret `String` parameter with overwrite enabled. The service resolves the scale-set ID from the group and scale-set names; if the named scale set does not exist, it registers it in the resolved runner group and uses the ID returned by GitHub. `scaleSetId` is only an optional legacy pin for an already-known ID. `expectedRunnerGroupId` can be omitted or null; when supplied, it is treated as an additional consistency check. Optional fields are `scaleSetId`, `runnerGroupIdParameterName`, `expectedRunnerGroupId`, `sessionOwner`, `workFolder`, `forceGhes`, `sslVerify`, and `userAgent`. `sslVerify` defaults to `true`; when false, the service uses a reconciler-scoped Undici dispatcher for both GitHub App token and scale-set requests without changing `NODE_TLS_REJECT_UNAUTHORIZED` or the global dispatcher. `userAgent` becomes the `system` identity inside the required structured scale-set protocol User-Agent rather than replacing that header. `bootTimeoutMinutes` defaults to `10`; it is orchestration-owned and is passed to the selected compute provider on every reconciliation. + +GitHub App ID and private-key values are reloaded from SSM whenever an installation token is requested, so key rotation does not require a task restart. `installationIdParameterName` is optional; when it is absent or its parameter is not present, the service creates a short-lived App JWT and discovers the installation by matching the configured organization or enterprise account through `GET /app/installations`. This works with GitHub.com, GHES, and GitHub Enterprise Cloud data-residency API hosts derived from `githubConfigUrl`. A SHA-256 credential fingerprint keeps the same Octokit auth instance—and its token cache—while the values remain unchanged, and replaces it after rotation. Private keys, installation tokens, App JWTs, message-session tokens, message bodies, and JIT configurations are never accepted as manifest values and are redacted from logs. + +`SCALE_SET_CONTROLLER_MANIFEST` is supported only as a bounded local/test convenience. It contains `{ "version": 1, "groupName": "...", "reconcilers": [...] }` and uses the same reconciler objects. + +Runtime settings: + +| Environment variable | Default | +| --------------------------------------------- | ------- | +| `SCALE_SET_HEALTH_PORT` | `8080` | +| `SCALE_SET_HEALTH_STALE_AFTER_SECONDS` | `180` | +| `SCALE_SET_SHUTDOWN_TIMEOUT_SECONDS` | `110` | +| `SCALE_SET_SESSION_CLOSE_TIMEOUT_SECONDS` | `10` | +| `SCALE_SET_RECONNECT_INITIAL_BACKOFF_SECONDS` | `1` | +| `SCALE_SET_RECONNECT_MAX_BACKOFF_SECONDS` | `30` | +| `LOG_LEVEL` | `info` | + +## Reconciliation and health + +Demand is calculated as `max(totalAssignedJobs, min(maxRunners, minRunners + totalAssignedJobs))`. The maximum therefore bounds requested idle capacity without ever requesting scale-down below work GitHub has already assigned. Job-started and job-completed messages maintain a bounded in-memory lifecycle cache. After a restart, lifecycle state is unknown, but provider-owned EC2 tags still identify current capacity. The aggregate scale-set busy count protects unknown runners from scale-down until it reaches zero. Runner deletion executes inside the serialized reconcile loop and re-checks the exact Actions-service runner identity by name before removal. + +The public GitHub runner inventory is not fetched by the scale-set service. The selected compute provider reconciles its own capacity inventory, while the scale-set session remains the source of demand and aggregate busy-runner statistics. This keeps the service aligned with the upstream scale-set API and supports GitHub.com, GHES, and data-residency endpoints without relying on a separate public REST runner endpoint. + +Messages follow the upstream scale-set listener order: acknowledge first, then acquire available jobs, update lifecycle state, and reconcile compute. Provider failures are reported with the provider result and then retried through session recreation with bounded backoff; the acknowledged message is recovered from the next session's statistics snapshot. A typed busy/unknown retention remains a successful reconciliation. Session and transport failures are handled separately by bounded client retries or session recreation. + +The EC2 provider uses the tagged, `config-published` EC2 instances as its capacity inventory. The GitHub Actions scale-set session supplies `desiredRunners` through `totalAssignedJobs` and the aggregate `totalBusyRunners` count. When capacity is above desired and the aggregate busy count is zero, tagged runners may be removed through the Actions service and their matching EC2 instances terminated; busy, contradictory, or unknown identities are retained. The provider uses the boot window (`bootTimeoutMinutes`, default `10`) for newly launched instances and keeps interrupted publication states from being counted as serving. EC2 ownership includes a SHA-256 hash of the canonical GitHub configuration scope, preventing the same runner-config name and numeric scale-set ID in another GitHub scope from colliding. No public GitHub REST runner inventory call is required. + +- `GET /healthz` reports controller liveness and is used by Docker/ECS. External GitHub outages remain live but degraded to avoid restart loops. +- `GET /readyz` reports readiness and returns 503 unless every reconciler is ready. + +## Container + +Build from the repository root: + +```shell +docker build --target runtime -f lambdas/services/scale-set/Dockerfile -t scale-set-controller . +``` + +The image supports `linux/amd64` and `linux/arm64`, uses a digest-pinned multi-stage Node image, runs as the unprivileged `node` user, includes a Node-based health check, and does not require filesystem writes. Deploy with a read-only root filesystem, all Linux capabilities dropped, no Docker socket, and only the task-role permissions required by the selected group. + +The module's official GHCR package must allow anonymous pulls so the default image works without registry credentials. Production deployments should select a released image by digest and verify its provenance/attestation. A private ECR override requires `container.ecr_repository.arn`; private non-ECR registry credentials are not currently exposed by the Terraform orchestration module. diff --git a/lambdas/services/scale-set/healthcheck.cjs b/lambdas/services/scale-set/healthcheck.cjs new file mode 100644 index 0000000000..66c833829c --- /dev/null +++ b/lambdas/services/scale-set/healthcheck.cjs @@ -0,0 +1,13 @@ +'use strict'; + +const http = require('node:http'); +const port = Number(process.env.SCALE_SET_HEALTH_PORT || '8080'); +const request = http.get( + { host: '127.0.0.1', port, path: '/healthz', timeout: 4000, headers: { Connection: 'close' } }, + (response) => { + response.resume(); + process.exit(response.statusCode === 200 ? 0 : 1); + }, +); +request.on('timeout', () => request.destroy(new Error('health check timed out'))); +request.on('error', () => process.exit(1)); diff --git a/lambdas/services/scale-set/package.json b/lambdas/services/scale-set/package.json new file mode 100644 index 0000000000..a5a0af9ed9 --- /dev/null +++ b/lambdas/services/scale-set/package.json @@ -0,0 +1,43 @@ +{ + "name": "@aws-github-runner/scale-set-service", + "version": "0.1.0", + "private": true, + "main": "dist/index.js", + "type": "module", + "license": "MIT", + "scripts": { + "build": "ncc build src/main.ts -o dist --minify", + "typecheck": "tsc --noEmit", + "test": "NODE_ENV=test nx test", + "test:watch": "NODE_ENV=test nx test --watch", + "lint": "eslint src", + "format": "prettier --write \"**/*.ts\"", + "format-check": "prettier --check \"**/*.ts\"", + "all": "yarn typecheck && yarn build && yarn format-check && yarn lint && yarn test" + }, + "devDependencies": { + "@types/node": "^22.19.3", + "@vercel/ncc": "0.38.4", + "typescript": "^5.9.3" + }, + "dependencies": { + "@aws-github-runner/aws-ssm-util": "*", + "@aws-github-runner/compute-providers": "*", + "@aws-github-runner/github-actions-scale-set": "*", + "@aws-sdk/client-ssm": "^3.1009.0", + "@aws-sdk/credential-providers": "^3.1009.0", + "@octokit/auth-app": "8.2.0", + "@octokit/request": "^9.2.2", + "undici": "^6.19.2" + }, + "nx": { + "includedScripts": [ + "build", + "typecheck", + "format", + "format-check", + "lint", + "all" + ] + } +} diff --git a/lambdas/services/scale-set/src/config.test.ts b/lambdas/services/scale-set/src/config.test.ts new file mode 100644 index 0000000000..1d7af53d2e --- /dev/null +++ b/lambdas/services/scale-set/src/config.test.ts @@ -0,0 +1,222 @@ +import { + MAX_MANIFEST_BYTES, + parseScaleSetControllerManifest, + parseScaleSetReconcilerConfig, + parseScaleSetServiceConfig, +} from './config'; + +function runnerConfig(overrides: Record = {}) { + return { + schemaVersion: 1, + runnerConfigName: 'linux-x64', + githubConfigUrl: 'https://github.com/example', + scaleSetId: 123, + expectedScaleSetName: 'linux-x64', + expectedRunnerGroupId: null, + minRunners: 0, + maxRunners: 20, + githubApp: { + appIdParameterName: '/runner/app/id', + privateKeyParameterName: '/runner/app/key', + installationIdParameterName: '/runner/app/installation-id', + }, + computeProvider: { type: 'ec2', configuration: { subnetIds: ['subnet-1'] } }, + ...overrides, + }; +} + +describe('scale-set service configuration', () => { + it('parses the production SSM group source and runtime defaults', () => { + expect( + parseScaleSetServiceConfig({ + SCALE_SET_CONTROLLER_GROUP_NAME: 'ec2-default', + SCALE_SET_CONTROLLER_GROUP_CONFIG_PATH: '/runner/groups/ec2-default', + SCALE_SET_CONTROLLER_GROUP_CONFIG_REVISION: '42', + }), + ).toEqual({ + groupName: 'ec2-default', + groupConfigPath: '/runner/groups/ec2-default', + groupRevision: '42', + healthPort: 8080, + healthStaleAfterMs: 180000, + shutdownTimeoutMs: 110000, + sessionCloseTimeoutMs: 10000, + reconnectInitialBackoffMs: 1000, + reconnectMaxBackoffMs: 30000, + }); + }); + + it('supports bounded inline manifests for local use', () => { + const manifest = JSON.stringify({ version: 1, groupName: 'local', reconcilers: [runnerConfig()] }); + expect(parseScaleSetServiceConfig({ SCALE_SET_CONTROLLER_MANIFEST: manifest }).manifest).toBe(manifest); + expect(() => + parseScaleSetServiceConfig({ + SCALE_SET_CONTROLLER_MANIFEST: manifest, + SCALE_SET_CONTROLLER_GROUP_CONFIG_PATH: '/both', + }), + ).toThrow('provide exactly one'); + expect(() => + parseScaleSetServiceConfig({ SCALE_SET_CONTROLLER_MANIFEST: 'x'.repeat(MAX_MANIFEST_BYTES + 1) }), + ).toThrow('must not exceed'); + }); + + it('validates production selectors and numeric runtime settings', () => { + expect(() => parseScaleSetServiceConfig({})).toThrow('provide exactly one'); + expect(() => + parseScaleSetServiceConfig({ + SCALE_SET_CONTROLLER_GROUP_NAME: 'bad name', + SCALE_SET_CONTROLLER_GROUP_CONFIG_PATH: '/valid', + SCALE_SET_CONTROLLER_GROUP_CONFIG_REVISION: '1', + }), + ).toThrow('group name'); + expect(() => + parseScaleSetServiceConfig({ + SCALE_SET_CONTROLLER_GROUP_NAME: 'valid', + SCALE_SET_CONTROLLER_GROUP_CONFIG_PATH: '/valid', + SCALE_SET_CONTROLLER_GROUP_CONFIG_REVISION: '1', + SCALE_SET_RECONNECT_INITIAL_BACKOFF_SECONDS: '31', + SCALE_SET_RECONNECT_MAX_BACKOFF_SECONDS: '30', + }), + ).toThrow('must not exceed'); + }); +}); + +describe('parseScaleSetControllerManifest', () => { + it('parses the frozen flat runner-config schema and defaults', () => { + expect(parseScaleSetReconcilerConfig(runnerConfig(), 0, 'group')).toMatchObject({ + schemaVersion: 1, + runnerConfigName: 'linux-x64', + scaleSetName: 'linux-x64', + runnerLabels: ['linux-x64'], + bootTimeoutMinutes: 10, + sessionOwner: 'group.linux-x64', + workFolder: '_work', + forceGhes: false, + sslVerify: true, + computeProvider: { type: 'ec2', configuration: { subnetIds: ['subnet-1'] } }, + }); + }); + + it('normalizes explicit optional settings', () => { + expect( + parseScaleSetReconcilerConfig( + runnerConfig({ + expectedRunnerGroupId: 7, + sessionOwner: 'owner/group', + workFolder: 'runner/_work', + forceGhes: true, + sslVerify: false, + userAgent: 'github-aws-runners/test', + bootTimeoutMinutes: 30, + }), + 0, + 'group', + ), + ).toMatchObject({ + expectedRunnerGroupId: 7, + sessionOwner: 'owner/group', + workFolder: 'runner/_work', + forceGhes: true, + sslVerify: false, + bootTimeoutMinutes: 30, + }); + }); + + it('accepts a runner-group name for runtime ID resolution', () => { + expect( + parseScaleSetReconcilerConfig(runnerConfig({ runnerGroupName: 'self-hosted-linux' }), 0, 'group'), + ).toMatchObject({ runnerGroupName: 'self-hosted-linux' }); + }); + + it('accepts an optional compute-provider role ARN', () => { + expect( + parseScaleSetReconcilerConfig( + runnerConfig({ + computeProvider: { + type: 'ec2', + roleArn: 'arn:aws:iam::123456789012:role/scale-set-compute', + configuration: { subnetIds: ['subnet-1'] }, + }, + }), + 0, + 'group', + ).computeProvider, + ).toMatchObject({ type: 'ec2', roleArn: 'arn:aws:iam::123456789012:role/scale-set-compute' }); + }); + + it('accepts scaleSetName without a GitHub-generated scale-set ID', () => { + const parsed = parseScaleSetReconcilerConfig( + runnerConfig({ + scaleSetName: 'linux-x64', + expectedScaleSetName: undefined, + runnerGroupName: 'self-hosted-linux', + scaleSetId: undefined, + }), + 0, + 'group', + ); + expect(parsed).toMatchObject({ scaleSetName: 'linux-x64', runnerGroupName: 'self-hosted-linux' }); + expect(parsed).not.toHaveProperty('scaleSetId'); + }); + + it('bounds the derived session owner for maximum-length names', () => { + const parsed = parseScaleSetReconcilerConfig( + runnerConfig({ runnerConfigName: 'r'.repeat(128) }), + 0, + 'g'.repeat(128), + ); + expect(parsed.sessionOwner).toHaveLength(256); + expect(parsed.sessionOwner).toMatch(/\.[a-f0-9]{16}$/); + }); + + it('rejects unsafe URLs, unknown fields, prototype keys, and schema drift', () => { + expect(() => + parseScaleSetReconcilerConfig(runnerConfig({ githubConfigUrl: 'http://github.com/example' }), 0, 'g'), + ).toThrow('must use HTTPS'); + expect(() => parseScaleSetReconcilerConfig(runnerConfig({ extra: true }), 0, 'g')).toThrow('unknown field'); + expect(() => parseScaleSetReconcilerConfig(runnerConfig({ schemaVersion: 2 }), 0, 'g')).toThrow('schemaVersion'); + expect(() => parseScaleSetReconcilerConfig(runnerConfig({ bootTimeoutMinutes: 0 }), 0, 'g')).toThrow( + 'bootTimeoutMinutes', + ); + expect(() => parseScaleSetReconcilerConfig(runnerConfig({ bootTimeoutMinutes: 121 }), 0, 'g')).toThrow( + 'bootTimeoutMinutes', + ); + const polluted = JSON.parse('{"__proto__":{"admin":true}}') as unknown; + expect(() => + parseScaleSetReconcilerConfig( + runnerConfig({ computeProvider: { type: 'ec2', configuration: polluted } }), + 0, + 'g', + ), + ).toThrow('forbidden field'); + }); + + it('rejects duplicate runner names and scale-set IDs within an equivalent GitHub scope', () => { + expect(() => + parseScaleSetControllerManifest({ version: 1, groupName: 'g', reconcilers: [runnerConfig(), runnerConfig()] }), + ).toThrow('duplicated'); + expect(() => + parseScaleSetControllerManifest({ + version: 1, + groupName: 'g', + reconcilers: [ + runnerConfig(), + runnerConfig({ runnerConfigName: 'other', githubConfigUrl: 'https://GITHUB.com/example/' }), + ], + }), + ).toThrow('duplicated'); + }); + + it('allows the same numeric scale-set ID in different GitHub scopes', () => { + expect( + parseScaleSetControllerManifest({ + version: 1, + groupName: 'g', + reconcilers: [ + runnerConfig(), + runnerConfig({ runnerConfigName: 'other', githubConfigUrl: 'https://github.com/another' }), + ], + }).reconcilers, + ).toHaveLength(2); + }); +}); diff --git a/lambdas/services/scale-set/src/config.ts b/lambdas/services/scale-set/src/config.ts new file mode 100644 index 0000000000..5461f4ca84 --- /dev/null +++ b/lambdas/services/scale-set/src/config.ts @@ -0,0 +1,545 @@ +import { createHash } from 'node:crypto'; + +export const SCALE_SET_CONTROLLER_MANIFEST_VERSION = 1; +export const MAX_MANIFEST_BYTES = 256 * 1024; + +export type JsonPrimitive = boolean | number | string | null; +export type JsonValue = JsonPrimitive | JsonValue[] | { readonly [key: string]: JsonValue }; + +export interface GitHubAppParameterReferences { + appIdParameterName: string; + installationIdParameterName?: string; + privateKeyParameterName: string; +} + +export interface ScaleSetReconcilerConfig { + schemaVersion: 1; + runnerConfigName: string; + scaleSetId?: number; + scaleSetName: string; + runnerLabels: readonly string[]; + runnerGroupName?: string; + runnerGroupIdParameterName?: string; + expectedRunnerGroupId?: number; + githubConfigUrl: string; + githubApp: GitHubAppParameterReferences; + computeProvider: { + type: string; + roleArn?: string; + configuration: Readonly>; + }; + minRunners: number; + maxRunners: number; + bootTimeoutMinutes: number; + sessionOwner: string; + workFolder: string; + forceGhes: boolean; + sslVerify: boolean; + userAgent?: string; +} + +export interface ScaleSetControllerManifest { + version: typeof SCALE_SET_CONTROLLER_MANIFEST_VERSION; + groupName: string; + revision?: string; + reconcilers: readonly ScaleSetReconcilerConfig[]; +} + +export interface ScaleSetServiceConfig { + manifest?: string; + groupConfigPath?: string; + groupName?: string; + groupRevision?: string; + healthPort: number; + healthStaleAfterMs: number; + shutdownTimeoutMs: number; + sessionCloseTimeoutMs: number; + reconnectInitialBackoffMs: number; + reconnectMaxBackoffMs: number; +} + +export type ScaleSetServiceEnvironment = Readonly>; + +const MAX_SCALE_SET_CAPACITY = 2_147_483_647; +const DEFAULT_BOOT_TIMEOUT_MINUTES = 10; +const MAX_BOOT_TIMEOUT_MINUTES = 120; +const DEFAULT_HEALTH_PORT = 8080; +const DEFAULT_HEALTH_STALE_AFTER_SECONDS = 180; +const DEFAULT_SHUTDOWN_TIMEOUT_SECONDS = 110; +const DEFAULT_SESSION_CLOSE_TIMEOUT_SECONDS = 10; +const DEFAULT_RECONNECT_INITIAL_BACKOFF_SECONDS = 1; +const DEFAULT_RECONNECT_MAX_BACKOFF_SECONDS = 30; +const MAX_RECONCILERS = 1000; +const MAX_PROVIDER_CONFIG_NODES = 10_000; +const MAX_PROVIDER_CONFIG_DEPTH = 32; +const SAFE_NAME = /^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$/; +const SAFE_PROVIDER_TYPE = /^[a-z][a-z0-9_-]{0,63}$/; +const SAFE_SSM_PARAMETER = /^\/[A-Za-z0-9_.\-/]{1,2047}$/; +const PROTOTYPE_KEYS = new Set(['__proto__', 'constructor', 'prototype']); + +export class ScaleSetConfigurationError extends Error { + constructor(message: string, options?: ErrorOptions) { + super(message, options); + this.name = 'ScaleSetConfigurationError'; + } +} + +function parseInteger( + environment: ScaleSetServiceEnvironment, + name: string, + options: { defaultValue: number; minimum: number; maximum: number }, +): number { + const raw = environment[name]?.trim(); + if (!raw) return options.defaultValue; + if (!/^\d+$/.test(raw)) throw new ScaleSetConfigurationError(`${name} must be an integer`); + const value = Number(raw); + if (!Number.isSafeInteger(value) || value < options.minimum || value > options.maximum) { + throw new ScaleSetConfigurationError(`${name} must be between ${options.minimum} and ${options.maximum}`); + } + return value; +} + +export function parseScaleSetServiceConfig(environment: ScaleSetServiceEnvironment): ScaleSetServiceConfig { + const manifest = environment.SCALE_SET_CONTROLLER_MANIFEST?.trim(); + const groupConfigPath = environment.SCALE_SET_CONTROLLER_GROUP_CONFIG_PATH?.trim(); + if ((manifest === undefined || manifest === '') === (groupConfigPath === undefined || groupConfigPath === '')) { + throw new ScaleSetConfigurationError( + 'provide exactly one of SCALE_SET_CONTROLLER_MANIFEST or SCALE_SET_CONTROLLER_GROUP_CONFIG_PATH', + ); + } + if (manifest !== undefined && Buffer.byteLength(manifest, 'utf8') > MAX_MANIFEST_BYTES) { + throw new ScaleSetConfigurationError(`SCALE_SET_CONTROLLER_MANIFEST must not exceed ${MAX_MANIFEST_BYTES} bytes`); + } + let groupName: string | undefined; + let groupRevision: string | undefined; + if (groupConfigPath !== undefined) { + validateSsmParameterName(groupConfigPath, 'group config path'); + groupName = validateSafeName(environment.SCALE_SET_CONTROLLER_GROUP_NAME?.trim() ?? '', 'group name'); + groupRevision = environment.SCALE_SET_CONTROLLER_GROUP_CONFIG_REVISION?.trim(); + if (!groupRevision || !/^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$/.test(groupRevision)) { + throw new ScaleSetConfigurationError('SCALE_SET_CONTROLLER_GROUP_CONFIG_REVISION is invalid'); + } + } + + const reconnectInitialBackoffMs = + parseInteger(environment, 'SCALE_SET_RECONNECT_INITIAL_BACKOFF_SECONDS', { + defaultValue: DEFAULT_RECONNECT_INITIAL_BACKOFF_SECONDS, + minimum: 1, + maximum: 300, + }) * 1000; + const reconnectMaxBackoffMs = + parseInteger(environment, 'SCALE_SET_RECONNECT_MAX_BACKOFF_SECONDS', { + defaultValue: DEFAULT_RECONNECT_MAX_BACKOFF_SECONDS, + minimum: 1, + maximum: 3600, + }) * 1000; + if (reconnectInitialBackoffMs > reconnectMaxBackoffMs) { + throw new ScaleSetConfigurationError( + 'SCALE_SET_RECONNECT_INITIAL_BACKOFF_SECONDS must not exceed SCALE_SET_RECONNECT_MAX_BACKOFF_SECONDS', + ); + } + + return { + ...(manifest ? { manifest } : {}), + ...(groupConfigPath ? { groupConfigPath, groupName, groupRevision } : {}), + healthPort: parseInteger(environment, 'SCALE_SET_HEALTH_PORT', { + defaultValue: DEFAULT_HEALTH_PORT, + minimum: 1, + maximum: 65535, + }), + healthStaleAfterMs: + parseInteger(environment, 'SCALE_SET_HEALTH_STALE_AFTER_SECONDS', { + defaultValue: DEFAULT_HEALTH_STALE_AFTER_SECONDS, + minimum: 30, + maximum: 3600, + }) * 1000, + shutdownTimeoutMs: + parseInteger(environment, 'SCALE_SET_SHUTDOWN_TIMEOUT_SECONDS', { + defaultValue: DEFAULT_SHUTDOWN_TIMEOUT_SECONDS, + minimum: 1, + maximum: 300, + }) * 1000, + sessionCloseTimeoutMs: + parseInteger(environment, 'SCALE_SET_SESSION_CLOSE_TIMEOUT_SECONDS', { + defaultValue: DEFAULT_SESSION_CLOSE_TIMEOUT_SECONDS, + minimum: 1, + maximum: 60, + }) * 1000, + reconnectInitialBackoffMs, + reconnectMaxBackoffMs, + }; +} + +function objectValue(value: unknown, path: string): Record { + if (typeof value !== 'object' || value === null || Array.isArray(value)) { + throw new ScaleSetConfigurationError(`${path} must be an object`); + } + return value as Record; +} + +function exactKeys(value: Record, allowed: readonly string[], path: string): void { + const allowedSet = new Set(allowed); + const unknown = Object.keys(value).filter((key) => !allowedSet.has(key)); + if (unknown.length > 0) + throw new ScaleSetConfigurationError(`${path} contains unknown field ${JSON.stringify(unknown[0])}`); +} + +function requiredString(value: Record, key: string, path: string): string { + const result = value[key]; + if (typeof result !== 'string' || result.trim() === '') { + throw new ScaleSetConfigurationError(`${path}.${key} must be a non-empty string`); + } + return result.trim(); +} + +function optionalString(value: Record, key: string, path: string): string | undefined { + const result = value[key]; + if (result === undefined) return undefined; + if (typeof result !== 'string' || result.trim() === '') { + throw new ScaleSetConfigurationError(`${path}.${key} must be a non-empty string when set`); + } + return result.trim(); +} + +function containsControlCharacter(value: string): boolean { + return [...value].some((character) => { + const code = character.charCodeAt(0); + return code <= 0x1f || code === 0x7f; + }); +} + +function parseRunnerLabels(value: unknown, path: string, fallback: string): readonly string[] { + if (value === undefined) return [fallback]; + if (!Array.isArray(value) || value.length > 100) { + throw new ScaleSetConfigurationError(`${path}.runnerLabels must be an array with at most 100 entries`); + } + const labels = value.map((label, index) => { + if (typeof label !== 'string' || label.length === 0 || label.length > 255 || containsControlCharacter(label)) { + throw new ScaleSetConfigurationError(`${path}.runnerLabels[${index}] is invalid`); + } + return label; + }); + return labels.length === 0 ? [fallback] : [...new Set(labels)]; +} + +function integer(value: Record, key: string, path: string, minimum: number, maximum: number): number { + const result = value[key]; + if (!Number.isSafeInteger(result) || (result as number) < minimum || (result as number) > maximum) { + throw new ScaleSetConfigurationError(`${path}.${key} must be an integer between ${minimum} and ${maximum}`); + } + return result as number; +} + +function optionalBoolean(value: Record, key: string, path: string, fallback: boolean): boolean { + const result = value[key]; + if (result === undefined) return fallback; + if (typeof result !== 'boolean') throw new ScaleSetConfigurationError(`${path}.${key} must be a boolean`); + return result; +} + +function validateSafeName(value: string, path: string): string { + if (!SAFE_NAME.test(value)) { + throw new ScaleSetConfigurationError( + `${path} must start with an ASCII letter or digit and contain only letters, digits, dots, underscores, or hyphens`, + ); + } + return value; +} + +function validateSsmParameterName(value: string, path: string): string { + if (!SAFE_SSM_PARAMETER.test(value) || value.includes('//') || value.endsWith('/')) { + throw new ScaleSetConfigurationError(`${path} must be an absolute SSM parameter name`); + } + return value; +} + +function validateGitHubConfigUrl(raw: string, path: string): string { + let url: URL; + try { + url = new URL(raw); + } catch (error) { + throw new ScaleSetConfigurationError(`${path} must be a valid URL`, { cause: error }); + } + if (url.protocol !== 'https:') throw new ScaleSetConfigurationError(`${path} must use HTTPS`); + if (url.username || url.password || url.search || url.hash) { + throw new ScaleSetConfigurationError(`${path} must not contain credentials, a query, or a fragment`); + } + const parts = url.pathname + .replace(/^\/+|\/+$/g, '') + .split('/') + .filter(Boolean); + if (parts.length < 1 || parts.length > 2 || (parts[0].toLowerCase() === 'enterprises' && parts.length !== 2)) { + throw new ScaleSetConfigurationError(`${path} must identify a GitHub organization, repository, or enterprise`); + } + url.pathname = `/${parts.join('/')}`; + return url.toString().replace(/\/$/, ''); +} + +function validateWorkFolder(value: string, path: string): string { + if ( + value.length > 128 || + value.startsWith('/') || + value.includes('\\') || + value.split('/').some((part) => part === '' || part === '.' || part === '..') || + !/^[A-Za-z0-9._/-]+$/.test(value) + ) { + throw new ScaleSetConfigurationError(`${path} must be a safe relative path`); + } + return value; +} + +function validateUserAgent(value: string | undefined, path: string): string | undefined { + if (value === undefined) return undefined; + if (value.length > 256 || !/^[\x20-\x7E]+$/.test(value)) { + throw new ScaleSetConfigurationError(`${path} must contain at most 256 visible ASCII characters`); + } + return value; +} + +function validateJsonValue(value: unknown, path: string, depth = 0, counter = { value: 0 }): JsonValue { + counter.value += 1; + if (counter.value > MAX_PROVIDER_CONFIG_NODES || depth > MAX_PROVIDER_CONFIG_DEPTH) { + throw new ScaleSetConfigurationError(`${path} exceeds the provider configuration complexity limit`); + } + if (value === null || typeof value === 'boolean' || typeof value === 'string') return value; + if (typeof value === 'number') { + if (!Number.isFinite(value)) throw new ScaleSetConfigurationError(`${path} contains a non-finite number`); + return value; + } + if (Array.isArray(value)) + return value.map((item, index) => validateJsonValue(item, `${path}[${index}]`, depth + 1, counter)); + const record = objectValue(value, path); + const result: Record = Object.create(null) as Record; + for (const [key, child] of Object.entries(record)) { + if (PROTOTYPE_KEYS.has(key)) + throw new ScaleSetConfigurationError(`${path} contains forbidden field ${JSON.stringify(key)}`); + if (key.length === 0 || key.length > 128) + throw new ScaleSetConfigurationError(`${path} contains an invalid field name`); + result[key] = validateJsonValue(child, `${path}.${key}`, depth + 1, counter); + } + return result; +} + +function parseGitHubApp(value: unknown, path: string): GitHubAppParameterReferences { + const record = objectValue(value, path); + exactKeys(record, ['appIdParameterName', 'installationIdParameterName', 'privateKeyParameterName'], path); + const installationIdParameterName = optionalString(record, 'installationIdParameterName', path); + return { + appIdParameterName: validateSsmParameterName( + requiredString(record, 'appIdParameterName', path), + `${path}.appIdParameterName`, + ), + ...(installationIdParameterName === undefined + ? {} + : { + installationIdParameterName: validateSsmParameterName( + installationIdParameterName, + `${path}.installationIdParameterName`, + ), + }), + privateKeyParameterName: validateSsmParameterName( + requiredString(record, 'privateKeyParameterName', path), + `${path}.privateKeyParameterName`, + ), + }; +} + +function parseComputeProvider(value: unknown, path: string): ScaleSetReconcilerConfig['computeProvider'] { + const record = objectValue(value, path); + exactKeys(record, ['type', 'roleArn', 'configuration'], path); + const type = requiredString(record, 'type', path); + if (!SAFE_PROVIDER_TYPE.test(type)) throw new ScaleSetConfigurationError(`${path}.type is invalid`); + const roleArn = record.roleArn === undefined ? undefined : requiredString(record, 'roleArn', path); + if (roleArn !== undefined && !/^arn:[A-Za-z0-9-]+:iam::[0-9]{12}:role\/[A-Za-z0-9+=,.@_/-]{1,512}$/.test(roleArn)) { + throw new ScaleSetConfigurationError(`${path}.roleArn is invalid`); + } + const configuration = validateJsonValue(record.configuration, `${path}.configuration`); + if (typeof configuration !== 'object' || configuration === null || Array.isArray(configuration)) { + throw new ScaleSetConfigurationError(`${path}.configuration must be an object`); + } + return roleArn === undefined ? { type, configuration } : { type, roleArn, configuration }; +} + +export function parseScaleSetReconcilerConfig( + value: unknown, + index: number, + groupName: string, + basePath = 'manifest.reconcilers', +): ScaleSetReconcilerConfig { + const path = `${basePath}[${index}]`; + const record = objectValue(value, path); + exactKeys( + record, + [ + 'schemaVersion', + 'runnerConfigName', + 'scaleSetId', + 'scaleSetName', + 'runnerLabels', + 'expectedScaleSetName', + 'runnerGroupName', + 'runnerGroupIdParameterName', + 'expectedRunnerGroupId', + 'githubConfigUrl', + 'githubApp', + 'computeProvider', + 'minRunners', + 'maxRunners', + 'bootTimeoutMinutes', + 'sessionOwner', + 'workFolder', + 'forceGhes', + 'sslVerify', + 'userAgent', + ], + path, + ); + const runnerConfigName = validateSafeName( + requiredString(record, 'runnerConfigName', path), + `${path}.runnerConfigName`, + ); + const minRunners = integer(record, 'minRunners', path, 0, MAX_SCALE_SET_CAPACITY); + const maxRunners = integer(record, 'maxRunners', path, 0, MAX_SCALE_SET_CAPACITY); + const bootTimeoutMinutes = + record.bootTimeoutMinutes === undefined + ? DEFAULT_BOOT_TIMEOUT_MINUTES + : integer(record, 'bootTimeoutMinutes', path, 1, MAX_BOOT_TIMEOUT_MINUTES); + if (minRunners > maxRunners) throw new ScaleSetConfigurationError(`${path}.minRunners must not exceed maxRunners`); + const sessionOwner = optionalString(record, 'sessionOwner', path) ?? defaultSessionOwner(groupName, runnerConfigName); + if (!/^[A-Za-z0-9][A-Za-z0-9._:/-]{0,255}$/.test(sessionOwner)) { + throw new ScaleSetConfigurationError(`${path}.sessionOwner is invalid`); + } + const userAgent = validateUserAgent(optionalString(record, 'userAgent', path), `${path}.userAgent`); + if (record.schemaVersion !== 1) throw new ScaleSetConfigurationError(`${path}.schemaVersion must be 1`); + const expectedRunnerGroupId = + record.expectedRunnerGroupId === undefined || record.expectedRunnerGroupId === null + ? undefined + : integer(record, 'expectedRunnerGroupId', path, 1, MAX_SCALE_SET_CAPACITY); + const runnerGroupName = + record.runnerGroupName === undefined + ? undefined + : validateScaleSetName(requiredString(record, 'runnerGroupName', path), `${path}.runnerGroupName`); + const runnerGroupIdParameterName = + record.runnerGroupIdParameterName === undefined + ? undefined + : validateSsmParameterName( + requiredString(record, 'runnerGroupIdParameterName', path), + `${path}.runnerGroupIdParameterName`, + ); + if (record.scaleSetName !== undefined && record.expectedScaleSetName !== undefined) { + throw new ScaleSetConfigurationError(`${path} must configure only one of scaleSetName or expectedScaleSetName`); + } + const scaleSetName = validateScaleSetName( + requiredString(record, record.scaleSetName === undefined ? 'expectedScaleSetName' : 'scaleSetName', path), + `${path}.scaleSetName`, + ); + const scaleSetId = + record.scaleSetId === undefined ? undefined : integer(record, 'scaleSetId', path, 1, MAX_SCALE_SET_CAPACITY); + const runnerLabels = parseRunnerLabels(record.runnerLabels, path, scaleSetName); + if (scaleSetId === undefined && runnerGroupName === undefined) { + throw new ScaleSetConfigurationError(`${path}.runnerGroupName is required when scaleSetId is omitted`); + } + return { + schemaVersion: 1, + runnerConfigName, + ...(scaleSetId === undefined ? {} : { scaleSetId }), + scaleSetName, + runnerLabels, + ...(runnerGroupName === undefined ? {} : { runnerGroupName }), + ...(runnerGroupIdParameterName === undefined ? {} : { runnerGroupIdParameterName }), + ...(expectedRunnerGroupId === undefined ? {} : { expectedRunnerGroupId }), + githubConfigUrl: validateGitHubConfigUrl( + requiredString(record, 'githubConfigUrl', path), + `${path}.githubConfigUrl`, + ), + githubApp: parseGitHubApp(record.githubApp, `${path}.githubApp`), + computeProvider: parseComputeProvider(record.computeProvider, `${path}.computeProvider`), + minRunners, + maxRunners, + bootTimeoutMinutes, + sessionOwner, + workFolder: validateWorkFolder(optionalString(record, 'workFolder', path) ?? '_work', `${path}.workFolder`), + forceGhes: optionalBoolean(record, 'forceGhes', path, false), + sslVerify: optionalBoolean(record, 'sslVerify', path, true), + ...(userAgent === undefined ? {} : { userAgent }), + }; +} + +function defaultSessionOwner(groupName: string, runnerConfigName: string): string { + const candidate = `${groupName}.${runnerConfigName}`; + if (candidate.length <= 256) return candidate; + const suffix = createHash('sha256').update(candidate).digest('hex').slice(0, 16); + return `${candidate.slice(0, 239)}.${suffix}`; +} + +function validateScaleSetName(value: string, path: string): string { + if (value.length > 128 || !/^[\x20-\x7E]+$/.test(value)) { + throw new ScaleSetConfigurationError(`${path} must contain at most 128 visible ASCII characters`); + } + return value; +} + +export function parseScaleSetControllerManifest(input: string | unknown): ScaleSetControllerManifest { + let parsed = input; + if (typeof input === 'string') { + if (Buffer.byteLength(input, 'utf8') > MAX_MANIFEST_BYTES) { + throw new ScaleSetConfigurationError(`controller manifest must not exceed ${MAX_MANIFEST_BYTES} bytes`); + } + try { + parsed = JSON.parse(input) as unknown; + } catch (error) { + throw new ScaleSetConfigurationError('controller manifest must contain valid JSON', { cause: error }); + } + } + const manifest = objectValue(parsed, 'manifest'); + exactKeys(manifest, ['version', 'groupName', 'revision', 'reconcilers'], 'manifest'); + if (manifest.version !== SCALE_SET_CONTROLLER_MANIFEST_VERSION) { + throw new ScaleSetConfigurationError(`manifest.version must be ${SCALE_SET_CONTROLLER_MANIFEST_VERSION}`); + } + const groupName = validateSafeName(requiredString(manifest, 'groupName', 'manifest'), 'manifest.groupName'); + if ( + !Array.isArray(manifest.reconcilers) || + manifest.reconcilers.length < 1 || + manifest.reconcilers.length > MAX_RECONCILERS + ) { + throw new ScaleSetConfigurationError(`manifest.reconcilers must contain between 1 and ${MAX_RECONCILERS} entries`); + } + const revision = optionalString(manifest, 'revision', 'manifest'); + if (revision !== undefined && !/^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$/.test(revision)) { + throw new ScaleSetConfigurationError('manifest.revision is invalid'); + } + const reconcilers = manifest.reconcilers.map((value, index) => + parseScaleSetReconcilerConfig(value, index, groupName), + ); + validateUniqueReconcilers(reconcilers); + return { + version: SCALE_SET_CONTROLLER_MANIFEST_VERSION, + groupName, + ...(revision === undefined ? {} : { revision }), + reconcilers, + }; +} + +export function validateUniqueReconcilers(reconcilers: readonly ScaleSetReconcilerConfig[]): void { + const names = new Set(); + const scopedScaleSets = new Set(); + for (const reconciler of reconcilers) { + if (names.has(reconciler.runnerConfigName)) { + throw new ScaleSetConfigurationError( + `runner config ${JSON.stringify(reconciler.runnerConfigName)} is duplicated`, + ); + } + const scopedScaleSet = [ + reconciler.githubConfigUrl, + reconciler.runnerGroupName ?? String(reconciler.expectedRunnerGroupId ?? ''), + reconciler.scaleSetName, + ].join('\u0000'); + if (scopedScaleSets.has(scopedScaleSet)) { + throw new ScaleSetConfigurationError( + `scale set ${JSON.stringify(reconciler.scaleSetName)} is duplicated within GitHub scope ${JSON.stringify(reconciler.githubConfigUrl)}`, + ); + } + names.add(reconciler.runnerConfigName); + scopedScaleSets.add(scopedScaleSet); + } +} diff --git a/lambdas/services/scale-set/src/controller.ts b/lambdas/services/scale-set/src/controller.ts new file mode 100644 index 0000000000..5cbd2ac229 --- /dev/null +++ b/lambdas/services/scale-set/src/controller.ts @@ -0,0 +1,51 @@ +import type { ScaleSetControllerManifest, ScaleSetServiceConfig } from './config'; +import { ScaleSetControllerHealth } from './health'; +import type { ScaleSetLogger } from './logger'; +import { ScaleSetReconciler, type ScaleSetReconcilerDependencies } from './reconciler'; + +export class ScaleSetController { + readonly health: ScaleSetControllerHealth; + + constructor( + private readonly manifest: ScaleSetControllerManifest, + private readonly serviceConfig: ScaleSetServiceConfig, + private readonly dependencies: ScaleSetReconcilerDependencies, + private readonly controllerLogger: ScaleSetLogger, + ) { + this.health = new ScaleSetControllerHealth( + manifest.groupName, + manifest.reconcilers.map(({ runnerConfigName }) => runnerConfigName), + serviceConfig.healthStaleAfterMs, + ); + } + + async run(signal: AbortSignal): Promise { + this.controllerLogger.debug('scale_set_reconcilers_starting', { + reconcilerCount: this.manifest.reconcilers.length, + runnerConfigNames: this.manifest.reconcilers.map(({ runnerConfigName }) => runnerConfigName), + }); + const completions = this.manifest.reconcilers.map(async (config) => { + const status = this.health.reporter(config.runnerConfigName); + try { + await new ScaleSetReconciler(config, this.serviceConfig, this.dependencies).run(signal, status); + } catch (error) { + status.markFailed(error); + this.controllerLogger.error('scale_set_reconciler_uncaught_failure', { + runnerConfigName: config.runnerConfigName, + scaleSetId: config.scaleSetId, + error, + }); + } + }); + + await Promise.race([Promise.all(completions), waitForAbort(signal)]); + if (!signal.aborted) await waitForAbort(signal); + this.health.markStopping(); + await Promise.all(completions); + } +} + +async function waitForAbort(signal: AbortSignal): Promise { + if (signal.aborted) return; + await new Promise((resolve) => signal.addEventListener('abort', () => resolve(), { once: true })); +} diff --git a/lambdas/services/scale-set/src/credentials.test.ts b/lambdas/services/scale-set/src/credentials.test.ts new file mode 100644 index 0000000000..54ba60faff --- /dev/null +++ b/lambdas/services/scale-set/src/credentials.test.ts @@ -0,0 +1,200 @@ +const authMocks = vi.hoisted(() => ({ createAppAuth: vi.fn(), requestDefaults: vi.fn() })); +vi.mock('@octokit/auth-app', () => ({ createAppAuth: authMocks.createAppAuth })); +vi.mock('@octokit/request', () => ({ request: { defaults: authMocks.requestDefaults } })); + +import type { ScaleSetFetch } from '@aws-github-runner/github-actions-scale-set'; + +import { createGitHubAppAccessTokenProvider, loadGitHubAppCredentials, type ParameterStore } from './credentials'; + +const references = { + appIdParameterName: '/app/id', + privateKeyParameterName: '/app/key', + installationIdParameterName: '/app/installation', +}; + +function encodedKey(body: string): string { + return Buffer.from(`-----BEGIN PRIVATE KEY-----\n${body}\n-----END PRIVATE KEY-----\n`).toString('base64'); +} + +describe('GitHub App credentials', () => { + beforeEach(() => { + vi.clearAllMocks(); + authMocks.requestDefaults.mockReturnValue(vi.fn()); + }); + + it('validates and decodes referenced SSM values', async () => { + const store: ParameterStore = { + get: vi.fn().mockResolvedValue( + new Map([ + ['/app/id', '123'], + ['/app/installation', '456'], + ['/app/key', encodedKey('abc')], + ]), + ), + put: vi.fn(), + }; + await expect(loadGitHubAppCredentials(references, store)).resolves.toMatchObject({ + appId: '123', + installationId: 456, + privateKey: expect.stringContaining('BEGIN PRIVATE KEY'), + }); + }); + + it('reuses auth for unchanged credentials and recreates it after rotation', async () => { + const values = [encodedKey('first'), encodedKey('first'), encodedKey('second')]; + const store: ParameterStore = { + get: vi.fn( + async () => + new Map([ + ['/app/id', '123'], + ['/app/installation', '456'], + ['/app/key', values.shift() as string], + ]), + ), + }; + const firstAuth = vi.fn().mockResolvedValue({ token: 'token-one', expiresAt: '2099-01-01T00:00:00Z' }); + const secondAuth = vi.fn().mockResolvedValue({ token: 'token-two', expiresAt: '2099-01-01T00:00:00Z' }); + const fetchImplementation = vi.fn(); + authMocks.createAppAuth.mockReturnValueOnce(firstAuth).mockReturnValueOnce(secondAuth); + + const provider = await createGitHubAppAccessTokenProvider( + references, + 'https://github.com/example', + false, + store, + fetchImplementation, + ); + await provider(); + await provider(); + await provider(); + + expect(store.get).toHaveBeenCalledTimes(3); + expect(authMocks.createAppAuth).toHaveBeenCalledTimes(2); + expect(authMocks.requestDefaults).toHaveBeenCalledWith({ + baseUrl: 'https://api.github.com', + request: { fetch: fetchImplementation }, + }); + expect(firstAuth).toHaveBeenCalledTimes(2); + expect(secondAuth).toHaveBeenCalledTimes(1); + }); + + it('discovers the installation ID from the configured organization when SSM does not provide one', async () => { + const store: ParameterStore = { + get: vi.fn().mockResolvedValue( + new Map([ + ['/app/id', '123'], + ['/app/key', encodedKey('abc')], + ]), + ), + put: vi.fn(), + }; + const appAuth = vi.fn().mockResolvedValue({ token: 'app-jwt' }); + const installationAuth = vi + .fn() + .mockResolvedValue({ token: 'installation-token', expiresAt: '2099-01-01T00:00:00Z' }); + const fetchImplementation = vi.fn().mockResolvedValue( + new Response(JSON.stringify([{ id: 456, account: { login: 'example' } }]), { + status: 200, + headers: { 'content-type': 'application/json' }, + }), + ); + authMocks.createAppAuth.mockReturnValueOnce(appAuth).mockReturnValueOnce(installationAuth); + + const provider = await createGitHubAppAccessTokenProvider( + references, + 'https://github.com/example', + false, + store, + fetchImplementation, + ); + + await expect(provider()).resolves.toMatchObject({ token: 'installation-token' }); + await expect(provider()).resolves.toMatchObject({ token: 'installation-token' }); + expect(appAuth).toHaveBeenCalledWith({ type: 'app' }); + expect(installationAuth).toHaveBeenCalledWith({ type: 'installation', installationId: 456 }); + expect(appAuth).toHaveBeenCalledTimes(1); + expect(fetchImplementation).toHaveBeenCalledTimes(1); + expect(store.put).toHaveBeenCalledWith('/app/installation', '456'); + expect(fetchImplementation).toHaveBeenCalledWith( + expect.objectContaining({ + href: 'https://api.github.com/app/installations?per_page=100&page=1', + }), + expect.objectContaining({ + headers: expect.objectContaining({ Authorization: 'Bearer app-jwt' }), + }), + ); + }); + + it('preserves the GHES API prefix during installation discovery', async () => { + const store: ParameterStore = { + get: vi.fn().mockResolvedValue( + new Map([ + ['/app/id', '123'], + ['/app/key', encodedKey('abc')], + ]), + ), + put: vi.fn(), + }; + const appAuth = vi.fn().mockResolvedValue({ token: 'app-jwt' }); + const installationAuth = vi + .fn() + .mockResolvedValue({ token: 'installation-token', expiresAt: '2099-01-01T00:00:00Z' }); + const fetchImplementation = vi.fn().mockResolvedValue( + new Response(JSON.stringify([{ id: 456, account: { login: 'example' } }]), { + status: 200, + headers: { 'content-type': 'application/json' }, + }), + ); + authMocks.createAppAuth.mockReturnValueOnce(appAuth).mockReturnValueOnce(installationAuth); + + const provider = await createGitHubAppAccessTokenProvider( + references, + 'https://github.example.com/example', + true, + store, + fetchImplementation, + ); + + await expect(provider()).resolves.toMatchObject({ token: 'installation-token' }); + expect(fetchImplementation).toHaveBeenCalledWith( + expect.objectContaining({ + href: 'https://github.example.com/api/v3/app/installations?per_page=100&page=1', + }), + expect.objectContaining({ + headers: expect.objectContaining({ Authorization: 'Bearer app-jwt' }), + }), + ); + }); + + it.each([ + [new Map([['/app/id', '123']]), 'was not returned'], + [ + new Map([ + ['/app/id', 'bad id'], + ['/app/installation', '1'], + ['/app/key', encodedKey('abc')], + ]), + 'App ID', + ], + [ + new Map([ + ['/app/id', '1'], + ['/app/installation', 'zero'], + ['/app/key', encodedKey('abc')], + ]), + 'positive integer', + ], + [ + new Map([ + ['/app/id', '1'], + ['/app/installation', '2'], + ['/app/key', 'not-base64'], + ]), + 'canonical base64', + ], + ])('rejects malformed credential parameters', async (values, message) => { + await expect(loadGitHubAppCredentials(references, { get: vi.fn().mockResolvedValue(values) })).rejects.toThrow( + message, + ); + }); +}); diff --git a/lambdas/services/scale-set/src/credentials.ts b/lambdas/services/scale-set/src/credentials.ts new file mode 100644 index 0000000000..7e4cacecd0 --- /dev/null +++ b/lambdas/services/scale-set/src/credentials.ts @@ -0,0 +1,209 @@ +import { createAppAuth } from '@octokit/auth-app'; +import { request } from '@octokit/request'; +import { createHash } from 'node:crypto'; + +import { + githubApiUrl, + parseGitHubConfigUrl, + type AccessToken, + type ScaleSetFetch, +} from '@aws-github-runner/github-actions-scale-set'; + +import { ScaleSetConfigurationError, type GitHubAppParameterReferences } from './config'; + +export interface ParameterStore { + get(names: readonly string[]): Promise>; + put?(name: string, value: string): Promise; +} + +interface GitHubAppCredentials { + appId: string; + installationId?: number; + privateKey: string; +} + +interface GitHubAppInstallation { + id?: unknown; + account?: { login?: unknown }; +} + +const MAX_PRIVATE_KEY_BYTES = 64 * 1024; + +function requiredParameter(values: ReadonlyMap, name: string): string { + const value = values.get(name); + if (value === undefined || value === '') { + throw new ScaleSetConfigurationError(`required SSM parameter ${JSON.stringify(name)} was not returned`); + } + return value; +} + +function decodePrivateKey(encoded: string): string { + if ( + encoded.length === 0 || + encoded.length > Math.ceil((MAX_PRIVATE_KEY_BYTES * 4) / 3) + 4 || + encoded.length % 4 !== 0 || + !/^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/.test(encoded) + ) { + throw new ScaleSetConfigurationError('GitHub App private key parameter must contain canonical base64'); + } + const decoded = Buffer.from(encoded, 'base64').toString('utf8').replace(/\\n/g, '\n'); + if (Buffer.byteLength(decoded, 'utf8') > MAX_PRIVATE_KEY_BYTES) { + throw new ScaleSetConfigurationError('GitHub App private key is too large'); + } + if (!/^-----BEGIN (?:RSA )?PRIVATE KEY-----\n[\s\S]+\n-----END (?:RSA )?PRIVATE KEY-----\n?$/.test(decoded)) { + throw new ScaleSetConfigurationError('GitHub App private key parameter is not a supported PEM private key'); + } + return decoded; +} + +export async function loadGitHubAppCredentials( + references: GitHubAppParameterReferences, + parameterStore: ParameterStore, +): Promise { + const names = [references.appIdParameterName, references.privateKeyParameterName]; + if (references.installationIdParameterName !== undefined) names.splice(1, 0, references.installationIdParameterName); + const values = await parameterStore.get(names); + const appId = requiredParameter(values, references.appIdParameterName).trim(); + if (!/^[A-Za-z0-9_-]{1,128}$/.test(appId)) { + throw new ScaleSetConfigurationError('GitHub App ID parameter is invalid'); + } + let installationId: number | undefined; + if (references.installationIdParameterName !== undefined) { + const installationIdRaw = values.get(references.installationIdParameterName)?.trim(); + if (installationIdRaw !== undefined && installationIdRaw !== '') { + if (!/^\d+$/.test(installationIdRaw)) { + throw new ScaleSetConfigurationError('GitHub App installation ID parameter must be a positive integer'); + } + installationId = Number(installationIdRaw); + if (!Number.isSafeInteger(installationId) || installationId <= 0) { + throw new ScaleSetConfigurationError('GitHub App installation ID parameter must be a positive integer'); + } + } + } + return { + appId, + installationId, + privateKey: decodePrivateKey(requiredParameter(values, references.privateKeyParameterName)), + }; +} + +async function discoverGitHubAppInstallationId( + credentials: Pick, + target: string, + apiBaseUrl: string, + fetchImplementation: ScaleSetFetch, +): Promise { + const appRequest = request.defaults({ baseUrl: apiBaseUrl, request: { fetch: fetchImplementation } }); + const appAuth = createAppAuth({ + appId: credentials.appId, + privateKey: credentials.privateKey, + request: appRequest, + }); + const appAuthentication = await appAuth({ type: 'app' }); + for (let page = 1; page <= 100; page += 1) { + const url = new URL('app/installations', `${apiBaseUrl}/`); + url.searchParams.set('per_page', '100'); + url.searchParams.set('page', String(page)); + const response = await fetchImplementation(url, { + headers: { + Accept: 'application/vnd.github+json', + Authorization: `Bearer ${appAuthentication.token}`, + 'User-Agent': 'github-aws-runners/scale-set-controller', + }, + }); + if (!response.ok) { + throw new ScaleSetConfigurationError(`GitHub App installation discovery failed with HTTP ${response.status}`); + } + let payload: unknown; + try { + payload = await response.json(); + } catch (error) { + throw new ScaleSetConfigurationError('GitHub App installation discovery returned invalid JSON', { cause: error }); + } + if (!Array.isArray(payload)) { + throw new ScaleSetConfigurationError('GitHub App installation discovery returned an invalid response'); + } + for (const value of payload as unknown[]) { + if (typeof value !== 'object' || value === null || Array.isArray(value)) continue; + const candidate = value as GitHubAppInstallation; + if ( + Number.isSafeInteger(candidate.id) && + typeof candidate.account?.login === 'string' && + candidate.account.login.toLowerCase() === target.toLowerCase() + ) { + return candidate.id as number; + } + } + if (payload.length < 100) break; + } + throw new ScaleSetConfigurationError(`GitHub App is not installed for ${JSON.stringify(target)}`); +} + +export async function createGitHubAppAccessTokenProvider( + references: GitHubAppParameterReferences, + githubConfigUrl: string, + forceGhes: boolean, + parameterStore: ParameterStore, + fetchImplementation: ScaleSetFetch = globalThis.fetch, +): Promise<() => Promise> { + const parsedConfig = parseGitHubConfigUrl(githubConfigUrl, forceGhes); + const apiBaseUrl = githubApiUrl(parsedConfig, '/').toString().replace(/\/$/, ''); + let cached: + | { + fingerprint: string; + installationId: number; + auth: ReturnType; + } + | undefined; + let discovered: + | { + fingerprint: string; + installationId: number; + } + | undefined; + + return async () => { + // Reload references for rotation visibility, but preserve the Octokit auth + // instance while credentials are unchanged so its installation-token cache + // remains effective. + const credentials = await loadGitHubAppCredentials(references, parameterStore); + const credentialFingerprint = createHash('sha256') + .update(credentials.appId) + .update('\u0000') + .update(credentials.privateKey) + .digest('base64url'); + const target = parsedConfig.organization ?? parsedConfig.enterprise; + if (credentials.installationId === undefined && target === undefined) { + throw new ScaleSetConfigurationError( + 'GitHub App installation discovery requires an organization or enterprise URL', + ); + } + let installationId = credentials.installationId; + if (installationId === undefined) { + if (discovered?.fingerprint === credentialFingerprint) { + installationId = discovered.installationId; + } else { + installationId = await discoverGitHubAppInstallationId(credentials, target!, apiBaseUrl, fetchImplementation); + discovered = { fingerprint: credentialFingerprint, installationId }; + if (references.installationIdParameterName !== undefined && parameterStore.put !== undefined) { + await parameterStore.put(references.installationIdParameterName, String(installationId)); + } + } + } + const fingerprint = `${credentialFingerprint}\u0000${installationId}`; + if (cached?.fingerprint !== fingerprint) { + cached = { + fingerprint, + installationId, + auth: createAppAuth({ + appId: credentials.appId, + installationId, + privateKey: credentials.privateKey, + request: request.defaults({ baseUrl: apiBaseUrl, request: { fetch: fetchImplementation } }), + }), + }; + } + const installation = await cached.auth({ type: 'installation', installationId: cached.installationId }); + return { token: installation.token, expiresAt: installation.expiresAt }; + }; +} diff --git a/lambdas/services/scale-set/src/github-http.test.ts b/lambdas/services/scale-set/src/github-http.test.ts new file mode 100644 index 0000000000..870399ac84 --- /dev/null +++ b/lambdas/services/scale-set/src/github-http.test.ts @@ -0,0 +1,58 @@ +const undiciMocks = vi.hoisted(() => ({ + close: vi.fn().mockResolvedValue(undefined), + createAgent: vi.fn(), +})); + +vi.mock('undici', () => ({ + Agent: class MockAgent { + constructor(options: unknown) { + undiciMocks.createAgent(options); + } + + close = undiciMocks.close; + }, +})); + +import type { ScaleSetFetch } from '@aws-github-runner/github-actions-scale-set'; + +import { createScaleSetGitHubHttp } from './github-http'; + +describe('scale-set GitHub HTTP isolation', () => { + beforeEach(() => { + vi.clearAllMocks(); + }); + + it('uses the supplied verified fetch without changing process TLS settings', async () => { + const original = process.env.NODE_TLS_REJECT_UNAUTHORIZED; + const fetchImplementation = vi.fn().mockResolvedValue(new Response(null, { status: 204 })); + const http = createScaleSetGitHubHttp(fetchImplementation); + + await http.fetch(true)('https://github.example/_apis/runtime/runnerscalesets'); + await http.close(); + + expect(fetchImplementation).toHaveBeenCalledWith('https://github.example/_apis/runtime/runnerscalesets'); + expect(undiciMocks.createAgent).not.toHaveBeenCalled(); + expect(process.env.NODE_TLS_REJECT_UNAUTHORIZED).toBe(original); + }); + + it('uses one scoped insecure dispatcher and closes it without mutating global TLS state', async () => { + const original = process.env.NODE_TLS_REJECT_UNAUTHORIZED; + const fetchImplementation = vi.fn().mockResolvedValue(new Response(null, { status: 204 })); + const http = createScaleSetGitHubHttp(fetchImplementation); + const first = http.fetch(false); + const second = http.fetch(false); + + await first('https://github.example/_apis/runtime/runnerscalesets', { method: 'GET' }); + await http.close(); + + expect(first).toBe(second); + expect(undiciMocks.createAgent).toHaveBeenCalledOnce(); + expect(undiciMocks.createAgent).toHaveBeenCalledWith({ connect: { rejectUnauthorized: false } }); + expect(fetchImplementation).toHaveBeenCalledWith( + 'https://github.example/_apis/runtime/runnerscalesets', + expect.objectContaining({ method: 'GET', dispatcher: expect.anything() }), + ); + expect(undiciMocks.close).toHaveBeenCalledOnce(); + expect(process.env.NODE_TLS_REJECT_UNAUTHORIZED).toBe(original); + }); +}); diff --git a/lambdas/services/scale-set/src/github-http.ts b/lambdas/services/scale-set/src/github-http.ts new file mode 100644 index 0000000000..85a60626b1 --- /dev/null +++ b/lambdas/services/scale-set/src/github-http.ts @@ -0,0 +1,34 @@ +import { Agent, type Dispatcher } from 'undici'; + +import type { ScaleSetFetch } from '@aws-github-runner/github-actions-scale-set'; + +type DispatcherRequestInit = RequestInit & { dispatcher: Dispatcher }; + +export interface ScaleSetGitHubHttp { + fetch(sslVerify: boolean): ScaleSetFetch; + close(): Promise; +} + +/** + * Creates fetch implementations whose TLS policy is scoped to one controller + * process. Disabling verification never mutates NODE_TLS_REJECT_UNAUTHORIZED + * or the global Undici dispatcher, so verified and unverified GHES runner + * configurations may safely share one grouped task. + */ +export function createScaleSetGitHubHttp(fetchImplementation: ScaleSetFetch = globalThis.fetch): ScaleSetGitHubHttp { + let insecureAgent: Agent | undefined; + let insecureFetch: ScaleSetFetch | undefined; + + return { + fetch(sslVerify) { + if (sslVerify) return fetchImplementation; + insecureAgent ??= new Agent({ connect: { rejectUnauthorized: false } }); + insecureFetch ??= async (input, init = {}) => + await fetchImplementation(input, { ...init, dispatcher: insecureAgent } as DispatcherRequestInit); + return insecureFetch; + }, + async close() { + await insecureAgent?.close(); + }, + }; +} diff --git a/lambdas/services/scale-set/src/health-server.test.ts b/lambdas/services/scale-set/src/health-server.test.ts new file mode 100644 index 0000000000..16e65ab953 --- /dev/null +++ b/lambdas/services/scale-set/src/health-server.test.ts @@ -0,0 +1,15 @@ +import { startScaleSetHealthServer } from './health-server'; + +describe('health server', () => { + it('separates liveness and readiness on loopback', async () => { + const health = { snapshot: vi.fn(() => ({ live: true, ready: false, state: 'degraded' })) }; + const server = await startScaleSetHealthServer(health, 0); + try { + await expect(fetch(`http://127.0.0.1:${server.port}/healthz`)).resolves.toMatchObject({ status: 200 }); + await expect(fetch(`http://127.0.0.1:${server.port}/readyz`)).resolves.toMatchObject({ status: 503 }); + await expect(fetch(`http://127.0.0.1:${server.port}/other`)).resolves.toMatchObject({ status: 404 }); + } finally { + await server.close(); + } + }); +}); diff --git a/lambdas/services/scale-set/src/health-server.ts b/lambdas/services/scale-set/src/health-server.ts new file mode 100644 index 0000000000..f9b967aaec --- /dev/null +++ b/lambdas/services/scale-set/src/health-server.ts @@ -0,0 +1,53 @@ +import { createServer, type Server } from 'node:http'; + +import type { ScaleSetControllerHealth } from './health'; + +export interface ScaleSetHealthServer { + port: number; + close(): Promise; +} + +export async function startScaleSetHealthServer( + health: Pick, + port: number, +): Promise { + const server = createServer((request, response) => { + response.setHeader('Cache-Control', 'no-store'); + response.setHeader('Connection', 'close'); + response.setHeader('Content-Type', 'application/json; charset=utf-8'); + response.setHeader('X-Content-Type-Options', 'nosniff'); + + if (request.method !== 'GET' || (request.url !== '/healthz' && request.url !== '/readyz')) { + response.statusCode = 404; + response.end(JSON.stringify({ status: 'not-found' })); + return; + } + + const snapshot = health.snapshot(); + const healthy = request.url === '/readyz' ? snapshot.ready : snapshot.live; + response.statusCode = healthy ? 200 : 503; + response.end(JSON.stringify(snapshot)); + }); + await listen(server, port); + const address = server.address(); + if (address === null || typeof address === 'string') throw new Error('health server did not bind a TCP address'); + return { + port: address.port, + close: async () => { + server.closeAllConnections(); + if (!server.listening) return; + await new Promise((resolve, reject) => server.close((error) => (error ? reject(error) : resolve()))); + }, + }; +} + +async function listen(server: Server, port: number): Promise { + await new Promise((resolve, reject) => { + const onError = (error: Error) => reject(error); + server.once('error', onError); + server.listen(port, '127.0.0.1', () => { + server.removeListener('error', onError); + resolve(); + }); + }); +} diff --git a/lambdas/services/scale-set/src/health.test.ts b/lambdas/services/scale-set/src/health.test.ts new file mode 100644 index 0000000000..b2d5294e11 --- /dev/null +++ b/lambdas/services/scale-set/src/health.test.ts @@ -0,0 +1,46 @@ +import { ScaleSetControllerHealth } from './health'; + +describe('ScaleSetControllerHealth', () => { + it('aggregates independent readiness while reconnect heartbeats stay live', () => { + let now = 0; + const health = new ScaleSetControllerHealth('group', ['a', 'b'], 100, () => now); + const a = health.reporter('a'); + const b = health.reporter('b'); + a.markSessionReady(); + b.markSessionReady(); + a.markProgress(); + b.markProgress(); + expect(health.snapshot()).toMatchObject({ state: 'ready', live: true, ready: true }); + + now = 200; + expect(health.snapshot()).toMatchObject({ state: 'degraded', live: true, ready: false }); + expect(health.snapshot().reconcilers.a).toMatchObject({ state: 'ready', live: true, ready: false }); + + a.markReconnecting(new Error('outage')); + expect(health.snapshot()).toMatchObject({ state: 'degraded', live: true, ready: false }); + expect(health.snapshot().reconcilers.a).toMatchObject({ state: 'reconnecting', live: true, ready: false }); + }); + + it('contains one terminal reconciler failure while another stays ready', () => { + const health = new ScaleSetControllerHealth('group', ['a', 'b'], 100); + health.reporter('a').markFailed(new TypeError('bad config')); + health.reporter('b').markProgress(); + expect(health.snapshot()).toMatchObject({ state: 'degraded', live: true, ready: false }); + expect(health.snapshot().reconcilers.a).toMatchObject({ state: 'failed', lastErrorName: 'TypeError' }); + }); + + it('marks all reporters stopping without reviving failures', () => { + const health = new ScaleSetControllerHealth('group', ['a'], 100); + health.reporter('a').markFailed(); + health.markStopping(); + expect(health.snapshot()).toMatchObject({ state: 'stopping', live: true, ready: false }); + expect(health.snapshot().reconcilers.a.state).toBe('failed'); + }); + + it('rejects duplicate and unknown reporter names', () => { + expect(() => new ScaleSetControllerHealth('g', [], 1)).toThrow('at least one'); + expect(() => new ScaleSetControllerHealth('g', ['a', 'a'], 1)).toThrow('duplicate'); + const health = new ScaleSetControllerHealth('g', ['a'], 1); + expect(() => health.reporter('b')).toThrow('unknown runner config'); + }); +}); diff --git a/lambdas/services/scale-set/src/health.ts b/lambdas/services/scale-set/src/health.ts new file mode 100644 index 0000000000..6cad43d10d --- /dev/null +++ b/lambdas/services/scale-set/src/health.ts @@ -0,0 +1,137 @@ +export type ScaleSetReconcilerState = 'starting' | 'ready' | 'reconnecting' | 'failed' | 'stopping'; +export type ScaleSetControllerState = 'starting' | 'ready' | 'degraded' | 'failed' | 'stopping'; + +export interface ScaleSetReconcilerHealthSnapshot { + state: ScaleSetReconcilerState; + live: boolean; + ready: boolean; + lastActivityAt: string; + consecutiveFailures: number; + lastErrorName?: string; +} + +export interface ScaleSetControllerHealthSnapshot { + groupName: string; + state: ScaleSetControllerState; + live: boolean; + ready: boolean; + reconcilers: Readonly>; +} + +export interface ScaleSetReconcilerStatusReporter { + markSessionReady(): void; + markProgress(): void; + markReconnecting(error?: unknown): void; + markFailed(error?: unknown): void; + markStopping(): void; +} + +interface MutableHealth { + state: ScaleSetReconcilerState; + lastActivityAt: number; + consecutiveFailures: number; + lastErrorName?: string; +} + +function errorName(error: unknown): string | undefined { + if (error === undefined) return undefined; + return error instanceof Error ? error.name : typeof error; +} + +export class ScaleSetControllerHealth { + private readonly states = new Map(); + private stopping = false; + + constructor( + readonly groupName: string, + runnerConfigNames: readonly string[], + private readonly staleAfterMs: number, + private readonly now: () => number = Date.now, + ) { + const startedAt = now(); + for (const name of runnerConfigNames) { + if (this.states.has(name)) throw new Error(`duplicate health reporter for ${JSON.stringify(name)}`); + this.states.set(name, { state: 'starting', lastActivityAt: startedAt, consecutiveFailures: 0 }); + } + if (this.states.size === 0) throw new Error('at least one reconciler health reporter is required'); + } + + reporter(runnerConfigName: string): ScaleSetReconcilerStatusReporter { + const state = this.states.get(runnerConfigName); + if (!state) throw new Error(`unknown runner config ${JSON.stringify(runnerConfigName)}`); + return { + markSessionReady: () => { + if (state.state === 'starting') { + state.state = 'ready'; + state.lastActivityAt = this.now(); + } else if (state.state === 'reconnecting') { + state.state = 'ready'; + } + }, + markProgress: () => { + if (state.state === 'failed' || state.state === 'stopping') return; + state.state = 'ready'; + state.lastActivityAt = this.now(); + state.consecutiveFailures = 0; + state.lastErrorName = undefined; + }, + markReconnecting: (error) => { + if (state.state === 'failed' || state.state === 'stopping') return; + state.state = 'reconnecting'; + state.lastActivityAt = this.now(); + state.consecutiveFailures += 1; + state.lastErrorName = errorName(error); + }, + markFailed: (error) => { + if (state.state === 'stopping') return; + state.state = 'failed'; + state.consecutiveFailures += 1; + state.lastErrorName = errorName(error); + }, + markStopping: () => { + if (state.state !== 'failed') state.state = 'stopping'; + }, + }; + } + + markStopping(): void { + this.stopping = true; + for (const name of this.states.keys()) this.reporter(name).markStopping(); + } + + snapshot(): ScaleSetControllerHealthSnapshot { + const now = this.now(); + const reconcilers: Record = Object.create(null) as Record< + string, + ScaleSetReconcilerHealthSnapshot + >; + for (const [name, state] of this.states) { + const stale = now - state.lastActivityAt > this.staleAfterMs; + // Staleness means the reconciler is not ready, but it is not a process + // liveness failure. A single bounded GitHub/AWS request can legitimately + // outlive the readiness window; restarting the task would only churn its + // message sessions and reset the provider retry policy. + const live = state.state === 'stopping' || state.state !== 'failed'; + reconcilers[name] = { + state: state.state, + live, + ready: state.state === 'ready' && !stale, + lastActivityAt: new Date(state.lastActivityAt).toISOString(), + consecutiveFailures: state.consecutiveFailures, + ...(state.lastErrorName === undefined ? {} : { lastErrorName: state.lastErrorName }), + }; + } + const values = Object.values(reconcilers); + const liveCount = values.filter(({ live }) => live).length; + const readyCount = values.filter(({ ready }) => ready).length; + const live = this.stopping || liveCount > 0; + const ready = !this.stopping && readyCount === values.length; + let state: ScaleSetControllerState; + if (this.stopping) state = 'stopping'; + else if (ready) state = 'ready'; + else if (liveCount === 0) state = 'failed'; + else if (values.every((value) => value.state === 'starting')) state = 'starting'; + else state = 'degraded'; + return { groupName: this.groupName, state, live, ready, reconcilers }; + } +} diff --git a/lambdas/services/scale-set/src/index.ts b/lambdas/services/scale-set/src/index.ts new file mode 100644 index 0000000000..6074b49499 --- /dev/null +++ b/lambdas/services/scale-set/src/index.ts @@ -0,0 +1,9 @@ +export * from './config'; +export * from './controller'; +export * from './credentials'; +export * from './health'; +export * from './health-server'; +export * from './lifecycle'; +export * from './logger'; +export * from './parameter-store'; +export * from './reconciler'; diff --git a/lambdas/services/scale-set/src/lifecycle.test.ts b/lambdas/services/scale-set/src/lifecycle.test.ts new file mode 100644 index 0000000000..a43c31fe5c --- /dev/null +++ b/lambdas/services/scale-set/src/lifecycle.test.ts @@ -0,0 +1,45 @@ +import { ScaleSetServiceRuntime } from './lifecycle'; + +describe('ScaleSetServiceRuntime', () => { + it('starts once and performs idempotent bounded shutdown', async () => { + const health = { markStopping: vi.fn(), snapshot: vi.fn() }; + const run = vi.fn(async (signal: AbortSignal) => { + if (!signal.aborted) + await new Promise((resolve) => signal.addEventListener('abort', () => resolve(), { once: true })); + }); + const runtime = new ScaleSetServiceRuntime({ shutdownTimeoutMs: 100 }, { run, health } as never); + const completion = runtime.run(); + expect(() => runtime.run()).toThrow('already started'); + const shutdown = runtime.shutdown(); + expect(runtime.shutdown()).toBe(shutdown); + await shutdown; + await completion; + expect(health.markStopping).toHaveBeenCalledOnce(); + }); + + it('allows shutdown before start and prevents a later start', async () => { + const health = { markStopping: vi.fn() }; + const runtime = new ScaleSetServiceRuntime({ shutdownTimeoutMs: 100 }, { run: vi.fn(), health } as never); + await runtime.shutdown(); + expect(() => runtime.run()).toThrow('already stopping'); + }); + + it('rejects when a controller ignores cancellation past the timeout', async () => { + vi.useFakeTimers(); + try { + const health = { markStopping: vi.fn() }; + const runtime = new ScaleSetServiceRuntime({ shutdownTimeoutMs: 10 }, { + run: vi.fn(() => new Promise(() => undefined)), + health, + } as never); + void runtime.run(); + await Promise.resolve(); + const shutdown = runtime.shutdown(); + const expectation = expect(shutdown).rejects.toThrow('did not stop within 10ms'); + await vi.advanceTimersByTimeAsync(10); + await expectation; + } finally { + vi.useRealTimers(); + } + }); +}); diff --git a/lambdas/services/scale-set/src/lifecycle.ts b/lambdas/services/scale-set/src/lifecycle.ts new file mode 100644 index 0000000000..d4c4a600c0 --- /dev/null +++ b/lambdas/services/scale-set/src/lifecycle.ts @@ -0,0 +1,51 @@ +import type { ScaleSetServiceConfig } from './config'; +import type { ScaleSetController } from './controller'; + +export class ScaleSetServiceRuntime { + private readonly abortController = new AbortController(); + private completion: Promise | undefined; + private shutdownCompletion: Promise | undefined; + + constructor( + private readonly config: Pick, + private readonly controller: Pick, + ) {} + + get health() { + return this.controller.health; + } + + run(): Promise { + if (this.shutdownCompletion !== undefined) throw new Error('Scale-set service runtime is already stopping'); + if (this.completion !== undefined) throw new Error('Scale-set service runtime has already started'); + this.completion = Promise.resolve().then(async () => { + if (!this.abortController.signal.aborted) await this.controller.run(this.abortController.signal); + }); + return this.completion; + } + + shutdown(reason: unknown = new Error('Scale-set service shutdown requested')): Promise { + this.shutdownCompletion ??= this.shutdownOnce(reason); + return this.shutdownCompletion; + } + + private async shutdownOnce(reason: unknown): Promise { + this.controller.health.markStopping(); + this.abortController.abort(reason); + if (this.completion === undefined) return; + let timeout: ReturnType | undefined; + try { + await Promise.race([ + this.completion, + new Promise((_resolve, reject) => { + timeout = setTimeout( + () => reject(new Error(`Scale-set controller did not stop within ${this.config.shutdownTimeoutMs}ms`)), + this.config.shutdownTimeoutMs, + ); + }), + ]); + } finally { + if (timeout !== undefined) clearTimeout(timeout); + } + } +} diff --git a/lambdas/services/scale-set/src/logger.test.ts b/lambdas/services/scale-set/src/logger.test.ts new file mode 100644 index 0000000000..a2bf9fef05 --- /dev/null +++ b/lambdas/services/scale-set/src/logger.test.ts @@ -0,0 +1,50 @@ +import { createScaleSetLogger, logger, sanitizeLogAttributes } from './logger'; + +describe('redacted structured logging', () => { + it('emits debug records when LOG_LEVEL is debug', () => { + const spy = vi.spyOn(console, 'debug').mockImplementation(() => undefined); + createScaleSetLogger({ LOG_LEVEL: 'debug' }).debug('debug_event', { reconcilerCount: 2 }); + expect(spy).toHaveBeenCalledWith(expect.stringContaining('"event":"debug_event"')); + spy.mockRestore(); + }); + + it('does not emit debug records at the default info level', () => { + const spy = vi.spyOn(console, 'debug').mockImplementation(() => undefined); + createScaleSetLogger({}).debug('hidden_debug_event'); + expect(spy).not.toHaveBeenCalled(); + spy.mockRestore(); + }); + + it('redacts nested secrets and strips log-injection characters', () => { + expect( + sanitizeLogAttributes({ + runnerConfig: 'linux\nforged', + privateKey: 'secret', + nested: { authorization: 'Bearer secret', safe: 'ok' }, + }), + ).toEqual({ + runnerConfig: 'linux forged', + privateKey: '[REDACTED]', + nested: { authorization: '[REDACTED]', safe: 'ok' }, + }); + }); + + it('logs errors without their potentially sensitive message', () => { + const spy = vi.spyOn(console, 'error').mockImplementation(() => undefined); + logger.error('failed', { error: new Error('token=secret') }); + expect(spy).toHaveBeenCalledOnce(); + expect(spy.mock.calls[0][0]).not.toContain('token=secret'); + expect(spy.mock.calls[0][0]).toContain('token=[REDACTED]'); + expect(JSON.parse(spy.mock.calls[0][0] as string)).toMatchObject({ level: 'error', event: 'failed' }); + spy.mockRestore(); + }); + + it('includes generic error messages without provider-specific error handling', () => { + expect(sanitizeLogAttributes({ error: new Error('invalid scale-set configuration') })).toEqual({ + error: { + name: 'Error', + message: 'invalid scale-set configuration', + }, + }); + }); +}); diff --git a/lambdas/services/scale-set/src/logger.ts b/lambdas/services/scale-set/src/logger.ts new file mode 100644 index 0000000000..9230ad12b7 --- /dev/null +++ b/lambdas/services/scale-set/src/logger.ts @@ -0,0 +1,91 @@ +const REDACTED = '[REDACTED]'; +const SENSITIVE_KEY = /(authorization|credential|encodedjit|jitconfig|password|private.?key|secret|sessionid|token)/i; +const SENSITIVE_MESSAGE_VALUE = + /((?:authorization|credential|encodedjit|jitconfig|password|private.?key|secret|sessionid|token)\s*[=:]\s*)[^\s,;]+/gi; +const MAX_LOG_STRING_LENGTH = 1024; +const MAX_LOG_DEPTH = 4; +const LOG_LEVEL_PRIORITY = { debug: 10, info: 20, warn: 30, error: 40 } as const; + +export type ScaleSetLogLevel = keyof typeof LOG_LEVEL_PRIORITY; + +export interface ScaleSetLogger { + debug(event: string, attributes?: Readonly>): void; + info(event: string, attributes?: Readonly>): void; + warn(event: string, attributes?: Readonly>): void; + error(event: string, attributes?: Readonly>): void; +} + +function sanitizeString(value: string): string { + return value.replace(/[\r\n\u2028\u2029]/g, ' ').slice(0, MAX_LOG_STRING_LENGTH); +} + +function sanitizeErrorMessage(value: string): string { + return sanitizeString(value).replace(SENSITIVE_MESSAGE_VALUE, '$1' + REDACTED); +} + +function sanitize(value: unknown, key: string, depth: number): unknown { + if (SENSITIVE_KEY.test(key)) return REDACTED; + if (depth > MAX_LOG_DEPTH) return '[TRUNCATED]'; + if (value === null || typeof value === 'boolean' || typeof value === 'number') return value; + if (typeof value === 'string') return sanitizeString(value); + if (value instanceof Error) { + const status = 'status' in value && typeof value.status === 'number' ? value.status : undefined; + const code = 'code' in value && typeof value.code === 'string' ? sanitizeString(value.code) : undefined; + return { + name: sanitizeString(value.name), + message: sanitizeErrorMessage(value.message), + ...(status === undefined ? {} : { status }), + ...(code ? { code } : {}), + }; + } + if (Array.isArray(value)) return value.slice(0, 100).map((item) => sanitize(item, key, depth + 1)); + if (typeof value === 'object') { + const result: Record = Object.create(null) as Record; + for (const [childKey, childValue] of Object.entries(value).slice(0, 100)) { + result[sanitizeString(childKey)] = sanitize(childValue, childKey, depth + 1); + } + return result; + } + return sanitizeString(typeof value); +} + +export function sanitizeLogAttributes(attributes: Readonly> = {}): Record { + return sanitize(attributes, '', 0) as Record; +} + +function parseLogLevel(value: string | undefined): ScaleSetLogLevel { + return value !== undefined && value in LOG_LEVEL_PRIORITY ? (value as ScaleSetLogLevel) : 'info'; +} + +function write( + level: ScaleSetLogLevel, + minimumLevel: ScaleSetLogLevel, + event: string, + attributes?: Readonly>, +): void { + if (LOG_LEVEL_PRIORITY[level] < LOG_LEVEL_PRIORITY[minimumLevel]) return; + const record = JSON.stringify({ + timestamp: new Date().toISOString(), + level, + event: sanitizeString(event), + ...sanitizeLogAttributes(attributes), + }); + if (level === 'error') console.error(record); + else if (level === 'warn') console.warn(record); + else if (level === 'info') console.info(record); + else console.debug(record); +} + +export function createScaleSetLogger( + environment: Readonly> = process.env, +): ScaleSetLogger { + const minimumLevel = parseLogLevel(environment.LOG_LEVEL); + return { + debug: (event, attributes) => write('debug', minimumLevel, event, attributes), + info: (event, attributes) => write('info', minimumLevel, event, attributes), + warn: (event, attributes) => write('warn', minimumLevel, event, attributes), + error: (event, attributes) => write('error', minimumLevel, event, attributes), + }; +} + +export const logger = createScaleSetLogger(); diff --git a/lambdas/services/scale-set/src/main.ts b/lambdas/services/scale-set/src/main.ts new file mode 100644 index 0000000000..3f3f08389f --- /dev/null +++ b/lambdas/services/scale-set/src/main.ts @@ -0,0 +1,144 @@ +import { GitHubActionsScaleSetClient } from '@aws-github-runner/github-actions-scale-set'; +import { fromNodeProviderChain, fromTemporaryCredentials } from '@aws-sdk/credential-providers'; +import { createScaleSetComputeProviderRegistry } from '@aws-github-runner/compute-providers/scale-set'; + +import { parseScaleSetServiceConfig } from './config'; +import { ScaleSetController } from './controller'; +import { createGitHubAppAccessTokenProvider } from './credentials'; +import { createScaleSetGitHubHttp } from './github-http'; +import { startScaleSetHealthServer, type ScaleSetHealthServer } from './health-server'; +import { ScaleSetServiceRuntime } from './lifecycle'; +import { logger } from './logger'; +import { createDefaultControllerManifestLoader, defaultParameterStore } from './parameter-store'; +import { abortableSleep, type ScaleSetReconcilerDependencies } from './reconciler'; + +async function main(): Promise { + logger.info('scale_set_controller_configuration_loading', { + manifestConfigured: Boolean(process.env.SCALE_SET_CONTROLLER_MANIFEST?.trim()), + groupConfigConfigured: Boolean(process.env.SCALE_SET_CONTROLLER_GROUP_CONFIG_PATH?.trim()), + }); + const serviceConfig = parseScaleSetServiceConfig(process.env); + const manifest = await createDefaultControllerManifestLoader().load(serviceConfig); + logger.info('scale_set_controller_manifest_loaded', { + groupName: manifest.groupName, + revision: manifest.revision, + reconcilerCount: manifest.reconcilers.length, + runnerConfigNames: manifest.reconcilers.map(({ runnerConfigName }) => runnerConfigName), + }); + logger.debug('scale_set_controller_reconcilers_loaded', { + groupName: manifest.groupName, + reconcilerCount: manifest.reconcilers.length, + runnerConfigNames: manifest.reconcilers.map(({ runnerConfigName }) => runnerConfigName), + }); + const computeProviders = createScaleSetComputeProviderRegistry(); + const githubHttp = createScaleSetGitHubHttp(); + const dependencies: ScaleSetReconcilerDependencies = { + computeProviders, + createAccessTokenProvider: async (config) => + await createGitHubAppAccessTokenProvider( + config.githubApp, + config.githubConfigUrl, + config.forceGhes, + defaultParameterStore, + githubHttp.fetch(config.sslVerify), + ), + createClient: (config, accessTokenProvider) => + new GitHubActionsScaleSetClient({ + gitHubConfigUrl: config.githubConfigUrl, + accessTokenProvider, + fetch: githubHttp.fetch(config.sslVerify), + forceGhes: config.forceGhes, + systemInfo: { + system: config.userAgent ?? 'github-aws-runners', + version: '1', + scaleSetId: config.scaleSetId ?? 0, + subsystem: 'scale-set-controller', + }, + }), + logger, + parameterStore: defaultParameterStore, + createComputeProviderCredentials: (roleArn) => { + const taskCredentials = fromNodeProviderChain(); + if (roleArn === undefined) { + logger.info('scale_set_compute_provider_credentials_selected', { + credentialSource: 'task_role', + }); + return taskCredentials; + } + + logger.info('scale_set_compute_provider_credentials_selected', { + credentialSource: 'assumed_compute_role', + roleArn, + }); + const assumedRoleCredentials = fromTemporaryCredentials({ + masterCredentials: taskCredentials, + params: { + RoleArn: roleArn, + RoleSessionName: 'scale-set-controller', + DurationSeconds: 3600, + }, + clientConfig: { + maxAttempts: 3, + retryMode: 'standard', + }, + }); + let roleAssumptionLogged = false; + return async () => { + try { + const credentials = await assumedRoleCredentials(); + if (!roleAssumptionLogged) { + roleAssumptionLogged = true; + logger.info('scale_set_compute_provider_role_assumed', { + credentialSource: 'assumed_compute_role', + roleArn, + }); + } + return credentials; + } catch (error) { + logger.error('scale_set_compute_provider_role_assume_failed', { roleArn, error }); + throw error; + } + }; + }, + sleep: abortableSleep, + random: Math.random, + closeSignal: AbortSignal.timeout, + }; + const controller = new ScaleSetController(manifest, serviceConfig, dependencies, logger); + const runtime = new ScaleSetServiceRuntime(serviceConfig, controller); + let healthServer: ScaleSetHealthServer | undefined; + + const shutdown = (signal: NodeJS.Signals) => { + logger.info('scale_set_controller_shutdown_requested', { signal, groupName: manifest.groupName }); + void runtime.shutdown(new Error(`received ${signal}`)).catch((error) => { + logger.error('scale_set_controller_shutdown_failed', { error, groupName: manifest.groupName }); + process.exitCode = 1; + }); + }; + const onSigterm = () => shutdown('SIGTERM'); + const onSigint = () => shutdown('SIGINT'); + process.once('SIGTERM', onSigterm); + process.once('SIGINT', onSigint); + + try { + healthServer = await startScaleSetHealthServer(runtime.health, serviceConfig.healthPort); + logger.info('scale_set_controller_started', { + groupName: manifest.groupName, + revision: manifest.revision, + reconcilerCount: manifest.reconcilers.length, + healthPort: healthServer.port, + }); + await runtime.run(); + } finally { + await runtime.shutdown().catch(() => undefined); + await healthServer?.close(); + await githubHttp.close(); + process.removeListener('SIGTERM', onSigterm); + process.removeListener('SIGINT', onSigint); + } +} + +void main().catch((error) => { + logger.error('scale_set_controller_fatal_failure', { error }); + process.exitCode = 1; +}); diff --git a/lambdas/services/scale-set/src/parameter-store.test.ts b/lambdas/services/scale-set/src/parameter-store.test.ts new file mode 100644 index 0000000000..f18cc2ef7e --- /dev/null +++ b/lambdas/services/scale-set/src/parameter-store.test.ts @@ -0,0 +1,78 @@ +import { createControllerManifestLoader, type ParametersByPathClient } from './parameter-store'; +import type { ScaleSetServiceConfig } from './config'; + +function leaf(name: string, id: number): string { + return JSON.stringify({ + schemaVersion: 1, + runnerConfigName: name, + githubConfigUrl: 'https://github.com/example', + scaleSetId: id, + expectedScaleSetName: name, + expectedRunnerGroupId: null, + minRunners: 0, + maxRunners: 10, + sslVerify: true, + githubApp: { + appIdParameterName: '/app/id', + privateKeyParameterName: '/app/key', + installationIdParameterName: '/app/installation', + }, + computeProvider: { type: 'ec2', configuration: {} }, + }); +} + +const config: ScaleSetServiceConfig = { + groupName: 'group', + groupConfigPath: '/groups/group', + groupRevision: 'rev-1', + healthPort: 8080, + healthStaleAfterMs: 1000, + shutdownTimeoutMs: 1000, + sessionCloseTimeoutMs: 1000, + reconnectInitialBackoffMs: 100, + reconnectMaxBackoffMs: 1000, +}; + +describe('createControllerManifestLoader', () => { + it('paginates direct children, sorts them, and returns a versioned group', async () => { + const send = vi + .fn() + .mockResolvedValueOnce({ Parameters: [{ Name: '/groups/group/b', Value: leaf('b', 2) }], NextToken: 'next' }) + .mockResolvedValueOnce({ Parameters: [{ Name: '/groups/group/a', Value: leaf('a', 1) }] }); + const manifest = await createControllerManifestLoader({ send } as ParametersByPathClient).load(config); + expect(manifest).toMatchObject({ version: 1, groupName: 'group', revision: 'rev-1' }); + expect(manifest.reconcilers.map(({ runnerConfigName }) => runnerConfigName)).toEqual(['a', 'b']); + expect(send).toHaveBeenCalledTimes(2); + }); + + it('uses the injected inline manifest loader path for local tests', async () => { + const inline = JSON.stringify({ version: 1, groupName: 'local', reconcilers: [JSON.parse(leaf('a', 1))] }); + const send = vi.fn(); + await expect( + createControllerManifestLoader({ send } as ParametersByPathClient).load({ ...config, manifest: inline }), + ).resolves.toMatchObject({ + groupName: 'local', + }); + expect(send).not.toHaveBeenCalled(); + }); + + it.each([ + [{ Parameters: [] }, 'contains no runner configs'], + [{ Parameters: [{ Name: '/groups/group/nested/a', Value: leaf('a', 1) }] }, 'outside the direct group path'], + [{ Parameters: [{ Name: '/groups/group/wrong', Value: leaf('a', 1) }] }, 'must match runnerConfigName'], + [{ Parameters: [{ Name: '/groups/group/a', Value: '{' }] }, 'contains invalid JSON'], + [{ Parameters: [{ Name: undefined, Value: leaf('a', 1) }] }, 'incomplete parameter'], + ])('rejects malformed SSM group pages', async (page, message) => { + await expect( + createControllerManifestLoader({ send: vi.fn().mockResolvedValue(page) }).load(config), + ).rejects.toThrow(message); + }); + + it('rejects repeated pagination tokens', async () => { + const send = vi + .fn() + .mockResolvedValueOnce({ Parameters: [{ Name: '/groups/group/a', Value: leaf('a', 1) }], NextToken: 'same' }) + .mockResolvedValueOnce({ Parameters: [], NextToken: 'same' }); + await expect(createControllerManifestLoader({ send }).load(config)).rejects.toThrow('repeated token'); + }); +}); diff --git a/lambdas/services/scale-set/src/parameter-store.ts b/lambdas/services/scale-set/src/parameter-store.ts new file mode 100644 index 0000000000..3df49818b8 --- /dev/null +++ b/lambdas/services/scale-set/src/parameter-store.ts @@ -0,0 +1,141 @@ +import { GetParametersByPathCommand, PutParameterCommand, SSMClient } from '@aws-sdk/client-ssm'; + +import { getParameters, ssmClient } from '@aws-github-runner/aws-ssm-util'; + +import { + MAX_MANIFEST_BYTES, + SCALE_SET_CONTROLLER_MANIFEST_VERSION, + ScaleSetConfigurationError, + parseScaleSetControllerManifest, + parseScaleSetReconcilerConfig, + validateUniqueReconcilers, + type ScaleSetControllerManifest, + type ScaleSetServiceConfig, +} from './config'; +import type { ParameterStore } from './credentials'; + +const MAX_GROUP_PARAMETERS = 1000; +const MAX_PARAMETER_BYTES = 64 * 1024; +const MAX_GROUP_BYTES = 4 * 1024 * 1024; + +export const defaultParameterStore: ParameterStore = { + get: async (names) => await getParameters([...names]), + put: async (name, value) => { + await ssmClient().send( + new PutParameterCommand({ + Name: name, + Value: value, + Type: 'String', + Overwrite: true, + }), + ); + }, +}; + +export interface ControllerManifestLoader { + load(config: ScaleSetServiceConfig): Promise; +} + +export interface ParametersByPathClient { + send(command: GetParametersByPathCommand): Promise<{ + Parameters?: Array<{ Name?: string; Value?: string }>; + NextToken?: string; + }>; +} + +export function createControllerManifestLoader(client: ParametersByPathClient): ControllerManifestLoader { + return { + load: async (config) => { + if (config.manifest !== undefined) return parseScaleSetControllerManifest(config.manifest); + if (!config.groupConfigPath || !config.groupName || !config.groupRevision) { + throw new ScaleSetConfigurationError('SSM group configuration source is incomplete'); + } + const prefix = `${config.groupConfigPath.replace(/\/$/, '')}/`; + const parameters: Array<{ name: string; value: string }> = []; + const seenTokens = new Set(); + let nextToken: string | undefined; + let totalBytes = 0; + do { + if (nextToken !== undefined && seenTokens.has(nextToken)) { + throw new ScaleSetConfigurationError('SSM pagination returned a repeated token'); + } + if (nextToken !== undefined) seenTokens.add(nextToken); + const response = await client.send( + new GetParametersByPathCommand({ + Path: config.groupConfigPath, + Recursive: false, + WithDecryption: false, + MaxResults: 10, + ...(nextToken === undefined ? {} : { NextToken: nextToken }), + }), + ); + for (const parameter of response.Parameters ?? []) { + if (!parameter.Name || parameter.Value === undefined) { + throw new ScaleSetConfigurationError('SSM group configuration returned an incomplete parameter'); + } + if (!parameter.Name.startsWith(prefix) || parameter.Name.slice(prefix.length).includes('/')) { + throw new ScaleSetConfigurationError( + 'SSM group configuration returned a parameter outside the direct group path', + ); + } + const size = Buffer.byteLength(parameter.Value, 'utf8'); + if (size > MAX_PARAMETER_BYTES || size > MAX_MANIFEST_BYTES) { + throw new ScaleSetConfigurationError( + `runner config parameter ${JSON.stringify(parameter.Name)} is too large`, + ); + } + totalBytes += size; + if (totalBytes > MAX_GROUP_BYTES) + throw new ScaleSetConfigurationError('SSM controller group configuration is too large'); + parameters.push({ name: parameter.Name, value: parameter.Value }); + if (parameters.length > MAX_GROUP_PARAMETERS) { + throw new ScaleSetConfigurationError(`SSM controller group exceeds ${MAX_GROUP_PARAMETERS} runner configs`); + } + } + nextToken = response.NextToken; + } while (nextToken !== undefined); + + if (parameters.length === 0) + throw new ScaleSetConfigurationError('SSM controller group contains no runner configs'); + parameters.sort((left, right) => left.name.localeCompare(right.name)); + const reconcilers = parameters.map(({ name, value }, index) => { + let parsed: unknown; + try { + parsed = JSON.parse(value) as unknown; + } catch (error) { + throw new ScaleSetConfigurationError( + `runner config parameter ${JSON.stringify(name)} contains invalid JSON`, + { + cause: error, + }, + ); + } + const reconciler = parseScaleSetReconcilerConfig(parsed, index, config.groupName as string, 'ssmRunnerConfigs'); + const leafName = name.slice(prefix.length); + if (leafName !== reconciler.runnerConfigName) { + throw new ScaleSetConfigurationError( + `runner config parameter ${JSON.stringify(name)} must match runnerConfigName ${JSON.stringify(reconciler.runnerConfigName)}`, + ); + } + return reconciler; + }); + validateUniqueReconcilers(reconcilers); + return { + version: SCALE_SET_CONTROLLER_MANIFEST_VERSION, + groupName: config.groupName, + revision: config.groupRevision, + reconcilers, + }; + }, + }; +} + +export function createDefaultControllerManifestLoader(): ControllerManifestLoader { + return createControllerManifestLoader( + new SSMClient({ + region: process.env.AWS_REGION, + maxAttempts: 10, + retryMode: 'adaptive', + }), + ); +} diff --git a/lambdas/services/scale-set/src/reconciler.test.ts b/lambdas/services/scale-set/src/reconciler.test.ts new file mode 100644 index 0000000000..2575816e9e --- /dev/null +++ b/lambdas/services/scale-set/src/reconciler.test.ts @@ -0,0 +1,539 @@ +import { + ScaleSetProtocolError, + type MessageSessionClient, + type RunnerScaleSetMessage, +} from '@aws-github-runner/github-actions-scale-set'; +import type { ScaleSetComputeProvider, ScaleSetReconcileResult } from '@aws-github-runner/compute-providers/scale-set'; +import { describe, expect, it, vi } from 'vitest'; + +import type { ScaleSetReconcilerConfig, ScaleSetServiceConfig } from './config'; +import type { ScaleSetReconcilerStatusReporter } from './health'; +import { + ScaleSetReconciler, + calculateDesiredRunners, + validateProviderResult, + type ScaleSetReconcilerClient, + type ScaleSetReconcilerDependencies, +} from './reconciler'; + +const config: ScaleSetReconcilerConfig = { + schemaVersion: 1, + runnerConfigName: 'linux', + scaleSetId: 42, + scaleSetName: 'linux', + runnerLabels: ['self-hosted', 'linux', 'x64'], + githubConfigUrl: 'https://github.com/example', + githubApp: { + appIdParameterName: '/app/id', + privateKeyParameterName: '/app/key', + installationIdParameterName: '/app/installation', + }, + computeProvider: { type: 'ec2', configuration: {} }, + minRunners: 0, + maxRunners: 10, + bootTimeoutMinutes: 10, + sessionOwner: 'group.linux', + workFolder: '_work', + forceGhes: false, +}; + +const serviceConfig: Pick< + ScaleSetServiceConfig, + 'sessionCloseTimeoutMs' | 'reconnectInitialBackoffMs' | 'reconnectMaxBackoffMs' +> = { sessionCloseTimeoutMs: 100, reconnectInitialBackoffMs: 1, reconnectMaxBackoffMs: 10 }; + +function result(overrides: Partial = {}): ScaleSetReconcileResult { + return { + status: 'converged', + desiredRunners: 1, + currentRunners: 1, + actions: { launched: 0, terminated: 0, retainedBusy: 0, retainedUnknown: 0 }, + errors: [], + ...overrides, + }; +} + +function reporter(): ScaleSetReconcilerStatusReporter { + return { + markSessionReady: vi.fn(), + markProgress: vi.fn(), + markReconnecting: vi.fn(), + markFailed: vi.fn(), + markStopping: vi.fn(), + }; +} + +function message(): RunnerScaleSetMessage { + return { + messageId: 7, + statistics: { + totalAvailableJobs: 1, + totalAcquiredJobs: 0, + totalAssignedJobs: 1, + totalRunningJobs: 0, + totalRegisteredRunners: 1, + totalBusyRunners: 0, + totalIdleRunners: 1, + }, + jobAvailableMessages: [{ runnerRequestId: 99 } as RunnerScaleSetMessage['jobAvailableMessages'][number]], + jobAssignedMessages: [], + jobStartedMessages: [ + { runnerId: 5, runnerName: 'runner-5' } as RunnerScaleSetMessage['jobStartedMessages'][number], + ], + jobCompletedMessages: [], + }; +} + +function fixture(options: { + session: Partial & { session: MessageSessionClient['session'] }; + reconcile?: ScaleSetComputeProvider['reconcile']; +}) { + const computeProvider: ScaleSetComputeProvider = { + reconcile: options.reconcile ?? vi.fn().mockResolvedValue(result()), + }; + const client: ScaleSetReconcilerClient = { + getRunnerScaleSetById: vi.fn().mockResolvedValue({ + id: 42, + name: 'linux', + runnerGroupId: 7, + labels: [{ name: 'linux' }, { name: 'self-hosted' }, { name: 'x64' }], + }), + getRunnerScaleSet: vi.fn().mockResolvedValue({ + id: 42, + name: 'linux', + runnerGroupId: 7, + labels: [{ name: 'linux' }, { name: 'self-hosted' }, { name: 'x64' }], + }), + createRunnerScaleSet: vi.fn().mockResolvedValue({ id: 42, name: 'linux', runnerGroupId: 7 }), + updateRunnerScaleSet: vi.fn().mockResolvedValue({ id: 42, name: 'linux', runnerGroupId: 7 }), + getRunnerGroupByName: vi.fn().mockResolvedValue({ + id: 7, + name: 'runner-group', + size: 0, + isDefaultGroup: false, + }), + createMessageSessionClient: vi.fn().mockResolvedValue(options.session as MessageSessionClient), + generateJitRunnerConfig: vi.fn(), + getRunnerByName: vi.fn(), + removeRunner: vi.fn(), + systemInfo: { scaleSetId: 42 }, + setSystemInfo: vi.fn(), + }; + const dependencies: ScaleSetReconcilerDependencies = { + createAccessTokenProvider: vi.fn().mockResolvedValue(async () => ({ token: 'not-a-real-token' })), + createClient: vi.fn().mockReturnValue(client), + computeProviders: { create: vi.fn().mockReturnValue(computeProvider) }, + logger: { debug: vi.fn(), info: vi.fn(), warn: vi.fn(), error: vi.fn() }, + sleep: vi.fn(async (_delay, signal) => { + if (!signal.aborted) + await new Promise((resolve) => signal.addEventListener('abort', () => resolve(), { once: true })); + }), + random: () => 0, + closeSignal: () => new AbortController().signal, + parameterStore: { get: vi.fn().mockResolvedValue(new Map()), put: vi.fn() }, + createComputeProviderCredentials: vi.fn(), + }; + return { client, computeProvider, dependencies }; +} + +describe('ScaleSetReconciler', () => { + it('resolves the GitHub runner-group and scale-set IDs from their names', async () => { + const abort = new AbortController(); + const session = { + session: { statistics: undefined }, + getMessage: vi.fn().mockResolvedValue(message()), + deleteMessage: vi.fn(), + acquireJobs: vi.fn(), + close: vi.fn(), + }; + const { client, dependencies } = fixture({ + session, + reconcile: vi.fn(async () => { + abort.abort(); + return result(); + }), + }); + vi.mocked(client.getRunnerScaleSet).mockResolvedValue({ id: 42, name: 'linux', runnerGroupId: 7 }); + + await new ScaleSetReconciler( + { + ...config, + scaleSetId: undefined, + runnerGroupName: 'runner-group', + runnerGroupIdParameterName: '/runner/group-id', + scaleSetName: 'linux', + }, + serviceConfig, + dependencies, + ).run(abort.signal, reporter()); + + expect(client.getRunnerGroupByName).toHaveBeenCalledWith('runner-group', { signal: abort.signal }); + expect(client.getRunnerScaleSet).toHaveBeenCalledWith(7, 'linux', { signal: abort.signal }); + expect(client.setSystemInfo).toHaveBeenCalledWith(expect.objectContaining({ scaleSetId: 42 })); + expect(dependencies.parameterStore.put).toHaveBeenCalledWith('/runner/group-id', '7'); + expect(dependencies.logger.info).toHaveBeenCalledWith( + 'scale_set_compute_provider_created', + expect.objectContaining({ computeProviderType: 'ec2' }), + ); + expect(dependencies.logger.info).toHaveBeenCalledWith( + 'scale_set_compute_provider_reconcile_started', + expect.objectContaining({ computeProviderType: 'ec2', desiredRunners: 1, busyRunners: 0 }), + ); + }); + + it('registers a missing scale set in the resolved runner group', async () => { + const abort = new AbortController(); + const session = { + session: { statistics: undefined }, + getMessage: vi.fn().mockResolvedValue(message()), + deleteMessage: vi.fn(), + acquireJobs: vi.fn(), + close: vi.fn(), + }; + const { client, dependencies } = fixture({ + session, + reconcile: vi.fn(async () => { + abort.abort(); + return result(); + }), + }); + vi.mocked(client.getRunnerScaleSet).mockResolvedValueOnce(null); + + await new ScaleSetReconciler( + { ...config, scaleSetId: undefined, runnerGroupName: 'runner-group', scaleSetName: 'linux' }, + serviceConfig, + dependencies, + ).run(abort.signal, reporter()); + + expect(client.createRunnerScaleSet).toHaveBeenCalledWith( + { + name: 'linux', + runnerGroupId: 7, + labels: [ + { name: 'linux', type: 'System' }, + { name: 'self-hosted', type: 'System' }, + { name: 'x64', type: 'System' }, + ], + runnerSetting: {}, + }, + { signal: abort.signal }, + ); + }); + + it('updates labels on an existing scale set when configuration changes', async () => { + const abort = new AbortController(); + const session = { + session: { statistics: undefined }, + getMessage: vi.fn().mockResolvedValue(message()), + deleteMessage: vi.fn(), + acquireJobs: vi.fn(), + close: vi.fn(), + }; + const { client, dependencies } = fixture({ + session, + reconcile: vi.fn(async () => { + abort.abort(); + return result(); + }), + }); + vi.mocked(client.getRunnerScaleSetById) + .mockResolvedValueOnce({ + id: 42, + name: 'linux', + runnerGroupId: 7, + labels: [{ name: 'linux' }, { name: 'self-hosted' }, { name: 'x64' }], + }) + .mockResolvedValueOnce({ id: 42, name: 'linux', runnerGroupId: 7, labels: [{ name: 'linux' }] }); + + await new ScaleSetReconciler(config, serviceConfig, dependencies).run(abort.signal, reporter()); + + expect(client.updateRunnerScaleSet).toHaveBeenCalledWith( + 42, + { + labels: [ + { name: 'linux', type: 'System' }, + { name: 'self-hosted', type: 'System' }, + { name: 'x64', type: 'System' }, + ], + runnerSetting: {}, + }, + { signal: abort.signal }, + ); + expect(dependencies.logger.info).toHaveBeenCalledWith( + 'scale_set_labels_updating', + expect.objectContaining({ currentLabels: ['linux'], desiredLabels: ['linux', 'self-hosted', 'x64'] }), + ); + expect(dependencies.logger.debug).toHaveBeenCalledWith( + 'scale_set_session_scale_set_loaded', + expect.objectContaining({ + scaleSetLabels: [ + { name: 'linux', type: 'System' }, + { name: 'self-hosted', type: 'System' }, + { name: 'x64', type: 'System' }, + ], + }), + ); + }); + + it('acknowledges before acquisition, lifecycle observation, and reconciliation', async () => { + const order: string[] = []; + const abort = new AbortController(); + const session = { + session: { statistics: undefined }, + getMessage: vi.fn().mockResolvedValue(message()), + acquireJobs: vi.fn(async () => { + order.push('acquire'); + return [99]; + }), + deleteMessage: vi.fn(async () => { + order.push('delete'); + }), + close: vi.fn(), + }; + const reconcile = vi.fn(async (request) => { + order.push('reconcile'); + expect(request.busyRunners).toBe(0); + expect(request.bootTimeoutMinutes).toBe(10); + expect(request.runnerStates).toContainEqual( + expect.objectContaining({ runnerId: 5, runnerName: 'runner-5', lifecycle: 'started' }), + ); + abort.abort(); + return result(); + }); + const { dependencies } = fixture({ session, reconcile }); + await new ScaleSetReconciler(config, serviceConfig, dependencies).run(abort.signal, reporter()); + expect(order).toEqual(['delete', 'acquire', 'reconcile']); + expect(session.deleteMessage).toHaveBeenCalledWith(7, { signal: abort.signal }); + expect(dependencies.computeProviders.create).toHaveBeenCalledWith('ec2', { + runnerConfigName: 'linux', + scaleSetId: 42, + githubScope: 'https://github.com/example', + configuration: {}, + }); + }); + + it('reconnects and retries when reconciliation rejects', async () => { + const abort = new AbortController(); + const session = { + session: { statistics: undefined }, + getMessage: vi.fn().mockResolvedValue(message()), + acquireJobs: vi.fn().mockResolvedValue([99]), + deleteMessage: vi.fn(), + close: vi.fn(), + }; + const reconcile = vi + .fn() + .mockRejectedValueOnce(new Error('provider failed')) + .mockImplementationOnce(async () => { + abort.abort(); + return result(); + }); + const { client, dependencies } = fixture({ session, reconcile }); + dependencies.sleep = vi.fn().mockResolvedValue(undefined); + const status = reporter(); + + await new ScaleSetReconciler(config, serviceConfig, dependencies).run(abort.signal, status); + + expect(client.createMessageSessionClient).toHaveBeenCalledTimes(2); + expect(session.close).toHaveBeenCalledTimes(2); + expect(reconcile).toHaveBeenCalledTimes(2); + expect(status.markReconnecting).toHaveBeenCalledWith(expect.any(Error)); + expect(status.markFailed).not.toHaveBeenCalled(); + expect(dependencies.sleep).toHaveBeenCalledOnce(); + }); + + it('does not process a message when acknowledgement fails', async () => { + const abort = new AbortController(); + const reconcile = vi.fn(); + const session = { + session: { statistics: undefined }, + getMessage: vi.fn().mockResolvedValue(message()), + acquireJobs: vi.fn(), + deleteMessage: vi.fn().mockRejectedValue(new Error('acknowledgement failed')), + close: vi.fn(), + }; + const { dependencies } = fixture({ session, reconcile }); + dependencies.sleep = vi.fn(async () => abort.abort()); + const status = reporter(); + + await new ScaleSetReconciler(config, serviceConfig, dependencies).run(abort.signal, status); + + expect(session.deleteMessage).toHaveBeenCalledOnce(); + expect(session.acquireJobs).not.toHaveBeenCalled(); + expect(reconcile).not.toHaveBeenCalled(); + expect(status.markReconnecting).toHaveBeenCalledOnce(); + }); + + it('reconnects after a scale-set protocol error', async () => { + const abort = new AbortController(); + const session = { + session: { statistics: undefined }, + getMessage: vi.fn().mockResolvedValue(message()), + acquireJobs: vi.fn().mockResolvedValue([99]), + deleteMessage: vi.fn(), + close: vi.fn(), + }; + const reconcile = vi.fn(async () => { + abort.abort(); + return result(); + }); + const { client, dependencies } = fixture({ session, reconcile }); + dependencies.sleep = vi.fn().mockResolvedValue(undefined); + vi.mocked(client.createMessageSessionClient).mockRejectedValueOnce(new ScaleSetProtocolError('invalid session')); + vi.mocked(client.createMessageSessionClient).mockResolvedValueOnce(session); + const status = reporter(); + + await new ScaleSetReconciler(config, serviceConfig, dependencies).run(abort.signal, status); + + expect(client.createMessageSessionClient).toHaveBeenCalledTimes(2); + expect(dependencies.logger.warn).toHaveBeenCalledWith( + 'scale_set_reconciler_reconnecting', + expect.objectContaining({ + error: expect.objectContaining({ name: 'ScaleSetProtocolError', message: 'invalid session' }), + }), + ); + expect(status.markFailed).not.toHaveBeenCalled(); + expect(status.markReconnecting).toHaveBeenCalledWith(expect.any(ScaleSetProtocolError)); + expect(dependencies.logger.info).not.toHaveBeenCalledWith('scale_set_reconciler_retry_stopped', expect.anything()); + }); + + it('reconnects and retries when the provider returns an error result', async () => { + const abort = new AbortController(); + const order: string[] = []; + const session = { + session: { statistics: undefined }, + getMessage: vi.fn().mockResolvedValue(message()), + acquireJobs: vi.fn().mockResolvedValue([99]), + deleteMessage: vi.fn(async () => { + order.push('delete'); + }), + close: vi.fn(), + }; + const reconcile = vi + .fn() + .mockImplementationOnce(async () => { + order.push('reconcile'); + return result({ + status: 'error', + currentRunners: 0, + errors: [{ operation: 'launch', code: 'ThrottlingException' }], + }); + }) + .mockImplementationOnce(async () => { + order.push('reconcile'); + abort.abort(); + return result(); + }); + const { dependencies } = fixture({ session, reconcile }); + dependencies.sleep = vi.fn().mockResolvedValue(undefined); + const status = reporter(); + + await new ScaleSetReconciler(config, serviceConfig, dependencies).run(abort.signal, status); + + expect(order).toEqual(['delete', 'reconcile', 'delete', 'reconcile']); + expect(reconcile).toHaveBeenCalledTimes(2); + expect(status.markReconnecting).toHaveBeenCalledOnce(); + expect(status.markFailed).not.toHaveBeenCalled(); + expect(dependencies.sleep).toHaveBeenCalledOnce(); + }); + + it('uses the Actions-service identity check without public runner verification', async () => { + const abort = new AbortController(); + const order: string[] = []; + const session = { + session: { statistics: undefined }, + getMessage: vi.fn().mockResolvedValue(message()), + acquireJobs: vi.fn(async () => { + order.push('acquire'); + return [99]; + }), + deleteMessage: vi.fn(async () => { + order.push('delete'); + }), + close: vi.fn(), + }; + const reconcile = vi.fn(async (request) => { + order.push('reconcile'); + await expect(request.removeRunner({ runnerId: 5, runnerName: 'runner-5', scaleSetId: 42 })).resolves.toEqual({ + status: 'removed', + }); + abort.abort(); + return result({ + status: 'retained', + currentRunners: 2, + actions: { launched: 0, terminated: 0, retainedBusy: 1, retainedUnknown: 0 }, + }); + }); + const { client, dependencies } = fixture({ session, reconcile }); + vi.mocked(client.getRunnerByName).mockImplementation(async () => { + order.push('actions-refetch'); + return { id: 5, name: 'runner-5', runnerScaleSetId: 42 }; + }); + await new ScaleSetReconciler(config, serviceConfig, dependencies).run(abort.signal, reporter()); + + expect(order).toEqual(['delete', 'acquire', 'reconcile', 'actions-refetch']); + expect(client.removeRunner).toHaveBeenCalledWith(5, { signal: abort.signal }); + expect(session.deleteMessage).toHaveBeenCalledOnce(); + expect(session.acquireJobs).toHaveBeenCalledTimes(1); + }); + + it('bounds the lifecycle cache per reconciler', () => { + const { dependencies } = fixture({ session: { session: {}, close: vi.fn() } }); + const reconciler = new ScaleSetReconciler(config, serviceConfig, dependencies) as unknown as { + rememberLifecycle(id: number, name: string, lifecycle: 'started'): void; + lifecycle: Map; + resolvedScaleSetId: number; + }; + reconciler.resolvedScaleSetId = 42; + for (let index = 0; index < 1100; index += 1) reconciler.rememberLifecycle(index + 1, `runner-${index}`, 'started'); + expect(reconciler.lifecycle.size).toBe(1000); + expect(reconciler.lifecycle.has('runner-0')).toBe(false); + }); +}); + +describe('reconciler helpers', () => { + it('calculates bounded desired capacity', () => { + expect(calculateDesiredRunners(5, 2, 6)).toBe(6); + expect(calculateDesiredRunners(8, 2, 5)).toBe(8); + expect(() => calculateDesiredRunners(-1, 0, 1)).toThrow('non-negative integer'); + }); + + it.each([ + { status: 'unexpected' }, + { status: 'retryable_error' }, + { status: 'non_retryable_error' }, + { retryable: true }, + { actions: { launched: 0, terminated: 0, retainedBusy: -1, retainedUnknown: 0 } }, + { actions: { launched: 0, terminated: 0, retainedBusy: 0, retainedUnknown: 0, retryable: true } }, + { status: 'converged', errors: [{ operation: 'list', code: 'UNEXPECTED_ERROR' }] }, + { status: 'retained', errors: [{ operation: 'list', code: 'UNEXPECTED_ERROR' }] }, + { status: 'error', errors: [] }, + { currentRunners: 0 }, + { currentRunners: 2 }, + { status: 'retained' }, + { errors: [{ operation: 'shell', code: 'BAD' }] }, + { errors: [{ operation: 'list', code: 'contains spaces' }] }, + { errors: [{ operation: 'list', code: 'BAD!CODE' }] }, + { errors: [{ operation: 'list', code: 'BAD\nCODE' }] }, + { errors: [{ operation: 'list', code: 'BAD', retryable: true }] }, + ])('rejects malformed compute-provider result metadata: %o', (overrides) => { + expect(() => validateProviderResult({ ...result(), ...overrides } as ScaleSetReconcileResult, 1)).toThrow( + /scale-set compute provider returned (?:an? )?invalid/, + ); + }); + + it('accepts bounded provider and AWS error codes', () => { + expect(() => + validateProviderResult( + result({ + status: 'error', + errors: [ + { operation: 'list', code: 'AccessDeniedException' }, + { operation: 'launch', code: 'ThrottlingException' }, + ], + }), + 1, + ), + ).not.toThrow(); + }); +}); diff --git a/lambdas/services/scale-set/src/reconciler.ts b/lambdas/services/scale-set/src/reconciler.ts new file mode 100644 index 0000000000..05163f92b8 --- /dev/null +++ b/lambdas/services/scale-set/src/reconciler.ts @@ -0,0 +1,743 @@ +import { + GitHubActionsScaleSetClient, + isScaleSetHttpError, + ScaleSetProtocolError, + type AccessToken, + type MessageSessionClient, + type RunnerScaleSet, + type RunnerScaleSetMessage, + type RunnerScaleSetStatistic, +} from '@aws-github-runner/github-actions-scale-set'; +import type { + ScaleSetComputeProvider, + ScaleSetComputeProviderCredentialProvider, + ScaleSetComputeProviderFactoryInput, + ScaleSetReconcileRequest, + ScaleSetReconcileResult, + ScaleSetRunnerLifecycle, + ScaleSetRunnerState, +} from '@aws-github-runner/compute-providers/scale-set'; + +import { ScaleSetConfigurationError, type ScaleSetReconcilerConfig, type ScaleSetServiceConfig } from './config'; +import type { ParameterStore } from './credentials'; +import type { ScaleSetReconcilerStatusReporter } from './health'; +import type { ScaleSetLogger } from './logger'; + +const MAX_JIT_CONFIGURATION_BYTES = 1024 * 1024; + +function uniqueLabelNames(labelNames: readonly string[]): string[] { + return [...new Set(labelNames)]; +} + +function normalizedScaleSetLabelNames(labels: RunnerScaleSet['labels']): string[] { + return [...new Set((labels ?? []).map(({ name }) => name))].sort(); +} + +function desiredScaleSetLabelNames(config: ScaleSetReconcilerConfig): string[] { + return uniqueLabelNames([config.scaleSetName, ...config.runnerLabels]); +} + +function desiredScaleSetLabels(config: ScaleSetReconcilerConfig): Array<{ name: string; type: 'System' }> { + return desiredScaleSetLabelNames(config).map((name) => ({ name, type: 'System' })); +} + +export interface ScaleSetComputeProviderFactory { + create(type: string, input: ScaleSetComputeProviderFactoryInput): ScaleSetComputeProvider; +} + +export type ScaleSetReconcilerClient = Pick< + GitHubActionsScaleSetClient, + | 'createMessageSessionClient' + | 'generateJitRunnerConfig' + | 'getRunnerGroupByName' + | 'getRunnerScaleSet' + | 'getRunnerScaleSetById' + | 'getRunnerByName' + | 'removeRunner' + | 'createRunnerScaleSet' + | 'updateRunnerScaleSet' + | 'setSystemInfo' + | 'systemInfo' +>; + +export interface ScaleSetReconcilerDependencies { + createAccessTokenProvider(config: ScaleSetReconcilerConfig): Promise<() => Promise>; + createClient(config: ScaleSetReconcilerConfig, provider: () => Promise): ScaleSetReconcilerClient; + computeProviders: ScaleSetComputeProviderFactory; + logger: ScaleSetLogger; + sleep(delayMs: number, signal: AbortSignal): Promise; + random(): number; + closeSignal(timeoutMs: number): AbortSignal; + parameterStore: ParameterStore; + createComputeProviderCredentials(roleArn?: string): ScaleSetComputeProviderCredentialProvider | undefined; +} + +interface LifecycleObservation { + runnerId: number; + runnerName: string; + scaleSetId: number; + lifecycle: ScaleSetRunnerLifecycle; +} + +export class ScaleSetProviderReconciliationError extends Error { + constructor( + readonly result?: ScaleSetReconcileResult, + options?: ErrorOptions, + ) { + super( + result === undefined + ? 'scale-set compute provider reconciliation failed' + : `scale-set compute provider returned ${result.status}`, + options, + ); + this.name = 'ScaleSetProviderReconciliationError'; + } +} + +export class ScaleSetReconciler { + private readonly lifecycle = new Map(); + private readonly lifecycleLimit: number; + private resolvedScaleSetId?: number; + private resolvedRunnerGroupId?: number; + + constructor( + private readonly config: ScaleSetReconcilerConfig, + private readonly serviceConfig: Pick< + ScaleSetServiceConfig, + 'sessionCloseTimeoutMs' | 'reconnectInitialBackoffMs' | 'reconnectMaxBackoffMs' + >, + private readonly dependencies: ScaleSetReconcilerDependencies, + ) { + this.lifecycleLimit = Math.min(20_000, Math.max(1000, config.maxRunners * 4)); + } + + async run(signal: AbortSignal, status: ScaleSetReconcilerStatusReporter): Promise { + let provider: ScaleSetComputeProvider; + let client: ScaleSetReconcilerClient; + try { + const accessTokenProvider = await this.dependencies.createAccessTokenProvider(this.config); + client = this.dependencies.createClient(this.config, accessTokenProvider); + const resolved = await this.resolveScaleSet(client, signal); + this.resolvedScaleSetId = resolved.scaleSetId; + this.resolvedRunnerGroupId = resolved.runnerGroupId; + client.setSystemInfo({ ...client.systemInfo, scaleSetId: resolved.scaleSetId }); + const credentials = this.dependencies.createComputeProviderCredentials(this.config.computeProvider.roleArn); + this.log('debug', 'scale_set_compute_provider_loading', { + computeProviderType: this.config.computeProvider.type, + }); + provider = this.dependencies.computeProviders.create(this.config.computeProvider.type, { + runnerConfigName: this.config.runnerConfigName, + scaleSetId: resolved.scaleSetId, + githubScope: this.config.githubConfigUrl, + ...(credentials === undefined ? {} : { credentials }), + configuration: this.config.computeProvider.configuration, + }); + this.log('info', 'scale_set_compute_provider_created', { + computeProviderType: this.config.computeProvider.type, + }); + this.log('debug', 'scale_set_compute_provider_loaded', { + computeProviderType: this.config.computeProvider.type, + }); + } catch (error) { + status.markFailed(error); + this.log('error', 'scale_set_reconciler_initialization_failed', { + computeProviderType: this.config.computeProvider.type, + ...httpErrorLogAttributes(error), + error: errorLogAttributes(error), + }); + return; + } + + let consecutiveFailures = 0; + while (!signal.aborted) { + let session: MessageSessionClient | undefined; + let madeProgress = false; + try { + const configuredScaleSet = await client.getRunnerScaleSetById(this.scaleSetId, { signal }); + if ( + configuredScaleSet === null || + configuredScaleSet.id !== this.scaleSetId || + configuredScaleSet.name !== this.config.scaleSetName || + (this.resolvedRunnerGroupId !== undefined && configuredScaleSet.runnerGroupId !== this.resolvedRunnerGroupId) + ) { + throw new ScaleSetConfigurationError('configured GitHub runner scale set identity does not match'); + } + const reconciledScaleSet = await this.reconcileScaleSetLabels(client, configuredScaleSet, signal); + session = await client.createMessageSessionClient(this.scaleSetId, this.config.sessionOwner, { signal }); + status.markSessionReady(); + this.log('info', 'scale_set_session_created'); + this.log('debug', 'scale_set_session_scale_set_loaded', { + scaleSetName: reconciledScaleSet.name, + scaleSetLabels: reconciledScaleSet.labels?.map(({ name, type }) => ({ name, type })), + }); + let latestStatistics = session.session.statistics ?? undefined; + let lastMessageId = 0; + if (latestStatistics !== undefined) { + await this.reconcile(client, provider, latestStatistics, signal); + madeProgress = true; + consecutiveFailures = 0; + status.markProgress(); + } + + while (!signal.aborted) { + const message = await session.getMessage(lastMessageId, this.config.maxRunners, { signal }); + if (message === null) { + if (latestStatistics === undefined) { + throw new ScaleSetProtocolError('message session returned no message and no statistics snapshot'); + } + await this.reconcile(client, provider, latestStatistics, signal); + } else { + if (message.statistics === null) { + throw new ScaleSetProtocolError(`scale-set message ${message.messageId} contains no statistics`); + } + latestStatistics = message.statistics; + lastMessageId = message.messageId; + await session.deleteMessage(message.messageId, { signal }); + const requestIds = uniqueRequestIds(message); + if (requestIds.length > 0) await session.acquireJobs(requestIds, { signal }); + this.observeLifecycle(message); + await this.reconcile(client, provider, latestStatistics, signal); + this.pruneCompletedLifecycle(message); + } + madeProgress = true; + consecutiveFailures = 0; + status.markProgress(); + } + } catch (error) { + if (signal.aborted) break; + if (isFatalReconcilerError(error)) { + status.markFailed(error); + this.log('info', 'scale_set_reconciler_retry_stopped', { + retryable: false, + reason: 'fatal_error', + error: errorLogAttributes(error), + }); + this.log('error', 'scale_set_reconciler_failed', { + ...httpErrorLogAttributes(error), + error: errorLogAttributes(error), + }); + return; + } + consecutiveFailures = madeProgress ? 1 : consecutiveFailures + 1; + status.markReconnecting(error); + this.log('warn', 'scale_set_reconciler_reconnecting', { consecutiveFailures, error }); + } finally { + if (session !== undefined) await this.closeSession(session); + } + + if (!signal.aborted) { + await this.dependencies.sleep( + calculateReconnectDelay( + consecutiveFailures, + this.serviceConfig.reconnectInitialBackoffMs, + this.serviceConfig.reconnectMaxBackoffMs, + this.dependencies.random, + ), + signal, + ); + } + } + status.markStopping(); + } + + private async resolveScaleSet( + client: ScaleSetReconcilerClient, + signal: AbortSignal, + ): Promise<{ scaleSetId: number; runnerGroupId?: number }> { + let runnerGroupId = this.config.expectedRunnerGroupId; + if (this.config.runnerGroupName !== undefined) { + const cachedRunnerGroupId = await this.loadCachedRunnerGroupId(); + const runnerGroup = + cachedRunnerGroupId === undefined + ? await client.getRunnerGroupByName(this.config.runnerGroupName, { signal }) + : { id: cachedRunnerGroupId }; + if (runnerGroupId !== undefined && runnerGroupId !== runnerGroup.id) { + throw new ScaleSetConfigurationError( + `runner group ${JSON.stringify(this.config.runnerGroupName)} resolved to ID ${runnerGroup.id}, expected ${runnerGroupId}`, + ); + } + runnerGroupId = runnerGroup.id; + if (cachedRunnerGroupId === undefined && this.config.runnerGroupIdParameterName !== undefined) { + await this.dependencies.parameterStore.put?.(this.config.runnerGroupIdParameterName, String(runnerGroupId)); + } + this.log('info', 'scale_set_runner_group_resolved', { + runnerConfigName: this.config.runnerConfigName, + runnerGroupName: this.config.runnerGroupName, + runnerGroupId, + }); + } + + if (this.config.scaleSetId === undefined && runnerGroupId === undefined) { + throw new ScaleSetConfigurationError('runner group ID was not resolved'); + } + let configuredScaleSet = + this.config.scaleSetId === undefined + ? await client.getRunnerScaleSet(runnerGroupId as number, this.config.scaleSetName, { signal }) + : await client.getRunnerScaleSetById(this.config.scaleSetId, { signal }); + if (configuredScaleSet === null && runnerGroupId !== undefined && this.config.scaleSetId === undefined) { + this.log('info', 'scale_set_registering', { + runnerConfigName: this.config.runnerConfigName, + scaleSetName: this.config.scaleSetName, + runnerGroupId, + }); + try { + configuredScaleSet = await client.createRunnerScaleSet( + { + name: this.config.scaleSetName, + runnerGroupId, + labels: desiredScaleSetLabels(this.config), + runnerSetting: {}, + }, + { signal }, + ); + } catch (error) { + const existingScaleSet = await client.getRunnerScaleSet(runnerGroupId, this.config.scaleSetName, { signal }); + if (existingScaleSet === null) throw error; + configuredScaleSet = existingScaleSet; + } + } + if (configuredScaleSet === null || configuredScaleSet.id === undefined) { + throw new ScaleSetConfigurationError( + `GitHub runner scale set ${JSON.stringify(this.config.scaleSetName)} was not found`, + ); + } + if ( + configuredScaleSet.name !== this.config.scaleSetName || + (runnerGroupId !== undefined && configuredScaleSet.runnerGroupId !== runnerGroupId) || + (this.config.scaleSetId !== undefined && configuredScaleSet.id !== this.config.scaleSetId) + ) { + throw new ScaleSetConfigurationError('configured GitHub runner scale set identity does not match'); + } + this.log('info', 'scale_set_resolved', { + runnerConfigName: this.config.runnerConfigName, + scaleSetName: this.config.scaleSetName, + scaleSetId: configuredScaleSet.id, + runnerGroupId, + }); + this.log('debug', 'scale_set_labels_resolved', { + scaleSetName: configuredScaleSet.name, + scaleSetLabels: configuredScaleSet.labels?.map(({ name, type }) => ({ name, type })), + }); + return { scaleSetId: configuredScaleSet.id, runnerGroupId }; + } + + private async reconcileScaleSetLabels( + client: ScaleSetReconcilerClient, + configuredScaleSet: RunnerScaleSet, + signal: AbortSignal, + ): Promise { + const desiredLabels = desiredScaleSetLabelNames(this.config); + const desiredLabelsWithTypes = desiredScaleSetLabels(this.config); + const currentLabels = normalizedScaleSetLabelNames(configuredScaleSet.labels); + if (currentLabels.join('\u0000') === [...desiredLabels].sort().join('\u0000')) return configuredScaleSet; + + this.log('info', 'scale_set_labels_updating', { + currentLabels, + desiredLabels, + }); + await client.updateRunnerScaleSet( + this.scaleSetId, + { + labels: desiredLabelsWithTypes, + runnerSetting: configuredScaleSet.runnerSetting ?? {}, + }, + { signal }, + ); + this.log('info', 'scale_set_labels_updated', { + scaleSetName: configuredScaleSet.name, + scaleSetLabels: desiredLabels, + }); + return { ...configuredScaleSet, labels: desiredLabelsWithTypes }; + } + + private async loadCachedRunnerGroupId(): Promise { + const parameterName = this.config.runnerGroupIdParameterName; + if (parameterName === undefined) return undefined; + const values = await this.dependencies.parameterStore.get([parameterName]); + const raw = values.get(parameterName)?.trim(); + if (raw === undefined || raw === '') return undefined; + if (!/^\d+$/.test(raw)) { + throw new ScaleSetConfigurationError( + `runner group ID parameter ${JSON.stringify(parameterName)} must contain a positive integer`, + ); + } + const id = Number(raw); + if (!Number.isSafeInteger(id) || id <= 0) { + throw new ScaleSetConfigurationError( + `runner group ID parameter ${JSON.stringify(parameterName)} must contain a positive integer`, + ); + } + return id; + } + + private get scaleSetId(): number { + if (this.resolvedScaleSetId === undefined) { + throw new ScaleSetConfigurationError('scale set ID was not resolved'); + } + return this.resolvedScaleSetId; + } + + private async reconcile( + client: ScaleSetReconcilerClient, + provider: ScaleSetComputeProvider, + statistics: RunnerScaleSetStatistic, + signal: AbortSignal, + ): Promise { + const desiredRunners = calculateDesiredRunners( + statistics.totalAssignedJobs, + this.config.minRunners, + this.config.maxRunners, + ); + const callbacks = this.createReconcileCallbacks(client, signal); + const result = await this.reconcileProvider(provider, { + desiredRunners, + busyRunners: statistics.totalBusyRunners, + bootTimeoutMinutes: this.config.bootTimeoutMinutes, + runnerStates: this.lifecycleStates(), + ...callbacks, + }); + validateProviderResult(result, desiredRunners); + this.logReconciliationResult(result, desiredRunners); + throwIfProviderError(result); + } + + private createReconcileCallbacks(client: ScaleSetReconcilerClient, signal: AbortSignal) { + return { + signal, + generateJitConfiguration: async ({ + runnerName, + signal: callbackSignal, + }: { + runnerName: string; + signal?: AbortSignal; + }) => { + const jit = await client.generateJitRunnerConfig( + { name: runnerName, workFolder: this.config.workFolder }, + this.scaleSetId, + { signal: callbackSignal ?? signal }, + ); + if ( + jit.runner === null || + jit.runner.name !== runnerName || + jit.runner.runnerScaleSetId !== this.scaleSetId || + !Number.isSafeInteger(jit.runner.id) || + jit.runner.id <= 0 + ) { + throw new ScaleSetProtocolError('GitHub returned a mismatched runner identity for JIT configuration'); + } + if ( + typeof jit.encodedJITConfig !== 'string' || + jit.encodedJITConfig === '' || + Buffer.byteLength(jit.encodedJITConfig, 'utf8') > MAX_JIT_CONFIGURATION_BYTES + ) { + throw new ScaleSetProtocolError('GitHub returned an invalid JIT configuration'); + } + return { + encodedJitConfiguration: jit.encodedJITConfig, + runnerId: jit.runner.id, + runnerName: jit.runner.name, + scaleSetId: jit.runner.runnerScaleSetId, + }; + }, + removeRunner: async (expected: { + runnerId: number; + runnerName: string; + scaleSetId: number; + signal?: AbortSignal; + }) => { + const callbackSignal = expected.signal ?? signal; + const runner = await client.getRunnerByName(expected.runnerName, { signal: callbackSignal }); + if (runner === null) return { status: 'retained_unknown' as const }; + if ( + runner.id !== expected.runnerId || + runner.name !== expected.runnerName || + runner.runnerScaleSetId !== expected.scaleSetId || + expected.scaleSetId !== this.scaleSetId + ) { + return { status: 'retained_unknown' as const }; + } + // Busy state comes from the aggregate scale-set statistics. The + // Actions-service runner reference above remains the exact identity + // check; no public GitHub REST runner call is required. + try { + await client.removeRunner(runner.id, { signal: callbackSignal }); + } catch (error) { + if (isScaleSetHttpError(error) && error.status === 404) return { status: 'removed' as const }; + throw error; + } + return { status: 'removed' as const }; + }, + }; + } + + private logReconciliationResult(result: ScaleSetReconcileResult, desiredRunners: number): void { + this.log('info', 'scale_set_reconciled', { + computeProviderType: this.config.computeProvider.type, + desiredRunners, + currentRunners: result.currentRunners, + status: result.status, + actions: result.actions, + errorCount: result.errors.length, + errors: result.errors, + }); + if (result.status === 'retained') { + this.log('warn', 'scale_set_capacity_retained', { + computeProviderType: this.config.computeProvider.type, + desiredRunners, + currentRunners: result.currentRunners, + retainedBusy: result.actions.retainedBusy, + retainedUnknown: result.actions.retainedUnknown, + }); + } + } + + private async reconcileProvider( + provider: ScaleSetComputeProvider, + request: ScaleSetReconcileRequest, + ): Promise { + this.log('info', 'scale_set_compute_provider_reconcile_started', { + computeProviderType: this.config.computeProvider.type, + desiredRunners: request.desiredRunners, + busyRunners: request.busyRunners, + }); + try { + return await provider.reconcile(request); + } catch (error) { + request.signal.throwIfAborted(); + this.log('error', 'scale_set_compute_provider_reconcile_failed', { + computeProviderType: this.config.computeProvider.type, + desiredRunners: request.desiredRunners, + busyRunners: request.busyRunners, + error: errorLogAttributes(error), + }); + throw new ScaleSetProviderReconciliationError(undefined, { cause: error }); + } + } + + private observeLifecycle(message: RunnerScaleSetMessage): void { + for (const runner of message.jobStartedMessages) { + if (runner.runnerId !== undefined && runner.runnerName !== undefined) + this.rememberLifecycle(runner.runnerId, runner.runnerName, 'started'); + } + for (const runner of message.jobCompletedMessages) { + if (runner.runnerId !== undefined && runner.runnerName !== undefined) + this.rememberLifecycle(runner.runnerId, runner.runnerName, 'completed'); + } + } + + private rememberLifecycle(runnerId: number, runnerName: string, lifecycle: ScaleSetRunnerLifecycle): void { + if (!Number.isSafeInteger(runnerId) || runnerId <= 0 || runnerName === '') return; + const current = this.lifecycle.get(runnerName); + if (current !== undefined && current.runnerId !== runnerId) { + this.lifecycle.delete(runnerName); + return; + } + this.lifecycle.set(runnerName, { runnerId, runnerName, scaleSetId: this.scaleSetId, lifecycle }); + while (this.lifecycle.size > this.lifecycleLimit) { + const oldest = this.lifecycle.keys().next().value as string | undefined; + if (oldest === undefined) break; + this.lifecycle.delete(oldest); + } + } + + private pruneCompletedLifecycle(message: RunnerScaleSetMessage): void { + for (const runner of message.jobCompletedMessages) { + if (runner.runnerId === undefined || runner.runnerName === undefined) continue; + const observation = this.lifecycle.get(runner.runnerName); + if (observation?.runnerId === runner.runnerId && observation.lifecycle === 'completed') { + this.lifecycle.delete(runner.runnerName); + } + } + } + + private lifecycleStates(): ScaleSetRunnerState[] { + return [...this.lifecycle.values()].map((observation) => ({ + ...observation, + status: 'unknown', + busy: undefined, + })); + } + + private async closeSession(session: Pick): Promise { + try { + await session.close({ signal: this.dependencies.closeSignal(this.serviceConfig.sessionCloseTimeoutMs) }); + } catch (error) { + this.log('warn', 'scale_set_session_close_failed', { error }); + } + } + + private log( + level: 'debug' | 'info' | 'warn' | 'error', + event: string, + attributes: Record = {}, + ): void { + this.dependencies.logger[level](event, { + groupRunnerConfig: this.config.runnerConfigName, + scaleSetId: this.resolvedScaleSetId, + ...attributes, + }); + } +} + +export function calculateDesiredRunners(totalAssignedJobs: number, minRunners: number, maxRunners: number): number { + if (!Number.isSafeInteger(totalAssignedJobs) || totalAssignedJobs < 0) { + throw new ScaleSetProtocolError('statistics.totalAssignedJobs must be a non-negative integer'); + } + // maxRunners bounds newly requested idle capacity, but an operator reducing + // it must never make already-assigned work a scale-down target. + return Math.max(totalAssignedJobs, Math.min(maxRunners, minRunners + totalAssignedJobs)); +} + +export function calculateReconnectDelay( + attempt: number, + initialBackoffMs: number, + maxBackoffMs: number, + random: () => number = Math.random, +): number { + if (!Number.isSafeInteger(attempt) || attempt <= 0) throw new Error('attempt must be a positive integer'); + const ceiling = Math.min(maxBackoffMs, initialBackoffMs * 2 ** Math.min(attempt - 1, 30)); + const value = Math.max(0, Math.min(1, random())); + return Math.floor(ceiling / 2 + (ceiling / 2) * value); +} + +export async function abortableSleep(delayMs: number, signal: AbortSignal): Promise { + if (signal.aborted || delayMs <= 0) return; + await new Promise((resolve) => { + const done = () => { + clearTimeout(timeout); + signal.removeEventListener('abort', done); + resolve(); + }; + const timeout = setTimeout(done, delayMs); + signal.addEventListener('abort', done, { once: true }); + }); +} + +function uniqueRequestIds(message: RunnerScaleSetMessage): number[] { + return [...new Set(message.jobAvailableMessages.map(({ runnerRequestId }) => runnerRequestId))]; +} + +export function validateProviderResult(result: ScaleSetReconcileResult, desiredRunners: number): void { + const value = result as unknown; + if (typeof value !== 'object' || value === null || Array.isArray(value)) { + throw new ScaleSetProtocolError('scale-set compute provider returned an invalid reconciliation result'); + } + const record = value as Record; + const resultFields = new Set(['status', 'desiredRunners', 'currentRunners', 'actions', 'errors']); + if (Object.keys(record).some((key) => !resultFields.has(key))) { + throw new ScaleSetProtocolError('scale-set compute provider returned an invalid reconciliation result'); + } + const statuses = new Set(['converged', 'retained', 'error']); + if (!statuses.has(record.status as string)) { + throw new ScaleSetProtocolError('scale-set compute provider returned an invalid status'); + } + if (record.desiredRunners !== desiredRunners || !boundedCount(record.currentRunners)) { + throw new ScaleSetProtocolError('scale-set compute provider returned invalid capacity counts'); + } + const actions = record.actions; + if (typeof actions !== 'object' || actions === null || Array.isArray(actions)) { + throw new ScaleSetProtocolError('scale-set compute provider returned invalid actions'); + } + const actionFields = new Set(['launched', 'terminated', 'retainedBusy', 'retainedUnknown']); + if (Object.keys(actions).some((key) => !actionFields.has(key))) { + throw new ScaleSetProtocolError('scale-set compute provider returned invalid action counts'); + } + for (const key of ['launched', 'terminated', 'retainedBusy', 'retainedUnknown']) { + if (!boundedCount((actions as Record)[key])) { + throw new ScaleSetProtocolError('scale-set compute provider returned invalid action counts'); + } + } + if (!Array.isArray(record.errors) || record.errors.length > 1000) { + throw new ScaleSetProtocolError('scale-set compute provider returned invalid errors'); + } + const operations = new Set([ + 'validate', + 'reconcile', + 'list', + 'launch', + 'generate_jit_configuration', + 'publish_jit_configuration', + 'remove_runner', + 'terminate', + ]); + const errorFields = new Set(['operation', 'code', 'runnerName', 'resourceId']); + for (const error of record.errors) { + if (typeof error !== 'object' || error === null || Array.isArray(error)) { + throw new ScaleSetProtocolError('scale-set compute provider returned invalid error metadata'); + } + const metadata = error as Record; + if ( + Object.keys(metadata).some((key) => !errorFields.has(key)) || + !operations.has(metadata.operation as string) || + typeof metadata.code !== 'string' || + !/^[A-Za-z][A-Za-z0-9._-]{0,127}$/.test(metadata.code) || + !optionalBoundedMetadata(metadata.runnerName) || + !optionalBoundedMetadata(metadata.resourceId) + ) { + throw new ScaleSetProtocolError('scale-set compute provider returned invalid error metadata'); + } + } + if ((record.status === 'error') !== record.errors.length > 0) { + throw new ScaleSetProtocolError('scale-set compute provider returned invalid error status'); + } + let expectedStatus: ScaleSetReconcileResult['status'] = 'converged'; + if (record.errors.length > 0 || (record.currentRunners as number) < desiredRunners) { + expectedStatus = 'error'; + } else if ((record.currentRunners as number) > desiredRunners) { + expectedStatus = 'retained'; + } + if (record.status !== expectedStatus) { + throw new ScaleSetProtocolError('scale-set compute provider returned invalid reconciliation status'); + } +} + +function throwIfProviderError(result: ScaleSetReconcileResult): void { + if (result.status === 'error') { + throw new ScaleSetProviderReconciliationError(result); + } +} + +function boundedCount(value: unknown): value is number { + return Number.isSafeInteger(value) && (value as number) >= 0 && (value as number) <= 2_147_483_647; +} + +function optionalBoundedMetadata(value: unknown): value is string | undefined { + return value === undefined || (typeof value === 'string' && value.length <= 256 && !hasAsciiControlCharacter(value)); +} + +function hasAsciiControlCharacter(value: string): boolean { + return [...value].some((character) => { + const code = character.charCodeAt(0); + return code <= 31 || code === 127; + }); +} + +function isFatalReconcilerError(error: unknown): boolean { + if (error instanceof ScaleSetConfigurationError) return true; + if (!isScaleSetHttpError(error)) return false; + return error.status >= 400 && error.status < 500 && ![408, 409, 425, 429].includes(error.status); +} + +function httpErrorLogAttributes(error: unknown): Record { + if (!isScaleSetHttpError(error)) return {}; + return { + requestMethod: error.method, + requestUrl: error.url, + requestStatus: error.status, + requestCode: error.code, + }; +} + +function errorLogAttributes(error: unknown, depth = 0): Record { + if (!(error instanceof Error)) return { message: String(error) }; + if (depth >= 3) return { name: error.name, message: error.message, cause: '[TRUNCATED]' }; + + const errorWithMetadata = error as Error & { code?: unknown; status?: unknown; cause?: unknown }; + return { + name: error.name, + message: error.message, + ...(typeof errorWithMetadata.code === 'string' ? { code: errorWithMetadata.code } : {}), + ...(typeof errorWithMetadata.status === 'number' ? { status: errorWithMetadata.status } : {}), + ...(errorWithMetadata.cause === undefined ? {} : { cause: errorLogAttributes(errorWithMetadata.cause, depth + 1) }), + }; +} diff --git a/lambdas/services/scale-set/tsconfig.json b/lambdas/services/scale-set/tsconfig.json new file mode 100644 index 0000000000..714aa27b6b --- /dev/null +++ b/lambdas/services/scale-set/tsconfig.json @@ -0,0 +1,8 @@ +{ + "extends": "../../tsconfig.json", + "compilerOptions": { + "types": ["node", "vitest/globals"] + }, + "include": ["src/**/*"], + "exclude": ["src/**/*.test.ts"] +} diff --git a/lambdas/services/scale-set/vitest.config.ts b/lambdas/services/scale-set/vitest.config.ts new file mode 100644 index 0000000000..28a41aa2aa --- /dev/null +++ b/lambdas/services/scale-set/vitest.config.ts @@ -0,0 +1,18 @@ +import { mergeConfig } from 'vitest/config'; +import defaultConfig from '../../vitest.base.config'; + +export default mergeConfig(defaultConfig, { + root: __dirname, + test: { + coverage: { + include: ['src/**/*.ts'], + exclude: ['src/**/*.test.ts', 'src/index.ts', 'src/main.ts'], + thresholds: { + statements: 80, + branches: 70, + functions: 80, + lines: 80, + }, + }, + }, +}); diff --git a/lambdas/yarn.lock b/lambdas/yarn.lock index a703ceaa7a..fcc1230e80 100644 --- a/lambdas/yarn.lock +++ b/lambdas/yarn.lock @@ -139,6 +139,7 @@ __metadata: "@aws-github-runner/aws-ssm-util": "npm:*" "@aws-github-runner/storage-providers": "npm:*" "@aws-sdk/client-ec2": "npm:^3.1009.0" + "@aws-sdk/client-ssm": "npm:^3.1009.0" "@octokit/rest": "npm:22.0.1" aws-sdk-client-mock: "npm:^4.1.0" aws-sdk-client-mock-jest: "npm:^4.1.0" @@ -201,6 +202,33 @@ __metadata: languageName: unknown linkType: soft +"@aws-github-runner/github-actions-scale-set@npm:*, @aws-github-runner/github-actions-scale-set@workspace:libs/github-actions-scale-set": + version: 0.0.0-use.local + resolution: "@aws-github-runner/github-actions-scale-set@workspace:libs/github-actions-scale-set" + dependencies: + "@types/node": "npm:^22.19.3" + typescript: "npm:^5.9.3" + languageName: unknown + linkType: soft + +"@aws-github-runner/scale-set-service@workspace:services/scale-set": + version: 0.0.0-use.local + resolution: "@aws-github-runner/scale-set-service@workspace:services/scale-set" + dependencies: + "@aws-github-runner/aws-ssm-util": "npm:*" + "@aws-github-runner/compute-providers": "npm:*" + "@aws-github-runner/github-actions-scale-set": "npm:*" + "@aws-sdk/client-ssm": "npm:^3.1009.0" + "@aws-sdk/credential-providers": "npm:^3.1009.0" + "@octokit/auth-app": "npm:8.2.0" + "@octokit/request": "npm:^9.2.2" + "@types/node": "npm:^22.19.3" + "@vercel/ncc": "npm:0.38.4" + typescript: "npm:^5.9.3" + undici: "npm:^6.19.2" + languageName: unknown + linkType: soft + "@aws-github-runner/storage-providers@npm:*, @aws-github-runner/storage-providers@workspace:libs/storage-providers": version: 0.0.0-use.local resolution: "@aws-github-runner/storage-providers@workspace:libs/storage-providers" @@ -624,6 +652,22 @@ __metadata: languageName: node linkType: hard +"@aws-sdk/core@npm:^3.977.9": + version: 3.977.9 + resolution: "@aws-sdk/core@npm:3.977.9" + dependencies: + "@aws-sdk/types": "npm:^3.974.5" + "@aws-sdk/xml-builder": "npm:^3.972.40" + "@aws/lambda-invoke-store": "npm:^0.3.0" + "@smithy/core": "npm:^3.33.3" + "@smithy/signature-v4": "npm:^5.6.12" + "@smithy/types": "npm:^4.17.2" + bowser: "npm:^2.11.0" + tslib: "npm:^2.6.2" + checksum: 10c0/9118a8d05b7c27fe55fb5e793840193d1b311b6c0c2958e591bcada391a71783536ed9c8f4913cc9c4b7258dba650d625ed782669a12875348544c33f35aa8d6 + languageName: node + linkType: hard + "@aws-sdk/crc64-nvme@npm:^3.972.5": version: 3.972.5 resolution: "@aws-sdk/crc64-nvme@npm:3.972.5" @@ -634,6 +678,19 @@ __metadata: languageName: node linkType: hard +"@aws-sdk/credential-provider-cognito-identity@npm:^3.972.69": + version: 3.972.69 + resolution: "@aws-sdk/credential-provider-cognito-identity@npm:3.972.69" + dependencies: + "@aws-sdk/nested-clients": "npm:^3.997.44" + "@aws-sdk/types": "npm:^3.974.5" + "@smithy/core": "npm:^3.33.3" + "@smithy/types": "npm:^4.17.2" + tslib: "npm:^2.6.2" + checksum: 10c0/b6e1a120c5e2690619b031a04826c8b9d058ea827ddf016726fcf3e9b7d45e04677dce3478d9683553bc565b6b43dac93842893ebd1b44220f9959370896a2ad + languageName: node + linkType: hard + "@aws-sdk/credential-provider-env@npm:^3.972.21": version: 3.972.21 resolution: "@aws-sdk/credential-provider-env@npm:3.972.21" @@ -647,6 +704,19 @@ __metadata: languageName: node linkType: hard +"@aws-sdk/credential-provider-env@npm:^3.972.70": + version: 3.972.70 + resolution: "@aws-sdk/credential-provider-env@npm:3.972.70" + dependencies: + "@aws-sdk/core": "npm:^3.977.9" + "@aws-sdk/types": "npm:^3.974.5" + "@smithy/core": "npm:^3.33.3" + "@smithy/types": "npm:^4.17.2" + tslib: "npm:^2.6.2" + checksum: 10c0/f5d8f3f1021a83911f617dd6b36277486d0bdb174c107c4f7776a7e2c609a101aa03ada1cca9d97229ac79aea7797ec6392fe449029b87f7cd013360592bddf3 + languageName: node + linkType: hard + "@aws-sdk/credential-provider-http@npm:^3.972.23": version: 3.972.23 resolution: "@aws-sdk/credential-provider-http@npm:3.972.23" @@ -665,6 +735,21 @@ __metadata: languageName: node linkType: hard +"@aws-sdk/credential-provider-http@npm:^3.972.72": + version: 3.972.72 + resolution: "@aws-sdk/credential-provider-http@npm:3.972.72" + dependencies: + "@aws-sdk/core": "npm:^3.977.9" + "@aws-sdk/types": "npm:^3.974.5" + "@smithy/core": "npm:^3.33.3" + "@smithy/fetch-http-handler": "npm:^5.7.2" + "@smithy/node-http-handler": "npm:^4.11.3" + "@smithy/types": "npm:^4.17.2" + tslib: "npm:^2.6.2" + checksum: 10c0/aaa67dc42ce00713d92f931c620e36cb199533e6f1f892a6be76553986c86977442924e76f3743732a7b65e6f6777271789a6edbc70d5a86e0a9b6b5a3ef25f3 + languageName: node + linkType: hard + "@aws-sdk/credential-provider-ini@npm:^3.972.23": version: 3.972.23 resolution: "@aws-sdk/credential-provider-ini@npm:3.972.23" @@ -687,6 +772,27 @@ __metadata: languageName: node linkType: hard +"@aws-sdk/credential-provider-ini@npm:^3.973.15": + version: 3.973.15 + resolution: "@aws-sdk/credential-provider-ini@npm:3.973.15" + dependencies: + "@aws-sdk/core": "npm:^3.977.9" + "@aws-sdk/credential-provider-env": "npm:^3.972.70" + "@aws-sdk/credential-provider-http": "npm:^3.972.72" + "@aws-sdk/credential-provider-login": "npm:^3.972.77" + "@aws-sdk/credential-provider-process": "npm:^3.972.70" + "@aws-sdk/credential-provider-sso": "npm:^3.973.14" + "@aws-sdk/credential-provider-web-identity": "npm:^3.972.76" + "@aws-sdk/nested-clients": "npm:^3.997.44" + "@aws-sdk/types": "npm:^3.974.5" + "@smithy/core": "npm:^3.33.3" + "@smithy/credential-provider-imds": "npm:^4.4.16" + "@smithy/types": "npm:^4.17.2" + tslib: "npm:^2.6.2" + checksum: 10c0/1fe5f8c84cb9877a62084f9cbedebaed605ecc34891c44254d979ccd1282a5e164840d6dbb340773c311c028568f9ea642649bebf4b38cebb2d6d0039ad9de12 + languageName: node + linkType: hard + "@aws-sdk/credential-provider-login@npm:^3.972.23": version: 3.972.23 resolution: "@aws-sdk/credential-provider-login@npm:3.972.23" @@ -703,6 +809,20 @@ __metadata: languageName: node linkType: hard +"@aws-sdk/credential-provider-login@npm:^3.972.77": + version: 3.972.77 + resolution: "@aws-sdk/credential-provider-login@npm:3.972.77" + dependencies: + "@aws-sdk/core": "npm:^3.977.9" + "@aws-sdk/nested-clients": "npm:^3.997.44" + "@aws-sdk/types": "npm:^3.974.5" + "@smithy/core": "npm:^3.33.3" + "@smithy/types": "npm:^4.17.2" + tslib: "npm:^2.6.2" + checksum: 10c0/9be54fdf406325d462abdc0a2a182ba1ed39009fa18b60035705e6c3fae92dfe1afe5dba24a4fa3c8376f7cd2e1dad7a91152909f3b96dde0b215a88dfcd5330 + languageName: node + linkType: hard + "@aws-sdk/credential-provider-node@npm:^3.972.24": version: 3.972.24 resolution: "@aws-sdk/credential-provider-node@npm:3.972.24" @@ -723,6 +843,25 @@ __metadata: languageName: node linkType: hard +"@aws-sdk/credential-provider-node@npm:^3.972.82": + version: 3.972.82 + resolution: "@aws-sdk/credential-provider-node@npm:3.972.82" + dependencies: + "@aws-sdk/credential-provider-env": "npm:^3.972.70" + "@aws-sdk/credential-provider-http": "npm:^3.972.72" + "@aws-sdk/credential-provider-ini": "npm:^3.973.15" + "@aws-sdk/credential-provider-process": "npm:^3.972.70" + "@aws-sdk/credential-provider-sso": "npm:^3.973.14" + "@aws-sdk/credential-provider-web-identity": "npm:^3.972.76" + "@aws-sdk/types": "npm:^3.974.5" + "@smithy/core": "npm:^3.33.3" + "@smithy/credential-provider-imds": "npm:^4.4.16" + "@smithy/types": "npm:^4.17.2" + tslib: "npm:^2.6.2" + checksum: 10c0/c79c2381c1e76fecadb5adf9a8623946d819b20bfc0eaa236a384115027cdea3fe1f0a294c75edbb6aee1a0ccbbc4ec606c2eee8e1c4d720ed2ef281aa1175c8 + languageName: node + linkType: hard + "@aws-sdk/credential-provider-process@npm:^3.972.21": version: 3.972.21 resolution: "@aws-sdk/credential-provider-process@npm:3.972.21" @@ -737,6 +876,19 @@ __metadata: languageName: node linkType: hard +"@aws-sdk/credential-provider-process@npm:^3.972.70": + version: 3.972.70 + resolution: "@aws-sdk/credential-provider-process@npm:3.972.70" + dependencies: + "@aws-sdk/core": "npm:^3.977.9" + "@aws-sdk/types": "npm:^3.974.5" + "@smithy/core": "npm:^3.33.3" + "@smithy/types": "npm:^4.17.2" + tslib: "npm:^2.6.2" + checksum: 10c0/87ea5f575f611461a538312b6c502baf86ce33eef6974bb2f9e85bd32147cb8f6b518e80f5a5d8ef371fceb03d517f4d3b6c07ddeb7f15d5a2f56c6dcbf8e8ba + languageName: node + linkType: hard + "@aws-sdk/credential-provider-sso@npm:^3.972.23": version: 3.972.23 resolution: "@aws-sdk/credential-provider-sso@npm:3.972.23" @@ -753,6 +905,21 @@ __metadata: languageName: node linkType: hard +"@aws-sdk/credential-provider-sso@npm:^3.973.14": + version: 3.973.14 + resolution: "@aws-sdk/credential-provider-sso@npm:3.973.14" + dependencies: + "@aws-sdk/core": "npm:^3.977.9" + "@aws-sdk/nested-clients": "npm:^3.997.44" + "@aws-sdk/token-providers": "npm:3.1116.0" + "@aws-sdk/types": "npm:^3.974.5" + "@smithy/core": "npm:^3.33.3" + "@smithy/types": "npm:^4.17.2" + tslib: "npm:^2.6.2" + checksum: 10c0/12132e57c07277b4c115c835bb7a14b2a62e4f30a69998eca501a171d46be469439b3e7b33c970165362bd261b8689389dd06ba0a959657d97330f3e5172ec52 + languageName: node + linkType: hard + "@aws-sdk/credential-provider-web-identity@npm:^3.972.23": version: 3.972.23 resolution: "@aws-sdk/credential-provider-web-identity@npm:3.972.23" @@ -768,6 +935,44 @@ __metadata: languageName: node linkType: hard +"@aws-sdk/credential-provider-web-identity@npm:^3.972.76": + version: 3.972.76 + resolution: "@aws-sdk/credential-provider-web-identity@npm:3.972.76" + dependencies: + "@aws-sdk/core": "npm:^3.977.9" + "@aws-sdk/nested-clients": "npm:^3.997.44" + "@aws-sdk/types": "npm:^3.974.5" + "@smithy/core": "npm:^3.33.3" + "@smithy/types": "npm:^4.17.2" + tslib: "npm:^2.6.2" + checksum: 10c0/5efcf6a4b10e49b5b3f9bf76b638ff0b0ce84d26126197f0d28cecbd01458f5d84a074a4cdc8817a7310a940a36a8ad164c1531eb031713bd1485d52bef8c877 + languageName: node + linkType: hard + +"@aws-sdk/credential-providers@npm:^3.1009.0": + version: 3.1127.0 + resolution: "@aws-sdk/credential-providers@npm:3.1127.0" + dependencies: + "@aws-sdk/core": "npm:^3.977.9" + "@aws-sdk/credential-provider-cognito-identity": "npm:^3.972.69" + "@aws-sdk/credential-provider-env": "npm:^3.972.70" + "@aws-sdk/credential-provider-http": "npm:^3.972.72" + "@aws-sdk/credential-provider-ini": "npm:^3.973.15" + "@aws-sdk/credential-provider-login": "npm:^3.972.77" + "@aws-sdk/credential-provider-node": "npm:^3.972.82" + "@aws-sdk/credential-provider-process": "npm:^3.972.70" + "@aws-sdk/credential-provider-sso": "npm:^3.973.14" + "@aws-sdk/credential-provider-web-identity": "npm:^3.972.76" + "@aws-sdk/nested-clients": "npm:^3.997.44" + "@aws-sdk/types": "npm:^3.974.5" + "@smithy/core": "npm:^3.33.3" + "@smithy/credential-provider-imds": "npm:^4.4.16" + "@smithy/types": "npm:^4.17.2" + tslib: "npm:^2.6.2" + checksum: 10c0/44e33959b07968db555a047101f56ac94554ea7518b7b29012c7498ca7dc280df12901a1c7971366745ab164eb0dc6e4e4831898c2859ab60544f77e5e7b334d + languageName: node + linkType: hard + "@aws-sdk/lib-storage@npm:^3.1009.0": version: 3.1014.0 resolution: "@aws-sdk/lib-storage@npm:3.1014.0" @@ -1006,6 +1211,22 @@ __metadata: languageName: node linkType: hard +"@aws-sdk/nested-clients@npm:^3.997.44": + version: 3.997.44 + resolution: "@aws-sdk/nested-clients@npm:3.997.44" + dependencies: + "@aws-sdk/core": "npm:^3.977.9" + "@aws-sdk/signature-v4-multi-region": "npm:^3.996.46" + "@aws-sdk/types": "npm:^3.974.5" + "@smithy/core": "npm:^3.33.3" + "@smithy/fetch-http-handler": "npm:^5.7.2" + "@smithy/node-http-handler": "npm:^4.11.3" + "@smithy/types": "npm:^4.17.2" + tslib: "npm:^2.6.2" + checksum: 10c0/947c2049a69a02399f600bd448f34bcaaf3a97b5bbb9b43e9ce932e84d4e9c3131687d330b9c48391da4365582890dd72bf4b8dfafe0693c041e48aa0f5c972d + languageName: node + linkType: hard + "@aws-sdk/region-config-resolver@npm:^3.972.9": version: 3.972.9 resolution: "@aws-sdk/region-config-resolver@npm:3.972.9" @@ -1033,6 +1254,18 @@ __metadata: languageName: node linkType: hard +"@aws-sdk/signature-v4-multi-region@npm:^3.996.46": + version: 3.996.46 + resolution: "@aws-sdk/signature-v4-multi-region@npm:3.996.46" + dependencies: + "@aws-sdk/types": "npm:^3.974.5" + "@smithy/signature-v4": "npm:^5.6.12" + "@smithy/types": "npm:^4.17.2" + tslib: "npm:^2.6.2" + checksum: 10c0/069dfb7a95663cad2e0aec1d87df8a800abad33cb49dfbe9412dad2d63ab6350328c6165166b12d50e609d8dbe5a5536aa6b1101be4d91584fbca76b2fea4d00 + languageName: node + linkType: hard + "@aws-sdk/token-providers@npm:3.1014.0": version: 3.1014.0 resolution: "@aws-sdk/token-providers@npm:3.1014.0" @@ -1048,6 +1281,20 @@ __metadata: languageName: node linkType: hard +"@aws-sdk/token-providers@npm:3.1116.0": + version: 3.1116.0 + resolution: "@aws-sdk/token-providers@npm:3.1116.0" + dependencies: + "@aws-sdk/core": "npm:^3.977.9" + "@aws-sdk/nested-clients": "npm:^3.997.44" + "@aws-sdk/types": "npm:^3.974.5" + "@smithy/core": "npm:^3.33.3" + "@smithy/types": "npm:^4.17.2" + tslib: "npm:^2.6.2" + checksum: 10c0/e720401f6b6d5682984cf1927d3e4c86663b750086bac3b58827a3b9b8585c8b06fb93af82a03a40d5a8ca48c89c0f47c1ac48ea81822c52ddecc3e36ef7cf17 + languageName: node + linkType: hard + "@aws-sdk/types@npm:^3.222.0, @aws-sdk/types@npm:^3.4.1, @aws-sdk/types@npm:^3.973.6": version: 3.973.6 resolution: "@aws-sdk/types@npm:3.973.6" @@ -1058,6 +1305,16 @@ __metadata: languageName: node linkType: hard +"@aws-sdk/types@npm:^3.974.5": + version: 3.974.5 + resolution: "@aws-sdk/types@npm:3.974.5" + dependencies: + "@smithy/types": "npm:^4.17.2" + tslib: "npm:^2.6.2" + checksum: 10c0/803aaaa1c0675dcb564803993f3c47d96302fad461af8af80e75afc40c72228ebf669593c18e44ca09fc5acf0d1bd25966261de07844d8f11ad82aa2650252d0 + languageName: node + linkType: hard + "@aws-sdk/util-arn-parser@npm:^3.972.3": version: 3.972.3 resolution: "@aws-sdk/util-arn-parser@npm:3.972.3" @@ -1143,6 +1400,16 @@ __metadata: languageName: node linkType: hard +"@aws-sdk/xml-builder@npm:^3.972.40": + version: 3.972.40 + resolution: "@aws-sdk/xml-builder@npm:3.972.40" + dependencies: + "@smithy/types": "npm:^4.17.2" + tslib: "npm:^2.6.2" + checksum: 10c0/5b06fa0466b5ddb0e33138dc16f6a30e11e52fc5ea71f3ed72af7b24621d29c9e8103df3eed9c4f14cef50f4d345dd9e7021242a8c155a67869ee4ce79982bb4 + languageName: node + linkType: hard + "@aws/lambda-invoke-store@npm:0.2.3, @aws/lambda-invoke-store@npm:^0.2.2": version: 0.2.3 resolution: "@aws/lambda-invoke-store@npm:0.2.3" @@ -1150,6 +1417,13 @@ __metadata: languageName: node linkType: hard +"@aws/lambda-invoke-store@npm:^0.3.0": + version: 0.3.0 + resolution: "@aws/lambda-invoke-store@npm:0.3.0" + checksum: 10c0/b4a2e6b3b5397bc606053e64270d26dc5c886336f88a98cad587b1592eec17058f8fb172f1827a9f0e591f3595cf8f01575c8c9b36cde38c06456f8a65204046 + languageName: node + linkType: hard + "@babel/code-frame@npm:^7.0.0, @babel/code-frame@npm:^7.12.13, @babel/code-frame@npm:^7.28.6, @babel/code-frame@npm:^7.29.0": version: 7.29.0 resolution: "@babel/code-frame@npm:7.29.0" @@ -1544,7 +1818,7 @@ __metadata: languageName: node linkType: hard -"@babel/parser@npm:^7.25.4, @babel/parser@npm:^7.28.0, @babel/parser@npm:^7.28.6, @babel/parser@npm:^7.29.0": +"@babel/parser@npm:^7.28.0, @babel/parser@npm:^7.28.6, @babel/parser@npm:^7.29.0": version: 7.29.3 resolution: "@babel/parser@npm:7.29.3" dependencies: @@ -2650,7 +2924,7 @@ __metadata: languageName: node linkType: hard -"@babel/types@npm:^7.22.15, @babel/types@npm:^7.22.19, @babel/types@npm:^7.22.5, @babel/types@npm:^7.23.0, @babel/types@npm:^7.25.4, @babel/types@npm:^7.28.0, @babel/types@npm:^7.28.4, @babel/types@npm:^7.28.6, @babel/types@npm:^7.29.0, @babel/types@npm:^7.4.4": +"@babel/types@npm:^7.22.15, @babel/types@npm:^7.22.19, @babel/types@npm:^7.22.5, @babel/types@npm:^7.23.0, @babel/types@npm:^7.28.0, @babel/types@npm:^7.28.4, @babel/types@npm:^7.28.6, @babel/types@npm:^7.29.0, @babel/types@npm:^7.4.4": version: 7.29.0 resolution: "@babel/types@npm:7.29.0" dependencies: @@ -3145,7 +3419,7 @@ __metadata: languageName: node linkType: hard -"@jridgewell/trace-mapping@npm:^0.3.23, @jridgewell/trace-mapping@npm:^0.3.24, @jridgewell/trace-mapping@npm:^0.3.28, @jridgewell/trace-mapping@npm:^0.3.31": +"@jridgewell/trace-mapping@npm:^0.3.24, @jridgewell/trace-mapping@npm:^0.3.28, @jridgewell/trace-mapping@npm:^0.3.31": version: 0.3.31 resolution: "@jridgewell/trace-mapping@npm:0.3.31" dependencies: @@ -3982,10 +4256,10 @@ __metadata: languageName: node linkType: hard -"@oxc-project/types@npm:=0.149.0": - version: 0.149.0 - resolution: "@oxc-project/types@npm:0.149.0" - checksum: 10c0/f91ff8101300ce5a29fbfc57bdaafd9c1c9d3d9b46c63093f437c479d047042eb615cb6c3ddac91a0d289701ce60c365632866a85d0f1e0d159b20180115187e +"@oxc-project/types@npm:=0.148.0": + version: 0.148.0 + resolution: "@oxc-project/types@npm:0.148.0" + checksum: 10c0/23700196086ec996dcaf8cb494d0d378257a4f6f8ac4dfa0ee732bdf9a8a519da535cb33c8654ee6ea771872ae5801f9963c9d402767a4d13c2fa128cf7c42d2 languageName: node linkType: hard @@ -4150,107 +4424,107 @@ __metadata: languageName: node linkType: hard -"@rolldown/binding-android-arm-eabi@npm:1.2.8": - version: 1.2.8 - resolution: "@rolldown/binding-android-arm-eabi@npm:1.2.8" +"@rolldown/binding-android-arm-eabi@npm:1.2.7": + version: 1.2.7 + resolution: "@rolldown/binding-android-arm-eabi@npm:1.2.7" conditions: os=android & cpu=arm languageName: node linkType: hard -"@rolldown/binding-android-arm64@npm:1.2.8": - version: 1.2.8 - resolution: "@rolldown/binding-android-arm64@npm:1.2.8" +"@rolldown/binding-android-arm64@npm:1.2.7": + version: 1.2.7 + resolution: "@rolldown/binding-android-arm64@npm:1.2.7" conditions: os=android & cpu=arm64 languageName: node linkType: hard -"@rolldown/binding-darwin-arm64@npm:1.2.8": - version: 1.2.8 - resolution: "@rolldown/binding-darwin-arm64@npm:1.2.8" +"@rolldown/binding-darwin-arm64@npm:1.2.7": + version: 1.2.7 + resolution: "@rolldown/binding-darwin-arm64@npm:1.2.7" conditions: os=darwin & cpu=arm64 languageName: node linkType: hard -"@rolldown/binding-darwin-x64@npm:1.2.8": - version: 1.2.8 - resolution: "@rolldown/binding-darwin-x64@npm:1.2.8" +"@rolldown/binding-darwin-x64@npm:1.2.7": + version: 1.2.7 + resolution: "@rolldown/binding-darwin-x64@npm:1.2.7" conditions: os=darwin & cpu=x64 languageName: node linkType: hard -"@rolldown/binding-freebsd-x64@npm:1.2.8": - version: 1.2.8 - resolution: "@rolldown/binding-freebsd-x64@npm:1.2.8" +"@rolldown/binding-freebsd-x64@npm:1.2.7": + version: 1.2.7 + resolution: "@rolldown/binding-freebsd-x64@npm:1.2.7" conditions: os=freebsd & cpu=x64 languageName: node linkType: hard -"@rolldown/binding-linux-arm-gnueabihf@npm:1.2.8": - version: 1.2.8 - resolution: "@rolldown/binding-linux-arm-gnueabihf@npm:1.2.8" +"@rolldown/binding-linux-arm-gnueabihf@npm:1.2.7": + version: 1.2.7 + resolution: "@rolldown/binding-linux-arm-gnueabihf@npm:1.2.7" conditions: os=linux & cpu=arm languageName: node linkType: hard -"@rolldown/binding-linux-arm64-gnu@npm:1.2.8": - version: 1.2.8 - resolution: "@rolldown/binding-linux-arm64-gnu@npm:1.2.8" +"@rolldown/binding-linux-arm64-gnu@npm:1.2.7": + version: 1.2.7 + resolution: "@rolldown/binding-linux-arm64-gnu@npm:1.2.7" conditions: os=linux & cpu=arm64 & libc=glibc languageName: node linkType: hard -"@rolldown/binding-linux-arm64-musl@npm:1.2.8": - version: 1.2.8 - resolution: "@rolldown/binding-linux-arm64-musl@npm:1.2.8" +"@rolldown/binding-linux-arm64-musl@npm:1.2.7": + version: 1.2.7 + resolution: "@rolldown/binding-linux-arm64-musl@npm:1.2.7" conditions: os=linux & cpu=arm64 & libc=musl languageName: node linkType: hard -"@rolldown/binding-linux-ppc64-gnu@npm:1.2.8": - version: 1.2.8 - resolution: "@rolldown/binding-linux-ppc64-gnu@npm:1.2.8" +"@rolldown/binding-linux-ppc64-gnu@npm:1.2.7": + version: 1.2.7 + resolution: "@rolldown/binding-linux-ppc64-gnu@npm:1.2.7" conditions: os=linux & cpu=ppc64 & libc=glibc languageName: node linkType: hard -"@rolldown/binding-linux-s390x-gnu@npm:1.2.8": - version: 1.2.8 - resolution: "@rolldown/binding-linux-s390x-gnu@npm:1.2.8" +"@rolldown/binding-linux-s390x-gnu@npm:1.2.7": + version: 1.2.7 + resolution: "@rolldown/binding-linux-s390x-gnu@npm:1.2.7" conditions: os=linux & cpu=s390x & libc=glibc languageName: node linkType: hard -"@rolldown/binding-linux-x64-gnu@npm:1.2.8": - version: 1.2.8 - resolution: "@rolldown/binding-linux-x64-gnu@npm:1.2.8" +"@rolldown/binding-linux-x64-gnu@npm:1.2.7": + version: 1.2.7 + resolution: "@rolldown/binding-linux-x64-gnu@npm:1.2.7" conditions: os=linux & cpu=x64 & libc=glibc languageName: node linkType: hard -"@rolldown/binding-linux-x64-musl@npm:1.2.8": - version: 1.2.8 - resolution: "@rolldown/binding-linux-x64-musl@npm:1.2.8" +"@rolldown/binding-linux-x64-musl@npm:1.2.7": + version: 1.2.7 + resolution: "@rolldown/binding-linux-x64-musl@npm:1.2.7" conditions: os=linux & cpu=x64 & libc=musl languageName: node linkType: hard -"@rolldown/binding-openharmony-arm64@npm:1.2.8": - version: 1.2.8 - resolution: "@rolldown/binding-openharmony-arm64@npm:1.2.8" +"@rolldown/binding-openharmony-arm64@npm:1.2.7": + version: 1.2.7 + resolution: "@rolldown/binding-openharmony-arm64@npm:1.2.7" conditions: os=openharmony & cpu=arm64 languageName: node linkType: hard -"@rolldown/binding-win32-arm64-msvc@npm:1.2.8": - version: 1.2.8 - resolution: "@rolldown/binding-win32-arm64-msvc@npm:1.2.8" +"@rolldown/binding-win32-arm64-msvc@npm:1.2.7": + version: 1.2.7 + resolution: "@rolldown/binding-win32-arm64-msvc@npm:1.2.7" conditions: os=win32 & cpu=arm64 languageName: node linkType: hard -"@rolldown/binding-win32-x64-msvc@npm:1.2.8": - version: 1.2.8 - resolution: "@rolldown/binding-win32-x64-msvc@npm:1.2.8" +"@rolldown/binding-win32-x64-msvc@npm:1.2.7": + version: 1.2.7 + resolution: "@rolldown/binding-win32-x64-msvc@npm:1.2.7" conditions: os=win32 & cpu=x64 languageName: node linkType: hard @@ -4566,6 +4840,16 @@ __metadata: languageName: node linkType: hard +"@smithy/core@npm:^3.33.2, @smithy/core@npm:^3.33.3": + version: 3.33.3 + resolution: "@smithy/core@npm:3.33.3" + dependencies: + "@smithy/types": "npm:^4.17.2" + tslib: "npm:^2.6.2" + checksum: 10c0/57c5c6c1834d84eddfff905932350973afa20e1006024b82d6599af4c8d1e75b243e06b93c998acc95946399f1342ddc298c6941e0030384e6541ceec4007376 + languageName: node + linkType: hard + "@smithy/credential-provider-imds@npm:^4.2.12": version: 4.2.12 resolution: "@smithy/credential-provider-imds@npm:4.2.12" @@ -4579,6 +4863,17 @@ __metadata: languageName: node linkType: hard +"@smithy/credential-provider-imds@npm:^4.4.16": + version: 4.5.2 + resolution: "@smithy/credential-provider-imds@npm:4.5.2" + dependencies: + "@smithy/core": "npm:^3.33.2" + "@smithy/types": "npm:^4.17.2" + tslib: "npm:^2.6.2" + checksum: 10c0/d5481a7797a1f485849d92f6c188cfb9411736ae2469e27394b863e107dd1024266baa8e3a62cfc3f75b3abfd84b3c389955a0e96b64c4ad90462c70bbe66ab0 + languageName: node + linkType: hard + "@smithy/eventstream-codec@npm:^4.2.12": version: 4.2.12 resolution: "@smithy/eventstream-codec@npm:4.2.12" @@ -4647,6 +4942,17 @@ __metadata: languageName: node linkType: hard +"@smithy/fetch-http-handler@npm:^5.7.2": + version: 5.8.0 + resolution: "@smithy/fetch-http-handler@npm:5.8.0" + dependencies: + "@smithy/core": "npm:^3.33.3" + "@smithy/types": "npm:^4.18.0" + tslib: "npm:^2.6.2" + checksum: 10c0/8035961bad01fd80de32caf2bb9b035cf6e102c8586790017660fa4440ba0e9ee126942b58f2ca456e94972e90c25d863378939c252b903d2741c7b05f202279 + languageName: node + linkType: hard + "@smithy/hash-blob-browser@npm:^4.2.13": version: 4.2.13 resolution: "@smithy/hash-blob-browser@npm:4.2.13" @@ -4799,6 +5105,17 @@ __metadata: languageName: node linkType: hard +"@smithy/node-http-handler@npm:^4.11.3": + version: 4.12.1 + resolution: "@smithy/node-http-handler@npm:4.12.1" + dependencies: + "@smithy/core": "npm:^3.33.3" + "@smithy/types": "npm:^4.18.0" + tslib: "npm:^2.6.2" + checksum: 10c0/a657259f8ebbff531cad854e9e12ff3f98b84c095965f4f9a6e8a3e1af4f5bbbb1bc8a7228df220663872c2eef74b5403d148dbe6be1edee06a3baa93cbeb5dc + languageName: node + linkType: hard + "@smithy/node-http-handler@npm:^4.5.0": version: 4.5.0 resolution: "@smithy/node-http-handler@npm:4.5.0" @@ -4897,6 +5214,17 @@ __metadata: languageName: node linkType: hard +"@smithy/signature-v4@npm:^5.6.12": + version: 5.7.3 + resolution: "@smithy/signature-v4@npm:5.7.3" + dependencies: + "@smithy/core": "npm:^3.33.3" + "@smithy/types": "npm:^4.17.2" + tslib: "npm:^2.6.2" + checksum: 10c0/6976142320c7c112ee817f5329d0adc45c1ec101a095a70e4afcb7e3fa9f18cd4f544dcbb481e1932fb877197ec9e1037054bacda422cccfd8de7eed55905319 + languageName: node + linkType: hard + "@smithy/smithy-client@npm:^4.12.7": version: 4.12.7 resolution: "@smithy/smithy-client@npm:4.12.7" @@ -4930,6 +5258,15 @@ __metadata: languageName: node linkType: hard +"@smithy/types@npm:^4.17.2, @smithy/types@npm:^4.18.0": + version: 4.18.0 + resolution: "@smithy/types@npm:4.18.0" + dependencies: + tslib: "npm:^2.6.2" + checksum: 10c0/f948eaf2c6004ce919a5a203615da4d2d4923465df764c2f6ab982fcacde10c81e1fd23c40f983387459ffdad056f8e827ebecaa776a4331ed4f6431ad8bdd34 + languageName: node + linkType: hard + "@smithy/url-parser@npm:^4.2.12": version: 4.2.12 resolution: "@smithy/url-parser@npm:4.2.12" @@ -5876,28 +6213,27 @@ __metadata: languageName: node linkType: hard -"@vitest/coverage-v8@npm:^4.0.5": - version: 4.0.5 - resolution: "@vitest/coverage-v8@npm:4.0.5" +"@vitest/coverage-v8@npm:^4.1.11": + version: 4.1.11 + resolution: "@vitest/coverage-v8@npm:4.1.11" dependencies: "@bcoe/v8-coverage": "npm:^1.0.2" - "@vitest/utils": "npm:4.0.5" - ast-v8-to-istanbul: "npm:^0.3.5" - debug: "npm:^4.4.3" + "@vitest/utils": "npm:4.1.11" + ast-v8-to-istanbul: "npm:^1.0.0" istanbul-lib-coverage: "npm:^3.2.2" istanbul-lib-report: "npm:^3.0.1" - istanbul-lib-source-maps: "npm:^5.0.6" istanbul-reports: "npm:^3.2.0" - magicast: "npm:^0.3.5" - std-env: "npm:^3.9.0" - tinyrainbow: "npm:^3.0.3" + magicast: "npm:^0.5.2" + obug: "npm:^2.1.1" + std-env: "npm:^4.0.0-rc.1" + tinyrainbow: "npm:^3.1.0" peerDependencies: - "@vitest/browser": 4.0.5 - vitest: 4.0.5 + "@vitest/browser": 4.1.11 + vitest: 4.1.11 peerDependenciesMeta: "@vitest/browser": optional: true - checksum: 10c0/6c93ff8a4c38f9a1cb8044eaae9553badfd7e82a8f46c642769fef47ebdd7ea9cc1f05e9e9c31feeb76f32f65ca76396ab35a285604648e02283793c8bd655a4 + checksum: 10c0/91127fd40f445b506cc661c54e26defd75d970fa1983cb83442856dd7f295542f0378e0bca847036a7a5be871b3b17c27167d21f277e47368cf7409eafaa1b40 languageName: node linkType: hard @@ -5948,15 +6284,6 @@ __metadata: languageName: node linkType: hard -"@vitest/pretty-format@npm:4.0.5": - version: 4.0.5 - resolution: "@vitest/pretty-format@npm:4.0.5" - dependencies: - tinyrainbow: "npm:^3.0.3" - checksum: 10c0/76b36512ba8978475223a4f15041f66aeda32a54b9426b372d75f3584243521e3a8976eeb82b50534c48271f30023ee6345213e22add750ffb49a69098bc8619 - languageName: node - linkType: hard - "@vitest/pretty-format@npm:4.1.0": version: 4.1.0 resolution: "@vitest/pretty-format@npm:4.1.0" @@ -6011,16 +6338,6 @@ __metadata: languageName: node linkType: hard -"@vitest/utils@npm:4.0.5": - version: 4.0.5 - resolution: "@vitest/utils@npm:4.0.5" - dependencies: - "@vitest/pretty-format": "npm:4.0.5" - tinyrainbow: "npm:^3.0.3" - checksum: 10c0/1b772533bb7020c14c22036f94027afa9b51aad683abf048f377af776186ecc41d6abd716daf18ac7f5654b4569409c5f5668b8e0f2ac3a33fe291bcd839cb8c - languageName: node - linkType: hard - "@vitest/utils@npm:4.1.0": version: 4.1.0 resolution: "@vitest/utils@npm:4.1.0" @@ -6249,14 +6566,14 @@ __metadata: languageName: node linkType: hard -"ast-v8-to-istanbul@npm:^0.3.5": - version: 0.3.8 - resolution: "ast-v8-to-istanbul@npm:0.3.8" +"ast-v8-to-istanbul@npm:^1.0.0": + version: 1.0.6 + resolution: "ast-v8-to-istanbul@npm:1.0.6" dependencies: "@jridgewell/trace-mapping": "npm:^0.3.31" estree-walker: "npm:^3.0.3" - js-tokens: "npm:^9.0.1" - checksum: 10c0/6f7d74fc36011699af6d4ad88ecd8efc7d74bd90b8e8dbb1c69d43c8f4bec0ed361fb62a5b5bd98bbee02ee87c62cd8bcc25a39634964e45476bf5489dfa327f + js-tokens: "npm:^10.0.0" + checksum: 10c0/d0f5c45f3c3054d2f142f460df3dfb26a17b910e1ca95f3d379fc505877c7610baba635c4a7e6f8c837d6ad0f0aa0e94daee4252200cb3d42f420e518a8c5d3e languageName: node linkType: hard @@ -8321,17 +8638,6 @@ __metadata: languageName: node linkType: hard -"istanbul-lib-source-maps@npm:^5.0.6": - version: 5.0.6 - resolution: "istanbul-lib-source-maps@npm:5.0.6" - dependencies: - "@jridgewell/trace-mapping": "npm:^0.3.23" - debug: "npm:^4.1.1" - istanbul-lib-coverage: "npm:^3.0.0" - checksum: 10c0/ffe75d70b303a3621ee4671554f306e0831b16f39ab7f4ab52e54d356a5d33e534d97563e318f1333a6aae1d42f91ec49c76b6cd3f3fb378addcb5c81da0255f - languageName: node - linkType: hard - "istanbul-reports@npm:^3.2.0": version: 3.2.0 resolution: "istanbul-reports@npm:3.2.0" @@ -8450,6 +8756,13 @@ __metadata: languageName: node linkType: hard +"js-tokens@npm:^10.0.0": + version: 10.0.0 + resolution: "js-tokens@npm:10.0.0" + checksum: 10c0/a93498747812ba3e0c8626f95f75ab29319f2a13613a0de9e610700405760931624433a0de59eb7c27ff8836e526768fb20783861b86ef89be96676f2c996b64 + languageName: node + linkType: hard + "js-tokens@npm:^4.0.0": version: 4.0.0 resolution: "js-tokens@npm:4.0.0" @@ -8457,14 +8770,7 @@ __metadata: languageName: node linkType: hard -"js-tokens@npm:^9.0.1": - version: 9.0.1 - resolution: "js-tokens@npm:9.0.1" - checksum: 10c0/68dcab8f233dde211a6b5fd98079783cbcd04b53617c1250e3553ee16ab3e6134f5e65478e41d82f6d351a052a63d71024553933808570f04dbf828d7921e80e - languageName: node - linkType: hard - -"js-yaml@npm:^3.10.0, js-yaml@npm:^3.13.1": +"js-yaml@npm:^3.15.2": version: 3.15.2 resolution: "js-yaml@npm:3.15.2" dependencies: @@ -8476,17 +8782,6 @@ __metadata: languageName: node linkType: hard -"js-yaml@npm:^4.1.0": - version: 4.1.0 - resolution: "js-yaml@npm:4.1.0" - dependencies: - argparse: "npm:^2.0.1" - bin: - js-yaml: bin/js-yaml.js - checksum: 10c0/184a24b4eaacfce40ad9074c64fd42ac83cf74d8c8cd137718d456ced75051229e5061b8633c3366b8aada17945a7a356b337828c19da92b51ae62126575018f - languageName: node - linkType: hard - "jsbn@npm:1.1.0": version: 1.1.0 resolution: "jsbn@npm:1.1.0" @@ -8600,7 +8895,7 @@ __metadata: "@trivago/prettier-plugin-sort-imports": "npm:^6.0.0" "@typescript-eslint/eslint-plugin": "npm:^8.47.0" "@typescript-eslint/parser": "npm:^8.46.2" - "@vitest/coverage-v8": "npm:^4.0.5" + "@vitest/coverage-v8": "npm:^4.1.11" chalk: "npm:^5.6.2" eslint: "npm:^9.39.2" eslint-plugin-prettier: "npm:5.5.4" @@ -8837,14 +9132,14 @@ __metadata: languageName: node linkType: hard -"magicast@npm:^0.3.5": - version: 0.3.5 - resolution: "magicast@npm:0.3.5" +"magicast@npm:^0.5.2": + version: 0.5.4 + resolution: "magicast@npm:0.5.4" dependencies: - "@babel/parser": "npm:^7.25.4" - "@babel/types": "npm:^7.25.4" - source-map-js: "npm:^1.2.0" - checksum: 10c0/a6cacc0a848af84f03e3f5bda7b0de75e4d0aa9ddce5517fd23ed0f31b5ddd51b2d0ff0b7e09b51f7de0f4053c7a1107117edda6b0732dca3e9e39e6c5a68c64 + "@babel/parser": "npm:^7.29.7" + "@babel/types": "npm:^7.29.7" + source-map-js: "npm:^1.2.1" + checksum: 10c0/f6a3b33d1c994cace3999fc96876a9fd06429deff8266563ccdc1d731c9edc4e1fb59d724128d7d4f3382eb457cd5a39ae1ec7b1e1864e068297ee2a30c12a3d languageName: node linkType: hard @@ -9837,7 +10132,7 @@ __metadata: languageName: node linkType: hard -"picomatch@npm:^4.0.7": +"picomatch@npm:^4.0.5": version: 4.0.7 resolution: "picomatch@npm:4.0.7" checksum: 10c0/beb6ae02c43ae44e84883b90830196d9046b1726ead292adcf7f57945e0bb0d992d68563d87e03b484b6f3c9a5c6defda7523477f047d7f0e663f126cc01787f @@ -9851,7 +10146,7 @@ __metadata: languageName: node linkType: hard -"postcss@npm:^8.5.28": +"postcss@npm:^8.5.26": version: 8.5.28 resolution: "postcss@npm:8.5.28" dependencies: @@ -10179,26 +10474,26 @@ __metadata: languageName: node linkType: hard -"rolldown@npm:~1.2.6": - version: 1.2.8 - resolution: "rolldown@npm:1.2.8" - dependencies: - "@oxc-project/types": "npm:=0.149.0" - "@rolldown/binding-android-arm-eabi": "npm:1.2.8" - "@rolldown/binding-android-arm64": "npm:1.2.8" - "@rolldown/binding-darwin-arm64": "npm:1.2.8" - "@rolldown/binding-darwin-x64": "npm:1.2.8" - "@rolldown/binding-freebsd-x64": "npm:1.2.8" - "@rolldown/binding-linux-arm-gnueabihf": "npm:1.2.8" - "@rolldown/binding-linux-arm64-gnu": "npm:1.2.8" - "@rolldown/binding-linux-arm64-musl": "npm:1.2.8" - "@rolldown/binding-linux-ppc64-gnu": "npm:1.2.8" - "@rolldown/binding-linux-s390x-gnu": "npm:1.2.8" - "@rolldown/binding-linux-x64-gnu": "npm:1.2.8" - "@rolldown/binding-linux-x64-musl": "npm:1.2.8" - "@rolldown/binding-openharmony-arm64": "npm:1.2.8" - "@rolldown/binding-win32-arm64-msvc": "npm:1.2.8" - "@rolldown/binding-win32-x64-msvc": "npm:1.2.8" +"rolldown@npm:~1.2.4": + version: 1.2.7 + resolution: "rolldown@npm:1.2.7" + dependencies: + "@oxc-project/types": "npm:=0.148.0" + "@rolldown/binding-android-arm-eabi": "npm:1.2.7" + "@rolldown/binding-android-arm64": "npm:1.2.7" + "@rolldown/binding-darwin-arm64": "npm:1.2.7" + "@rolldown/binding-darwin-x64": "npm:1.2.7" + "@rolldown/binding-freebsd-x64": "npm:1.2.7" + "@rolldown/binding-linux-arm-gnueabihf": "npm:1.2.7" + "@rolldown/binding-linux-arm64-gnu": "npm:1.2.7" + "@rolldown/binding-linux-arm64-musl": "npm:1.2.7" + "@rolldown/binding-linux-ppc64-gnu": "npm:1.2.7" + "@rolldown/binding-linux-s390x-gnu": "npm:1.2.7" + "@rolldown/binding-linux-x64-gnu": "npm:1.2.7" + "@rolldown/binding-linux-x64-musl": "npm:1.2.7" + "@rolldown/binding-openharmony-arm64": "npm:1.2.7" + "@rolldown/binding-win32-arm64-msvc": "npm:1.2.7" + "@rolldown/binding-win32-x64-msvc": "npm:1.2.7" "@rolldown/pluginutils": "npm:^1.0.0" dependenciesMeta: "@rolldown/binding-android-arm-eabi": @@ -10233,7 +10528,7 @@ __metadata: optional: true bin: rolldown: ./bin/cli.mjs - checksum: 10c0/5a9c6b30a7ef0af257a963e6b0b95d5a60b23512b7dbcca617273bd63fb0892b88be6b33dc9a6cb66350f47d260eff6456af0b7afef34898767f3527ce0d967d + checksum: 10c0/e3b73addf51981d8b8430990ff59c14197994cfe9e1aae443dce47d353b409e7ffd478946185a15769588ab2e3d504557319fd58ab650ac9ac37c299384da10a languageName: node linkType: hard @@ -10592,7 +10887,7 @@ __metadata: languageName: node linkType: hard -"source-map-js@npm:^1.2.0, source-map-js@npm:^1.2.1": +"source-map-js@npm:^1.2.1": version: 1.2.1 resolution: "source-map-js@npm:1.2.1" checksum: 10c0/7bda1fc4c197e3c6ff17de1b8b2c20e60af81b63a52cb32ec5a5d67a20a7d42651e2cb34ebe93833c5a2a084377e17455854fee3e21e7925c64a51b6a52b0faf @@ -10679,13 +10974,6 @@ __metadata: languageName: node linkType: hard -"std-env@npm:^3.9.0": - version: 3.9.0 - resolution: "std-env@npm:3.9.0" - checksum: 10c0/4a6f9218aef3f41046c3c7ecf1f98df00b30a07f4f35c6d47b28329bc2531eef820828951c7d7b39a1c5eb19ad8a46e3ddfc7deb28f0a2f3ceebee11bab7ba50 - languageName: node - linkType: hard - "std-env@npm:^4.0.0-rc.1": version: 4.1.0 resolution: "std-env@npm:4.1.0" @@ -11085,6 +11373,13 @@ __metadata: languageName: node linkType: hard +"undici@npm:^6.19.2": + version: 6.28.0 + resolution: "undici@npm:6.28.0" + checksum: 10c0/3029a70df06b38b5b2f30732932a1e92544c753cd82c8abdf0d35afad48e0ba91612e79fe3a442dbbb9434d6a9eba2b714d5ea28984c903dda2b5d5444f38354 + languageName: node + linkType: hard + "unicode-canonical-property-names-ecmascript@npm:^2.0.0": version: 2.0.0 resolution: "unicode-canonical-property-names-ecmascript@npm:2.0.0" @@ -11200,18 +11495,18 @@ __metadata: linkType: hard "vite@npm:^6.0.0 || ^7.0.0 || ^8.0.0": - version: 8.3.0 - resolution: "vite@npm:8.3.0" + version: 8.2.2 + resolution: "vite@npm:8.2.2" dependencies: fsevents: "npm:~2.3.3" lightningcss: "npm:^1.33.0" - picomatch: "npm:^4.0.7" - postcss: "npm:^8.5.28" - rolldown: "npm:~1.2.6" + picomatch: "npm:^4.0.5" + postcss: "npm:^8.5.26" + rolldown: "npm:~1.2.4" tinyglobby: "npm:^0.2.17" peerDependencies: "@types/node": ^20.19.0 || >=22.12.0 - "@vitejs/devtools": ^0.7.1 + "@vitejs/devtools": ^0.4.0 || ^0.5.0 esbuild: ^0.27.0 || ^0.28.0 jiti: ">=1.21.0" less: ^4.0.0 @@ -11252,7 +11547,7 @@ __metadata: optional: true bin: vite: bin/vite.js - checksum: 10c0/28d9a1c8b5018a865b8d8b2330859b11b035c3c578349d6ef0fc4a69b7a99460fc65aff75926d2ac18a7b217cd9a847901e2f7bcb32875f37319ccd81cd9064a + checksum: 10c0/94cbbbdc38ad500dcb86b6202ddd14aa41d05c80739766cada9bbe250b410d1a27be433c9c491ed39744019471ac1e27a59908616a88c42f9787bdb6bdca49d2 languageName: node linkType: hard diff --git a/tests/ministack/README.md b/tests/ministack/README.md index 706b5157a2..1a00a7e32c 100644 --- a/tests/ministack/README.md +++ b/tests/ministack/README.md @@ -89,3 +89,9 @@ override the hostname with `MINISTACK_GITHUB_MOCK_HOST` when using a different container runtime. When MiniStack is exposed on a non-default local port, use a host address reachable from its container for `AWS_ENDPOINT_URL`, for example `AWS_ENDPOINT_URL=http://:14568`, instead of `127.0.0.1`. + +The workflow also runs `run-scale-set-integration.sh`. It applies the +`multi-runner-scale-set` example and verifies the managed ECS controller, +Fargate task hardening, scale-set environment contract, and reconciler SSM +parameter through MiniStack's AWS-compatible APIs. It does not send webhook +events or exercise webhook scale-up, scale-down, or pool handlers. diff --git a/tests/ministack/initializerJson.json b/tests/ministack/initializerJson.json new file mode 100644 index 0000000000..443ddb6201 --- /dev/null +++ b/tests/ministack/initializerJson.json @@ -0,0 +1,138 @@ +[ + { + "httpRequest": { + "method": "POST", + "path": "/api/v3/app/installations/456/access_tokens" + }, + "httpResponse": { + "statusCode": 201, + "headers": { "Content-Type": ["application/json"] }, + "body": "{\"token\":\"fake-installation-token\",\"expires_at\":\"2099-01-01T00:00:00Z\"}" + } + }, + { + "httpRequest": { + "method": "POST", + "path": "/api/v3/orgs/example/actions/runners/registration-token" + }, + "httpResponse": { + "statusCode": 201, + "headers": { "Content-Type": ["application/json"] }, + "body": "{\"token\":\"fake-registration-token\"}" + } + }, + { + "httpRequest": { + "method": "POST", + "path": "/api/v3/actions/runner-registration" + }, + "httpResponse": { + "statusCode": 200, + "headers": { "Content-Type": ["application/json"] }, + "body": "{\"url\":\"https://mockserver:1080/tenant/123\",\"token\":\"header.eyJleHAiOjQwNzA5MDg4MDB9.signature\"}" + } + }, + { + "httpRequest": { + "method": "GET", + "path": "/tenant/123/_apis/runtime/runnergroups/" + }, + "httpResponse": { + "statusCode": 200, + "headers": { "Content-Type": ["application/json"] }, + "body": "{\"count\":1,\"value\":[{\"id\":48,\"name\":\"experimental-euw1-sl-cicd-forge-emu\"}]}" + } + }, + { + "httpRequest": { + "method": "GET", + "path": "/tenant/123/_apis/runtime/runnerscalesets" + }, + "httpResponse": { + "statusCode": 200, + "headers": { "Content-Type": ["application/json"] }, + "body": "{\"count\":1,\"value\":[{\"id\":223,\"name\":\"medium\",\"runnerGroupId\":48,\"labels\":[{\"name\":\"medium\",\"type\":\"system\"}],\"runnerSetting\":{}}]}" + } + }, + { + "httpRequest": { + "method": "GET", + "path": "/tenant/123/_apis/runtime/runnerscalesets/223" + }, + "httpResponse": { + "statusCode": 200, + "headers": { "Content-Type": ["application/json"] }, + "body": "{\"id\":223,\"name\":\"medium\",\"runnerGroupId\":48,\"labels\":[{\"name\":\"medium\",\"type\":\"system\"}],\"runnerSetting\":{}}" + } + }, + { + "httpRequest": { + "method": "PATCH", + "path": "/tenant/123/_apis/runtime/runnerscalesets/223" + }, + "httpResponse": { + "statusCode": 200, + "headers": { "Content-Type": ["application/json"] }, + "body": "{\"id\":223,\"name\":\"medium\",\"runnerGroupId\":48,\"labels\":[{\"name\":\"medium\",\"type\":\"system\"},{\"name\":\"linux\",\"type\":\"user\"},{\"name\":\"scale-set\",\"type\":\"user\"},{\"name\":\"self-hosted\",\"type\":\"user\"},{\"name\":\"x64\",\"type\":\"user\"}],\"runnerSetting\":{}}" + } + }, + { + "httpRequest": { + "method": "POST", + "path": "/tenant/123/_apis/runtime/runnerscalesets/223/generatejitconfig" + }, + "httpResponseTemplate": { + "templateType": "MUSTACHE", + "template": "{\"statusCode\":200,\"headers\":{\"Content-Type\":[\"application/json\"]},\"body\":\"{\\\"runner\\\":{\\\"id\\\":321,\\\"name\\\":\\\"{{#jsonPath}}$.name{{/jsonPath}}{{jsonPathResult}}\\\",\\\"runnerScaleSetId\\\":223},\\\"encodedJITConfig\\\":\\\"test-only-jit-configuration\\\"}\"}" + } + }, + { + "httpRequest": { + "method": "POST", + "path": "/tenant/123/_apis/runtime/runnerscalesets/223/sessions" + }, + "httpResponse": { + "statusCode": 200, + "headers": { "Content-Type": ["application/json"] }, + "body": "{\"sessionId\":\"11111111-1111-1111-1111-111111111111\",\"ownerName\":\"local.medium\",\"runnerScaleSet\":{\"id\":223,\"name\":\"medium\",\"runnerGroupId\":48},\"messageQueueUrl\":\"https://mockserver:1080/messages?sessionId=11111111-1111-1111-1111-111111111111&api-version=6.0-preview\",\"messageQueueAccessToken\":\"fake-queue-token\",\"statistics\":{\"totalAvailableJobs\":0,\"totalAcquiredJobs\":0,\"totalAssignedJobs\":0,\"totalRunningJobs\":0,\"totalRegisteredRunners\":0,\"totalBusyRunners\":0,\"totalIdleRunners\":0}}" + } + }, + { + "httpRequest": { + "method": "GET", + "path": "/messages" + }, + "httpResponse": { + "statusCode": 202, + "delay": { "timeUnit": "MILLISECONDS", "value": 1000 } + } + }, + { + "httpRequest": { + "method": "GET", + "path": "/tenant/123/_apis/distributedtask/pools/0/agents" + }, + "httpResponseTemplate": { + "templateType": "MUSTACHE", + "template": "{\"statusCode\":200,\"headers\":{\"Content-Type\":[\"application/json\"]},\"body\":\"{\\\"count\\\":1,\\\"value\\\":[{\\\"id\\\":321,\\\"name\\\":\\\"{{ request.queryStringParameters.agentName.0 }}\\\",\\\"runnerScaleSetId\\\":223}]}\"}" + } + }, + { + "httpRequest": { + "method": "DELETE", + "path": "/tenant/123/_apis/distributedtask/pools/0/agents/321" + }, + "httpResponse": { + "statusCode": 204 + } + }, + { + "httpRequest": { + "method": "DELETE", + "path": "/tenant/123/_apis/runtime/runnerscalesets/223/sessions/11111111-1111-1111-1111-111111111111" + }, + "httpResponse": { + "statusCode": 204 + } + } +] diff --git a/tests/ministack/multi-runner-scale-set.tfvars b/tests/ministack/multi-runner-scale-set.tfvars index 6a47126ad6..7dd1a0ad39 100644 --- a/tests/ministack/multi-runner-scale-set.tfvars +++ b/tests/ministack/multi-runner-scale-set.tfvars @@ -54,4 +54,4 @@ scale_set = { container = { image = "localhost:4566/scale-set-controller:smoke" } -} \ No newline at end of file +} diff --git a/tests/ministack/run-scale-set-integration.sh b/tests/ministack/run-scale-set-integration.sh new file mode 100755 index 0000000000..08e0d156de --- /dev/null +++ b/tests/ministack/run-scale-set-integration.sh @@ -0,0 +1,600 @@ +#!/bin/sh + +set -eu + +export AWS_ACCESS_KEY_ID="${AWS_ACCESS_KEY_ID:-000000000000}" +export AWS_SECRET_ACCESS_KEY="${AWS_SECRET_ACCESS_KEY:-test-only}" +export AWS_DEFAULT_REGION="${AWS_DEFAULT_REGION:-eu-west-1}" +export AWS_REGION="${AWS_REGION:-eu-west-1}" +export AWS_ENDPOINT_URL="${AWS_ENDPOINT_URL:-http://127.0.0.1:4566}" +export AWS_EC2_METADATA_DISABLED="${AWS_EC2_METADATA_DISABLED:-true}" + +script_dir=$(CDPATH='' cd -- "$(dirname -- "$0")" && pwd) +source_root=$(CDPATH='' cd -- "$script_dir/../.." && pwd) +example="multi-runner-scale-set" +base_tfvars="$script_dir/$example.tfvars" +tfvars_file=$(mktemp "${TMPDIR:-/tmp}/terraform-aws-github-runner-scale-set.XXXXXX") +app_key_file=$(mktemp "${TMPDIR:-/tmp}/terraform-aws-github-runner-scale-set-key.XXXXXX") +log_file=$(mktemp "${TMPDIR:-/tmp}/terraform-aws-github-runner-scale-set-logs.XXXXXX") +controller_log_file=$(mktemp "${TMPDIR:-/tmp}/terraform-aws-github-runner-scale-set-controller-logs.XXXXXX") +config_file=$(mktemp "${TMPDIR:-/tmp}/terraform-aws-github-runner-scale-set-config.XXXXXX") +task_definition_file=$(mktemp "${TMPDIR:-/tmp}/terraform-aws-github-runner-scale-set-task-definition.XXXXXX") +instance_file=$(mktemp "${TMPDIR:-/tmp}/terraform-aws-github-runner-scale-set-instances.XXXXXX") +scale_down_config_file=$(mktemp "${TMPDIR:-/tmp}/terraform-aws-github-runner-scale-set-scale-down-config.XXXXXX") +initializer_file=$(mktemp "${TMPDIR:-/tmp}/terraform-aws-github-runner-scale-set-mockserver.XXXXXX") +repository_name="scale-set-controller" +image_reference="localhost:4566/${repository_name}:smoke" +mockserver_host="${MINISTACK_GITHUB_MOCK_HOST:-127.0.0.1}" +mockserver_port="${MINISTACK_GITHUB_MOCK_PORT:-1080}" +mockserver_url="${MINISTACK_GITHUB_MOCK_URL:-http://127.0.0.1:${mockserver_port}}" +controller_mock_url="https://${mockserver_host}:${mockserver_port}" +terraform_state_exists=false + +cleanup() { + cleanup_status=$? + set +e + + if [ "$terraform_state_exists" = true ]; then + "$source_root/tests/ministack/run-example.sh" destroy "$example" "$tfvars_file" >/dev/null 2>&1 + fi + + rm -f "$tfvars_file" "$app_key_file" "$log_file" "$controller_log_file" "$config_file" "$task_definition_file" "$instance_file" "$scale_down_config_file" "$initializer_file" + exit "$cleanup_status" +} +trap cleanup EXIT INT TERM + +require_command() { + if ! command -v "$1" >/dev/null 2>&1; then + echo "$1 is required to run the scale-set MiniStack smoke test." >&2 + exit 69 + fi +} + +for command in aws curl docker openssl python3 terraform; do + require_command "$command" +done + +ministack_aws() { + aws --endpoint-url "$AWS_ENDPOINT_URL" --region "$AWS_DEFAULT_REGION" "$@" +} + +wait_for_http() { + url="$1" + attempts=90 + while ! curl -fsS --max-time 2 "$url" >/dev/null 2>&1; do + attempts=$((attempts - 1)) + if [ "$attempts" -le 0 ]; then + echo "Timed out waiting for $url." >&2 + exit 70 + fi + sleep 1 + done +} + +wait_for_scale_set_runner() { + attempts=90 + while :; do + ministack_aws ec2 describe-instances --output json > "$instance_file" + instance_id=$(python3 - "$instance_file" <<'PY' +import json +import sys + +with open(sys.argv[1], encoding="utf-8") as instance_file: + response = json.load(instance_file) + +required_tags = { + "ghr:Application": "github-action-runner", + "ghr:created_by": "scale-set-service", + "ghr:environment": "ministack-scale-set-linux-scale-set", + "ghr:Type": "Org", + "ghr:Owner": "example", + "ghr:scale_set_state": "config-published", + "ghr:github_runner_id": "321", +} +active_states = {"pending", "running", "stopping", "stopped", "shutting-down"} +matches = [] +for reservation in response.get("Reservations", []): + for instance in reservation.get("Instances", []): + if instance.get("State", {}).get("Name") not in active_states: + continue + tags = {tag.get("Key"): tag.get("Value") for tag in instance.get("Tags", [])} + if all(tags.get(key) == value for key, value in required_tags.items()) and tags.get("ghr:runner_name", "").startswith("scale-set-"): + matches.append(instance["InstanceId"]) + +if len(matches) == 1: + print(matches[0]) +PY +) + if [ -n "$instance_id" ]; then + printf ' [PASS] MiniStack created and registered scale-set EC2 runner %s\n' "$instance_id" + return 0 + fi + attempts=$((attempts - 1)) + if [ "$attempts" -le 0 ]; then + echo "Timed out waiting for the scale-set EC2 runner to reach config-published state." >&2 + cat "$instance_file" >&2 || true + exit 1 + fi + sleep 2 + done +} + +wait_for_no_scale_set_runners() { + attempts=90 + while :; do + ministack_aws ec2 describe-instances --output json > "$instance_file" + active_count=$(python3 - "$instance_file" <<'PY' +import json +import sys + +with open(sys.argv[1], encoding="utf-8") as instance_file: + response = json.load(instance_file) + +required_tags = { + "ghr:Application": "github-action-runner", + "ghr:created_by": "scale-set-service", + "ghr:environment": "ministack-scale-set-linux-scale-set", + "ghr:Type": "Org", + "ghr:Owner": "example", +} +active_states = {"pending", "running", "stopping", "stopped", "shutting-down"} +count = 0 +for reservation in response.get("Reservations", []): + for instance in reservation.get("Instances", []): + if instance.get("State", {}).get("Name") not in active_states: + continue + tags = {tag.get("Key"): tag.get("Value") for tag in instance.get("Tags", [])} + if all(tags.get(key) == value for key, value in required_tags.items()): + count += 1 +print(count) +PY +) + if [ "$active_count" = "0" ]; then + printf '%s\n' ' [PASS] scale-set EC2 runner was terminated after the minimum changed to zero' + return 0 + fi + attempts=$((attempts - 1)) + if [ "$attempts" -le 0 ]; then + echo "Timed out waiting for the scale-set EC2 runner to terminate; active count is $active_count." >&2 + cat "$instance_file" >&2 || true + exit 1 + fi + sleep 2 + done +} + +wait_for_http "$AWS_ENDPOINT_URL/_ministack/health" +attempts=90 +while ! curl -fsS --max-time 2 -X PUT "$mockserver_url/mockserver/status" >/dev/null 2>&1; do + attempts=$((attempts - 1)) + if [ "$attempts" -le 0 ]; then + echo "Timed out waiting for $mockserver_url/mockserver/status." >&2 + exit 70 + fi + sleep 1 +done +curl -fsS -X PUT "$mockserver_url/mockserver/reset" >/dev/null +CONTROLLER_MOCK_URL="$controller_mock_url" python3 - "$source_root/tests/ministack/initializerJson.json" "$initializer_file" <<'PY' +import os +import sys + +source, destination = sys.argv[1:] +with open(source, encoding="utf-8") as source_file: + fixture = source_file.read() +fixture = fixture.replace("https://mockserver:1080", os.environ["CONTROLLER_MOCK_URL"]) +with open(destination, "w", encoding="utf-8") as destination_file: + destination_file.write(fixture) +PY +curl -fsS -X PUT \ + "$mockserver_url/mockserver/expectation" \ + -H 'Content-Type: application/json' \ + --data-binary "@$initializer_file" \ + >/dev/null + +repository_policy=$(python3 - <<'PY' +import json + +print(json.dumps({ + "Version": "2012-10-17", + "Statement": [{ + "Sid": "AllowAccountPull", + "Effect": "Allow", + "Principal": {"AWS": "arn:aws:iam::000000000000:root"}, + "Action": [ + "ecr:BatchCheckLayerAvailability", + "ecr:BatchGetImage", + "ecr:GetDownloadUrlForLayer", + ], + }], +})) +PY +) + +ministack_aws ecr create-repository \ + --repository-name "$repository_name" \ + --image-tag-mutability IMMUTABLE \ + --image-scanning-configuration scanOnPush=false \ + >/dev/null +ministack_aws ecr set-repository-policy \ + --repository-name "$repository_name" \ + --policy-text "$repository_policy" \ + >/dev/null + +docker build \ + --target runtime \ + --file "$source_root/lambdas/services/scale-set/Dockerfile" \ + --tag "$image_reference" \ + "$source_root" + +ministack_aws ecr get-login-password | docker login \ + --username AWS \ + --password-stdin localhost:4566 >/dev/null +docker push "$image_reference" + +ministack_aws ecr describe-images \ + --repository-name "$repository_name" \ + --image-ids imageTag=smoke \ + >/dev/null + +openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out "$app_key_file" 2>/dev/null +app_key_base64=$(base64 < "$app_key_file" | tr -d '\n') +APP_KEY_BASE64="$app_key_base64" GITHUB_CONFIG_URL="$controller_mock_url" python3 - "$base_tfvars" "$tfvars_file" <<'PY' +import os +import sys + +source, destination = sys.argv[1:] +replacement = os.environ["APP_KEY_BASE64"] +github_config_url = os.environ["GITHUB_CONFIG_URL"] +with open(source, encoding="utf-8") as source_file: + lines = source_file.readlines() +with open(destination, "w", encoding="utf-8") as destination_file: + for line in lines: + if line.lstrip().startswith("key_base64"): + destination_file.write(f' key_base64 = "{replacement}"\n') + elif line.lstrip().startswith("config_url"): + destination_file.write(f' config_url = "{github_config_url}"\n') + else: + destination_file.write(line) +PY +unset app_key_base64 APP_KEY_BASE64 + +terraform_state_exists=true +"$source_root/tests/ministack/run-example.sh" apply "$example" "$tfvars_file" + +config_path="/ministack-scale-set/scale-set-controller/linux-scale-set/linux-scale-set" +ministack_aws ssm get-parameter \ + --name "$config_path" \ + --query 'Parameter.Value' \ + --output text > "$config_file" +EXPECTED_GITHUB_CONFIG_URL="$controller_mock_url/example" python3 - "$config_file" <<'PY' +import json +import os +import sys + +with open(sys.argv[1], encoding="utf-8") as config_file: + config = json.load(config_file) + +assert config["githubConfigUrl"] == os.environ["EXPECTED_GITHUB_CONFIG_URL"] +assert config["forceGhes"] is True +assert config["sslVerify"] is False +assert config["minRunners"] == 1 +assert config["githubApp"]["appIdParameterName"] +assert config["githubApp"]["installationIdParameterName"] +assert config["githubApp"]["privateKeyParameterName"] +print(" [PASS] SSM manifest has the expected MockServer and GitHub App settings") +PY + +task_definition=$(ministack_aws ecs list-task-definitions \ + --family-prefix ministack-scale-set-ss-linux-scale-se- \ + --sort DESC \ + --query 'taskDefinitionArns[0]' \ + --output text) +if [ -z "$task_definition" ] || [ "$task_definition" = "None" ]; then + echo "The scale-set task definition was not registered." >&2 + exit 1 +fi + +actual_image=$(ministack_aws ecs describe-task-definition \ + --task-definition "$task_definition" \ + --query 'taskDefinition.containerDefinitions[?name==`scale-set-controller`].image | [0]' \ + --output text) +if [ "$actual_image" != "$image_reference" ]; then + echo "Expected the ECS task to use $image_reference, got $actual_image." >&2 + exit 1 +fi +printf '%s\n' ' [PASS] ECS task definition uses the image pushed to MiniStack ECR' + +log_driver=$(ministack_aws ecs describe-task-definition \ + --task-definition "$task_definition" \ + --query 'taskDefinition.containerDefinitions[?name==`scale-set-controller`].logConfiguration.logDriver | [0]' \ + --output text) +if [ "$log_driver" != "awslogs" ]; then + echo "Expected the ECS task to request the awslogs driver, got $log_driver." >&2 + exit 1 +fi +printf '%s\n' ' [PASS] ECS task definition requests the awslogs driver' + +log_group=$(ministack_aws logs describe-log-groups \ + --log-group-name-prefix "/aws/ecs/ministack-scale-set-ss-linux-scale-se-" \ + --query 'logGroups[0].logGroupName' \ + --output text) +if [ -z "$log_group" ] || [ "$log_group" = "None" ]; then + echo "The scale-set CloudWatch log group was not created." >&2 + exit 1 +fi +printf '%s\n' ' [PASS] CloudWatch log group was created for the ECS controller' + +task_family=$(ministack_aws ecs describe-task-definition \ + --task-definition "$task_definition" \ + --query 'taskDefinition.family' \ + --output text) + +# MiniStack exposes ECS credentials through the gateway's container IP. The +# Node.js AWS SDK intentionally rejects that address in +# AWS_CONTAINER_CREDENTIALS_FULL_URI because non-HTTPS full URIs are limited +# to loopback and the real ECS metadata address. Use only synthetic, +# test-scoped credentials in a temporary task-definition revision so the +# smoke test still exercises the pushed image, ECS service, and controller +# lifecycle without changing the production task definition or application. +ministack_aws ecs describe-task-definition \ + --task-definition "$task_definition" \ + --query 'taskDefinition' \ + --output json > "$task_definition_file" +TASK_DEFINITION_FILE="$task_definition_file" python3 - <<'PY' +import json +import os + +path = os.environ["TASK_DEFINITION_FILE"] +with open(path, encoding="utf-8") as task_definition_file: + task_definition = json.load(task_definition_file) + +for field in ( + "taskDefinitionArn", + "revision", + "status", + "requiresAttributes", + "compatibilities", + "registeredAt", + "registeredBy", +): + task_definition.pop(field, None) + +for container in task_definition["containerDefinitions"]: + if container["name"] != "scale-set-controller": + continue + environment = container.setdefault("environment", []) + environment.extend([ + {"name": "AWS_ACCESS_KEY_ID", "value": "000000000000"}, + {"name": "AWS_SECRET_ACCESS_KEY", "value": "test-only"}, + ]) + +with open(path, "w", encoding="utf-8") as task_definition_file: + json.dump(task_definition, task_definition_file) +PY + +environment_name=$(sed -n 's/^environment[[:space:]]*=[[:space:]]*"\([^"]*\)".*/\1/p' "$base_tfvars") +cluster_name="${environment_name}-scale-set" +task_definition=$(ministack_aws ecs register-task-definition \ + --cli-input-json "file://$task_definition_file" \ + --query 'taskDefinition.taskDefinitionArn' \ + --output text) +ministack_aws ecs update-service \ + --cluster "$cluster_name" \ + --service "$task_family" \ + --task-definition "$task_definition" \ + >/dev/null + +task_revision=$(ministack_aws ecs describe-task-definition \ + --task-definition "$task_definition" \ + --query 'taskDefinition.revision' \ + --output text) +controller_container="" +attempts=90 +while [ -z "$controller_container" ]; do + controller_container=$(docker ps -a \ + --filter "label=com.amazonaws.ecs.task-definition-family=$task_family" \ + --filter "label=com.amazonaws.ecs.task-definition-version=$task_revision" \ + --filter 'name=scale-set-controller' \ + --format '{{.ID}}' | sed -n '1p') + if [ -n "$controller_container" ]; then + break + fi + attempts=$((attempts - 1)) + if [ "$attempts" -le 0 ]; then + echo "Timed out waiting for the MiniStack ECS controller container." >&2 + docker ps -a --format '{{.ID}} {{.Image}} {{.Status}} {{.Names}}' >&2 || true + exit 1 + fi + sleep 2 +done +printf ' [PASS] MiniStack started ECS controller container %s\n' "$controller_container" + +wait_for_controller_log_event() { + marker="$1" + required_text="${2:-}" + attempts=90 + while :; do + docker logs "$controller_container" > "$controller_log_file" 2>&1 || true + if python3 - "$controller_log_file" "$marker" "$required_text" <<'PY' +import sys + +marker, required = sys.argv[2:] +with open(sys.argv[1], encoding="utf-8") as log_file: + messages = log_file.read().splitlines() +if any(marker in message and (not required or required in message) for message in messages): + raise SystemExit(0) +raise SystemExit(1) +PY + then + printf ' [PASS] CloudWatch logs contain %s\n' "$marker" + return 0 + fi + attempts=$((attempts - 1)) + if [ "$attempts" -le 0 ]; then + echo "Timed out waiting for controller log marker '$marker'." >&2 + cat "$controller_log_file" >&2 || true + echo "Controller AWS/ECS metadata environment:" >&2 + docker inspect --format '{{range .Config.Env}}{{println .}}{{end}}' "$controller_container" \ + | sed -E 's/^(AWS_CONTAINER_AUTHORIZATION_TOKEN|AWS_CONTAINER_CREDENTIALS_FULL_URI)=.*/\1=/' \ + | grep -E '^(AWS_|ECS_)' >&2 || true + echo "Controller network attachments:" >&2 + docker inspect --format '{{json .NetworkSettings.Networks}}' "$controller_container" >&2 || true + echo "Controller credential endpoint probe:" >&2 + docker exec "$controller_container" node -e \ + 'fetch(process.env.AWS_CONTAINER_CREDENTIALS_FULL_URI, {headers: {Authorization: process.env.AWS_CONTAINER_AUTHORIZATION_TOKEN}}).then((response) => { console.error(`status=${response.status}`); process.exit(response.ok ? 0 : 1); }).catch((error) => { console.error(`${error.name}:${error.message}`); process.exit(1); })' \ + >&2 || true + exit 1 + fi + sleep 2 + done +} + +printf '%s\n' ' [INFO] MiniStack 1.5.12 does not emit ECS awslogs streams; validating controller runtime logs instead' +wait_for_controller_log_event 'scale_set_controller_started' +wait_for_controller_log_event 'scale_set_session_created' +wait_for_controller_log_event 'scale_set_reconciled' '"desiredRunners":1' +wait_for_controller_log_event 'scale_set_reconciled' '"status":"converged"' +wait_for_scale_set_runner + +wait_for_mock_route() { + method="$1" + route="$2" + body=$(REQUEST_METHOD="$method" REQUEST_PATH="$route" python3 - <<'PY' +import json +import os + +print(json.dumps({ + "httpRequest": { + "method": os.environ["REQUEST_METHOD"], + "path": os.environ["REQUEST_PATH"], + }, + "times": {"atLeast": 1}, +})) +PY + ) + attempts=45 + while ! curl -fsS --max-time 5 -X PUT \ + http://127.0.0.1:1080/mockserver/verify \ + -H 'Content-Type: application/json' \ + --data "$body" >/dev/null 2>&1; do + attempts=$((attempts - 1)) + if [ "$attempts" -le 0 ]; then + echo "Timed out waiting for MockServer route: $method $route" >&2 + curl -sS --max-time 5 \ + 'http://127.0.0.1:1080/mockserver/retrieve?type=REQUESTS&format=JSON' >&2 || true + exit 1 + fi + sleep 2 + done + printf ' [PASS] MockServer received %s %s\n' "$method" "$route" +} + +wait_for_mock_route POST '/api/v3/app/installations/456/access_tokens' +wait_for_mock_route POST '/api/v3/orgs/example/actions/runners/registration-token' +wait_for_mock_route POST '/api/v3/actions/runner-registration' +wait_for_mock_route GET '/tenant/123/_apis/runtime/runnergroups/' +wait_for_mock_route GET '/tenant/123/_apis/runtime/runnerscalesets' +wait_for_mock_route GET '/tenant/123/_apis/runtime/runnerscalesets/223' +wait_for_mock_route PATCH '/tenant/123/_apis/runtime/runnerscalesets/223' +wait_for_mock_route POST '/tenant/123/_apis/runtime/runnerscalesets/223/generatejitconfig' +wait_for_mock_route POST '/tenant/123/_apis/runtime/runnerscalesets/223/sessions' +wait_for_mock_route GET '/messages' + +python3 - "$config_file" "$scale_down_config_file" <<'PY' +import json +import sys + +with open(sys.argv[1], encoding="utf-8") as config_file: + reconciler = json.load(config_file) + +assert reconciler["minRunners"] == 1 +reconciler["minRunners"] = 0 +with open(sys.argv[2], "w", encoding="utf-8") as config_file: + json.dump(reconciler, config_file) +PY +ministack_aws ssm put-parameter \ + --name "$config_path" \ + --type String \ + --value "$(cat "$scale_down_config_file")" \ + --overwrite \ + >/dev/null +printf '%s\n' ' [PASS] SSM manifest minimum changed from one runner to zero' + +SCALE_DOWN_CONFIG_FILE="$scale_down_config_file" TASK_DEFINITION_FILE="$task_definition_file" python3 - <<'PY' +import json +import os + +with open(os.environ["SCALE_DOWN_CONFIG_FILE"], encoding="utf-8") as config_file: + reconciler = json.load(config_file) +with open(os.environ["TASK_DEFINITION_FILE"], encoding="utf-8") as task_definition_file: + task_definition = json.load(task_definition_file) + +for container in task_definition["containerDefinitions"]: + if container["name"] != "scale-set-controller": + continue + for environment in container.get("environment", []): + if environment["name"] == "SCALE_SET_CONTROLLER_MANIFEST": + manifest = json.loads(environment["value"]) + assert len(manifest.get("reconcilers", [])) == 1 + manifest["reconcilers"][0]["minRunners"] = reconciler["minRunners"] + environment["value"] = json.dumps(manifest, separators=(",", ":")) + break + else: + raise RuntimeError("scale-set controller task definition has no inline manifest") + +with open(os.environ["TASK_DEFINITION_FILE"], "w", encoding="utf-8") as task_definition_file: + json.dump(task_definition, task_definition_file) +PY +task_definition=$(ministack_aws ecs register-task-definition \ + --cli-input-json "file://$task_definition_file" \ + --query 'taskDefinition.taskDefinitionArn' \ + --output text) +task_revision=$(ministack_aws ecs describe-task-definition \ + --task-definition "$task_definition" \ + --query 'taskDefinition.revision' \ + --output text) +ministack_aws ecs update-service \ + --cluster "$cluster_name" \ + --service "$task_family" \ + --task-definition "$task_definition" \ + --force-new-deployment \ + >/dev/null + +old_controller_container="$controller_container" +new_controller_container="" +attempts=90 +while [ -z "$new_controller_container" ]; do + for candidate in $(docker ps -a \ + --filter "label=com.amazonaws.ecs.task-definition-family=$task_family" \ + --filter "label=com.amazonaws.ecs.task-definition-version=$task_revision" \ + --filter 'name=scale-set-controller' \ + --format '{{.ID}}'); do + if [ "$candidate" != "$old_controller_container" ]; then + new_controller_container="$candidate" + break + fi + done + if [ -n "$new_controller_container" ]; then + break + fi + attempts=$((attempts - 1)) + if [ "$attempts" -le 0 ]; then + echo 'Timed out waiting for the ECS service to deploy the scale-down task.' >&2 + docker ps -a --format '{{.ID}} {{.Image}} {{.Status}} {{.Names}}' >&2 || true + exit 1 + fi + sleep 2 +done +controller_container="$new_controller_container" +printf ' [PASS] ECS service deployed a fresh controller container %s for scale-down\n' "$controller_container" +wait_for_controller_log_event 'scale_set_controller_started' +wait_for_controller_log_event 'scale_set_session_created' +wait_for_controller_log_event 'scale_set_reconciled' '"desiredRunners":0' +wait_for_controller_log_event 'scale_set_reconciled' '"status":"converged"' +wait_for_no_scale_set_runners + +"$source_root/tests/ministack/run-example.sh" destroy "$example" "$tfvars_file" +terraform_state_exists=false +wait_for_mock_route DELETE '/tenant/123/_apis/runtime/runnerscalesets/223/sessions/11111111-1111-1111-1111-111111111111' + +echo 'Scale-set MiniStack ECS/MockServer smoke test passed.' From cf6b86ad8a92f3b638ca4e477c4512d5c9129a25 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 21 Sep 2026 12:51:33 +0000 Subject: [PATCH 20/20] docs: auto update terraform docs --- README.md | 3 ++- modules/compute-providers/aws/ec2/README.md | 2 +- modules/multi-runner/README.md | 2 +- modules/runner-config/README.md | 2 +- modules/runners/README.md | 1 + modules/webhook/README.md | 2 +- 6 files changed, 7 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 9f5c9041b1..8ba80adff0 100644 --- a/README.md +++ b/README.md @@ -113,7 +113,7 @@ Join our discord community via [this invite link](https://discord.gg/bxgXW8jJGh) | [ami\_housekeeper\_lambda\_timeout](#input\_ami\_housekeeper\_lambda\_timeout) | Time out of the lambda in seconds. | `number` | `300` | no | | [ami\_housekeeper\_lambda\_zip](#input\_ami\_housekeeper\_lambda\_zip) | File location of the lambda zip file. | `string` | `null` | no | | [associate\_public\_ipv4\_address](#input\_associate\_public\_ipv4\_address) | Associate public IPv4 with the runner. Only tested with IPv4 | `bool` | `false` | no | -| [aws\_dynamic\_labels\_policy](#input\_aws\_dynamic\_labels\_policy) | Experimental! Can be removed / changed without trigger a major release.
Optional AWS dynamic label policy evaluated by the webhook dispatcher.
Only effective when `enable_dynamic_labels = true`.

Jobs whose provider-specific dynamic labels violate the policy are rejected
with a 202 and a warning is logged. Currently this policy applies to EC2
override labels using the `ghr-ec2-*` prefix.

Evaluation:
1. Keys in `blocked_keys` are always rejected.
2. Keys in `restricted_keys` are allowed only when their value passes the rule.
3. Keys not listed in `blocked_keys` or `restricted_keys` are allowed.

Schema:
- `blocked_keys`: keys to reject outright.
- `restricted_keys`: map of key to value rule:
`{ allowed = [globs], denied = [globs], max = number|string }`.

Keys use the provider dynamic label suffix, not the full label. For example,
use `instance-type` for `ghr-ec2-instance-type`. | `any` | `null` | no | +| [aws\_dynamic\_labels\_policy](#input\_aws\_dynamic\_labels\_policy) | Experimental! Can be removed / changed without trigger a major release.
Optional AWS dynamic label policy evaluated by the webhook dispatcher.
Only effective when `enable_dynamic_labels = true`.

Jobs whose provider-specific dynamic labels violate the policy are rejected
with a 202 and a warning is logged. Currently this policy applies to EC2
override labels using the `ghr-ec2-*` prefix.

Evaluation:
1. If `allowed_keys` is set (non-empty), any key not listed in it is rejected;
everything else in the policy still applies to the keys it does allow.
2. Keys in `blocked_keys` are always rejected. Cannot be used together with
`allowed_keys` — see `docs/configuration.md` for why.
3. Keys in `restricted_keys` are allowed only when their value passes the rule.
4. A key not listed anywhere above is allowed.

Schema:
- `allowed_keys`: only these keys are accepted; every other key is rejected.
- `blocked_keys`: keys to reject outright.
- `restricted_keys`: map of key to value rule:
`{ allowed = [globs], denied = [globs], max = number|string }`.

Keys use the provider dynamic label suffix, not the full label. For example,
use `instance-type` for `ghr-ec2-instance-type`. | `any` | `null` | no | | [aws\_partition](#input\_aws\_partition) | (optiona) partition in the arn namespace to use if not 'aws' | `string` | `"aws"` | no | | [aws\_region](#input\_aws\_region) | AWS region. | `string` | n/a | yes | | [block\_device\_mappings](#input\_block\_device\_mappings) | The EC2 instance block device configuration. Takes the following keys: `device_name`, `delete_on_termination`, `volume_type`, `volume_size`, `encrypted`, `iops`, `throughput`, `kms_key_id`, `snapshot_id`, `volume_initialization_rate`. | 
list(object({
    delete_on_termination      = optional(bool, true)
    device_name                = optional(string, "/dev/xvda")
    encrypted                  = optional(bool, true)
    iops                       = optional(number)
    kms_key_id                 = optional(string)
    snapshot_id                = optional(string)
    throughput                 = optional(number)
    volume_initialization_rate = optional(number)
    volume_size                = number
    volume_type                = optional(string, "gp3")
  }))
| 
[
  {
    "volume_size": 30
  }
]
| no | @@ -210,6 +210,7 @@ Join our discord community via [this invite link](https://discord.gg/bxgXW8jJGh) | [runner\_log\_files](#input\_runner\_log\_files) | (optional) List of logfiles to send to CloudWatch, will only be used if `enable_cloudwatch_agent` is set to true. Object description: `log_group_name`: Name of the log group, `prefix_log_group`: If true, the log group name will be prefixed with `/github-self-hosted-runners/`, `file_path`: path to the log file, `log_stream_name`: name of the log stream, `log_class`: The log class of the log group. Valid values are `STANDARD` or `INFREQUENT_ACCESS`. Defaults to `STANDARD`. | 
list(object({
    log_group_name   = string
    prefix_log_group = bool
    file_path        = string
    log_stream_name  = string
    log_class        = optional(string, "STANDARD")
  }))
| `null` | no | | [runner\_metadata\_options](#input\_runner\_metadata\_options) | Metadata options for the ec2 runner instances. By default, the module uses metadata tags for bootstrapping the runner, only disable `instance_metadata_tags` when using custom scripts for starting the runner. | `map(any)` | 
{
  "http_endpoint": "enabled",
  "http_put_response_hop_limit": 1,
  "http_tokens": "required",
  "instance_metadata_tags": "enabled"
}
| no | | [runner\_name\_prefix](#input\_runner\_name\_prefix) | The prefix used for the GitHub runner name. The prefix will be used in the default start script to prefix the instance name when register the runner in GitHub. The value is available via an EC2 tag 'ghr:runner\_name\_prefix'. | `string` | `""` | no | +| [runner\_network\_interfaces](#input\_runner\_network\_interfaces) | Advanced network interface configuration for the runner launch template. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/launch_template#network-interfaces for details. Leave unset (default) to keep using associate\_public\_ipv4\_address for a simple single-interface setup; set this to fully control one or more interfaces. | 
list(object({
    associate_carrier_ip_address = optional(bool)
    associate_public_ip_address  = optional(bool)
    delete_on_termination        = optional(bool)
    description                  = optional(string)
    device_index                 = optional(number)
    interface_type               = optional(string)
    ipv4_address_count           = optional(number)
    ipv4_addresses               = optional(list(string))
    ipv4_prefix_count            = optional(number)
    ipv4_prefixes                = optional(list(string))
    ipv6_address_count           = optional(number)
    ipv6_addresses               = optional(list(string))
    ipv6_prefix_count            = optional(number)
    ipv6_prefixes                = optional(list(string))
    network_card_index           = optional(number)
    network_interface_id         = optional(string)
    primary_ipv6                 = optional(bool)
    private_ip_address           = optional(string)
    security_groups              = optional(list(string))
    subnet_id                    = optional(string)
    connection_tracking_specification = optional(object({
      tcp_established_timeout = optional(number)
      udp_stream_timeout      = optional(number)
      udp_timeout             = optional(number)
    }))
    ena_srd_specification = optional(object({
      ena_srd_enabled = optional(bool)
      ena_srd_udp_specification = optional(object({
        ena_srd_udp_enabled = optional(bool)
      }))
    }))
  }))
| `[]` | no | | [runner\_os](#input\_runner\_os) | The EC2 Operating System type to use for action runner instances (linux, osx, windows). | `string` | `"linux"` | no | | [runner\_placement](#input\_runner\_placement) | The placement options for the instance. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/launch_template#placement for details. | 
object({
    affinity                = optional(string)
    availability_zone       = optional(string)
    group_id                = optional(string)
    group_name              = optional(string)
    host_id                 = optional(string)
    host_resource_group_arn = optional(string)
    spread_domain           = optional(string)
    tenancy                 = optional(string)
    partition_number        = optional(number)
  })
| `null` | no | | [runner\_run\_as](#input\_runner\_run\_as) | Run the GitHub actions agent as user. | `string` | `"ec2-user"` | no | diff --git a/modules/compute-providers/aws/ec2/README.md b/modules/compute-providers/aws/ec2/README.md index ec44a82fa8..6c692445c9 100644 --- a/modules/compute-providers/aws/ec2/README.md +++ b/modules/compute-providers/aws/ec2/README.md @@ -61,7 +61,7 @@ No modules. |------|-------------|------|---------|:--------:| | [aws\_partition](#input\_aws\_partition) | AWS partition used to construct IAM ARNs. | `string` | `"aws"` | no | | [aws\_region](#input\_aws\_region) | AWS region used by compute-provider resources and policy documents. | `string` | n/a | yes | -| [config](#input\_config) | EC2 compute-provider configuration. Paths match `compute_provider.aws.ec2` in the runner configuration.

- `ami`: Optional AMI discovery and encryption configuration. Null selects defaults for `runner.os` and `runner.architecture`.
- `ami.filter`: AMI filter names mapped to accepted values and merged over the provider defaults.
- `ami.owners`: AWS account IDs or aliases allowed to own the selected AMI.
- `ami.id_ssm_parameter`: Optional externally managed SSM parameter containing the AMI ID. Its object presence is the plan-time ownership discriminator.
- `ami.id_ssm_parameter.arn`: ARN of the external AMI-ID parameter. The ARN may remain unknown until apply.
- `ami.kms_key`: Optional customer-managed KMS key required for encrypted AMIs or snapshots. Its object presence is the plan-time policy discriminator.
- `ami.kms_key.arn`: ARN of the AMI KMS key. The ARN may remain unknown until apply.
- `vpc_id`: VPC in which runner networking resources are created.
- `subnet_ids`: Subnets from which the control plane may launch runners.
- `overrides.name_runner`: Optional Name tag override for runner compute resources.
- `overrides.name_sg`: Optional Name tag override for the managed security group.
- `instance_profile`: Optional externally managed instance profile. Its object presence is the plan-time ownership discriminator.
- `instance_profile.name`: Name of the external instance profile. The name may remain unknown until apply.
- `instance_profile_path`: IAM path for the provider-managed instance profile. Null derives the path from `prefix`.
- `binaries_syncer.enabled`: Uses the synchronized runner distribution from S3 during bootstrap.
- `binaries_syncer.s3`: S3 object containing the synchronized runner distribution. Required when synchronization is enabled.
- `binaries_syncer.s3.arn`: Runner-distribution bucket ARN used by IAM policies.
- `binaries_syncer.s3.id`: Runner-distribution bucket name used in the bootstrap URI.
- `binaries_syncer.s3.key`: Runner-distribution object key.
- `block_device_mappings`: EBS mappings added to the launch template.
- `block_device_mappings[].delete_on_termination`: Deletes the volume when its runner terminates.
- `block_device_mappings[].device_name`: Device name exposed to the runner instance.
- `block_device_mappings[].encrypted`: Enables EBS encryption.
- `block_device_mappings[].iops`: Provisioned IOPS for volume types that support configurable IOPS.
- `block_device_mappings[].kms_key_id`: KMS key ID or ARN used to encrypt the volume.
- `block_device_mappings[].snapshot_id`: Snapshot used to initialize the volume.
- `block_device_mappings[].throughput`: Provisioned throughput for volume types that support it.
- `block_device_mappings[].volume_initialization_rate`: Fixed initialization rate for supported snapshot-backed volumes.
- `block_device_mappings[].volume_size`: EBS volume size in GiB.
- `block_device_mappings[].volume_type`: EBS volume type.
- `ebs_optimized`: Requests EBS-optimized instances.
- `instance_target_capacity_type`: Primary capacity type, either `spot` or `on-demand`.
- `instance_allocation_strategy`: EC2 Fleet allocation strategy.
- `instance_type_priorities`: Optional numeric priorities keyed by instance type.
- `instance_max_spot_price`: Optional maximum hourly Spot price.
- `instance_types`: EC2 instance types available to the control plane.
- `user_data`: Runner bootstrap user-data configuration.
- `user_data.enabled`: Enables launch-template user data.
- `user_data.template`: Optional path to a custom user-data template.
- `user_data.content`: Optional complete user-data content used instead of a template.
- `user_data.pre_install`: Script inserted before runner installation.
- `user_data.post_install`: Script inserted after runner installation.
- `user_data.debug_logging_enabled`: Enables verbose user-data tracing, which can expose secrets.
- `ssm_enabled`: Includes Session Manager permissions in the provider's runner policy group.
- `create_service_linked_role_spot`: Allows scale-up to create the EC2 Spot service-linked role.
- `cloudwatch_agent.enabled`: Enables CloudWatch agent configuration for runner instances.
- `cloudwatch_agent.config`: Optional complete CloudWatch agent configuration.
- `managed_security_group_enabled`: Creates and attaches the provider-managed security group.
- `log_files`: Optional files collected by the CloudWatch agent. Null uses provider defaults.
- `log_files[].log_group_name`: CloudWatch log-group name before optional prefixing.
- `log_files[].prefix_log_group`: Prefixes the log-group name with the runner configuration path.
- `log_files[].file_path`: File or glob read by the CloudWatch agent.
- `log_files[].log_stream_name`: CloudWatch log-stream name template.
- `log_files[].log_class`: CloudWatch log-group class for the collected file.
- `key_name`: Optional EC2 key-pair name.
- `additional_security_group_ids`: Existing security groups attached to runners.
- `detailed_monitoring_enabled`: Enables detailed EC2 monitoring.
- `egress_rules`: Rules created on the managed security group.
- `egress_rules[].cidr_blocks`: IPv4 CIDR destinations.
- `egress_rules[].ipv6_cidr_blocks`: IPv6 CIDR destinations.
- `egress_rules[].prefix_list_ids`: AWS prefix-list destinations.
- `egress_rules[].from_port`: First destination port in the permitted range.
- `egress_rules[].protocol`: IP protocol name or number. Use `-1` for all protocols.
- `egress_rules[].security_groups`: Destination security-group IDs.
- `egress_rules[].self`: Allows traffic to the managed security group itself.
- `egress_rules[].to_port`: Last destination port in the permitted range.
- `egress_rules[].description`: Optional rule description.
- `tags`: Runner instance, volume, network-interface, and eligible Spot-request tags. Provider-required bootstrap tags take final precedence.
- `metadata_options`: Instance Metadata Service configuration.
- `metadata_options.instance_metadata_tags`: Exposes instance tags through Instance Metadata Service when enabled.
- `metadata_options.http_endpoint`: Enables or disables the Instance Metadata Service endpoint.
- `metadata_options.http_tokens`: Controls whether IMDSv2 session tokens are optional or required.
- `metadata_options.http_put_response_hop_limit`: Network hop limit for Instance Metadata Service token responses.
- `credit_specification`: CPU credit mode for burstable instance types.
- `cpu_options`: CPU topology and processor-feature configuration.
- `cpu_options.core_count`: Number of CPU cores exposed to the runner instance.
- `cpu_options.threads_per_core`: Number of hardware threads exposed per CPU core.
- `cpu_options.amd_sev_snp`: Enables or disables AMD SEV-SNP on supported instance types.
- `cpu_options.nested_virtualization`: Enables or disables nested virtualization on supported instance types.
- `placement`: EC2 placement configuration.
- `placement.affinity`: Dedicated Host affinity setting.
- `placement.availability_zone`: Availability Zone in which runner instances are placed.
- `placement.group_id`: Placement-group ID.
- `placement.group_name`: Placement-group name.
- `placement.host_id`: Dedicated Host ID.
- `placement.host_resource_group_arn`: ARN of the host resource group used for placement.
- `placement.spread_domain`: Spread-domain placement value.
- `placement.tenancy`: Instance tenancy, such as `default`, `dedicated`, or `host`.
- `placement.partition_number`: Placement-group partition number.
- `license_specifications`: License Manager configurations added to the launch template.
- `license_specifications[].license_configuration_arn`: ARN of an AWS License Manager license configuration.
- `associate_public_ipv4_address`: Associates a public IPv4 address with runner network interfaces.
- `on_demand_failover_for_errors`: EC2 errors that trigger on-demand fallback after a Spot failure.
- `scale_errors`: EC2 errors treated as retryable scale-up failures.
- `use_dedicated_host`: Enables the dedicated-host launch path. | 
object({
    ami = optional(object({
      filter = optional(map(list(string)), { state = ["available"] })
      owners = optional(list(string), ["amazon"])
      id_ssm_parameter = optional(object({
        arn = string
      }), null)
      kms_key = optional(object({
        arn = string
      }), null)
    }), null)
    vpc_id     = string
    subnet_ids = list(string)
    overrides = optional(object({
      name_runner = optional(string, "")
      name_sg     = optional(string, "")
    }), {})
    instance_profile = optional(object({
      name = string
    }), null)
    instance_profile_path = optional(string, null)
    binaries_syncer = optional(object({
      enabled = optional(bool, true)
      s3 = optional(object({
        arn = string
        id  = string
        key = string
      }), null)
    }), {})
    block_device_mappings = optional(list(object({
      delete_on_termination      = optional(bool, true)
      device_name                = optional(string, "/dev/xvda")
      encrypted                  = optional(bool, true)
      iops                       = optional(number)
      kms_key_id                 = optional(string)
      snapshot_id                = optional(string)
      throughput                 = optional(number)
      volume_initialization_rate = optional(number)
      volume_size                = number
      volume_type                = optional(string, "gp3")
    })), [{ volume_size = 30 }])
    ebs_optimized                 = optional(bool, false)
    instance_target_capacity_type = optional(string, "spot")
    instance_allocation_strategy  = optional(string, "lowest-price")
    instance_type_priorities      = optional(map(number), null)
    instance_max_spot_price       = optional(string, null)
    instance_types                = list(string)
    user_data = optional(object({
      enabled               = optional(bool, true)
      template              = optional(string, null)
      content               = optional(string, null)
      pre_install           = optional(string, "")
      post_install          = optional(string, "")
      debug_logging_enabled = optional(bool, false)
    }), {})
    ssm_enabled                     = optional(bool, false)
    create_service_linked_role_spot = optional(bool, false)
    cloudwatch_agent = optional(object({
      enabled = optional(bool, true)
      config  = optional(string, null)
    }), {})
    managed_security_group_enabled = optional(bool, true)
    log_files = optional(list(object({
      log_group_name   = string
      prefix_log_group = bool
      file_path        = string
      log_stream_name  = string
      log_class        = optional(string, "STANDARD")
    })), null)
    key_name                      = optional(string, null)
    additional_security_group_ids = optional(list(string), [])
    detailed_monitoring_enabled   = optional(bool, false)
    egress_rules = optional(list(object({
      cidr_blocks      = list(string)
      ipv6_cidr_blocks = list(string)
      prefix_list_ids  = list(string)
      from_port        = number
      protocol         = string
      security_groups  = list(string)
      self             = bool
      to_port          = number
      description      = string
      })), [{
      cidr_blocks      = ["0.0.0.0/0"]
      ipv6_cidr_blocks = ["::/0"]
      prefix_list_ids  = null
      from_port        = 0
      protocol         = "-1"
      security_groups  = null
      self             = null
      to_port          = 0
      description      = null
    }])
    tags = optional(map(string), {})
    metadata_options = optional(object({
      instance_metadata_tags      = optional(string, "enabled")
      http_endpoint               = optional(string, "enabled")
      http_tokens                 = optional(string, "required")
      http_put_response_hop_limit = optional(number, 1)
    }), {})
    credit_specification = optional(string, null)
    cpu_options = optional(object({
      core_count            = optional(number)
      threads_per_core      = optional(number)
      amd_sev_snp           = optional(string)
      nested_virtualization = optional(string)
    }), null)
    placement = optional(object({
      affinity                = optional(string)
      availability_zone       = optional(string)
      group_id                = optional(string)
      group_name              = optional(string)
      host_id                 = optional(string)
      host_resource_group_arn = optional(string)
      spread_domain           = optional(string)
      tenancy                 = optional(string)
      partition_number        = optional(number)
    }), null)
    license_specifications = optional(list(object({
      license_configuration_arn = string
    })), [])
    associate_public_ipv4_address = optional(bool, false)
    on_demand_failover_for_errors = optional(list(string), [])
    scale_errors = optional(list(string), [
      "UnfulfillableCapacity",
      "MaxSpotInstanceCountExceeded",
      "TargetCapacityLimitExceededException",
      "RequestLimitExceeded",
      "ResourceLimitExceeded",
      "MaxSpotInstanceCountExceeded",
      "MaxSpotFleetRequestCountExceeded",
      "InsufficientInstanceCapacity",
      "InsufficientCapacityOnHost",
    ])
    use_dedicated_host = optional(bool, false)
  })
| n/a | yes | +| [config](#input\_config) | EC2 compute-provider configuration. Paths match `compute_provider.aws.ec2` in the runner configuration.

- `ami`: Optional AMI discovery and encryption configuration. Null selects defaults for `runner.os` and `runner.architecture`.
- `ami.filter`: AMI filter names mapped to accepted values and merged over the provider defaults.
- `ami.owners`: AWS account IDs or aliases allowed to own the selected AMI.
- `ami.id_ssm_parameter`: Optional externally managed SSM parameter containing the AMI ID. Its object presence is the plan-time ownership discriminator.
- `ami.id_ssm_parameter.arn`: ARN of the external AMI-ID parameter. The ARN may remain unknown until apply.
- `ami.kms_key`: Optional customer-managed KMS key required for encrypted AMIs or snapshots. Its object presence is the plan-time policy discriminator.
- `ami.kms_key.arn`: ARN of the AMI KMS key. The ARN may remain unknown until apply.
- `vpc_id`: VPC in which runner networking resources are created.
- `subnet_ids`: Subnets from which the control plane may launch runners.
- `overrides.name_runner`: Optional Name tag override for runner compute resources.
- `overrides.name_sg`: Optional Name tag override for the managed security group.
- `instance_profile`: Optional externally managed instance profile. Its object presence is the plan-time ownership discriminator.
- `instance_profile.name`: Name of the external instance profile. The name may remain unknown until apply.
- `instance_profile_path`: IAM path for the provider-managed instance profile. Null derives the path from `prefix`.
- `binaries_syncer.enabled`: Uses the synchronized runner distribution from S3 during bootstrap.
- `binaries_syncer.s3`: S3 object containing the synchronized runner distribution. Required when synchronization is enabled.
- `binaries_syncer.s3.arn`: Runner-distribution bucket ARN used by IAM policies.
- `binaries_syncer.s3.id`: Runner-distribution bucket name used in the bootstrap URI.
- `binaries_syncer.s3.key`: Runner-distribution object key.
- `block_device_mappings`: EBS mappings added to the launch template.
- `block_device_mappings[].delete_on_termination`: Deletes the volume when its runner terminates.
- `block_device_mappings[].device_name`: Device name exposed to the runner instance.
- `block_device_mappings[].encrypted`: Enables EBS encryption.
- `block_device_mappings[].iops`: Provisioned IOPS for volume types that support configurable IOPS.
- `block_device_mappings[].kms_key_id`: KMS key ID or ARN used to encrypt the volume.
- `block_device_mappings[].snapshot_id`: Snapshot used to initialize the volume.
- `block_device_mappings[].throughput`: Provisioned throughput for volume types that support it.
- `block_device_mappings[].volume_initialization_rate`: Fixed initialization rate for supported snapshot-backed volumes.
- `block_device_mappings[].volume_size`: EBS volume size in GiB.
- `block_device_mappings[].volume_type`: EBS volume type.
- `ebs_optimized`: Requests EBS-optimized instances.
- `instance_target_capacity_type`: Primary capacity type, either `spot` or `on-demand`.
- `instance_allocation_strategy`: EC2 Fleet allocation strategy.
- `instance_type_priorities`: Optional numeric priorities keyed by instance type.
- `instance_max_spot_price`: Optional maximum hourly Spot price.
- `instance_types`: EC2 instance types available to the control plane.
- `user_data`: Runner bootstrap user-data configuration.
- `user_data.enabled`: Enables launch-template user data.
- `user_data.template`: Optional path to a custom user-data template.
- `user_data.content`: Optional complete user-data content used instead of a template.
- `user_data.pre_install`: Script inserted before runner installation.
- `user_data.post_install`: Script inserted after runner installation.
- `user_data.debug_logging_enabled`: Enables verbose user-data tracing, which can expose secrets.
- `ssm_enabled`: Includes Session Manager permissions in the provider's runner policy group.
- `create_service_linked_role_spot`: Allows scale-up to create the EC2 Spot service-linked role.
- `cloudwatch_agent.enabled`: Enables CloudWatch agent configuration for runner instances.
- `cloudwatch_agent.config`: Optional complete CloudWatch agent configuration.
- `managed_security_group_enabled`: Creates and attaches the provider-managed security group.
- `log_files`: Optional files collected by the CloudWatch agent. Null uses provider defaults.
- `log_files[].log_group_name`: CloudWatch log-group name before optional prefixing.
- `log_files[].prefix_log_group`: Prefixes the log-group name with the runner configuration path.
- `log_files[].file_path`: File or glob read by the CloudWatch agent.
- `log_files[].log_stream_name`: CloudWatch log-stream name template.
- `log_files[].log_class`: CloudWatch log-group class for the collected file.
- `key_name`: Optional EC2 key-pair name.
- `additional_security_group_ids`: Existing security groups attached to runners.
- `detailed_monitoring_enabled`: Enables detailed EC2 monitoring.
- `egress_rules`: Rules created on the managed security group.
- `egress_rules[].cidr_blocks`: IPv4 CIDR destinations.
- `egress_rules[].ipv6_cidr_blocks`: IPv6 CIDR destinations.
- `egress_rules[].prefix_list_ids`: AWS prefix-list destinations.
- `egress_rules[].from_port`: First destination port in the permitted range.
- `egress_rules[].protocol`: IP protocol name or number. Use `-1` for all protocols.
- `egress_rules[].security_groups`: Destination security-group IDs.
- `egress_rules[].self`: Allows traffic to the managed security group itself.
- `egress_rules[].to_port`: Last destination port in the permitted range.
- `egress_rules[].description`: Optional rule description.
- `tags`: Runner instance, volume, network-interface, and eligible Spot-request tags. Provider-required bootstrap tags take final precedence.
- `metadata_options`: Instance Metadata Service configuration.
- `metadata_options.instance_metadata_tags`: Exposes instance tags through Instance Metadata Service when enabled.
- `metadata_options.http_endpoint`: Enables or disables the Instance Metadata Service endpoint.
- `metadata_options.http_tokens`: Controls whether IMDSv2 session tokens are optional or required.
- `metadata_options.http_put_response_hop_limit`: Network hop limit for Instance Metadata Service token responses.
- `credit_specification`: CPU credit mode for burstable instance types.
- `cpu_options`: CPU topology and processor-feature configuration.
- `cpu_options.core_count`: Number of CPU cores exposed to the runner instance.
- `cpu_options.threads_per_core`: Number of hardware threads exposed per CPU core.
- `cpu_options.amd_sev_snp`: Enables or disables AMD SEV-SNP on supported instance types.
- `cpu_options.nested_virtualization`: Enables or disables nested virtualization on supported instance types.
- `placement`: EC2 placement configuration.
- `placement.affinity`: Dedicated Host affinity setting.
- `placement.availability_zone`: Availability Zone in which runner instances are placed.
- `placement.group_id`: Placement-group ID.
- `placement.group_name`: Placement-group name.
- `placement.host_id`: Dedicated Host ID.
- `placement.host_resource_group_arn`: ARN of the host resource group used for placement.
- `placement.spread_domain`: Spread-domain placement value.
- `placement.tenancy`: Instance tenancy, such as `default`, `dedicated`, or `host`.
- `placement.partition_number`: Placement-group partition number.
- `license_specifications`: License Manager configurations added to the launch template.
- `license_specifications[].license_configuration_arn`: ARN of an AWS License Manager license configuration.
- `associate_public_ipv4_address`: Associates a public IPv4 address with runner network interfaces.
- `network_interfaces`: Advanced network interface configuration for the launch template. Leave empty to keep using `associate_public_ipv4_address` for a simple single-interface setup.
- `on_demand_failover_for_errors`: EC2 errors that trigger on-demand fallback after a Spot failure.
- `scale_errors`: EC2 errors treated as retryable scale-up failures.
- `use_dedicated_host`: Enables the dedicated-host launch path. | 
object({
    ami = optional(object({
      filter = optional(map(list(string)), { state = ["available"] })
      owners = optional(list(string), ["amazon"])
      id_ssm_parameter = optional(object({
        arn = string
      }), null)
      kms_key = optional(object({
        arn = string
      }), null)
    }), null)
    vpc_id     = string
    subnet_ids = list(string)
    overrides = optional(object({
      name_runner = optional(string, "")
      name_sg     = optional(string, "")
    }), {})
    instance_profile = optional(object({
      name = string
    }), null)
    instance_profile_path = optional(string, null)
    binaries_syncer = optional(object({
      enabled = optional(bool, true)
      s3 = optional(object({
        arn = string
        id  = string
        key = string
      }), null)
    }), {})
    block_device_mappings = optional(list(object({
      delete_on_termination      = optional(bool, true)
      device_name                = optional(string, "/dev/xvda")
      encrypted                  = optional(bool, true)
      iops                       = optional(number)
      kms_key_id                 = optional(string)
      snapshot_id                = optional(string)
      throughput                 = optional(number)
      volume_initialization_rate = optional(number)
      volume_size                = number
      volume_type                = optional(string, "gp3")
    })), [{ volume_size = 30 }])
    ebs_optimized                 = optional(bool, false)
    instance_target_capacity_type = optional(string, "spot")
    instance_allocation_strategy  = optional(string, "lowest-price")
    instance_type_priorities      = optional(map(number), null)
    instance_max_spot_price       = optional(string, null)
    instance_types                = list(string)
    user_data = optional(object({
      enabled               = optional(bool, true)
      template              = optional(string, null)
      content               = optional(string, null)
      pre_install           = optional(string, "")
      post_install          = optional(string, "")
      debug_logging_enabled = optional(bool, false)
    }), {})
    ssm_enabled                     = optional(bool, false)
    create_service_linked_role_spot = optional(bool, false)
    cloudwatch_agent = optional(object({
      enabled = optional(bool, true)
      config  = optional(string, null)
    }), {})
    managed_security_group_enabled = optional(bool, true)
    log_files = optional(list(object({
      log_group_name   = string
      prefix_log_group = bool
      file_path        = string
      log_stream_name  = string
      log_class        = optional(string, "STANDARD")
    })), null)
    key_name                      = optional(string, null)
    additional_security_group_ids = optional(list(string), [])
    detailed_monitoring_enabled   = optional(bool, false)
    egress_rules = optional(list(object({
      cidr_blocks      = list(string)
      ipv6_cidr_blocks = list(string)
      prefix_list_ids  = list(string)
      from_port        = number
      protocol         = string
      security_groups  = list(string)
      self             = bool
      to_port          = number
      description      = string
      })), [{
      cidr_blocks      = ["0.0.0.0/0"]
      ipv6_cidr_blocks = ["::/0"]
      prefix_list_ids  = null
      from_port        = 0
      protocol         = "-1"
      security_groups  = null
      self             = null
      to_port          = 0
      description      = null
    }])
    tags = optional(map(string), {})
    metadata_options = optional(object({
      instance_metadata_tags      = optional(string, "enabled")
      http_endpoint               = optional(string, "enabled")
      http_tokens                 = optional(string, "required")
      http_put_response_hop_limit = optional(number, 1)
    }), {})
    credit_specification = optional(string, null)
    cpu_options = optional(object({
      core_count            = optional(number)
      threads_per_core      = optional(number)
      amd_sev_snp           = optional(string)
      nested_virtualization = optional(string)
    }), null)
    placement = optional(object({
      affinity                = optional(string)
      availability_zone       = optional(string)
      group_id                = optional(string)
      group_name              = optional(string)
      host_id                 = optional(string)
      host_resource_group_arn = optional(string)
      spread_domain           = optional(string)
      tenancy                 = optional(string)
      partition_number        = optional(number)
    }), null)
    license_specifications = optional(list(object({
      license_configuration_arn = string
    })), [])
    associate_public_ipv4_address = optional(bool, false)
    network_interfaces = optional(list(object({
      associate_carrier_ip_address = optional(bool)
      associate_public_ip_address  = optional(bool)
      delete_on_termination        = optional(bool)
      description                  = optional(string)
      device_index                 = optional(number)
      interface_type               = optional(string)
      ipv4_address_count           = optional(number)
      ipv4_addresses               = optional(list(string))
      ipv4_prefix_count            = optional(number)
      ipv4_prefixes                = optional(list(string))
      ipv6_address_count           = optional(number)
      ipv6_addresses               = optional(list(string))
      ipv6_prefix_count            = optional(number)
      ipv6_prefixes                = optional(list(string))
      network_card_index           = optional(number)
      network_interface_id         = optional(string)
      primary_ipv6                 = optional(bool)
      private_ip_address           = optional(string)
      security_groups              = optional(list(string))
      subnet_id                    = optional(string)
      connection_tracking_specification = optional(object({
        tcp_established_timeout = optional(number)
        udp_stream_timeout      = optional(number)
        udp_timeout             = optional(number)
      }))
      ena_srd_specification = optional(object({
        ena_srd_enabled = optional(bool)
        ena_srd_udp_specification = optional(object({
          ena_srd_udp_enabled = optional(bool)
        }))
      }))
    })), [])
    on_demand_failover_for_errors = optional(list(string), [])
    scale_errors = optional(list(string), [
      "UnfulfillableCapacity",
      "MaxSpotInstanceCountExceeded",
      "TargetCapacityLimitExceededException",
      "RequestLimitExceeded",
      "ResourceLimitExceeded",
      "MaxSpotInstanceCountExceeded",
      "MaxSpotFleetRequestCountExceeded",
      "InsufficientInstanceCapacity",
      "InsufficientCapacityOnHost",
    ])
    use_dedicated_host = optional(bool, false)
  })
| n/a | yes | | [github](#input\_github) | GitHub Enterprise Server settings available to compute-provider bootstrap data.

- `enterprise_server.url`: Optional GitHub Enterprise Server base URL. Null selects GitHub.com.
- `enterprise_server.ssl_verify`: Enables TLS certificate verification for GitHub Enterprise Server. | 
object({
    enterprise_server = optional(object({
      url        = optional(string, null)
      ssl_verify = optional(bool, true)
    }), {})
  })
| `{}` | no | | [observability](#input\_observability) | CloudWatch Logs settings available to compute-provider runner log groups.

- `logs.retention_in_days`: Retention period for provider-owned runner log groups.
- `logs.kms_key_id`: Optional KMS key ID or ARN used to encrypt runner log groups.
- `logs.tags`: Shared log-group tags that override module-level `tags`. | 
object({
    logs = optional(object({
      retention_in_days = optional(number, 180)
      kms_key_id        = optional(string, null)
      tags              = optional(map(string), {})
    }), {})
  })
| `{}` | no | | [prefix](#input\_prefix) | Prefix used to identify resources created for the runner configuration. | `string` | `"github-actions"` | no | diff --git a/modules/multi-runner/README.md b/modules/multi-runner/README.md index 57f3e84b0d..4e5be93d5c 100644 --- a/modules/multi-runner/README.md +++ b/modules/multi-runner/README.md @@ -189,7 +189,7 @@ module "multi-runner" { | [logging\_retention\_in\_days](#input\_logging\_retention\_in\_days) | Specifies the number of days you want to retain log events for the lambda log group. Possible values are: 0, 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, and 3653. | `number` | `180` | no | | [matcher\_config\_parameter\_store\_tier](#input\_matcher\_config\_parameter\_store\_tier) | The tier of the parameter store for the matcher configuration. Valid values are `Standard`, and `Advanced`. | `string` | `"Standard"` | no | | [metrics](#input\_metrics) | Configuration for metrics created by the module, by default metrics are disabled to avoid additional costs. When metrics are enable all metrics are created unless explicit configured otherwise. | 
object({
    enable    = optional(bool, false)
    namespace = optional(string, "GitHub Runners")
    metric = optional(object({
      enable_github_app_rate_limit    = optional(bool, true)
      enable_job_retry                = optional(bool, true)
      enable_spot_termination_warning = optional(bool, true)
    }), {})
  })
| `{}` | no | -| [multi\_runner\_config](#input\_multi\_runner\_config) | Accepts either the stable v1 runner configuration shape or the provider-boundary v2 shape. Entries with `runner_config` use the v1 shape; entries without `runner_config` use the v2 shape. A v2 entry does not need matcher configuration. A v2 entry must be acknowledged with `experimental_features = ["multi-runner-v2"]`; the v2 shape is experimental and may change before graduation.

multi\_runner\_config = {
runner\_config: {
runner\_os: "The EC2 Operating System type to use for action runner instances (linux, osx, windows)."
runner\_architecture: "The platform architecture of the runner instance\_type."
runner\_metadata\_options: "(Optional) Metadata options for the ec2 runner instances."
ami: "(Optional) AMI configuration for the action runner instances. This object allows you to specify all AMI-related settings in one place."
create\_service\_linked\_role\_spot: (Optional) create the serviced linked role for spot instances that is required by the scale-up lambda.
credit\_specification: "(Optional) The credit specification of the runner instance\_type. Can be unset, `standard` or `unlimited`.
delay\_webhook\_event: "The number of seconds the event accepted by the webhook is invisible on the queue before the scale up lambda will receive the event."
disable\_runner\_autoupdate: "Disable the auto update of the github runner agent. Be aware there is a grace period of 30 days, see also the [GitHub article](https://github.blog/changelog/2022-02-01-github-actions-self-hosted-runners-can-now-disable-automatic-updates/)"
ebs\_optimized: "The EC2 EBS optimized configuration."
enable\_ephemeral\_runners: "Enable ephemeral runners, runners will only be used once."
enable\_job\_queued\_check: Enables JIT configuration for creating runners instead of registration token based registraton. JIT configuration will only be applied for ephemeral runners. By default JIT configuration is enabled for ephemeral runners an can be disabled via this override. When running on GHES without support for JIT configuration this variable should be set to true for ephemeral runners."
enable\_on\_demand\_failover\_for\_errors: "Enable on-demand failover. For example to fall back to on demand when no spot capacity is available the variable can be set to `InsufficientInstanceCapacity`. When not defined the default behavior is to retry later."
scale\_errors: "List of AWS error codes that should trigger retry during scale up. This list replaces the module default scale-up retry errors"
enable\_organization\_runners: "Register runners to organization, instead of repo level"
enable\_runner\_binaries\_syncer: "Option to disable the lambda to sync GitHub runner distribution, useful when using a pre-build AMI."
enable\_ssm\_on\_runners: "Enable to allow access the runner instances for debugging purposes via SSM. Note that this adds additional permissions to the runner instances."
enable\_userdata: "Should the userdata script be enabled for the runner. Set this to false if you are using your own prebuilt AMI."
instance\_allocation\_strategy: "The allocation strategy for creating instances. For spot, AWS recommends `price-capacity-optimized`; for on-demand, use `lowest-price` or `prioritized`. The AWS default is `lowest-price`."
instance\_type\_priorities: "A map of instance type to priority for the `prioritized` and `capacity-optimized-prioritized` allocation strategies. Lower numbers mean higher priority. If not provided, priorities are assigned based on the order of `instance_types`."
instance\_max\_spot\_price: "Max price price for spot instances per hour. This variable will be passed to the create fleet as max spot price for the fleet."
instance\_target\_capacity\_type: "Default lifecycle used for runner instances, can be either `spot` or `on-demand`."
instance\_types: "List of instance types for the action runner. Defaults are based on runner\_os (al2023 for linux, macOS Sequoia for osx, Windows Server Core for win)."
job\_queue\_retention\_in\_seconds: "The number of seconds the job is held in the queue before it is purged"
minimum\_running\_time\_in\_minutes: "The time an ec2 action runner should be running at minimum before terminated if not busy."
pool\_runner\_owner: "The pool will deploy runners to the GitHub org ID, set this value to the org to which you want the runners deployed. Repo level is not supported."
runner\_additional\_security\_group\_ids: "List of additional security groups IDs to apply to the runner. If added outside the multi\_runner\_config block, the additional security group(s) will be applied to all runner configs. If added inside the multi\_runner\_config, the additional security group(s) will be applied to the individual runner."
runner\_as\_root: "Run the action runner under the root user. Variable `runner_run_as` will be ignored."
runner\_boot\_time\_in\_minutes: "The minimum time for an EC2 runner to boot and register as a runner."
scale\_down\_idle\_confirmation\_seconds: "Number of seconds a runner must consistently report not-busy before scale-down terminates it. GitHub's busy flag can be stale, so a single not-busy reading is not sufficient evidence a runner is idle. 0 keeps the previous single-reading behaviour."
runner\_disable\_default\_labels: "Disable default labels for the runners (os, architecture and `self-hosted`). If enabled, the runner will only have the extra labels provided in `runner_extra_labels`. In case you on own start script is used, this configuration parameter needs to be parsed via SSM."
runner\_extra\_labels: "Extra (custom) labels for the runners (GitHub). Separate each label by a comma. Labels checks on the webhook can be enforced by setting `multi_runner_config.matcherConfig.exactMatch`. GitHub read-only labels should not be provided."
runner\_group\_name: "Name of the runner group."
runner\_name\_prefix: "Prefix for the GitHub runner name."
runner\_run\_as: "Run the GitHub actions agent as user."
runners\_maximum\_count: "The maximum number of runners that will be created. Setting the variable to `-1` disables the maximum check."
scale\_down\_schedule\_expression: "Scheduler expression to check every x for scale down."
scale\_up\_reserved\_concurrent\_executions: "Amount of reserved concurrent executions for the scale-up lambda function. A value of 0 disables lambda from being triggered and -1 removes any concurrency limitations."
lambda\_event\_source\_mapping\_batch\_size: "(Optional) Maximum number of records per Lambda invocation for this runner flavor. Overrides the module-level `lambda_event_source_mapping_batch_size` when set."
lambda\_event\_source\_mapping\_maximum\_batching\_window\_in\_seconds: "(Optional) Maximum seconds to gather records before invoking Lambda for this runner flavor. Overrides the module-level `lambda_event_source_mapping_maximum_batching_window_in_seconds` when set."
userdata\_template: "Alternative user-data template, replacing the default template. By providing your own user\_data you have to take care of installing all required software, including the action runner. Variables userdata\_pre/post\_install are ignored."
enable\_jit\_config: "Overwrite the default behavior for JIT configuration. By default JIT configuration is enabled for ephemeral runners and disabled for non-ephemeral runners. In case of GHES check first if the JIT config API is available. In case you are upgrading from 3.x to 4.x you can set `enable_jit_config` to `false` to avoid a breaking change when having your own AMI."
enable\_runner\_detailed\_monitoring: "Should detailed monitoring be enabled for the runner. Set this to true if you want to use detailed monitoring. See https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-cloudwatch-new.html for details."
enable\_cloudwatch\_agent: "Enabling the cloudwatch agent on the ec2 runner instances, the runner contains default config. Configuration can be overridden via `cloudwatch_config`."
cloudwatch\_config: "(optional) Replaces the module default cloudwatch log config. See https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-Configuration-File-Details.html for details."
userdata\_pre\_install: "Script to be ran before the GitHub Actions runner is installed on the EC2 instances"
userdata\_post\_install: "Script to be ran after the GitHub Actions runner is installed on the EC2 instances"
runner\_hook\_job\_started: "Script to be ran in the runner environment at the beginning of every job"
runner\_hook\_job\_completed: "Script to be ran in the runner environment at the end of every job"
runner\_ec2\_tags: "Map of tags that will be added to the launch template instance tag specifications."
runner\_iam\_role\_managed\_policy\_arns: "Attach AWS or customer-managed IAM policies (by ARN) to the runner IAM role"
vpc\_id: "The VPC for security groups of the action runners. If not set uses the value of `var.vpc_id`."
subnet\_ids: "List of subnets in which the action runners will be launched, the subnets needs to be subnets in the `vpc_id`. If not set, uses the value of `var.subnet_ids`."
idle\_config: "List of time period that can be defined as cron expression to keep a minimum amount of runners active instead of scaling down to 0. By defining this list you can ensure that in time periods that match the cron expression within 5 seconds a runner is kept idle."
license\_specifications: "Optional EC2 License Manager license configuration ARNs for the runner launch template. Required for macOS dedicated-host runners when the host resource group uses a Mac dedicated host license configuration."
use\_dedicated\_host: "Experimental! Can be removed / changed without trigger a major release. Whether to use EC2 dedicated hosts for the runners. Needed for macos runners Note that using dedicated hosts can increase cost significantly."
runner\_log\_files: "(optional) Replaces the module default cloudwatch log config. See https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-Configuration-File-Details.html for details."
block\_device\_mappings: "The EC2 instance block device configuration. Takes the following keys: `device_name`, `delete_on_termination`, `volume_type`, `volume_size`, `encrypted`, `iops`, `throughput`, `kms_key_id`, `snapshot_id`, `volume_initialization_rate`."
job\_retry: "Experimental! Can be removed / changed without trigger a major release. Configure job retries. The configuration enables job retries (for ephemeral runners). After creating the instances a message will be published to a job retry queue. The job retry check lambda is checking after a delay if the job is queued. If not the message will be published again on the scale-up (build queue). Using this feature can impact the rate limit of the GitHub app."
pool\_config: "The configuration for updating the pool. The `pool_size` to adjust to by the events triggered by the `schedule_expression`. For example you can configure a cron expression for week days to adjust the pool to 10 and another expression for the weekend to adjust the pool to 1. Use `schedule_expression_timezone` to override the schedule time zone (defaults to UTC)."
iam\_overrides: "Allows to (optionally) override the instance profile and runner role created by the module. Set `override_instance_profile` to true and provide the `instance_profile_name` to use an existing instance profile. Set `override_runner_role` to true and provide the `runner_role_arn` to use an existing role for the runner instances."
}
# V2 contract
tags: "Tags applied to resources created for this runner configuration."
runner: "Runner settings such as the operating system, architecture, labels, hooks, runner group, name prefix, and IAM role configuration."
lambda: "Lambda settings such as runtime, architecture, networking, tags, and execution-role options for this runner configuration."
# Webhook, queue, and scale-up/scale-down orchestration settings.
orchestration\_provider: {
webhook: {
matcherConfig: "Label matching and dynamic-label policy used to route workflow jobs to this runner configuration."
runner: "Runner lifecycle settings including boot time, ephemeral mode, JIT configuration, and maximum runner count."
queue: "Build queue delay, retention, visibility timeout, redrive, and tags."
}
}
ssm: "SSM parameter paths, tags, and housekeeper settings for runner configuration storage."
observability: "Logging, tracing, and metric settings for the resources in this runner configuration."
# Compute settings for the runner provider.
compute\_provider: {
aws: {
ec2: "AWS EC2 runner settings, including AMI selection, instance types, capacity strategy, VPC and subnet placement, storage, user data, and runner access."
}
}
matcherConfig: {
labelMatchers: "The list of list of labels supported by the runner configuration. `[[self-hosted, linux, x64, example]]`"
exactMatch: "DEPRECATED: Use `bidirectionalLabelMatch` instead. If set to true all labels in the workflow job must match the GitHub labels (os, architecture and `self-hosted`). When false if __any__ workflow label matches it will trigger the webhook. Note: this only checks that workflow labels are a subset of runner labels, not the reverse."
bidirectionalLabelMatch: "If set to true, the runner labels and workflow job labels must be an exact two-way match (same set, any order, no extras or missing labels). This is stricter than `exactMatch` which only checks that workflow labels are a subset of runner labels. When false, if __any__ workflow label matches it will trigger the webhook."
priority: "If set it defines the priority of the matcher, the matcher with the lowest priority will be evaluated first. Default is 999, allowed values 0-999."
enableDynamicLabels: "Experimental! When true the dispatcher allows `ghr-*` dynamic labels for jobs routed to this runner. Default false."
awsDynamicLabelsPolicy: "Optional AWS dynamic label policy evaluated by the dispatcher. Only effective when `enableDynamicLabels = true`. Jobs whose provider dynamic labels violate every matching runner's policy are rejected with a 202 (a warning is logged). Evaluation: keys in `blocked_keys` are always rejected; keys in `restricted_keys` are allowed only when their value passes the rule; unlisted keys are allowed. Schema: `{ blocked_keys = [], restricted_keys = { = { allowed = [globs], denied = [globs], max = number|string } } }`. Keys use the dynamic label suffix, e.g. `instance-type` for `ghr-ec2-instance-type`."
}
redrive\_build\_queue: "Set options to attach (optional) a dead letter queue to the build queue, the queue between the webhook and the scale up lambda. You have the following options. 1. Disable by setting `enabled` to false. 2. Enable by setting `enabled` to `true`, `maxReceiveCount` to a number of max retries."
} | 
map(object({
    # V1 contract
    runner_config = optional(object({
      runner_os           = string
      runner_architecture = string
      runner_metadata_options = optional(map(any), {
        instance_metadata_tags      = "enabled"
        http_endpoint               = "enabled"
        http_tokens                 = "required"
        http_put_response_hop_limit = 1
      })
      ami = optional(object({
        filter               = optional(map(list(string)), { state = ["available"] })
        owners               = optional(list(string), ["amazon"])
        id_ssm_parameter_arn = optional(string, null)
        kms_key_arn          = optional(string, null)
      }), null)
      create_service_linked_role_spot      = optional(bool, false)
      credit_specification                 = optional(string, null)
      delay_webhook_event                  = optional(number, 30)
      disable_runner_autoupdate            = optional(bool, false)
      ebs_optimized                        = optional(bool, false)
      enable_ephemeral_runners             = optional(bool, false)
      enable_job_queued_check              = optional(bool, null)
      enable_on_demand_failover_for_errors = optional(list(string), [])
      scale_errors = optional(list(string), [
        "UnfulfillableCapacity",
        "MaxSpotInstanceCountExceeded",
        "TargetCapacityLimitExceededException",
        "RequestLimitExceeded",
        "ResourceLimitExceeded",
        "MaxSpotInstanceCountExceeded",
        "MaxSpotFleetRequestCountExceeded",
        "InsufficientInstanceCapacity",
        "InsufficientCapacityOnHost",
      ])
      enable_organization_runners                                    = optional(bool, false)
      enable_runner_binaries_syncer                                  = optional(bool, true)
      enable_ssm_on_runners                                          = optional(bool, false)
      enable_userdata                                                = optional(bool, true)
      instance_allocation_strategy                                   = optional(string, "lowest-price")
      instance_type_priorities                                       = optional(map(number), null)
      instance_max_spot_price                                        = optional(string, null)
      instance_target_capacity_type                                  = optional(string, "spot")
      instance_types                                                 = list(string)
      job_queue_retention_in_seconds                                 = optional(number, 86400)
      minimum_running_time_in_minutes                                = optional(number, null)
      pool_runner_owner                                              = optional(string, null)
      runner_as_root                                                 = optional(bool, false)
      runner_boot_time_in_minutes                                    = optional(number, 5)
      scale_down_idle_confirmation_seconds                           = optional(number, 0)
      runner_disable_default_labels                                  = optional(bool, false)
      runner_extra_labels                                            = optional(list(string), [])
      runner_group_name                                              = optional(string, "Default")
      runner_name_prefix                                             = optional(string, "")
      runner_run_as                                                  = optional(string, "ec2-user")
      runners_maximum_count                                          = number
      runner_additional_security_group_ids                           = optional(list(string), [])
      scale_down_schedule_expression                                 = optional(string, "cron(*/5 * * * ? *)")
      scale_up_reserved_concurrent_executions                        = optional(number, 1)
      lambda_event_source_mapping_batch_size                         = optional(number, null)
      lambda_event_source_mapping_maximum_batching_window_in_seconds = optional(number, null)
      userdata_template                                              = optional(string, null)
      userdata_content                                               = optional(string, null)
      enable_jit_config                                              = optional(bool, null)
      enable_runner_detailed_monitoring                              = optional(bool, false)
      enable_cloudwatch_agent                                        = optional(bool, true)
      cloudwatch_config                                              = optional(string, null)
      userdata_pre_install                                           = optional(string, "")
      userdata_post_install                                          = optional(string, "")
      runner_hook_job_started                                        = optional(string, "")
      runner_hook_job_completed                                      = optional(string, "")
      runner_ec2_tags                                                = optional(map(string), {})
      runner_iam_role_managed_policy_arns                            = optional(list(string), [])
      vpc_id                                                         = optional(string, null)
      subnet_ids                                                     = optional(list(string), null)
      idle_config = optional(list(object({
        cron             = string
        timeZone         = string
        idleCount        = number
        evictionStrategy = optional(string, "oldest_first")
      })), [])
      cpu_options = optional(object({
        core_count            = optional(number)
        threads_per_core      = optional(number)
        amd_sev_snp           = optional(string)
        nested_virtualization = optional(string)
      }), null)
      placement = optional(object({
        affinity                = optional(string)
        availability_zone       = optional(string)
        group_id                = optional(string)
        group_name              = optional(string)
        host_id                 = optional(string)
        host_resource_group_arn = optional(string)
        spread_domain           = optional(string)
        tenancy                 = optional(string)
        partition_number        = optional(number)
      }), null)
      license_specifications = optional(list(object({
        license_configuration_arn = string
      })), [])
      use_dedicated_host = optional(bool, false)
      runner_log_files = optional(list(object({
        log_group_name   = string
        prefix_log_group = bool
        file_path        = string
        log_stream_name  = string
        log_class        = optional(string, "STANDARD")
      })), null)
      block_device_mappings = optional(list(object({
        delete_on_termination      = optional(bool, true)
        device_name                = optional(string, "/dev/xvda")
        encrypted                  = optional(bool, true)
        iops                       = optional(number)
        kms_key_id                 = optional(string)
        snapshot_id                = optional(string)
        throughput                 = optional(number)
        volume_initialization_rate = optional(number)
        volume_size                = number
        volume_type                = optional(string, "gp3")
        })), [{
        volume_size = 30
      }])
      pool_config = optional(list(object({
        schedule_expression          = string
        schedule_expression_timezone = optional(string)
        size                         = number
      })), [])
      job_retry = optional(object({
        enable             = optional(bool, false)
        delay_in_seconds   = optional(number, 300)
        delay_backoff      = optional(number, 2)
        lambda_memory_size = optional(number, 256)
        lambda_timeout     = optional(number, 30)
        max_attempts       = optional(number, 1)
      }), {})
      iam_overrides = optional(object({
        override_instance_profile = optional(bool, null)
        instance_profile_name     = optional(string, null)
        override_runner_role      = optional(bool, null)
        runner_role_arn           = optional(string, null)
        }), {
        override_instance_profile = false
        instance_profile_name     = null
        override_runner_role      = false
        runner_role_arn           = null
      })
    }), null)
    matcherConfig = optional(object({
      labelMatchers           = list(list(string))
      exactMatch              = optional(bool, false)
      bidirectionalLabelMatch = optional(bool, false)
      priority                = optional(number, 999)
      enableDynamicLabels     = optional(bool, false)
      awsDynamicLabelsPolicy  = optional(any, null)
    }), null)
    redrive_build_queue = optional(object({
      enabled         = bool
      maxReceiveCount = number
      }), {
      enabled         = false
      maxReceiveCount = null
    })

    # V2 Contract
    tags = optional(map(string), {})

    runner = optional(object({
      os                     = optional(string, null)
      architecture           = optional(string, null)
      disable_default_labels = optional(bool, null)
      extra_labels           = optional(list(string), null)
      group_name             = optional(string, null)
      name_prefix            = optional(string, null)
      run_as_root            = optional(bool, null)
      run_as                 = optional(string, null)
      auto_update_disabled   = optional(bool, null)
      tags                   = optional(map(string), {})
      hooks = optional(object({
        job_started   = optional(string, null)
        job_completed = optional(string, null)
      }), {})
      iam = optional(object({
        role = optional(object({
          arn = string
        }), null)
        managed_policy_arns          = optional(map(string), null)
        additional_trust_policy_json = optional(string, null)
        path                         = optional(string, null)
        permissions_boundary         = optional(string, null)
      }), {})
    }), {})

    lambda = optional(object({
      runtime            = optional(string, null)
      architecture       = optional(string, null)
      subnet_ids         = optional(list(string), null)
      security_group_ids = optional(list(string), null)
      tags               = optional(map(string), {})
      role = optional(object({
        path                 = optional(string, null)
        permissions_boundary = optional(string, null)
      }), {})
    }), {})

    orchestration_provider = optional(object({
      webhook = optional(object({
        runner = optional(object({
          boot_time_in_minutes = optional(number, null)
          ephemeral            = optional(bool, null)
          jit_config_enabled   = optional(bool, null)
          maximum_count        = optional(number, null)
        }), {})
        github = optional(object({
          organization_runners = optional(bool, false)
        }), {})
        matcherConfig = optional(object({
          labelMatchers           = list(list(string))
          exactMatch              = optional(bool, false)
          bidirectionalLabelMatch = optional(bool, false)
          priority                = optional(number, 999)
          dynamic_labels_enabled  = optional(bool, false)
          awsDynamicLabelsPolicy = optional(object({
            blocked_keys = optional(list(string), [])
            restricted_keys = optional(map(object({
              allowed = optional(list(string), [])
              denied  = optional(list(string), [])
              max     = optional(string, null)
            })), {})
          }), null)
        }), null)
        queue = optional(object({
          delay_webhook_event            = optional(number, null)
          job_queue_retention_in_seconds = optional(number, null)
          visibility_timeout_seconds     = optional(number, null)
          redrive_build_queue = optional(object({
            enabled         = optional(bool, null)
            maxReceiveCount = optional(number, null)
          }), null)
          tags = optional(map(string), {})
        }), {})
        lambda = optional(object({
          scale = optional(object({
            up = optional(object({
              memory_size                    = optional(number, null)
              timeout                        = optional(number, null)
              reserved_concurrent_executions = optional(number, null)
              job_queued_check_enabled       = optional(bool, null)
              event_source_mapping = optional(object({
                batch_size                         = optional(number, null)
                maximum_batching_window_in_seconds = optional(number, null)
              }), {})
              tags = optional(map(string), {})
            }), {})
            down = optional(object({
              memory_size                     = optional(number, null)
              timeout                         = optional(number, null)
              schedule_expression             = optional(string, null)
              minimum_running_time_in_minutes = optional(number, null)
              idle_confirmation_seconds       = optional(number, null)
              idle_config = optional(list(object({
                cron             = string
                timeZone         = string
                idleCount        = number
                evictionStrategy = optional(string, "oldest_first")
              })), null)
              tags = optional(map(string), {})
            }), {})
          }), {})
          pool = optional(object({
            memory_size                    = optional(number, null)
            timeout                        = optional(number, null)
            reserved_concurrent_executions = optional(number, null)
            config = optional(list(object({
              schedule_expression          = string
              schedule_expression_timezone = optional(string)
              size                         = number
            })), null)
            include_busy_runners = optional(bool, null)
            runner_owner         = optional(string, null)
            tags                 = optional(map(string), {})
          }), {})
        }), {})
        job_retry = optional(object({
          enabled          = optional(bool, false)
          delay_in_seconds = optional(number, 300)
          delay_backoff    = optional(number, 2)
          max_attempts     = optional(number, 1)
          tags             = optional(map(string), {})
          lambda = optional(object({
            memory_size                    = optional(number, 256)
            reserved_concurrent_executions = optional(number, 1)
            timeout                        = optional(number, 30)
          }), {})
        }), {})
      }), null)
      scale_set = optional(object({
        name = string
        runner = optional(object({
          min_runners          = optional(number, 0)
          max_runners          = optional(number, 10)
          boot_time_in_minutes = optional(number, 10)
        }), {})
      }), null)
    }), {})

    ssm = optional(object({
      paths = optional(object({
        root   = optional(string, null)
        tokens = optional(string, null)
        config = optional(string, null)
      }), {})
      tags = optional(map(string), {})
      parameters = optional(object({
        tags = optional(map(string), {})
      }), {})
      housekeeper = optional(object({
        schedule_expression = optional(string, null)
        state               = optional(string, null)
        tags                = optional(map(string), {})
        lambda = optional(object({
          artifact = optional(object({
            zip = optional(string, null)
            s3 = optional(object({
              key            = string
              object_version = optional(string, null)
            }), null)
          }), {})
          memory_size = optional(number, null)
          timeout     = optional(number, null)
        }), {})
        config = optional(object({
          tokenPath      = optional(string, null)
          minimumDaysOld = optional(number, null)
          dryRun         = optional(bool, null)
        }), {})
      }), {})
    }), {})

    observability = optional(object({
      logs = optional(object({
        level             = optional(string, null)
        retention_in_days = optional(number, null)
        kms_key_id        = optional(string, null)
        class             = optional(string, null)
        tags              = optional(map(string), {})
      }), {})
      tracing = optional(object({
        mode                  = optional(string, null)
        capture_http_requests = optional(bool, null)
        capture_error         = optional(bool, null)
      }), {})
      metrics = optional(object({
        enabled   = optional(bool, null)
        namespace = optional(string, null)
        metric = optional(object({
          github_app_rate_limit = optional(object({
            enabled = optional(bool, null)
          }), {})
          job_retry = optional(object({
            enabled = optional(bool, null)
          }), {})
          spot_termination_warning = optional(object({
            enabled = optional(bool, null)
          }), {})
        }), {})
      }), {})
    }), {})

    compute_provider = optional(object({
      aws = optional(object({
        ec2 = optional(object({
          metadata_options = optional(object({
            instance_metadata_tags      = optional(string, "enabled")
            http_endpoint               = optional(string, "enabled")
            http_tokens                 = optional(string, "required")
            http_put_response_hop_limit = optional(number, 1)
          }), {})
          ami = optional(object({
            filter = optional(map(list(string)), { state = ["available"] })
            owners = optional(list(string), ["amazon"])
            id_ssm_parameter = optional(object({
              arn = string
            }), null)
            kms_key = optional(object({
              arn = string
            }), null)
          }), null)
          block_device_mappings = optional(list(object({
            delete_on_termination      = optional(bool, true)
            device_name                = optional(string, "/dev/xvda")
            encrypted                  = optional(bool, true)
            iops                       = optional(number)
            kms_key_id                 = optional(string)
            snapshot_id                = optional(string)
            throughput                 = optional(number)
            volume_initialization_rate = optional(number)
            volume_size                = number
            volume_type                = optional(string, "gp3")
          })), [{ volume_size = 30 }])
          create_service_linked_role_spot = optional(bool, false)
          credit_specification            = optional(string, null)
          ebs_optimized                   = optional(bool, false)
          cloudwatch_agent = optional(object({
            enabled = optional(bool, true)
            config  = optional(string, null)
          }), {})
          binaries_syncer = optional(object({
            enabled = optional(bool, null)
          }), {})
          detailed_monitoring_enabled = optional(bool, false)
          ssm_enabled                 = optional(bool, false)
          user_data = optional(object({
            enabled               = optional(bool, true)
            template              = optional(string, null)
            content               = optional(string, null)
            pre_install           = optional(string, "")
            post_install          = optional(string, "")
            debug_logging_enabled = optional(bool, false)
          }), {})
          instance_allocation_strategy   = optional(string, "lowest-price")
          instance_max_spot_price        = optional(string, null)
          instance_target_capacity_type  = optional(string, "spot")
          instance_type_priorities       = optional(map(number), null)
          instance_types                 = optional(list(string), [])
          additional_security_group_ids  = optional(list(string), null)
          managed_security_group_enabled = optional(bool, null)
          egress_rules = optional(list(object({
            cidr_blocks      = list(string)
            ipv6_cidr_blocks = list(string)
            prefix_list_ids  = list(string)
            from_port        = number
            protocol         = string
            security_groups  = list(string)
            self             = bool
            to_port          = number
            description      = string
          })), null)
          instance_profile_path         = optional(string, null)
          key_name                      = optional(string, null)
          associate_public_ipv4_address = optional(bool, null)
          instance_profile = optional(object({
            name = string
          }), null)
          on_demand_failover_for_errors = optional(list(string), [])
          scale_errors = optional(list(string), [
            "UnfulfillableCapacity",
            "MaxSpotInstanceCountExceeded",
            "TargetCapacityLimitExceededException",
            "RequestLimitExceeded",
            "ResourceLimitExceeded",
            "MaxSpotInstanceCountExceeded",
            "MaxSpotFleetRequestCountExceeded",
            "InsufficientInstanceCapacity",
            "InsufficientCapacityOnHost",
          ])
          subnet_ids = optional(list(string), null)
          vpc_id     = optional(string, null)
          cpu_options = optional(object({
            core_count            = optional(number)
            threads_per_core      = optional(number)
            amd_sev_snp           = optional(string)
            nested_virtualization = optional(string)
          }), null)
          placement = optional(object({
            affinity                = optional(string)
            availability_zone       = optional(string)
            group_id                = optional(string)
            group_name              = optional(string)
            host_id                 = optional(string)
            host_resource_group_arn = optional(string)
            spread_domain           = optional(string)
            tenancy                 = optional(string)
            partition_number        = optional(number)
          }), null)
          license_specifications = optional(list(object({
            license_configuration_arn = string
          })), [])
          use_dedicated_host = optional(bool, false)
          log_files = optional(list(object({
            log_group_name   = string
            prefix_log_group = bool
            file_path        = string
            log_stream_name  = string
            log_class        = optional(string, "STANDARD")
          })), null)
          tags = optional(map(string), {})
        }), null)
      }), {})
    }), {})
  }))
| `{}` | no | +| [multi\_runner\_config](#input\_multi\_runner\_config) | Accepts either the stable v1 runner configuration shape or the provider-boundary v2 shape. Entries with `runner_config` use the v1 shape; entries without `runner_config` use the v2 shape. A v2 entry does not need matcher configuration. A v2 entry must be acknowledged with `experimental_features = ["multi-runner-v2"]`; the v2 shape is experimental and may change before graduation.

multi\_runner\_config = {
runner\_config: {
runner\_os: "The EC2 Operating System type to use for action runner instances (linux, osx, windows)."
runner\_architecture: "The platform architecture of the runner instance\_type."
runner\_metadata\_options: "(Optional) Metadata options for the ec2 runner instances."
ami: "(Optional) AMI configuration for the action runner instances. This object allows you to specify all AMI-related settings in one place."
create\_service\_linked\_role\_spot: (Optional) create the serviced linked role for spot instances that is required by the scale-up lambda.
credit\_specification: "(Optional) The credit specification of the runner instance\_type. Can be unset, `standard` or `unlimited`.
delay\_webhook\_event: "The number of seconds the event accepted by the webhook is invisible on the queue before the scale up lambda will receive the event."
disable\_runner\_autoupdate: "Disable the auto update of the github runner agent. Be aware there is a grace period of 30 days, see also the [GitHub article](https://github.blog/changelog/2022-02-01-github-actions-self-hosted-runners-can-now-disable-automatic-updates/)"
ebs\_optimized: "The EC2 EBS optimized configuration."
enable\_ephemeral\_runners: "Enable ephemeral runners, runners will only be used once."
enable\_job\_queued\_check: Enables JIT configuration for creating runners instead of registration token based registraton. JIT configuration will only be applied for ephemeral runners. By default JIT configuration is enabled for ephemeral runners an can be disabled via this override. When running on GHES without support for JIT configuration this variable should be set to true for ephemeral runners."
enable\_on\_demand\_failover\_for\_errors: "Enable on-demand failover. For example to fall back to on demand when no spot capacity is available the variable can be set to `InsufficientInstanceCapacity`. When not defined the default behavior is to retry later."
scale\_errors: "List of AWS error codes that should trigger retry during scale up. This list replaces the module default scale-up retry errors"
enable\_organization\_runners: "Register runners to organization, instead of repo level"
enable\_runner\_binaries\_syncer: "Option to disable the lambda to sync GitHub runner distribution, useful when using a pre-build AMI."
enable\_ssm\_on\_runners: "Enable to allow access the runner instances for debugging purposes via SSM. Note that this adds additional permissions to the runner instances."
enable\_userdata: "Should the userdata script be enabled for the runner. Set this to false if you are using your own prebuilt AMI."
instance\_allocation\_strategy: "The allocation strategy for creating instances. For spot, AWS recommends `price-capacity-optimized`; for on-demand, use `lowest-price` or `prioritized`. The AWS default is `lowest-price`."
instance\_type\_priorities: "A map of instance type to priority for the `prioritized` and `capacity-optimized-prioritized` allocation strategies. Lower numbers mean higher priority. If not provided, priorities are assigned based on the order of `instance_types`."
instance\_max\_spot\_price: "Max price price for spot instances per hour. This variable will be passed to the create fleet as max spot price for the fleet."
instance\_target\_capacity\_type: "Default lifecycle used for runner instances, can be either `spot` or `on-demand`."
instance\_types: "List of instance types for the action runner. Defaults are based on runner\_os (al2023 for linux, macOS Sequoia for osx, Windows Server Core for win)."
job\_queue\_retention\_in\_seconds: "The number of seconds the job is held in the queue before it is purged"
minimum\_running\_time\_in\_minutes: "The time an ec2 action runner should be running at minimum before terminated if not busy."
pool\_runner\_owner: "The pool will deploy runners to the GitHub org ID, set this value to the org to which you want the runners deployed. Repo level is not supported."
runner\_additional\_security\_group\_ids: "List of additional security groups IDs to apply to the runner. If added outside the multi\_runner\_config block, the additional security group(s) will be applied to all runner configs. If added inside the multi\_runner\_config, the additional security group(s) will be applied to the individual runner."
runner\_as\_root: "Run the action runner under the root user. Variable `runner_run_as` will be ignored."
runner\_boot\_time\_in\_minutes: "The minimum time for an EC2 runner to boot and register as a runner."
scale\_down\_idle\_confirmation\_seconds: "Number of seconds a runner must consistently report not-busy before scale-down terminates it. GitHub's busy flag can be stale, so a single not-busy reading is not sufficient evidence a runner is idle. 0 keeps the previous single-reading behaviour."
runner\_disable\_default\_labels: "Disable default labels for the runners (os, architecture and `self-hosted`). If enabled, the runner will only have the extra labels provided in `runner_extra_labels`. In case you on own start script is used, this configuration parameter needs to be parsed via SSM."
runner\_extra\_labels: "Extra (custom) labels for the runners (GitHub). Separate each label by a comma. Labels checks on the webhook can be enforced by setting `multi_runner_config.matcherConfig.exactMatch`. GitHub read-only labels should not be provided."
runner\_group\_name: "Name of the runner group."
runner\_name\_prefix: "Prefix for the GitHub runner name."
runner\_run\_as: "Run the GitHub actions agent as user."
runners\_maximum\_count: "The maximum number of runners that will be created. Setting the variable to `-1` disables the maximum check."
scale\_down\_schedule\_expression: "Scheduler expression to check every x for scale down."
scale\_up\_reserved\_concurrent\_executions: "Amount of reserved concurrent executions for the scale-up lambda function. A value of 0 disables lambda from being triggered and -1 removes any concurrency limitations."
lambda\_event\_source\_mapping\_batch\_size: "(Optional) Maximum number of records per Lambda invocation for this runner flavor. Overrides the module-level `lambda_event_source_mapping_batch_size` when set."
lambda\_event\_source\_mapping\_maximum\_batching\_window\_in\_seconds: "(Optional) Maximum seconds to gather records before invoking Lambda for this runner flavor. Overrides the module-level `lambda_event_source_mapping_maximum_batching_window_in_seconds` when set."
userdata\_template: "Alternative user-data template, replacing the default template. By providing your own user\_data you have to take care of installing all required software, including the action runner. Variables userdata\_pre/post\_install are ignored."
enable\_jit\_config: "Overwrite the default behavior for JIT configuration. By default JIT configuration is enabled for ephemeral runners and disabled for non-ephemeral runners. In case of GHES check first if the JIT config API is available. In case you are upgrading from 3.x to 4.x you can set `enable_jit_config` to `false` to avoid a breaking change when having your own AMI."
enable\_runner\_detailed\_monitoring: "Should detailed monitoring be enabled for the runner. Set this to true if you want to use detailed monitoring. See https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-cloudwatch-new.html for details."
enable\_cloudwatch\_agent: "Enabling the cloudwatch agent on the ec2 runner instances, the runner contains default config. Configuration can be overridden via `cloudwatch_config`."
cloudwatch\_config: "(optional) Replaces the module default cloudwatch log config. See https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-Configuration-File-Details.html for details."
userdata\_pre\_install: "Script to be ran before the GitHub Actions runner is installed on the EC2 instances"
userdata\_post\_install: "Script to be ran after the GitHub Actions runner is installed on the EC2 instances"
runner\_hook\_job\_started: "Script to be ran in the runner environment at the beginning of every job"
runner\_hook\_job\_completed: "Script to be ran in the runner environment at the end of every job"
runner\_ec2\_tags: "Map of tags that will be added to the launch template instance tag specifications."
runner\_iam\_role\_managed\_policy\_arns: "Attach AWS or customer-managed IAM policies (by ARN) to the runner IAM role"
vpc\_id: "The VPC for security groups of the action runners. If not set uses the value of `var.vpc_id`."
subnet\_ids: "List of subnets in which the action runners will be launched, the subnets needs to be subnets in the `vpc_id`. If not set, uses the value of `var.subnet_ids`."
idle\_config: "List of time period that can be defined as cron expression to keep a minimum amount of runners active instead of scaling down to 0. By defining this list you can ensure that in time periods that match the cron expression within 5 seconds a runner is kept idle."
license\_specifications: "Optional EC2 License Manager license configuration ARNs for the runner launch template. Required for macOS dedicated-host runners when the host resource group uses a Mac dedicated host license configuration."
use\_dedicated\_host: "Experimental! Can be removed / changed without trigger a major release. Whether to use EC2 dedicated hosts for the runners. Needed for macos runners Note that using dedicated hosts can increase cost significantly."
runner\_log\_files: "(optional) Replaces the module default cloudwatch log config. See https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-Configuration-File-Details.html for details."
block\_device\_mappings: "The EC2 instance block device configuration. Takes the following keys: `device_name`, `delete_on_termination`, `volume_type`, `volume_size`, `encrypted`, `iops`, `throughput`, `kms_key_id`, `snapshot_id`, `volume_initialization_rate`."
job\_retry: "Experimental! Can be removed / changed without trigger a major release. Configure job retries. The configuration enables job retries (for ephemeral runners). After creating the instances a message will be published to a job retry queue. The job retry check lambda is checking after a delay if the job is queued. If not the message will be published again on the scale-up (build queue). Using this feature can impact the rate limit of the GitHub app."
pool\_config: "The configuration for updating the pool. The `pool_size` to adjust to by the events triggered by the `schedule_expression`. For example you can configure a cron expression for week days to adjust the pool to 10 and another expression for the weekend to adjust the pool to 1. Use `schedule_expression_timezone` to override the schedule time zone (defaults to UTC)."
iam\_overrides: "Allows to (optionally) override the instance profile and runner role created by the module. Set `override_instance_profile` to true and provide the `instance_profile_name` to use an existing instance profile. Set `override_runner_role` to true and provide the `runner_role_arn` to use an existing role for the runner instances."
}
# V2 contract
tags: "Tags applied to resources created for this runner configuration."
runner: "Runner settings such as the operating system, architecture, labels, hooks, runner group, name prefix, and IAM role configuration."
lambda: "Lambda settings such as runtime, architecture, networking, tags, and execution-role options for this runner configuration."
# Webhook, queue, and scale-up/scale-down orchestration settings.
orchestration\_provider: {
webhook: {
matcherConfig: "Label matching and dynamic-label policy used to route workflow jobs to this runner configuration."
runner: "Runner lifecycle settings including boot time, ephemeral mode, JIT configuration, and maximum runner count."
queue: "Build queue delay, retention, visibility timeout, redrive, and tags."
}
}
ssm: "SSM parameter paths, tags, and housekeeper settings for runner configuration storage."
observability: "Logging, tracing, and metric settings for the resources in this runner configuration."
# Compute settings for the runner provider.
compute\_provider: {
aws: {
ec2: "AWS EC2 runner settings, including AMI selection, instance types, capacity strategy, VPC and subnet placement, storage, user data, and runner access."
}
}
matcherConfig: {
labelMatchers: "The list of list of labels supported by the runner configuration. `[[self-hosted, linux, x64, example]]`"
exactMatch: "DEPRECATED: Use `bidirectionalLabelMatch` instead. If set to true all labels in the workflow job must match the GitHub labels (os, architecture and `self-hosted`). When false if __any__ workflow label matches it will trigger the webhook. Note: this only checks that workflow labels are a subset of runner labels, not the reverse."
bidirectionalLabelMatch: "If set to true, the runner labels and workflow job labels must be an exact two-way match (same set, any order, no extras or missing labels). This is stricter than `exactMatch` which only checks that workflow labels are a subset of runner labels. When false, if __any__ workflow label matches it will trigger the webhook."
priority: "If set it defines the priority of the matcher, the matcher with the lowest priority will be evaluated first. Default is 999, allowed values 0-999."
enableDynamicLabels: "Experimental! When true the dispatcher allows `ghr-*` dynamic labels for jobs routed to this runner. Default false."
awsDynamicLabelsPolicy: "Optional AWS dynamic label policy evaluated by the dispatcher. Only effective when `enableDynamicLabels = true`. Jobs whose provider dynamic labels violate every matching runner's policy are rejected with a 202 (a warning is logged). Evaluation: if `allowed_keys` is set, only those keys are accepted; keys in `blocked_keys` are always rejected (cannot be used together with `allowed_keys`); keys in `restricted_keys` are allowed only when their value passes the rule; a key not listed anywhere is allowed. Schema: `{ allowed_keys = [], blocked_keys = [], restricted_keys = { = { allowed = [globs], denied = [globs], max = number|string } } }`. Keys use the dynamic label suffix, e.g. `instance-type` for `ghr-ec2-instance-type`."
}
redrive\_build\_queue: "Set options to attach (optional) a dead letter queue to the build queue, the queue between the webhook and the scale up lambda. You have the following options. 1. Disable by setting `enabled` to false. 2. Enable by setting `enabled` to `true`, `maxReceiveCount` to a number of max retries."
} | 
map(object({
    # V1 contract
    runner_config = optional(object({
      runner_os           = string
      runner_architecture = string
      runner_metadata_options = optional(map(any), {
        instance_metadata_tags      = "enabled"
        http_endpoint               = "enabled"
        http_tokens                 = "required"
        http_put_response_hop_limit = 1
      })
      ami = optional(object({
        filter               = optional(map(list(string)), { state = ["available"] })
        owners               = optional(list(string), ["amazon"])
        id_ssm_parameter_arn = optional(string, null)
        kms_key_arn          = optional(string, null)
      }), null)
      create_service_linked_role_spot      = optional(bool, false)
      credit_specification                 = optional(string, null)
      delay_webhook_event                  = optional(number, 30)
      disable_runner_autoupdate            = optional(bool, false)
      ebs_optimized                        = optional(bool, false)
      enable_ephemeral_runners             = optional(bool, false)
      enable_job_queued_check              = optional(bool, null)
      enable_on_demand_failover_for_errors = optional(list(string), [])
      scale_errors = optional(list(string), [
        "UnfulfillableCapacity",
        "MaxSpotInstanceCountExceeded",
        "TargetCapacityLimitExceededException",
        "RequestLimitExceeded",
        "ResourceLimitExceeded",
        "MaxSpotInstanceCountExceeded",
        "MaxSpotFleetRequestCountExceeded",
        "InsufficientInstanceCapacity",
        "InsufficientCapacityOnHost",
      ])
      enable_organization_runners                                    = optional(bool, false)
      enable_runner_binaries_syncer                                  = optional(bool, true)
      enable_ssm_on_runners                                          = optional(bool, false)
      enable_userdata                                                = optional(bool, true)
      instance_allocation_strategy                                   = optional(string, "lowest-price")
      instance_type_priorities                                       = optional(map(number), null)
      instance_max_spot_price                                        = optional(string, null)
      instance_target_capacity_type                                  = optional(string, "spot")
      instance_types                                                 = list(string)
      job_queue_retention_in_seconds                                 = optional(number, 86400)
      minimum_running_time_in_minutes                                = optional(number, null)
      pool_runner_owner                                              = optional(string, null)
      runner_as_root                                                 = optional(bool, false)
      runner_boot_time_in_minutes                                    = optional(number, 5)
      scale_down_idle_confirmation_seconds                           = optional(number, 0)
      runner_disable_default_labels                                  = optional(bool, false)
      runner_extra_labels                                            = optional(list(string), [])
      runner_group_name                                              = optional(string, "Default")
      runner_name_prefix                                             = optional(string, "")
      runner_run_as                                                  = optional(string, "ec2-user")
      runners_maximum_count                                          = number
      runner_additional_security_group_ids                           = optional(list(string), [])
      scale_down_schedule_expression                                 = optional(string, "cron(*/5 * * * ? *)")
      scale_up_reserved_concurrent_executions                        = optional(number, 1)
      lambda_event_source_mapping_batch_size                         = optional(number, null)
      lambda_event_source_mapping_maximum_batching_window_in_seconds = optional(number, null)
      userdata_template                                              = optional(string, null)
      userdata_content                                               = optional(string, null)
      enable_jit_config                                              = optional(bool, null)
      enable_runner_detailed_monitoring                              = optional(bool, false)
      enable_cloudwatch_agent                                        = optional(bool, true)
      cloudwatch_config                                              = optional(string, null)
      userdata_pre_install                                           = optional(string, "")
      userdata_post_install                                          = optional(string, "")
      runner_hook_job_started                                        = optional(string, "")
      runner_hook_job_completed                                      = optional(string, "")
      runner_ec2_tags                                                = optional(map(string), {})
      runner_iam_role_managed_policy_arns                            = optional(list(string), [])
      vpc_id                                                         = optional(string, null)
      subnet_ids                                                     = optional(list(string), null)
      idle_config = optional(list(object({
        cron             = string
        timeZone         = string
        idleCount        = number
        evictionStrategy = optional(string, "oldest_first")
      })), [])
      cpu_options = optional(object({
        core_count            = optional(number)
        threads_per_core      = optional(number)
        amd_sev_snp           = optional(string)
        nested_virtualization = optional(string)
      }), null)
      network_interfaces = optional(list(object({
        associate_carrier_ip_address = optional(bool)
        associate_public_ip_address  = optional(bool)
        delete_on_termination        = optional(bool)
        description                  = optional(string)
        device_index                 = optional(number)
        interface_type               = optional(string)
        ipv4_address_count           = optional(number)
        ipv4_addresses               = optional(list(string))
        ipv4_prefix_count            = optional(number)
        ipv4_prefixes                = optional(list(string))
        ipv6_address_count           = optional(number)
        ipv6_addresses               = optional(list(string))
        ipv6_prefix_count            = optional(number)
        ipv6_prefixes                = optional(list(string))
        network_card_index           = optional(number)
        network_interface_id         = optional(string)
        primary_ipv6                 = optional(bool)
        private_ip_address           = optional(string)
        security_groups              = optional(list(string))
        subnet_id                    = optional(string)
        connection_tracking_specification = optional(object({
          tcp_established_timeout = optional(number)
          udp_stream_timeout      = optional(number)
          udp_timeout             = optional(number)
        }))
        ena_srd_specification = optional(object({
          ena_srd_enabled = optional(bool)
          ena_srd_udp_specification = optional(object({
            ena_srd_udp_enabled = optional(bool)
          }))
        }))
      })), [])
      placement = optional(object({
        affinity                = optional(string)
        availability_zone       = optional(string)
        group_id                = optional(string)
        group_name              = optional(string)
        host_id                 = optional(string)
        host_resource_group_arn = optional(string)
        spread_domain           = optional(string)
        tenancy                 = optional(string)
        partition_number        = optional(number)
      }), null)
      license_specifications = optional(list(object({
        license_configuration_arn = string
      })), [])
      use_dedicated_host = optional(bool, false)
      runner_log_files = optional(list(object({
        log_group_name   = string
        prefix_log_group = bool
        file_path        = string
        log_stream_name  = string
        log_class        = optional(string, "STANDARD")
      })), null)
      block_device_mappings = optional(list(object({
        delete_on_termination      = optional(bool, true)
        device_name                = optional(string, "/dev/xvda")
        encrypted                  = optional(bool, true)
        iops                       = optional(number)
        kms_key_id                 = optional(string)
        snapshot_id                = optional(string)
        throughput                 = optional(number)
        volume_initialization_rate = optional(number)
        volume_size                = number
        volume_type                = optional(string, "gp3")
        })), [{
        volume_size = 30
      }])
      pool_config = optional(list(object({
        schedule_expression          = string
        schedule_expression_timezone = optional(string)
        size                         = number
      })), [])
      job_retry = optional(object({
        enable             = optional(bool, false)
        delay_in_seconds   = optional(number, 300)
        delay_backoff      = optional(number, 2)
        lambda_memory_size = optional(number, 256)
        lambda_timeout     = optional(number, 30)
        max_attempts       = optional(number, 1)
      }), {})
      iam_overrides = optional(object({
        override_instance_profile = optional(bool, null)
        instance_profile_name     = optional(string, null)
        override_runner_role      = optional(bool, null)
        runner_role_arn           = optional(string, null)
        }), {
        override_instance_profile = false
        instance_profile_name     = null
        override_runner_role      = false
        runner_role_arn           = null
      })
    }), null)
    matcherConfig = optional(object({
      labelMatchers           = list(list(string))
      exactMatch              = optional(bool, false)
      bidirectionalLabelMatch = optional(bool, false)
      priority                = optional(number, 999)
      enableDynamicLabels     = optional(bool, false)
      awsDynamicLabelsPolicy  = optional(any, null)
    }), null)
    redrive_build_queue = optional(object({
      enabled         = bool
      maxReceiveCount = number
      }), {
      enabled         = false
      maxReceiveCount = null
    })

    # V2 Contract
    tags = optional(map(string), {})

    runner = optional(object({
      os                     = optional(string, null)
      architecture           = optional(string, null)
      disable_default_labels = optional(bool, null)
      extra_labels           = optional(list(string), null)
      group_name             = optional(string, null)
      name_prefix            = optional(string, null)
      run_as_root            = optional(bool, null)
      run_as                 = optional(string, null)
      auto_update_disabled   = optional(bool, null)
      tags                   = optional(map(string), {})
      hooks = optional(object({
        job_started   = optional(string, null)
        job_completed = optional(string, null)
      }), {})
      iam = optional(object({
        role = optional(object({
          arn = string
        }), null)
        managed_policy_arns          = optional(map(string), null)
        additional_trust_policy_json = optional(string, null)
        path                         = optional(string, null)
        permissions_boundary         = optional(string, null)
      }), {})
    }), {})

    lambda = optional(object({
      runtime            = optional(string, null)
      architecture       = optional(string, null)
      subnet_ids         = optional(list(string), null)
      security_group_ids = optional(list(string), null)
      tags               = optional(map(string), {})
      role = optional(object({
        path                 = optional(string, null)
        permissions_boundary = optional(string, null)
      }), {})
    }), {})

    orchestration_provider = optional(object({
      webhook = optional(object({
        runner = optional(object({
          boot_time_in_minutes = optional(number, null)
          ephemeral            = optional(bool, null)
          jit_config_enabled   = optional(bool, null)
          maximum_count        = optional(number, null)
        }), {})
        github = optional(object({
          organization_runners = optional(bool, false)
        }), {})
        matcherConfig = optional(object({
          labelMatchers           = list(list(string))
          exactMatch              = optional(bool, false)
          bidirectionalLabelMatch = optional(bool, false)
          priority                = optional(number, 999)
          dynamic_labels_enabled  = optional(bool, false)
          awsDynamicLabelsPolicy = optional(object({
            allowed_keys = optional(list(string), [])
            blocked_keys = optional(list(string), [])
            restricted_keys = optional(map(object({
              allowed = optional(list(string), [])
              denied  = optional(list(string), [])
              max     = optional(string, null)
            })), {})
          }), null)
        }), null)
        queue = optional(object({
          delay_webhook_event            = optional(number, null)
          job_queue_retention_in_seconds = optional(number, null)
          visibility_timeout_seconds     = optional(number, null)
          redrive_build_queue = optional(object({
            enabled         = optional(bool, null)
            maxReceiveCount = optional(number, null)
          }), null)
          tags = optional(map(string), {})
        }), {})
        lambda = optional(object({
          scale = optional(object({
            up = optional(object({
              memory_size                    = optional(number, null)
              timeout                        = optional(number, null)
              reserved_concurrent_executions = optional(number, null)
              job_queued_check_enabled       = optional(bool, null)
              event_source_mapping = optional(object({
                batch_size                         = optional(number, null)
                maximum_batching_window_in_seconds = optional(number, null)
              }), {})
              tags = optional(map(string), {})
            }), {})
            down = optional(object({
              memory_size                     = optional(number, null)
              timeout                         = optional(number, null)
              schedule_expression             = optional(string, null)
              minimum_running_time_in_minutes = optional(number, null)
              idle_confirmation_seconds       = optional(number, null)
              idle_config = optional(list(object({
                cron             = string
                timeZone         = string
                idleCount        = number
                evictionStrategy = optional(string, "oldest_first")
              })), null)
              tags = optional(map(string), {})
            }), {})
          }), {})
          pool = optional(object({
            memory_size                    = optional(number, null)
            timeout                        = optional(number, null)
            reserved_concurrent_executions = optional(number, null)
            config = optional(list(object({
              schedule_expression          = string
              schedule_expression_timezone = optional(string)
              size                         = number
            })), null)
            include_busy_runners = optional(bool, null)
            runner_owner         = optional(string, null)
            tags                 = optional(map(string), {})
          }), {})
        }), {})
        job_retry = optional(object({
          enabled          = optional(bool, false)
          delay_in_seconds = optional(number, 300)
          delay_backoff    = optional(number, 2)
          max_attempts     = optional(number, 1)
          tags             = optional(map(string), {})
          lambda = optional(object({
            memory_size                    = optional(number, 256)
            reserved_concurrent_executions = optional(number, 1)
            timeout                        = optional(number, 30)
          }), {})
        }), {})
      }), null)
      scale_set = optional(object({
        name = string
        runner = optional(object({
          min_runners          = optional(number, 0)
          max_runners          = optional(number, 10)
          boot_time_in_minutes = optional(number, 10)
        }), {})
      }), null)
    }), {})

    ssm = optional(object({
      paths = optional(object({
        root   = optional(string, null)
        tokens = optional(string, null)
        config = optional(string, null)
      }), {})
      tags = optional(map(string), {})
      parameters = optional(object({
        tags = optional(map(string), {})
      }), {})
      housekeeper = optional(object({
        schedule_expression = optional(string, null)
        state               = optional(string, null)
        tags                = optional(map(string), {})
        lambda = optional(object({
          artifact = optional(object({
            zip = optional(string, null)
            s3 = optional(object({
              key            = string
              object_version = optional(string, null)
            }), null)
          }), {})
          memory_size = optional(number, null)
          timeout     = optional(number, null)
        }), {})
        config = optional(object({
          tokenPath      = optional(string, null)
          minimumDaysOld = optional(number, null)
          dryRun         = optional(bool, null)
        }), {})
      }), {})
    }), {})

    observability = optional(object({
      logs = optional(object({
        level             = optional(string, null)
        retention_in_days = optional(number, null)
        kms_key_id        = optional(string, null)
        class             = optional(string, null)
        tags              = optional(map(string), {})
      }), {})
      tracing = optional(object({
        mode                  = optional(string, null)
        capture_http_requests = optional(bool, null)
        capture_error         = optional(bool, null)
      }), {})
      metrics = optional(object({
        enabled   = optional(bool, null)
        namespace = optional(string, null)
        metric = optional(object({
          github_app_rate_limit = optional(object({
            enabled = optional(bool, null)
          }), {})
          job_retry = optional(object({
            enabled = optional(bool, null)
          }), {})
          spot_termination_warning = optional(object({
            enabled = optional(bool, null)
          }), {})
        }), {})
      }), {})
    }), {})

    compute_provider = optional(object({
      aws = optional(object({
        ec2 = optional(object({
          metadata_options = optional(object({
            instance_metadata_tags      = optional(string, "enabled")
            http_endpoint               = optional(string, "enabled")
            http_tokens                 = optional(string, "required")
            http_put_response_hop_limit = optional(number, 1)
          }), {})
          ami = optional(object({
            filter = optional(map(list(string)), { state = ["available"] })
            owners = optional(list(string), ["amazon"])
            id_ssm_parameter = optional(object({
              arn = string
            }), null)
            kms_key = optional(object({
              arn = string
            }), null)
          }), null)
          block_device_mappings = optional(list(object({
            delete_on_termination      = optional(bool, true)
            device_name                = optional(string, "/dev/xvda")
            encrypted                  = optional(bool, true)
            iops                       = optional(number)
            kms_key_id                 = optional(string)
            snapshot_id                = optional(string)
            throughput                 = optional(number)
            volume_initialization_rate = optional(number)
            volume_size                = number
            volume_type                = optional(string, "gp3")
          })), [{ volume_size = 30 }])
          create_service_linked_role_spot = optional(bool, false)
          credit_specification            = optional(string, null)
          ebs_optimized                   = optional(bool, false)
          cloudwatch_agent = optional(object({
            enabled = optional(bool, true)
            config  = optional(string, null)
          }), {})
          binaries_syncer = optional(object({
            enabled = optional(bool, null)
          }), {})
          detailed_monitoring_enabled = optional(bool, false)
          ssm_enabled                 = optional(bool, false)
          user_data = optional(object({
            enabled               = optional(bool, true)
            template              = optional(string, null)
            content               = optional(string, null)
            pre_install           = optional(string, "")
            post_install          = optional(string, "")
            debug_logging_enabled = optional(bool, false)
          }), {})
          instance_allocation_strategy   = optional(string, "lowest-price")
          instance_max_spot_price        = optional(string, null)
          instance_target_capacity_type  = optional(string, "spot")
          instance_type_priorities       = optional(map(number), null)
          instance_types                 = optional(list(string), [])
          additional_security_group_ids  = optional(list(string), null)
          managed_security_group_enabled = optional(bool, null)
          egress_rules = optional(list(object({
            cidr_blocks      = list(string)
            ipv6_cidr_blocks = list(string)
            prefix_list_ids  = list(string)
            from_port        = number
            protocol         = string
            security_groups  = list(string)
            self             = bool
            to_port          = number
            description      = string
          })), null)
          instance_profile_path         = optional(string, null)
          key_name                      = optional(string, null)
          associate_public_ipv4_address = optional(bool, null)
          instance_profile = optional(object({
            name = string
          }), null)
          on_demand_failover_for_errors = optional(list(string), [])
          scale_errors = optional(list(string), [
            "UnfulfillableCapacity",
            "MaxSpotInstanceCountExceeded",
            "TargetCapacityLimitExceededException",
            "RequestLimitExceeded",
            "ResourceLimitExceeded",
            "MaxSpotInstanceCountExceeded",
            "MaxSpotFleetRequestCountExceeded",
            "InsufficientInstanceCapacity",
            "InsufficientCapacityOnHost",
          ])
          subnet_ids = optional(list(string), null)
          vpc_id     = optional(string, null)
          cpu_options = optional(object({
            core_count            = optional(number)
            threads_per_core      = optional(number)
            amd_sev_snp           = optional(string)
            nested_virtualization = optional(string)
          }), null)
          network_interfaces = optional(list(object({
            associate_carrier_ip_address = optional(bool)
            associate_public_ip_address  = optional(bool)
            delete_on_termination        = optional(bool)
            description                  = optional(string)
            device_index                 = optional(number)
            interface_type               = optional(string)
            ipv4_address_count           = optional(number)
            ipv4_addresses               = optional(list(string))
            ipv4_prefix_count            = optional(number)
            ipv4_prefixes                = optional(list(string))
            ipv6_address_count           = optional(number)
            ipv6_addresses               = optional(list(string))
            ipv6_prefix_count            = optional(number)
            ipv6_prefixes                = optional(list(string))
            network_card_index           = optional(number)
            network_interface_id         = optional(string)
            primary_ipv6                 = optional(bool)
            private_ip_address           = optional(string)
            security_groups              = optional(list(string))
            subnet_id                    = optional(string)
            connection_tracking_specification = optional(object({
              tcp_established_timeout = optional(number)
              udp_stream_timeout      = optional(number)
              udp_timeout             = optional(number)
            }))
            ena_srd_specification = optional(object({
              ena_srd_enabled = optional(bool)
              ena_srd_udp_specification = optional(object({
                ena_srd_udp_enabled = optional(bool)
              }))
            }))
          })), [])
          placement = optional(object({
            affinity                = optional(string)
            availability_zone       = optional(string)
            group_id                = optional(string)
            group_name              = optional(string)
            host_id                 = optional(string)
            host_resource_group_arn = optional(string)
            spread_domain           = optional(string)
            tenancy                 = optional(string)
            partition_number        = optional(number)
          }), null)
          license_specifications = optional(list(object({
            license_configuration_arn = string
          })), [])
          use_dedicated_host = optional(bool, false)
          log_files = optional(list(object({
            log_group_name   = string
            prefix_log_group = bool
            file_path        = string
            log_stream_name  = string
            log_class        = optional(string, "STANDARD")
          })), null)
          tags = optional(map(string), {})
        }), null)
      }), {})
    }), {})
  }))
| `{}` | no | | [parameter\_store\_tags](#input\_parameter\_store\_tags) | Map of tags that will be added to all the SSM Parameter Store parameters created by the Lambda function. | `map(string)` | `{}` | no | | [pool\_lambda\_reserved\_concurrent\_executions](#input\_pool\_lambda\_reserved\_concurrent\_executions) | Amount of reserved concurrent executions for the scale-up lambda function. A value of 0 disables lambda from being triggered and -1 removes any concurrency limitations. | `number` | `1` | no | | [pool\_lambda\_timeout](#input\_pool\_lambda\_timeout) | Time out for the pool lambda in seconds. | `number` | `60` | no | diff --git a/modules/runner-config/README.md b/modules/runner-config/README.md index 41c0b1a191..ea6fd45b4e 100644 --- a/modules/runner-config/README.md +++ b/modules/runner-config/README.md @@ -109,7 +109,7 @@ yarn run dist |------|-------------|------|---------|:--------:| | [aws\_partition](#input\_aws\_partition) | AWS partition used to construct ARNs. | `string` | `"aws"` | no | | [aws\_region](#input\_aws\_region) | AWS region. | `string` | n/a | yes | -| [compute\_provider](#input\_compute\_provider) | Typed compute-provider configuration. Provider-owned settings remain inside the selected compute-provider block.

Exactly one compute-provider block must be non-null. The populated block selects the provider, and its presence must be known during planning. Values inside the selected block may remain unknown until apply.

- `aws`: AWS compute-provider configurations.
- `aws.ec2`: EC2 compute-provider configuration.
- `aws.ec2.ami`: Optional AMI discovery or external AMI-parameter configuration. Null uses the operating-system and architecture defaults.
- `aws.ec2.ami.filter`: EC2 AMI filters combined with the provider's default AMI-name filter.
- `aws.ec2.ami.owners`: AWS account IDs or aliases allowed to own the selected AMI.
- `aws.ec2.ami.id_ssm_parameter`: Optional externally managed SSM parameter containing the AMI ID. Null creates a provider-managed AMI-ID parameter. The wrapper's presence is the plan-time ownership discriminator, so keep the object literal even when its ARN comes from another resource.
- `aws.ec2.ami.id_ssm_parameter.arn`: ARN of the externally managed SSM parameter. The ARN may be unknown until apply.
- `aws.ec2.ami.kms_key`: Optional KMS key required to launch encrypted AMIs or snapshots. The wrapper's presence is the plan-time policy discriminator.
- `aws.ec2.ami.kms_key.arn`: ARN of the KMS key. The ARN may be unknown until apply.
- `aws.ec2.vpc_id`: VPC in which runner networking resources are created.
- `aws.ec2.subnet_ids`: Subnets from which scale-up may launch runner instances.
- `aws.ec2.overrides`: Optional resource-name overrides.
- `aws.ec2.overrides.name_runner`: Name tag used for runner compute resources. An empty value uses the generated provider name.
- `aws.ec2.overrides.name_sg`: Name tag used for the managed runner security group. An empty value uses the generated provider name.
- `aws.ec2.instance_profile`: Optional externally managed instance profile used by the launch template.
- `aws.ec2.instance_profile.name`: Name of the externally managed instance profile.
- `aws.ec2.instance_profile_path`: IAM path for the provider-managed instance profile. Null uses a path derived from the runner-configuration prefix.
- `aws.ec2.binaries_syncer`: Runner-distribution synchronization configuration.
- `aws.ec2.binaries_syncer.enabled`: Enables use of a synchronized runner distribution from S3.
- `aws.ec2.binaries_syncer.s3`: S3 object containing the synchronized runner distribution. Required when synchronization is enabled.
- `aws.ec2.binaries_syncer.s3.arn`: ARN of the runner-distribution bucket, used by IAM policies.
- `aws.ec2.binaries_syncer.s3.id`: Bucket name used to construct the runner-distribution S3 URI.
- `aws.ec2.binaries_syncer.s3.key`: Object key of the runner distribution.
- `aws.ec2.block_device_mappings`: EBS mappings added to the runner launch template.
- `aws.ec2.block_device_mappings[].delete_on_termination`: Deletes the volume when its runner instance terminates.
- `aws.ec2.block_device_mappings[].device_name`: Device name exposed to the runner instance.
- `aws.ec2.block_device_mappings[].encrypted`: Enables EBS encryption.
- `aws.ec2.block_device_mappings[].iops`: Provisioned IOPS for volume types that support it.
- `aws.ec2.block_device_mappings[].kms_key_id`: KMS key ID or ARN used to encrypt the volume.
- `aws.ec2.block_device_mappings[].snapshot_id`: Snapshot used to initialize the volume.
- `aws.ec2.block_device_mappings[].throughput`: Provisioned throughput for volume types that support it.
- `aws.ec2.block_device_mappings[].volume_initialization_rate`: Fixed initialization rate in MiB/s for supported snapshot-backed volumes.
- `aws.ec2.block_device_mappings[].volume_size`: Volume size in GiB.
- `aws.ec2.block_device_mappings[].volume_type`: EBS volume type.
- `aws.ec2.ebs_optimized`: Requests EBS-optimized runner instances.
- `aws.ec2.instance_target_capacity_type`: Primary capacity type, either `spot` or `on-demand`.
- `aws.ec2.instance_allocation_strategy`: EC2 Fleet allocation strategy used to select instance capacity.
- `aws.ec2.instance_type_priorities`: Optional numeric priorities keyed by instance type.
- `aws.ec2.instance_max_spot_price`: Optional maximum hourly Spot price.
- `aws.ec2.instance_types`: EC2 instance types available to the scale-up and pool functions.
- `aws.ec2.user_data`: Runner bootstrap user-data configuration.
- `aws.ec2.user_data.enabled`: Enables launch-template user data.
- `aws.ec2.user_data.template`: Optional path to a custom user-data template.
- `aws.ec2.user_data.content`: Optional complete user-data content. When set, it is used instead of rendering a template.
- `aws.ec2.user_data.pre_install`: Script content inserted before runner installation in the default template.
- `aws.ec2.user_data.post_install`: Script content inserted after runner installation in the default template.
- `aws.ec2.user_data.debug_logging_enabled`: Enables verbose user-data tracing, which can expose secrets in logs.
- `aws.ec2.ssm_enabled`: Attaches runner permissions and policies required for AWS Systems Manager access.
- `aws.ec2.create_service_linked_role_spot`: Allows scale-up to create the EC2 Spot service-linked role.
- `aws.ec2.cloudwatch_agent`: CloudWatch agent configuration for runner instances.
- `aws.ec2.cloudwatch_agent.enabled`: Installs and configures the CloudWatch agent through the default bootstrap flow.
- `aws.ec2.cloudwatch_agent.config`: Optional complete CloudWatch agent configuration. Null renders the provider default from `log_files`.
- `aws.ec2.managed_security_group_enabled`: Creates and attaches the provider-managed runner security group.
- `aws.ec2.log_files`: Optional log files collected by the CloudWatch agent. Null uses the provider defaults.
- `aws.ec2.log_files[].log_group_name`: CloudWatch log-group name, before optional prefixing.
- `aws.ec2.log_files[].prefix_log_group`: Prefixes the log-group name with the runner configuration path when true.
- `aws.ec2.log_files[].file_path`: File or glob read by the CloudWatch agent.
- `aws.ec2.log_files[].log_stream_name`: CloudWatch log-stream name template.
- `aws.ec2.log_files[].log_class`: CloudWatch log-group class for the collected file.
- `aws.ec2.key_name`: Optional EC2 key-pair name added to the launch template.
- `aws.ec2.additional_security_group_ids`: Existing security groups attached in addition to the managed security group.
- `aws.ec2.detailed_monitoring_enabled`: Enables detailed EC2 monitoring for runner instances.
- `aws.ec2.egress_rules`: Egress rules created on the managed runner security group.
- `aws.ec2.egress_rules[].cidr_blocks`: IPv4 CIDR destinations.
- `aws.ec2.egress_rules[].ipv6_cidr_blocks`: IPv6 CIDR destinations.
- `aws.ec2.egress_rules[].prefix_list_ids`: AWS prefix-list destinations.
- `aws.ec2.egress_rules[].from_port`: First destination port in the permitted range.
- `aws.ec2.egress_rules[].protocol`: IP protocol name or number. Use `-1` for all protocols.
- `aws.ec2.egress_rules[].security_groups`: Destination security-group IDs.
- `aws.ec2.egress_rules[].self`: Allows traffic to the managed security group itself when true.
- `aws.ec2.egress_rules[].to_port`: Last destination port in the permitted range.
- `aws.ec2.egress_rules[].description`: Optional rule description.
- `aws.ec2.tags`: Additional tags for runner instances, EBS volumes, network interfaces, and eligible Spot instance requests created from the launch template. They override module-level tags and the generated runner `Name`; the provider-managed `ghr:environment`, `ghr:ssm_config_path`, and `ghr:runner_name_prefix` bootstrap tags take final precedence. These tags do not apply to static provider resources such as the launch template, security group, IAM resources, SSM parameters, or log groups.
- `aws.ec2.metadata_options`: Instance Metadata Service configuration in the launch template.
- `aws.ec2.metadata_options.instance_metadata_tags`: Exposes instance tags through Instance Metadata Service when `enabled`.
- `aws.ec2.metadata_options.http_endpoint`: Enables or disables the Instance Metadata Service endpoint.
- `aws.ec2.metadata_options.http_tokens`: Controls whether IMDSv2 session tokens are optional or required.
- `aws.ec2.metadata_options.http_put_response_hop_limit`: Network hop limit for Instance Metadata Service token responses.
- `aws.ec2.credit_specification`: CPU credit mode for burstable instance types, either `standard` or `unlimited`.
- `aws.ec2.cpu_options`: CPU topology and processor-feature configuration.
- `aws.ec2.cpu_options.core_count`: Number of CPU cores exposed to the runner instance.
- `aws.ec2.cpu_options.threads_per_core`: Number of hardware threads exposed per CPU core.
- `aws.ec2.cpu_options.amd_sev_snp`: Enables or disables AMD SEV-SNP on supported instance types.
- `aws.ec2.cpu_options.nested_virtualization`: Enables or disables nested virtualization on supported instance types.
- `aws.ec2.placement`: EC2 placement configuration for runner instances.
- `aws.ec2.placement.affinity`: Host affinity setting.
- `aws.ec2.placement.availability_zone`: Availability Zone in which the instance is placed.
- `aws.ec2.placement.group_id`: Placement-group ID.
- `aws.ec2.placement.group_name`: Placement-group name.
- `aws.ec2.placement.host_id`: Dedicated Host ID.
- `aws.ec2.placement.host_resource_group_arn`: ARN of the host resource group used for placement.
- `aws.ec2.placement.spread_domain`: Spread-domain placement value.
- `aws.ec2.placement.tenancy`: Instance tenancy, such as `default`, `dedicated`, or `host`.
- `aws.ec2.placement.partition_number`: Placement-group partition number.
- `aws.ec2.license_specifications`: License Manager configurations added to the launch template.
- `aws.ec2.license_specifications[].license_configuration_arn`: ARN of a License Manager license configuration.
- `aws.ec2.associate_public_ipv4_address`: Associates a public IPv4 address with runner network interfaces.
- `aws.ec2.on_demand_failover_for_errors`: EC2 error codes that trigger an on-demand fallback after a Spot launch failure.
- `aws.ec2.scale_errors`: EC2 error codes treated as retryable scale-up failures.
- `aws.ec2.use_dedicated_host`: Enables the dedicated-host launch path, required for macOS runners. | 
object({
    aws = optional(object({
      ec2 = optional(object({
        ami = optional(object({
          filter = optional(map(list(string)), { state = ["available"] })
          owners = optional(list(string), ["amazon"])
          id_ssm_parameter = optional(object({
            arn = string
          }), null)
          kms_key = optional(object({
            arn = string
          }), null)
        }), null)
        vpc_id     = string
        subnet_ids = list(string)
        overrides = optional(object({
          name_runner = optional(string, "")
          name_sg     = optional(string, "")
        }), {})
        instance_profile = optional(object({
          name = string
        }), null)
        instance_profile_path = optional(string, null)
        binaries_syncer = optional(object({
          enabled = optional(bool, true)
          s3 = optional(object({
            arn = string
            id  = string
            key = string
          }), null)
        }), {})
        block_device_mappings = optional(list(object({
          delete_on_termination      = optional(bool, true)
          device_name                = optional(string, "/dev/xvda")
          encrypted                  = optional(bool, true)
          iops                       = optional(number)
          kms_key_id                 = optional(string)
          snapshot_id                = optional(string)
          throughput                 = optional(number)
          volume_initialization_rate = optional(number)
          volume_size                = number
          volume_type                = optional(string, "gp3")
        })), [{ volume_size = 30 }])
        ebs_optimized                 = optional(bool, false)
        instance_target_capacity_type = optional(string, "spot")
        instance_allocation_strategy  = optional(string, "lowest-price")
        instance_type_priorities      = optional(map(number), null)
        instance_max_spot_price       = optional(string, null)
        instance_types                = list(string)
        user_data = optional(object({
          enabled               = optional(bool, true)
          template              = optional(string, null)
          content               = optional(string, null)
          pre_install           = optional(string, "")
          post_install          = optional(string, "")
          debug_logging_enabled = optional(bool, false)
        }), {})
        ssm_enabled                     = optional(bool, false)
        create_service_linked_role_spot = optional(bool, false)
        cloudwatch_agent = optional(object({
          enabled = optional(bool, true)
          config  = optional(string, null)
        }), {})
        managed_security_group_enabled = optional(bool, true)
        log_files = optional(list(object({
          log_group_name   = string
          prefix_log_group = bool
          file_path        = string
          log_stream_name  = string
          log_class        = optional(string, "STANDARD")
        })), null)
        key_name                      = optional(string, null)
        additional_security_group_ids = optional(list(string), [])
        detailed_monitoring_enabled   = optional(bool, false)
        egress_rules = optional(list(object({
          cidr_blocks      = list(string)
          ipv6_cidr_blocks = list(string)
          prefix_list_ids  = list(string)
          from_port        = number
          protocol         = string
          security_groups  = list(string)
          self             = bool
          to_port          = number
          description      = string
          })), [{
          cidr_blocks      = ["0.0.0.0/0"]
          ipv6_cidr_blocks = ["::/0"]
          prefix_list_ids  = null
          from_port        = 0
          protocol         = "-1"
          security_groups  = null
          self             = null
          to_port          = 0
          description      = null
        }])
        tags = optional(map(string), {})
        metadata_options = optional(object({
          instance_metadata_tags      = optional(string, "enabled")
          http_endpoint               = optional(string, "enabled")
          http_tokens                 = optional(string, "required")
          http_put_response_hop_limit = optional(number, 1)
        }), {})
        credit_specification = optional(string, null)
        cpu_options = optional(object({
          core_count            = optional(number)
          threads_per_core      = optional(number)
          amd_sev_snp           = optional(string)
          nested_virtualization = optional(string)
        }), null)
        placement = optional(object({
          affinity                = optional(string)
          availability_zone       = optional(string)
          group_id                = optional(string)
          group_name              = optional(string)
          host_id                 = optional(string)
          host_resource_group_arn = optional(string)
          spread_domain           = optional(string)
          tenancy                 = optional(string)
          partition_number        = optional(number)
        }), null)
        license_specifications = optional(list(object({
          license_configuration_arn = string
        })), [])
        associate_public_ipv4_address = optional(bool, false)
        on_demand_failover_for_errors = optional(list(string), [])
        scale_errors = optional(list(string), [
          "UnfulfillableCapacity",
          "MaxSpotInstanceCountExceeded",
          "TargetCapacityLimitExceededException",
          "RequestLimitExceeded",
          "ResourceLimitExceeded",
          "MaxSpotInstanceCountExceeded",
          "MaxSpotFleetRequestCountExceeded",
          "InsufficientInstanceCapacity",
          "InsufficientCapacityOnHost",
        ])
        use_dedicated_host = optional(bool, false)
      }), null)
    }), {})
  })
| n/a | yes | +| [compute\_provider](#input\_compute\_provider) | Typed compute-provider configuration. Provider-owned settings remain inside the selected compute-provider block.

Exactly one compute-provider block must be non-null. The populated block selects the provider, and its presence must be known during planning. Values inside the selected block may remain unknown until apply.

- `aws`: AWS compute-provider configurations.
- `aws.ec2`: EC2 compute-provider configuration.
- `aws.ec2.ami`: Optional AMI discovery or external AMI-parameter configuration. Null uses the operating-system and architecture defaults.
- `aws.ec2.ami.filter`: EC2 AMI filters combined with the provider's default AMI-name filter.
- `aws.ec2.ami.owners`: AWS account IDs or aliases allowed to own the selected AMI.
- `aws.ec2.ami.id_ssm_parameter`: Optional externally managed SSM parameter containing the AMI ID. Null creates a provider-managed AMI-ID parameter. The wrapper's presence is the plan-time ownership discriminator, so keep the object literal even when its ARN comes from another resource.
- `aws.ec2.ami.id_ssm_parameter.arn`: ARN of the externally managed SSM parameter. The ARN may be unknown until apply.
- `aws.ec2.ami.kms_key`: Optional KMS key required to launch encrypted AMIs or snapshots. The wrapper's presence is the plan-time policy discriminator.
- `aws.ec2.ami.kms_key.arn`: ARN of the KMS key. The ARN may be unknown until apply.
- `aws.ec2.vpc_id`: VPC in which runner networking resources are created.
- `aws.ec2.subnet_ids`: Subnets from which scale-up may launch runner instances.
- `aws.ec2.overrides`: Optional resource-name overrides.
- `aws.ec2.overrides.name_runner`: Name tag used for runner compute resources. An empty value uses the generated provider name.
- `aws.ec2.overrides.name_sg`: Name tag used for the managed runner security group. An empty value uses the generated provider name.
- `aws.ec2.instance_profile`: Optional externally managed instance profile used by the launch template.
- `aws.ec2.instance_profile.name`: Name of the externally managed instance profile.
- `aws.ec2.instance_profile_path`: IAM path for the provider-managed instance profile. Null uses a path derived from the runner-configuration prefix.
- `aws.ec2.binaries_syncer`: Runner-distribution synchronization configuration.
- `aws.ec2.binaries_syncer.enabled`: Enables use of a synchronized runner distribution from S3.
- `aws.ec2.binaries_syncer.s3`: S3 object containing the synchronized runner distribution. Required when synchronization is enabled.
- `aws.ec2.binaries_syncer.s3.arn`: ARN of the runner-distribution bucket, used by IAM policies.
- `aws.ec2.binaries_syncer.s3.id`: Bucket name used to construct the runner-distribution S3 URI.
- `aws.ec2.binaries_syncer.s3.key`: Object key of the runner distribution.
- `aws.ec2.block_device_mappings`: EBS mappings added to the runner launch template.
- `aws.ec2.block_device_mappings[].delete_on_termination`: Deletes the volume when its runner instance terminates.
- `aws.ec2.block_device_mappings[].device_name`: Device name exposed to the runner instance.
- `aws.ec2.block_device_mappings[].encrypted`: Enables EBS encryption.
- `aws.ec2.block_device_mappings[].iops`: Provisioned IOPS for volume types that support it.
- `aws.ec2.block_device_mappings[].kms_key_id`: KMS key ID or ARN used to encrypt the volume.
- `aws.ec2.block_device_mappings[].snapshot_id`: Snapshot used to initialize the volume.
- `aws.ec2.block_device_mappings[].throughput`: Provisioned throughput for volume types that support it.
- `aws.ec2.block_device_mappings[].volume_initialization_rate`: Fixed initialization rate in MiB/s for supported snapshot-backed volumes.
- `aws.ec2.block_device_mappings[].volume_size`: Volume size in GiB.
- `aws.ec2.block_device_mappings[].volume_type`: EBS volume type.
- `aws.ec2.ebs_optimized`: Requests EBS-optimized runner instances.
- `aws.ec2.instance_target_capacity_type`: Primary capacity type, either `spot` or `on-demand`.
- `aws.ec2.instance_allocation_strategy`: EC2 Fleet allocation strategy used to select instance capacity.
- `aws.ec2.instance_type_priorities`: Optional numeric priorities keyed by instance type.
- `aws.ec2.instance_max_spot_price`: Optional maximum hourly Spot price.
- `aws.ec2.instance_types`: EC2 instance types available to the scale-up and pool functions.
- `aws.ec2.user_data`: Runner bootstrap user-data configuration.
- `aws.ec2.user_data.enabled`: Enables launch-template user data.
- `aws.ec2.user_data.template`: Optional path to a custom user-data template.
- `aws.ec2.user_data.content`: Optional complete user-data content. When set, it is used instead of rendering a template.
- `aws.ec2.user_data.pre_install`: Script content inserted before runner installation in the default template.
- `aws.ec2.user_data.post_install`: Script content inserted after runner installation in the default template.
- `aws.ec2.user_data.debug_logging_enabled`: Enables verbose user-data tracing, which can expose secrets in logs.
- `aws.ec2.ssm_enabled`: Attaches runner permissions and policies required for AWS Systems Manager access.
- `aws.ec2.create_service_linked_role_spot`: Allows scale-up to create the EC2 Spot service-linked role.
- `aws.ec2.cloudwatch_agent`: CloudWatch agent configuration for runner instances.
- `aws.ec2.cloudwatch_agent.enabled`: Installs and configures the CloudWatch agent through the default bootstrap flow.
- `aws.ec2.cloudwatch_agent.config`: Optional complete CloudWatch agent configuration. Null renders the provider default from `log_files`.
- `aws.ec2.managed_security_group_enabled`: Creates and attaches the provider-managed runner security group.
- `aws.ec2.log_files`: Optional log files collected by the CloudWatch agent. Null uses the provider defaults.
- `aws.ec2.log_files[].log_group_name`: CloudWatch log-group name, before optional prefixing.
- `aws.ec2.log_files[].prefix_log_group`: Prefixes the log-group name with the runner configuration path when true.
- `aws.ec2.log_files[].file_path`: File or glob read by the CloudWatch agent.
- `aws.ec2.log_files[].log_stream_name`: CloudWatch log-stream name template.
- `aws.ec2.log_files[].log_class`: CloudWatch log-group class for the collected file.
- `aws.ec2.key_name`: Optional EC2 key-pair name added to the launch template.
- `aws.ec2.additional_security_group_ids`: Existing security groups attached in addition to the managed security group.
- `aws.ec2.detailed_monitoring_enabled`: Enables detailed EC2 monitoring for runner instances.
- `aws.ec2.egress_rules`: Egress rules created on the managed runner security group.
- `aws.ec2.egress_rules[].cidr_blocks`: IPv4 CIDR destinations.
- `aws.ec2.egress_rules[].ipv6_cidr_blocks`: IPv6 CIDR destinations.
- `aws.ec2.egress_rules[].prefix_list_ids`: AWS prefix-list destinations.
- `aws.ec2.egress_rules[].from_port`: First destination port in the permitted range.
- `aws.ec2.egress_rules[].protocol`: IP protocol name or number. Use `-1` for all protocols.
- `aws.ec2.egress_rules[].security_groups`: Destination security-group IDs.
- `aws.ec2.egress_rules[].self`: Allows traffic to the managed security group itself when true.
- `aws.ec2.egress_rules[].to_port`: Last destination port in the permitted range.
- `aws.ec2.egress_rules[].description`: Optional rule description.
- `aws.ec2.tags`: Additional tags for runner instances, EBS volumes, network interfaces, and eligible Spot instance requests created from the launch template. They override module-level tags and the generated runner `Name`; the provider-managed `ghr:environment`, `ghr:ssm_config_path`, and `ghr:runner_name_prefix` bootstrap tags take final precedence. These tags do not apply to static provider resources such as the launch template, security group, IAM resources, SSM parameters, or log groups.
- `aws.ec2.metadata_options`: Instance Metadata Service configuration in the launch template.
- `aws.ec2.metadata_options.instance_metadata_tags`: Exposes instance tags through Instance Metadata Service when `enabled`.
- `aws.ec2.metadata_options.http_endpoint`: Enables or disables the Instance Metadata Service endpoint.
- `aws.ec2.metadata_options.http_tokens`: Controls whether IMDSv2 session tokens are optional or required.
- `aws.ec2.metadata_options.http_put_response_hop_limit`: Network hop limit for Instance Metadata Service token responses.
- `aws.ec2.credit_specification`: CPU credit mode for burstable instance types, either `standard` or `unlimited`.
- `aws.ec2.cpu_options`: CPU topology and processor-feature configuration.
- `aws.ec2.cpu_options.core_count`: Number of CPU cores exposed to the runner instance.
- `aws.ec2.cpu_options.threads_per_core`: Number of hardware threads exposed per CPU core.
- `aws.ec2.cpu_options.amd_sev_snp`: Enables or disables AMD SEV-SNP on supported instance types.
- `aws.ec2.cpu_options.nested_virtualization`: Enables or disables nested virtualization on supported instance types.
- `aws.ec2.placement`: EC2 placement configuration for runner instances.
- `aws.ec2.placement.affinity`: Host affinity setting.
- `aws.ec2.placement.availability_zone`: Availability Zone in which the instance is placed.
- `aws.ec2.placement.group_id`: Placement-group ID.
- `aws.ec2.placement.group_name`: Placement-group name.
- `aws.ec2.placement.host_id`: Dedicated Host ID.
- `aws.ec2.placement.host_resource_group_arn`: ARN of the host resource group used for placement.
- `aws.ec2.placement.spread_domain`: Spread-domain placement value.
- `aws.ec2.placement.tenancy`: Instance tenancy, such as `default`, `dedicated`, or `host`.
- `aws.ec2.placement.partition_number`: Placement-group partition number.
- `aws.ec2.license_specifications`: License Manager configurations added to the launch template.
- `aws.ec2.license_specifications[].license_configuration_arn`: ARN of a License Manager license configuration.
- `aws.ec2.associate_public_ipv4_address`: Associates a public IPv4 address with runner network interfaces.
- `aws.ec2.network_interfaces`: Advanced network interface configuration for the launch template. Leave empty to keep using `associate_public_ipv4_address` for a simple single-interface setup.
- `aws.ec2.on_demand_failover_for_errors`: EC2 error codes that trigger an on-demand fallback after a Spot launch failure.
- `aws.ec2.scale_errors`: EC2 error codes treated as retryable scale-up failures.
- `aws.ec2.use_dedicated_host`: Enables the dedicated-host launch path, required for macOS runners. | 
object({
    aws = optional(object({
      ec2 = optional(object({
        ami = optional(object({
          filter = optional(map(list(string)), { state = ["available"] })
          owners = optional(list(string), ["amazon"])
          id_ssm_parameter = optional(object({
            arn = string
          }), null)
          kms_key = optional(object({
            arn = string
          }), null)
        }), null)
        vpc_id     = string
        subnet_ids = list(string)
        overrides = optional(object({
          name_runner = optional(string, "")
          name_sg     = optional(string, "")
        }), {})
        instance_profile = optional(object({
          name = string
        }), null)
        instance_profile_path = optional(string, null)
        binaries_syncer = optional(object({
          enabled = optional(bool, true)
          s3 = optional(object({
            arn = string
            id  = string
            key = string
          }), null)
        }), {})
        block_device_mappings = optional(list(object({
          delete_on_termination      = optional(bool, true)
          device_name                = optional(string, "/dev/xvda")
          encrypted                  = optional(bool, true)
          iops                       = optional(number)
          kms_key_id                 = optional(string)
          snapshot_id                = optional(string)
          throughput                 = optional(number)
          volume_initialization_rate = optional(number)
          volume_size                = number
          volume_type                = optional(string, "gp3")
        })), [{ volume_size = 30 }])
        ebs_optimized                 = optional(bool, false)
        instance_target_capacity_type = optional(string, "spot")
        instance_allocation_strategy  = optional(string, "lowest-price")
        instance_type_priorities      = optional(map(number), null)
        instance_max_spot_price       = optional(string, null)
        instance_types                = list(string)
        user_data = optional(object({
          enabled               = optional(bool, true)
          template              = optional(string, null)
          content               = optional(string, null)
          pre_install           = optional(string, "")
          post_install          = optional(string, "")
          debug_logging_enabled = optional(bool, false)
        }), {})
        ssm_enabled                     = optional(bool, false)
        create_service_linked_role_spot = optional(bool, false)
        cloudwatch_agent = optional(object({
          enabled = optional(bool, true)
          config  = optional(string, null)
        }), {})
        managed_security_group_enabled = optional(bool, true)
        log_files = optional(list(object({
          log_group_name   = string
          prefix_log_group = bool
          file_path        = string
          log_stream_name  = string
          log_class        = optional(string, "STANDARD")
        })), null)
        key_name                      = optional(string, null)
        additional_security_group_ids = optional(list(string), [])
        detailed_monitoring_enabled   = optional(bool, false)
        egress_rules = optional(list(object({
          cidr_blocks      = list(string)
          ipv6_cidr_blocks = list(string)
          prefix_list_ids  = list(string)
          from_port        = number
          protocol         = string
          security_groups  = list(string)
          self             = bool
          to_port          = number
          description      = string
          })), [{
          cidr_blocks      = ["0.0.0.0/0"]
          ipv6_cidr_blocks = ["::/0"]
          prefix_list_ids  = null
          from_port        = 0
          protocol         = "-1"
          security_groups  = null
          self             = null
          to_port          = 0
          description      = null
        }])
        tags = optional(map(string), {})
        metadata_options = optional(object({
          instance_metadata_tags      = optional(string, "enabled")
          http_endpoint               = optional(string, "enabled")
          http_tokens                 = optional(string, "required")
          http_put_response_hop_limit = optional(number, 1)
        }), {})
        credit_specification = optional(string, null)
        cpu_options = optional(object({
          core_count            = optional(number)
          threads_per_core      = optional(number)
          amd_sev_snp           = optional(string)
          nested_virtualization = optional(string)
        }), null)
        placement = optional(object({
          affinity                = optional(string)
          availability_zone       = optional(string)
          group_id                = optional(string)
          group_name              = optional(string)
          host_id                 = optional(string)
          host_resource_group_arn = optional(string)
          spread_domain           = optional(string)
          tenancy                 = optional(string)
          partition_number        = optional(number)
        }), null)
        license_specifications = optional(list(object({
          license_configuration_arn = string
        })), [])
        associate_public_ipv4_address = optional(bool, false)
        network_interfaces = optional(list(object({
          associate_carrier_ip_address = optional(bool)
          associate_public_ip_address  = optional(bool)
          delete_on_termination        = optional(bool)
          description                  = optional(string)
          device_index                 = optional(number)
          interface_type               = optional(string)
          ipv4_address_count           = optional(number)
          ipv4_addresses               = optional(list(string))
          ipv4_prefix_count            = optional(number)
          ipv4_prefixes                = optional(list(string))
          ipv6_address_count           = optional(number)
          ipv6_addresses               = optional(list(string))
          ipv6_prefix_count            = optional(number)
          ipv6_prefixes                = optional(list(string))
          network_card_index           = optional(number)
          network_interface_id         = optional(string)
          primary_ipv6                 = optional(bool)
          private_ip_address           = optional(string)
          security_groups              = optional(list(string))
          subnet_id                    = optional(string)
          connection_tracking_specification = optional(object({
            tcp_established_timeout = optional(number)
            udp_stream_timeout      = optional(number)
            udp_timeout             = optional(number)
          }))
          ena_srd_specification = optional(object({
            ena_srd_enabled = optional(bool)
            ena_srd_udp_specification = optional(object({
              ena_srd_udp_enabled = optional(bool)
            }))
          }))
        })), [])
        on_demand_failover_for_errors = optional(list(string), [])
        scale_errors = optional(list(string), [
          "UnfulfillableCapacity",
          "MaxSpotInstanceCountExceeded",
          "TargetCapacityLimitExceededException",
          "RequestLimitExceeded",
          "ResourceLimitExceeded",
          "MaxSpotInstanceCountExceeded",
          "MaxSpotFleetRequestCountExceeded",
          "InsufficientInstanceCapacity",
          "InsufficientCapacityOnHost",
        ])
        use_dedicated_host = optional(bool, false)
      }), null)
    }), {})
  })
| n/a | yes | | [compute\_provider\_key](#input\_compute\_provider\_key) | Optional plan-known compute-provider dispatch key. Null discovers the key from the exactly one populated compute\_provider block. | `string` | `null` | no | | [github](#input\_github) | GitHub API and runner-registration configuration.

- `app_parameters.key_base64`: Parameter Store reference for the primary GitHub App private key.
- `app_parameters.id`: Parameter Store reference for the primary GitHub App ID.
- `app_parameters.additional_apps_manifest`: Optional Parameter Store reference containing the additional GitHub App manifest.
- `app_parameters.additional_app_parameter_arns`: ARNs of the additional GitHub App credential parameters.
- `enterprise_server.url`: Optional GitHub Enterprise Server base URL. Null selects GitHub.com.
- `enterprise_server.ssl_verify`: Enables TLS certificate verification for GitHub Enterprise Server requests.
- `user_agent`: Optional User-Agent value added to GitHub API requests. | 
object({
    app_parameters = object({
      key_base64 = map(string)
      id         = map(string)
      additional_apps_manifest = optional(object({
        name = string
        arn  = string
      }), null)
      additional_app_parameter_arns = optional(list(string), [])
    })
    enterprise_server = optional(object({
      url        = optional(string, null)
      ssl_verify = optional(bool, true)
    }), {})
    user_agent = optional(string, null)
  })
| n/a | yes | | [lambda](#input\_lambda) | Common Lambda substrate independent of the selected runner orchestration provider.

- `artifact.s3.bucket`: Optional shared S3 bucket containing component-owned Lambda artifacts. An orchestration provider selects its own object key and version; the bucket alone selects no artifact.
- `runtime`: Runtime used by the control-plane Lambda functions.
- `architecture`: Instruction-set architecture used by the control-plane Lambda functions. Supported values are `arm64` and `x86_64`.
- `subnet_ids`: Subnets used for Lambda VPC configuration.
- `security_group_ids`: Security groups used for Lambda VPC configuration.
- `tags`: Shared tags applied to Lambda function resources only. These override module-level `tags`; component `tags` override this map when keys conflict.
- `principals`: Additional principals allowed to assume the control-plane Lambda roles.
- `role.path`: IAM path for module-managed Lambda execution roles. Defaults to a path derived from `prefix`.
- `role.permissions_boundary`: Permissions-boundary ARN applied to module-managed Lambda execution roles. | 
object({
    artifact = optional(object({
      s3 = optional(object({
        bucket = optional(string, null)
      }), {})
    }), {})
    runtime            = optional(string, "nodejs24.x")
    architecture       = optional(string, "arm64")
    subnet_ids         = optional(list(string), [])
    security_group_ids = optional(list(string), [])
    tags               = optional(map(string), {})
    principals = optional(list(object({
      type        = string
      identifiers = list(string)
    })), [])
    role = optional(object({
      path                 = optional(string, null)
      permissions_boundary = optional(string, null)
    }), {})
  })
| `{}` | no | diff --git a/modules/runners/README.md b/modules/runners/README.md index 4e4f650bc4..048360449b 100644 --- a/modules/runners/README.md +++ b/modules/runners/README.md @@ -195,6 +195,7 @@ yarn run dist | [metadata\_options](#input\_metadata\_options) | Metadata options for the ec2 runner instances. By default, the module uses metadata tags for bootstrapping the runner, only disable `instance_metadata_tags` when using custom scripts for starting the runner. | `map(any)` | 
{
  "http_endpoint": "enabled",
  "http_put_response_hop_limit": 1,
  "http_tokens": "required",
  "instance_metadata_tags": "enabled"
}
| no | | [metrics](#input\_metrics) | Configuration for metrics created by the module, by default metrics are disabled to avoid additional costs. When metrics are enable all metrics are created unless explicit configured otherwise. | 
object({
    enable    = optional(bool, false)
    namespace = optional(string, "GitHub Runners")
    metric = optional(object({
      enable_github_app_rate_limit    = optional(bool, true)
      enable_job_retry                = optional(bool, true)
      enable_spot_termination_warning = optional(bool, true)
    }), {})
  })
| `{}` | no | | [minimum\_running\_time\_in\_minutes](#input\_minimum\_running\_time\_in\_minutes) | The time an ec2 action runner should be running at minimum before terminated if non busy. If not set the default is calculated based on the OS. | `number` | `null` | no | +| [network\_interfaces](#input\_network\_interfaces) | Advanced network interface configuration for the runner launch template. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/launch_template#network-interfaces for details. Leave unset (default) to keep using associate\_public\_ipv4\_address for a simple single-interface setup; set this to fully control one or more interfaces. | 
list(object({
    associate_carrier_ip_address = optional(bool)
    associate_public_ip_address  = optional(bool)
    delete_on_termination        = optional(bool)
    description                  = optional(string)
    device_index                 = optional(number)
    interface_type               = optional(string)
    ipv4_address_count           = optional(number)
    ipv4_addresses               = optional(list(string))
    ipv4_prefix_count            = optional(number)
    ipv4_prefixes                = optional(list(string))
    ipv6_address_count           = optional(number)
    ipv6_addresses               = optional(list(string))
    ipv6_prefix_count            = optional(number)
    ipv6_prefixes                = optional(list(string))
    network_card_index           = optional(number)
    network_interface_id         = optional(string)
    primary_ipv6                 = optional(bool)
    private_ip_address           = optional(string)
    security_groups              = optional(list(string))
    subnet_id                    = optional(string)
    connection_tracking_specification = optional(object({
      tcp_established_timeout = optional(number)
      udp_stream_timeout      = optional(number)
      udp_timeout             = optional(number)
    }))
    ena_srd_specification = optional(object({
      ena_srd_enabled = optional(bool)
      ena_srd_udp_specification = optional(object({
        ena_srd_udp_enabled = optional(bool)
      }))
    }))
  }))
| `[]` | no | | [overrides](#input\_overrides) | This map provides the possibility to override some defaults. The following attributes are supported: `name_sg` overrides the `Name` tag for all security groups created by this module. `name_runner_agent_instance` overrides the `Name` tag for the ec2 instance defined in the auto launch configuration. `name_docker_machine_runners` overrides the `Name` tag spot instances created by the runner agent. | `map(string)` | 
{
  "name_runner": "",
  "name_sg": ""
}
| no | | [parameter\_store\_tags](#input\_parameter\_store\_tags) | Map of tags that will be added to all the SSM Parameter Store parameters created by the Lambda function. | `map(string)` | `{}` | no | | [placement](#input\_placement) | The placement options for the instance. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/launch_template#placement for details. | 
object({
    affinity                = optional(string)
    availability_zone       = optional(string)
    group_id                = optional(string)
    group_name              = optional(string)
    host_id                 = optional(string)
    host_resource_group_arn = optional(string)
    spread_domain           = optional(string)
    tenancy                 = optional(string)
    partition_number        = optional(number)
  })
| `null` | no | diff --git a/modules/webhook/README.md b/modules/webhook/README.md index f6a752da2c..04cdc762d8 100644 --- a/modules/webhook/README.md +++ b/modules/webhook/README.md @@ -89,7 +89,7 @@ yarn run dist | [repository\_white\_list](#input\_repository\_white\_list) | List of github repository full names (owner/repo\_name) that will be allowed to use the github app. Leave empty for no filtering. | `list(string)` | `[]` | no | | [role\_path](#input\_role\_path) | The path that will be added to the role; if not set, the environment name will be used. | `string` | `null` | no | | [role\_permissions\_boundary](#input\_role\_permissions\_boundary) | Permissions boundary that will be added to the created role for the lambda. | `string` | `null` | no | -| [runner\_matcher\_config](#input\_runner\_matcher\_config) | SQS queue to publish accepted build events based on the runner type. `computeProvider` defaults to `ec2`; EC2 is the only provider currently implemented. When exact match is disabled the webhook accepts the event if one of the workflow job labels is part of the matcher. The priority defines the order the matchers are applied. Optional `matcherConfig.enableDynamicLabels` and `matcherConfig.awsDynamicLabelsPolicy` are evaluated by the dispatcher to gate provider dynamic labels per runner. The policy supports `blocked_keys = []` and `restricted_keys = { = { allowed = [globs], denied = [globs], max = number|string } }`; keys use the provider dynamic label suffix form, for example `instance-type` for `ghr-ec2-instance-type`. | 
map(object({
    arn             = string
    id              = string
    computeProvider = optional(string, "ec2")
    matcherConfig = object({
      labelMatchers           = list(list(string))
      exactMatch              = bool
      bidirectionalLabelMatch = optional(bool, false)
      priority                = optional(number, 999)
      enableDynamicLabels     = optional(bool, false)
      awsDynamicLabelsPolicy  = optional(any, null)
    })
  }))
| n/a | yes | +| [runner\_matcher\_config](#input\_runner\_matcher\_config) | SQS queue to publish accepted build events based on the runner type. `computeProvider` defaults to `ec2`; EC2 is the only provider currently implemented. When exact match is disabled the webhook accepts the event if one of the workflow job labels is part of the matcher. The priority defines the order the matchers are applied. Optional `matcherConfig.enableDynamicLabels` and `matcherConfig.awsDynamicLabelsPolicy` are evaluated by the dispatcher to gate provider dynamic labels per runner. The policy supports `allowed_keys = []`, `blocked_keys = []` (cannot be used together with `allowed_keys`), and `restricted_keys = { = { allowed = [globs], denied = [globs], max = number|string } }`; keys use the provider dynamic label suffix form, for example `instance-type` for `ghr-ec2-instance-type`. | 
map(object({
    arn             = string
    id              = string
    computeProvider = optional(string, "ec2")
    matcherConfig = object({
      labelMatchers           = list(list(string))
      exactMatch              = bool
      bidirectionalLabelMatch = optional(bool, false)
      priority                = optional(number, 999)
      enableDynamicLabels     = optional(bool, false)
      awsDynamicLabelsPolicy = optional(object({
        allowed_keys = optional(list(string), [])
        blocked_keys = optional(list(string), [])
        restricted_keys = optional(map(object({
          allowed = optional(list(string), [])
          denied  = optional(list(string), [])
          max     = optional(string, null)
        })), {})
      }), null)
    })
  }))
| n/a | yes | | [ssm\_paths](#input\_ssm\_paths) | The root path used in SSM to store configuration and secrets. | 
object({
    root    = string
    webhook = string
  })
| n/a | yes | | [tags](#input\_tags) | Map of tags that will be added to created resources. By default resources will be tagged with name and environment. | `map(string)` | `{}` | no | | [tracing\_config](#input\_tracing\_config) | Configuration for lambda tracing. | 
object({
    mode                  = optional(string, null)
    capture_http_requests = optional(bool, false)
    capture_error         = optional(bool, false)
  })
| `{}` | no |