Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
129 changes: 129 additions & 0 deletions docs/microvm-runners.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
# Lambda MicroVM Runners (Experimental)

!!! warning
Lambda MicroVM runner support is experimental. The image build, lifecycle-hook server, control-plane integration, and AWS MicroVM APIs must be configured together. Validate the complete flow in a non-production environment before relying on it for workloads.

## Overview

Lambda MicroVM runners provide ephemeral GitHub Actions runners backed by
Lambda MicroVMs. The runner control plane receives demand, obtains the
one-time runner configuration, starts a MicroVM from a published image, and
passes the runtime execution role to the MicroVM.

The repository includes a combined [multi-runner webhook example](examples/multi-runner-webhook.md)
that places EC2 and Lambda MicroVM lanes behind one webhook endpoint. The
provider-specific lifecycle checks are shared where possible, so the same
deployment can validate both providers.

## Prerequisites

Before deploying the MicroVM runner lane, prepare all of the following in the
target AWS Region:

1. **MicroVM foundation.** Apply the
[MicroVM foundation example](examples/microvm-foundation.md). It creates the
regional artifact bucket, Lambda Network Connectors, the image-build role,
and the reusable MicroVM usage policy.
2. **Lifecycle-hook artifact.** Build and release
`lambdas/services/microvm-lifecycle-hooks` through the same workspace
artifact process used for the repository's Lambda services. The resulting
ZIP is embedded in the MicroVM image.
3. **Published MicroVM image.** Use the
[MicroVM Ubuntu image instructions](https://github.com/github-aws-runners/terraform-aws-github-runner/blob/main/images/microvm-ubuntu/README.md)
to build and publish an image with Packer. The image must contain the
compatible lifecycle-hook server and runner entrypoint.
4. **Runner execution role.** Configure the runner role through the runner
configuration. This is different from the foundation's build role. The
control-plane TypeScript passes the execution role to `RunMicrovm`, so the
Lambda that starts the MicroVM must have permission to pass it.
5. **Runner control plane and artifacts.** Deploy the runner control plane with
the published image ARN/version, Network Connector ARNs, GitHub App
configuration, and the runner-control and webhook Lambda ZIPs.

The foundation does not create the image or the runner execution role. The
image build does not choose the runtime role. These are separate dependencies
owned by the image build and runner-control-plane stages respectively.

## IAM roles

MicroVM deployments use two roles for two different operations:

| Role | Used by | Responsibility |
| --- | --- | --- |
| Build role (`build_role_arn`) | Packer/image publisher | Creates and publishes the MicroVM image and accesses the foundation build artifacts. |
| Execution role | Runner control plane and the MicroVM | Is passed to `RunMicrovm` and provides the permissions used by the ephemeral runner at runtime. |

Do not use the build role as the runner execution role. The control-plane
Lambda needs `iam:PassRole` for the configured execution role, and the
execution role must contain the runtime permissions required by the selected
runner lane.

## Deployment order

The complete dependency chain is:

```text
MicroVM foundation
|
v
Build/release lifecycle-hook server
|
v
Packer builds and publishes image
|
v
Runner control plane resolves execution role
|
v
RunMicrovm starts an ephemeral runner
```

The lifecycle-hook server is part of the image artifact. Updating the hook
server therefore requires building/releasing the artifact and publishing a
new compatible image before deploying that image version to the runner lane.

## Combined EC2 and MicroVM deployment

The [multi-runner webhook example](examples/multi-runner-webhook.md) accepts
explicit `runners_lambda_zip` and `webhook_lambda_zip` inputs and configures
both compute providers behind one webhook. Its MicroVM settings require a
published image:

```hcl
compute_provider = {
aws = {
microvm = {
image_arn = "arn:aws:lambda:eu-west-1:123456789012:microvm-image:gha-ubuntu-arm64"
image_version = null
ingress_network_connectors = []
egress_network_connectors = ["arn:aws:lambda:eu-west-1:123456789012:network-connector:example"]
}
}
}
```

Use the example's complete Terraform configuration as the source of truth for
the current input shape. The example deploys the control plane; it does not
build the foundation, lifecycle-hook artifact, or MicroVM image for you.

## Known limitations

- This integration is experimental and depends on AWS Lambda MicroVM APIs and
the lifecycle-hook protocol.
- A compatible lifecycle-hook server must be present in every image used by
the MicroVM provider.
- Image publication and activation are separate from Terraform deployment;
wait for the image version to become active before starting jobs.
- The build role and execution role are intentionally separate. Changes to
either role can affect a different stage of the lifecycle.
- The combined webhook example is useful for integration testing, but a real
deployment still needs a real MicroVM image and the network/runtime IAM
configuration described above.

## Repository examples

- [MicroVM foundation](examples/microvm-foundation.md)
- [MicroVM image build README](https://github.com/github-aws-runners/terraform-aws-github-runner/blob/main/images/microvm-ubuntu/README.md)
- [Lifecycle-hook service README](https://github.com/github-aws-runners/terraform-aws-github-runner/blob/main/lambdas/services/microvm-lifecycle-hooks/README.md)
- [Multi-runner webhook](examples/multi-runner-webhook.md)
- [MicroVM foundation module](modules/public/microvm-foundation.md)
22 changes: 19 additions & 3 deletions examples/microvm-foundation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,26 @@ documented in `../../images/microvm-ubuntu/README.md`. Use the outputs as the bu
- `connector_arns.ministack` -> `MICROVM_EGRESS_NETWORK_CONNECTOR_ARN`
- `usage_policy_arn` -> attach to the control-plane role used by the runner example

The deployment order is:

1. Apply this foundation to create the regional bucket, Network Connectors,
build role, and reusable runtime policy.
2. Build and release the lifecycle-hook service from
`lambdas/services/microvm-lifecycle-hooks` using the repository's normal
Lambda artifact process.
3. Build and publish the MicroVM image with Packer, passing the foundation
outputs and the released lifecycle-hook ZIP. The image builder uses the
**build role**.
4. Deploy the runner control plane, such as
`examples/multi-runner-webhook`, with the published image ARN/version. The
control plane resolves the **execution role** from the runner configuration
and passes it to `RunMicrovm` when it starts a job.

The two roles must not be conflated: the build role creates the image, while
the execution role runs the ephemeral GitHub Actions runner inside that image.
The foundation module owns regional storage, build IAM, Network Connectors,
and the reusable runtime policy. It does not publish an image or create the
runner control plane; those steps remain explicit and can be performed after
the foundation is available.
and the reusable runtime policy. It does not publish an image, create the
execution role, or create the runner control plane.

<!-- BEGIN_TF_DOCS -->
## Requirements
Expand Down
28 changes: 27 additions & 1 deletion examples/multi-runner-webhook/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,32 @@ terraform apply \
-var='webhook_lambda_zip=/path/to/webhook.zip'
```

## MicroVM prerequisites

The MicroVM lane expects an image that has already been built and published in
the target Region. The image is not created by this example. Prepare it in
this order:

1. Apply `examples/microvm-foundation`.
2. Build and release the lifecycle-hook service from
`lambdas/services/microvm-lifecycle-hooks` using the same artifact process
used for the repository's Lambda services.
3. Build the image with Packer from `images/microvm-ubuntu`, passing the
foundation's bucket, connector, build-role, and lifecycle-hook artifact.
4. Set `compute_provider.aws.microvm.image_arn` (and, when applicable,
`image_version`) to the published image.

The foundation's build role is used to create the image. It is different from
the execution role used by the runner job. The runner configuration owns that
execution role; the control-plane TypeScript passes it to `RunMicrovm` when it
starts an ephemeral runner. The control-plane Lambda therefore needs
permission to pass the configured execution role, and the role needs the
runtime permissions required by the selected runner lane.

This example deploys both EC2 and MicroVM lanes behind one webhook endpoint,
but it does not replace the foundation, image build, lifecycle-hook release,
or execution-role setup steps.

<!-- BEGIN_TF_DOCS -->
## Requirements

Expand Down Expand Up @@ -61,4 +87,4 @@ No resources.
|------|-------------|
| <a name="output_webhook_endpoint"></a> [webhook\_endpoint](#output\_webhook\_endpoint) | n/a |
| <a name="output_webhook_secret"></a> [webhook\_secret](#output\_webhook\_secret) | n/a |
<!-- END_TF_DOCS -->
<!-- END_TF_DOCS -->
21 changes: 15 additions & 6 deletions images/microvm-ubuntu/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,11 @@ Dockerfile, compiled lifecycle-hook ZIP contract, and image entrypoint.
Before building the image:

1. Apply `examples/microvm-foundation` in the target AWS Region.
2. Install Packer and set the required AWS, S3, IAM, connector, and
lifecycle-hook variables.
2. Build and release the lifecycle-hook service from
`lambdas/services/microvm-lifecycle-hooks`, then set
`MICROVM_LIFECYCLE_HOOK_ZIP` to the released artifact.
3. Install Packer and set the required AWS, S3, IAM, connector, and image
variables.

The image intentionally excludes the source repository's optional external
telemetry and Teleport services. It contains only the Actions runner,
Expand Down Expand Up @@ -40,7 +43,13 @@ packer build -color=false github_agent.microvm.ubuntu.pkr.hcl
```

The build role, artifact bucket, and network connector are created by the
foundation module. Keep the bucket private and versioned, use the module's
least-privilege policies, and do not put credentials in checked-in files. The
lifecycle-hook ZIP must contain the compiled `server.js` at its archive root;
any bundled dependencies must use safe relative paths.
foundation module. The build role is used only while Packer creates and
publishes the image; it is not baked into the image and is not the role used
by runner jobs. The execution role is selected by the runner control plane and
passed to `RunMicrovm` at launch time, so it is not configured by this image
build.

Keep the bucket private and versioned, use the module's least-privilege
policies, and do not put credentials in checked-in files. The lifecycle-hook
ZIP must contain the compiled `server.js` at its archive root; any bundled
dependencies must use safe relative paths.
14 changes: 14 additions & 0 deletions lambdas/services/microvm-lifecycle-hooks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,20 @@ yarn workspace @aws-github-runner/microvm-lifecycle-hooks start

`build` uses esbuild to create the self-contained CommonJS server bundle `dist/server.js`. It also writes `dist/package.json` with `type: commonjs` so the bundle remains executable after it is copied outside the Yarn workspace.

## Build and release with the Lambda artifacts

The lifecycle-hook server is a deployable image-build artifact, not a service
that is installed separately beside the runner control plane. Build and test it
through the normal Lambda workspace/release process, then provide the resulting
ZIP to the MicroVM image build as `MICROVM_LIFECYCLE_HOOK_ZIP`. Packer embeds
that artifact in the image; every published image used by the MicroVM provider
must contain a compatible hook server.

The hook server's release lifecycle is therefore separate from the MicroVM
execution role. The image build uses the foundation's build role. When a job
starts, the runner control plane supplies the runtime execution role to
`RunMicrovm`.

To build before invoking Docker, run the workspace build above. In the existing MicroVM runner Dockerfile, which already installs s6-overlay and the GitHub runner's Node 24 runtime, copy the complete artifact and replace the old hook command with:

```dockerfile
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ nav:
- Multi-runner v1 to v2 migration: multi-runner-v1-v2-migration.md
- Getting started: getting-started.md
- Security: security.md
- Lambda MicroVM runners (experimental): microvm-runners.md
- Architecture decisions:
- MiniStack for integration tests: adr/0001-use-ministack-for-terraform-integration-tests.md
- Modules:
Expand Down
18 changes: 18 additions & 0 deletions modules/microvm-foundation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,24 @@ It manages:
- A Lambda-trusted Network Connector operator role and propagation barrier.
- An unattached runtime usage policy for the reserved image namespace and connector inventory.

## Build role and execution role

MicroVM deployments use two different IAM roles with different lifecycles:

- The `build_role_arn` output is the **build role**. The image builder assumes
this role while it creates and publishes a MicroVM image. It grants the
image-build permissions for the foundation artifact bucket, logs, and any
configured ECR repositories. It is not the role used by a runner job.
- The **execution role** is attached to each MicroVM when the runner control
plane launches it. The control-plane TypeScript passes this role to
`RunMicrovm`; the Lambda that calls that API must be allowed to pass the
role. The MicroVM and the ephemeral runner use this role at runtime.

The execution role is resolved by the runner configuration and is intentionally
not created by this foundation module. The foundation creates the regional
build resources and the reusable `usage_policy_arn`; the runner/control-plane
configuration owns the runtime role and its provider-specific permissions.

The module does not create MicroVM images, runner execution roles, or the
runner control plane. Attach `usage_policy_arn` to the control-plane role that
owns the runtime launch operations. The caller must also grant the Terraform
Expand Down
Loading