Repository navigation
Multi-arch container publishing #3875
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| .git | ||
| **/bin | ||
| **/obj | ||
| **/out | ||
| **/TestResults | ||
| .env | ||
| .env.* | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| name: Container build | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - Dockerfile | ||
| - .dockerignore | ||
| - .github/workflows/container-build.yml | ||
| - scripts/publish-container.sh | ||
| - src/** | ||
| - schemas/** | ||
| - Nuget.config | ||
| - global.json | ||
| push: | ||
| branches: [main] | ||
| paths: | ||
| - Dockerfile | ||
| - .dockerignore | ||
| - .github/workflows/container-build.yml | ||
| - scripts/publish-container.sh | ||
| - src/** | ||
| - schemas/** | ||
| - Nuget.config | ||
| - global.json | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| build: | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| target: [runtime, runtime-nonroot] | ||
| runner: [ubuntu-24.04, ubuntu-24.04-arm] | ||
| include: | ||
| - runner: ubuntu-24.04 | ||
| platform: linux/amd64 | ||
| arch: x64 | ||
| - runner: ubuntu-24.04-arm | ||
| platform: linux/arm64 | ||
| arch: arm64 | ||
| runs-on: ${{ matrix.runner }} | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: docker/setup-buildx-action@v3 | ||
| - name: Check publishing script syntax | ||
| run: bash -n scripts/publish-container.sh | ||
| - name: Build image | ||
| uses: docker/build-push-action@v6 | ||
| with: | ||
| context: . | ||
| target: ${{ matrix.target }} | ||
| platforms: ${{ matrix.platform }} | ||
| load: true | ||
| tags: dab:container-test | ||
| - name: Check runtime and application architecture | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This step proves the .NET runtime starts and the app targets the right architecture, but it never starts DAB. So even on the ARM runner, DAB itself hasn't run on ARM64. Consider a short startup check:
Locally this took seconds per image. It would turn the remaining manual ARM64 check into CI coverage for every PR. |
||
| env: | ||
| ARCH: ${{ matrix.arch }} | ||
| TARGET: ${{ matrix.target }} | ||
| run: | | ||
| docker run --rm --entrypoint sh dab:container-test -ec ' | ||
| dotnet --info | ||
| test -s /App/dab-config.json | ||
| grep -F "\"name\": \".NETCoreApp,Version=v10.0/linux-$1\"" \ | ||
| /App/Azure.DataApiBuilder.Service.deps.json | ||
| if [ "$2" = runtime-nonroot ]; then | ||
| test "$(id -u)" = "$APP_UID" | ||
| touch /App/logs/container-test | ||
| else | ||
| test "$(id -u)" = 0 | ||
| fi | ||
| ' sh "$ARCH" "$TARGET" | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,11 +1,13 @@ | ||
| # Version values referenced from https://hub.docker.com/_/microsoft-dotnet-aspnet | ||
|
|
||
| FROM mcr.microsoft.com/dotnet/sdk:10.0-azurelinux3.0 AS build | ||
| FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:10.0-azurelinux3.0 AS build | ||
|
|
||
| ARG TARGETARCH | ||
|
|
||
| WORKDIR /src | ||
| COPY [".", "./"] | ||
| RUN dotnet build "./src/Service/Azure.DataApiBuilder.Service.csproj" -c Docker -o /out -r linux-x64 | ||
| RUN dotnet publish "./src/Service/Azure.DataApiBuilder.Service.csproj" \ | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
| -c Docker -o /out -a "$TARGETARCH" --self-contained false -p:UseAppHost=false | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Common runtime base. | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| # Building and publishing container images | ||
|
|
||
| The Dockerfile supports `linux/amd64` and `linux/arm64`. The SDK runs on the | ||
| builder's native architecture and cross-compiles for the requested platform; | ||
| the ASP.NET runtime base image is selected for that platform. Both the default | ||
| `runtime` target and the opt-in `runtime-nonroot` target support these platforms. | ||
|
|
||
| ## Local build | ||
|
|
||
| Build and load a single platform into the local Docker image store: | ||
|
|
||
| ```sh | ||
| docker buildx build --platform linux/arm64 --target runtime --load -t dab:local . | ||
| ``` | ||
|
|
||
| Omit `--platform` to build for the local builder's default platform. Running an | ||
| image for another architecture requires emulation. Cross-building | ||
| `runtime-nonroot` also requires emulation (or native builder nodes for both | ||
| architectures) because its permissions setup executes in the target image. | ||
|
|
||
| ## Release publishing | ||
|
|
||
| Use a Buildx builder with multi-platform support, for example: | ||
|
|
||
| ```sh | ||
| docker buildx create --name dab-multiarch --driver docker-container --use | ||
| docker buildx inspect --bootstrap | ||
| ``` | ||
|
|
||
| Authenticate to the release pipeline's **writable registry**, then publish all | ||
| tags for a variant in one build. Replace the example registry and version with | ||
| the release pipeline's existing destination and version: | ||
|
|
||
| ```sh | ||
| docker login <registry> | ||
| bash scripts/publish-container.sh runtime \ | ||
| <registry>/data-api-builder:<version> \ | ||
| <registry>/data-api-builder:latest | ||
| bash scripts/publish-container.sh runtime-nonroot \ | ||
| <registry>/data-api-builder:<version>-nonroot \ | ||
| <registry>/data-api-builder:latest-nonroot | ||
| ``` | ||
|
|
||
| The script pushes both platform images and their shared image index for each | ||
| supplied tag. Do not push the architectures separately to the same tag: the | ||
| last push would replace the earlier architecture instead of publishing an | ||
| index. A multi-platform build uses `--push`, not `--load`. | ||
|
|
||
| The official MCR release pipeline is not defined in this repository. Its | ||
| maintainers must replace the existing single-platform build/push with this | ||
| script (or the equivalent `docker buildx build --platform | ||
| linux/amd64,linux/arm64 --push` command), preserving their existing registry | ||
| authentication, tag selection, and MCR promotion process. Every promoted tag | ||
| must retain the multi-platform index and both referenced platform images. | ||
| Changing the Dockerfile alone does not make existing MCR tags multi-platform. | ||
|
|
||
| Verify the resulting tag in the writable registry and again after MCR promotion: | ||
|
|
||
| ```sh | ||
| docker buildx imagetools inspect <registry>/data-api-builder:<version> | ||
| docker buildx imagetools inspect mcr.microsoft.com/azure-databases/data-api-builder:<version> | ||
| ``` | ||
|
|
||
| Both must list `linux/amd64` and `linux/arm64`. Provenance attestations may also | ||
| appear as `unknown/unknown`; these do not replace the required platform images. | ||
| The `Container build` workflow builds both variants on native x64 and ARM64 | ||
| runners and checks the application's runtime identifier and non-root log access. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| #!/usr/bin/env bash | ||
| # Copyright (c) Microsoft Corporation. | ||
| # Licensed under the MIT License. | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| if [[ $# -lt 2 || ( "$1" != runtime && "$1" != runtime-nonroot ) ]]; then | ||
| echo "Usage: $0 <runtime|runtime-nonroot> <image:tag> [<image:tag> ...]" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| target=$1 | ||
| shift | ||
| tags=() | ||
| for tag in "$@"; do | ||
| tags+=(--tag "$tag") | ||
| done | ||
|
|
||
| repo_root=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd) | ||
| docker buildx build \ | ||
| --file "$repo_root/Dockerfile" \ | ||
| --platform linux/amd64,linux/arm64 \ | ||
| --target "$target" \ | ||
| "${tags[@]}" \ | ||
| --push \ | ||
| "$repo_root" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.dockerignorepatterns match from the context root, so.envand.env.*only exclude the root files. Our.gitignoreignores.envat any depth, and the CLI loads.envfrom the current directory.A nested
.envtherefore still reaches the builder, andCOPY [".", "./"]copies it into the build stage. It doesn't end up in the final image, but it can land in an exported build cache. Suggest: