Skip to content

Multi-arch container publishing - #3875

Open
Hans Bakker (hansmbakker) wants to merge 2 commits into
Azure:mainfrom
hansmbakker:fix/multiarch-container-images
Open

Hans Bakker (hansmbakker) wants to merge 2 commits into
Azure:mainfrom
hansmbakker:fix/multiarch-container-images

Conversation

@hansmbakker

Copy link
Copy Markdown

Why make this change?

  • Addresses [Enhancement]: Publish linux/arm64 variant of the container image #3854: the official image is published only for linux/amd64, forcing ARM64 users to rely on emulation or build their own images.
  • This prepares the repository-side changes. Official MCR publishing still requires maintainers to update their external release pipeline; this PR alone does not change existing MCR tags.

What is this change?

  • Run the SDK on BUILDPLATFORM and publish framework-dependent application output for TARGETARCH instead of hard-coding linux-x64. Preserve both root and non-root runtime targets.
  • Add a Buildx publishing script that pushes linux/amd64 and linux/arm64 together under each supplied tag, including their shared image index.
  • Add CI builds for both variants on native ubuntu-24.04 and ubuntu-24.04-arm runners, checking the application RID, runtime execution, and non-root log access without QEMU.
  • Exclude Git metadata, local build outputs, and environment files from the Docker context; document building, publishing, and index-preserving MCR promotion.

How was this tested?

  • Integration Tests
  • Unit Tests

Validation performed against the session changes (the pushed patch contains the same six changed files):

  • bash -n scripts/publish-container.sh — passed.
  • git diff --check — passed before committing.
  • Publishing-script command capture — passed: both platforms, multiple tags, correct target, --push, and rejection of missing/invalid arguments.
  • Workflow YAML parsing and expansion of all four runner/target combinations — passed. The GitHub Actions jobs themselves have not yet run.
  • docker buildx build --builder dab-multiarch --platform linux/amd64,linux/arm64 --target runtime --output type=oci,dest=/root/.copilot/session-state/e576fd408b1c412fbfcf67dc261ba454/files/dab-runtime.oci.tar --progress plain . — failed because the environment's NuGet TLS certificate chain was untrusted inside the build container.
  • Retried with a temporary validation-only Dockerfile matching the application build/runtime stages and mounting the host CA bundle as a BuildKit secret: docker buildx build --builder dab-multiarch --file /root/.copilot/session-state/e576fd408b1c412fbfcf67dc261ba454/files/Validation.Dockerfile --secret id=ca-bundle,src=/etc/ssl/certs/ca-certificates.crt --platform linux/amd64,linux/arm64 --target runtime --output type=oci,dest=/root/.copilot/session-state/e576fd408b1c412fbfcf67dc261ba454/files/dab-runtime.oci.tar --progress plain . — passed. The equivalent command with --target runtime-nonroot and a separate output archive also passed, using emulation for the ARM64 permissions step. No certificate changes are included in this PR.
  • python /root/.copilot/session-state/e576fd408b1c412fbfcf67dc261ba454/files/verify-oci.py /root/.copilot/session-state/e576fd408b1c412fbfcf67dc261ba454/files/dab-runtime.oci.tar /root/.copilot/session-state/e576fd408b1c412fbfcf67dc261ba454/files/dab-nonroot.oci.tar — passed: both archives contain amd64 and arm64 platform manifests; application RIDs and runtime ELF architectures match each platform.
  • Native amd64 smoke tests for both runtime targets — passed: queried /api/Book against an isolated PostgreSQL database and verified the expected row. Non-root UID and writable /App/logs checks passed.
  • Local ARM64 execution — not validated successfully: host binfmt registration was unavailable and the direct-QEMU attempt failed. Native ARM execution is covered by the added CI jobs, pending their execution.
  • Registry publishing and MCR promotion were not run: release credentials and the external MCR pipeline are not available in this session.

Sample Request(s)

# Load a single-platform local image.
docker buildx build --platform linux/arm64 --target runtime --load -t dab:local .

# After authenticating to a writable registry, publish both architectures.
bash scripts/publish-container.sh runtime \
  example.azurecr.io/data-api-builder:VERSION \
  example.azurecr.io/data-api-builder:latest

# Verify the published platform index.
docker buildx imagetools inspect example.azurecr.io/data-api-builder:VERSION

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@hansmbakker

Hans Bakker (hansmbakker) commented Oct 23, 2026 •

Copy link
Copy Markdown
Author

Here is a suggested contribution for multi-arch container image builds intending to fix #3854.

Since there was no existing container image publishing pipeline I could not build upon that.

I'd like to ask the maintainers of this repo to take it from here, as this is a process they should be owner of.
The maintainers are given permission to make edits to the branch and complete the PR.
This PR is intended as a starter generated by GitHub Copilot.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Native ARM64 execution and registry publishing remain unvalidated, and the new CI matrix still needs to pass.

No open issues 🎉

What changed in this PR

Prepares DAB containers for Linux AMD64 and ARM64 while preserving root and non-root variants. Official MCR publishing still requires external release-pipeline changes.

Changes:

  • Publish framework-dependent application output for the requested container architecture.
  • Add multi-platform publishing and native architecture CI checks.
  • Document build and release steps, and exclude local artifacts and environment files from the build context.
File Description
scripts/​publish-container.sh Pushes both architectures under each supplied tag.
README.md Links to container build and publishing guidance.
Dockerfile Uses the native SDK to publish for the target architecture.
docker/​README.md Documents local builds, publishing, and index-preserving MCR promotion.
.github/​workflows/​container-build.yml Adds native AMD64 and ARM64 checks for both runtime variants.
.dockerignore Excludes Git metadata, build artifacts, and root environment files.

Review effort: Balanced

@aaronburtle

Copy link
Copy Markdown
Contributor

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 6 pipeline(s).

@aaronburtle

Copy link
Copy Markdown
Contributor

/azp run CosmosDB_NoSql Integration Tests

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@aaronburtle aaronburtle left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Hans Bakker (@hansmbakker)! Before merging need to finish validation here's where thats at currently:

Check Status
Image builds for amd64 and arm64, root and non-root Validated (CI)
App in each image matches the image's architecture Validated (CI)
publish-container.sh publishes one tag covering both architectures Validated (local test registry)
DAB serves REST and GraphQL requests from the amd64 images Validated (local)
Non-root image runs as a non-root user and can write logs Validated (local)
No change for existing amd64 users Validated (only the native launcher is removed; see inline note)
DAB serves requests on ARM64 hardware Pending
Official MCR release pipeline still builds with these changes Pending (maintainers)

We'll hold the merge until the pending items are done. If you can build and run the image in your arm64 dev container, that would cover the ARM64 row. A few non-blocking notes are inline.

Comment thread .dockerignore
**/obj
**/out
**/TestResults
.env

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.dockerignore patterns match from the context root, so .env and .env.* only exclude the root files. Our .gitignore ignores .env at any depth, and the CLI loads .env from the current directory.

A nested .env therefore still reaches the builder, and COPY [".", "./"] copies it into the build stage. It doesn't end up in the final image, but it can land in an exported build cache. Suggest:

**/.env
**/.env.*

platforms: ${{ matrix.platform }}
load: true
tags: dab:container-test
- name: Check runtime and application architecture

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step proves the .NET runtime starts and the app targets the right architecture, but it never starts DAB. So even on the ARM runner, DAB itself hasn't run on ARM64.

Consider a short startup check:

  • a postgres service container seeded with samples/getting-started/azure-postgresql/library.postgresql.sql,
  • a minimal one-entity config,
  • then start the image and assert that / reports "status": "Healthy" and one REST or GraphQL read returns data.

Locally this took seconds per image. It would turn the remaining manual ARM64 check into CI coverage for every PR.

Comment thread Dockerfile
WORKDIR /src
COPY [".", "./"]
RUN dotnet build "./src/Service/Azure.DataApiBuilder.Service.csproj" -c Docker -o /out -r linux-x64
RUN dotnet publish "./src/Service/Azure.DataApiBuilder.Service.csproj" \

@aaronburtle aaronburtle Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-p:UseAppHost=false removes the native launcher (/App/Azure.DataApiBuilder.Service) that today's image ships. The default entrypoint doesn't use it, but anyone who starts DAB through it (for example a Kubernetes command or a derived Dockerfile) would break. Keeping it costs almost nothing: the arm64 build works without this flag and only adds a ~77 KB file. Could we drop -p:UseAppHost=false so the image contents stay the same as today? Was there a reason to drop the native launcher?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants