Repository navigation
Multi-arch container publishing - #3875
Hans Bakker (hansmbakker) wants to merge 2 commits into
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
|
Here is a suggested contribution for multi-arch container image builds intending to fix #3854. Since there was no existing container image publishing pipeline I could not build upon that. I'd like to ask the maintainers of this repo to take it from here, as this is a process they should be owner of. |
There was a problem hiding this comment.
🔵 Needs a closer look
Native ARM64 execution and registry publishing remain unvalidated, and the new CI matrix still needs to pass.
No open issues 🎉
What changed in this PR
Prepares DAB containers for Linux AMD64 and ARM64 while preserving root and non-root variants. Official MCR publishing still requires external release-pipeline changes.
Changes:
- Publish framework-dependent application output for the requested container architecture.
- Add multi-platform publishing and native architecture CI checks.
- Document build and release steps, and exclude local artifacts and environment files from the build context.
| File | Description |
|---|---|
scripts/publish-container.sh |
Pushes both architectures under each supplied tag. |
README.md |
Links to container build and publishing guidance. |
Dockerfile |
Uses the native SDK to publish for the target architecture. |
docker/README.md |
Documents local builds, publishing, and index-preserving MCR promotion. |
.github/workflows/container-build.yml |
Adds native AMD64 and ARM64 checks for both runtime variants. |
.dockerignore |
Excludes Git metadata, build artifacts, and root environment files. |
Review effort: Balanced
|
/azp run |
|
Azure Pipelines: Successfully started running 6 pipeline(s). |
|
/azp run CosmosDB_NoSql Integration Tests |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
aaronburtle
left a comment
There was a problem hiding this comment.
Thanks Hans Bakker (@hansmbakker)! Before merging need to finish validation here's where thats at currently:
| Check | Status |
|---|---|
| Image builds for amd64 and arm64, root and non-root | Validated (CI) |
| App in each image matches the image's architecture | Validated (CI) |
publish-container.sh publishes one tag covering both architectures |
Validated (local test registry) |
| DAB serves REST and GraphQL requests from the amd64 images | Validated (local) |
| Non-root image runs as a non-root user and can write logs | Validated (local) |
| No change for existing amd64 users | Validated (only the native launcher is removed; see inline note) |
| DAB serves requests on ARM64 hardware | Pending |
| Official MCR release pipeline still builds with these changes | Pending (maintainers) |
We'll hold the merge until the pending items are done. If you can build and run the image in your arm64 dev container, that would cover the ARM64 row. A few non-blocking notes are inline.
| **/obj | ||
| **/out | ||
| **/TestResults | ||
| .env |
There was a problem hiding this comment.
.dockerignore patterns match from the context root, so .env and .env.* only exclude the root files. Our .gitignore ignores .env at any depth, and the CLI loads .env from the current directory.
A nested .env therefore still reaches the builder, and COPY [".", "./"] copies it into the build stage. It doesn't end up in the final image, but it can land in an exported build cache. Suggest:
**/.env
**/.env.*
| platforms: ${{ matrix.platform }} | ||
| load: true | ||
| tags: dab:container-test | ||
| - name: Check runtime and application architecture |
There was a problem hiding this comment.
This step proves the .NET runtime starts and the app targets the right architecture, but it never starts DAB. So even on the ARM runner, DAB itself hasn't run on ARM64.
Consider a short startup check:
- a
postgresservice container seeded withsamples/getting-started/azure-postgresql/library.postgresql.sql, - a minimal one-entity config,
- then start the image and assert that
/reports"status": "Healthy"and one REST or GraphQL read returns data.
Locally this took seconds per image. It would turn the remaining manual ARM64 check into CI coverage for every PR.
| WORKDIR /src | ||
| COPY [".", "./"] | ||
| RUN dotnet build "./src/Service/Azure.DataApiBuilder.Service.csproj" -c Docker -o /out -r linux-x64 | ||
| RUN dotnet publish "./src/Service/Azure.DataApiBuilder.Service.csproj" \ |
There was a problem hiding this comment.
-p:UseAppHost=false removes the native launcher (/App/Azure.DataApiBuilder.Service) that today's image ships. The default entrypoint doesn't use it, but anyone who starts DAB through it (for example a Kubernetes command or a derived Dockerfile) would break. Keeping it costs almost nothing: the arm64 build works without this flag and only adds a ~77 KB file. Could we drop -p:UseAppHost=false so the image contents stay the same as today? Was there a reason to drop the native launcher?
Why make this change?
What is this change?
How was this tested?
Validation performed against the session changes (the pushed patch contains the same six changed files):
bash -n scripts/publish-container.sh— passed.git diff --check— passed before committing.docker buildx build --builder dab-multiarch --platform linux/amd64,linux/arm64 --target runtime --output type=oci,dest=/root/.copilot/session-state/e576fd408b1c412fbfcf67dc261ba454/files/dab-runtime.oci.tar --progress plain .— failed because the environment's NuGet TLS certificate chain was untrusted inside the build container.docker buildx build --builder dab-multiarch --file /root/.copilot/session-state/e576fd408b1c412fbfcf67dc261ba454/files/Validation.Dockerfile --secret id=ca-bundle,src=/etc/ssl/certs/ca-certificates.crt --platform linux/amd64,linux/arm64 --target runtime --output type=oci,dest=/root/.copilot/session-state/e576fd408b1c412fbfcf67dc261ba454/files/dab-runtime.oci.tar --progress plain .— passed. The equivalent command with--target runtime-nonrootand a separate output archive also passed, using emulation for the ARM64 permissions step. No certificate changes are included in this PR.python /root/.copilot/session-state/e576fd408b1c412fbfcf67dc261ba454/files/verify-oci.py /root/.copilot/session-state/e576fd408b1c412fbfcf67dc261ba454/files/dab-runtime.oci.tar /root/.copilot/session-state/e576fd408b1c412fbfcf67dc261ba454/files/dab-nonroot.oci.tar— passed: both archives contain amd64 and arm64 platform manifests; application RIDs and runtime ELF architectures match each platform./api/Bookagainst an isolated PostgreSQL database and verified the expected row. Non-root UID and writable/App/logschecks passed.Sample Request(s)