Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions .github/workflows/docker-build-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,71 @@ on:
pull_request:
paths:
- '**/Dockerfile*'
- 'docker-bake.hcl'
- '.dockerignore'
- 'hugegraph-server/hugegraph-dist/docker/**'
- 'hugegraph-server/hugegraph-dist/src/assembly/static/bin/util.sh'

jobs:
docker-bake-check:
runs-on: ubuntu-24.04
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Verify shared Maven stages stay identical
shell: bash
run: |
set -euo pipefail

extract_build_stage() {
awk '
/^FROM .* AS build$/ {
in_build = 1
}
in_build && seen && /^FROM / {
exit
}
in_build {
print
seen = 1
}
' "$1"
}

reference="hugegraph-pd/Dockerfile"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the stage header ever changes casing or name (as build is valid Dockerfile syntax), extract_build_stage returns empty for every file and the diffs pass vacuously, so the guard silently stops guarding. Reproduced this: rewriting AS build to as build in all four files makes the check pass even with divergent mvn lines. A partial rename is still caught. Cheap fix: assert the reference extraction is non-empty first.

Suggested change
reference="hugegraph-pd/Dockerfile"
reference="hugegraph-pd/Dockerfile"
[ -n "$(extract_build_stage "$reference")" ] || {
echo "ERROR: no 'AS build' stage found in $reference; guard would pass vacuously"
exit 1
}

for dockerfile in \
hugegraph-store/Dockerfile \
hugegraph-server/Dockerfile \
hugegraph-server/Dockerfile-hstore; do
diff -u \
<(extract_build_stage "$reference") \
<(extract_build_stage "$dockerfile")
done

- name: Validate shared image build graph
run: |
set -euo pipefail

docker buildx bake --print > /tmp/hugegraph-bake.json
jq -e '
.group.default.targets == [
"build-cache",
"pd",
"server-hstore",
"server-standalone",
"store"
] and
([.target[] | .platforms] |
all(. == ["linux/amd64", "linux/arm64"])) and
.target["build-cache"].target == "build" and
.target["build-cache"].output[0].type == "cacheonly" and
([.target | to_entries[] |
select(.key != "build-cache") |
.value.output[0].type] |
all(. == "docker"))
' /tmp/hugegraph-bake.json

docker-build:
runs-on: ubuntu-24.04
strategy:
Expand Down
116 changes: 116 additions & 0 deletions docker-bake.hcl
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

variable "MAVEN_ARGS" {
default = ""
}

variable "SOURCE_REVISION" {
default = "local"
}

variable "IMAGE_TAG" {
default = "local"
}

variable "CACHE_CHANNEL" {
default = "latest"
}

variable "EXPORT_CACHE" {
default = false
}

target "_common" {
context = "."
args = {
MAVEN_ARGS = MAVEN_ARGS
SOURCE_REVISION = SOURCE_REVISION
}
platforms = [
"linux/amd64",
"linux/arm64",
]
}

target "build-cache" {
inherits = ["_common"]
dockerfile = "hugegraph-pd/Dockerfile"
target = "build"
output = ["type=cacheonly"]
cache-from = [
"type=registry,ref=hugegraph/hugegraph:shared-${CACHE_CHANNEL}",
"type=registry,ref=hugegraph/pd:buildcache-${CACHE_CHANNEL}",
]
cache-to = EXPORT_CACHE ? [
"type=registry,ref=hugegraph/hugegraph:shared-${CACHE_CHANNEL},mode=max",
] : []
}

target "pd" {
inherits = ["_common"]
dockerfile = "hugegraph-pd/Dockerfile"
tags = ["hugegraph/pd:${IMAGE_TAG}"]
output = ["type=docker"]

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

‼️ critical — _common supplies both linux/amd64 and linux/arm64, but this target fixes the output to type=docker (and the other three runtime targets do the same). Buildx's Docker exporter only writes a single-platform image; with these two platforms the target cannot export the required manifest list, so the default Bake group cannot build the claimed multi-platform images. Please use a multi-platform-capable registry/image exporter for the publishing graph, or split out a single-platform local target, and make the CI guard validate the actual export contract.

cache-from = [
"type=registry,ref=hugegraph/hugegraph:shared-${CACHE_CHANNEL}",
"type=registry,ref=hugegraph/pd:buildcache-${CACHE_CHANNEL}",
]
}

target "store" {
inherits = ["_common"]
dockerfile = "hugegraph-store/Dockerfile"
tags = ["hugegraph/store:${IMAGE_TAG}"]
output = ["type=docker"]
cache-from = [
"type=registry,ref=hugegraph/hugegraph:shared-${CACHE_CHANNEL}",
"type=registry,ref=hugegraph/store:buildcache-${CACHE_CHANNEL}",
]
}

target "server-hstore" {
inherits = ["_common"]
dockerfile = "hugegraph-server/Dockerfile-hstore"
tags = ["hugegraph/server:${IMAGE_TAG}"]
output = ["type=docker"]
cache-from = [
"type=registry,ref=hugegraph/hugegraph:shared-${CACHE_CHANNEL}",
"type=registry,ref=hugegraph/server:buildcache-${CACHE_CHANNEL}",
]
}

target "server-standalone" {
inherits = ["_common"]
dockerfile = "hugegraph-server/Dockerfile"
tags = ["hugegraph/hugegraph:${IMAGE_TAG}"]
output = ["type=docker"]
cache-from = [
"type=registry,ref=hugegraph/hugegraph:shared-${CACHE_CHANNEL}",
"type=registry,ref=hugegraph/hugegraph:buildcache-${CACHE_CHANNEL}",
]
}

group "default" {
targets = [
"build-cache",
"pd",
"store",
"server-hstore",
"server-standalone",
]
}
4 changes: 4 additions & 0 deletions docker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,10 @@ Builds images locally from source Dockerfiles. Best for **developers** who want
Build the matching `hugegraph-toolchain` Hubble source as
`local/hugegraph-hubble:dev` before starting this stack.

The publishing pipeline uses the repository-root `docker-bake.hcl` to compile
the Java reactor once and build the PD, Store, HStore Server, and standalone
Server runtime images from that shared result.

```bash
(
cd docker
Expand Down
5 changes: 3 additions & 2 deletions hugegraph-pd/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,10 @@ WORKDIR /pkg
COPY . .

ARG MAVEN_ARGS
ARG SOURCE_REVISION=local

RUN --mount=type=cache,target=/root/.m2 \
mvn package $MAVEN_ARGS -e -B -ntp -Dmaven.test.skip=true -Dmaven.javadoc.skip=true \
RUN --mount=type=cache,id=hugegraph-maven-${SOURCE_REVISION},target=/root/.m2,sharing=locked \
mvn install $MAVEN_ARGS -e -B -ntp -Dmaven.test.skip=true -Dmaven.javadoc.skip=true \
&& rm ./hugegraph-server/*.tar.gz ./hugegraph-pd/*.tar.gz ./hugegraph-store/*.tar.gz

# 2nd stage: runtime env
Expand Down
5 changes: 3 additions & 2 deletions hugegraph-server/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,10 @@ WORKDIR /pkg
COPY . .

ARG MAVEN_ARGS
ARG SOURCE_REVISION=local

RUN --mount=type=cache,target=/root/.m2 \
mvn package $MAVEN_ARGS -e -B -ntp -Dmaven.test.skip=true -Dmaven.javadoc.skip=true \
RUN --mount=type=cache,id=hugegraph-maven-${SOURCE_REVISION},target=/root/.m2,sharing=locked \
mvn install $MAVEN_ARGS -e -B -ntp -Dmaven.test.skip=true -Dmaven.javadoc.skip=true \
&& rm ./hugegraph-server/*.tar.gz ./hugegraph-pd/*.tar.gz ./hugegraph-store/*.tar.gz

# 2nd stage: runtime env
Expand Down
5 changes: 3 additions & 2 deletions hugegraph-server/Dockerfile-hstore
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,10 @@ WORKDIR /pkg
COPY . .

ARG MAVEN_ARGS
ARG SOURCE_REVISION=local

RUN --mount=type=cache,target=/root/.m2 \
mvn package $MAVEN_ARGS -e -B -ntp -DskipTests -Dmaven.javadoc.skip=true \
RUN --mount=type=cache,id=hugegraph-maven-${SOURCE_REVISION},target=/root/.m2,sharing=locked \
mvn install $MAVEN_ARGS -e -B -ntp -Dmaven.test.skip=true -Dmaven.javadoc.skip=true \
&& rm ./hugegraph-server/*.tar.gz ./hugegraph-pd/*.tar.gz ./hugegraph-store/*.tar.gz

# 2nd stage: runtime env
Expand Down
5 changes: 3 additions & 2 deletions hugegraph-store/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,10 @@ WORKDIR /pkg
COPY . .

ARG MAVEN_ARGS
ARG SOURCE_REVISION=local

RUN --mount=type=cache,target=/root/.m2 \
mvn package $MAVEN_ARGS -e -B -ntp -Dmaven.test.skip=true -Dmaven.javadoc.skip=true \
RUN --mount=type=cache,id=hugegraph-maven-${SOURCE_REVISION},target=/root/.m2,sharing=locked \
mvn install $MAVEN_ARGS -e -B -ntp -Dmaven.test.skip=true -Dmaven.javadoc.skip=true \
&& rm ./hugegraph-server/*.tar.gz ./hugegraph-pd/*.tar.gz ./hugegraph-store/*.tar.gz

# 2nd stage: runtime env
Expand Down
Loading