diff --git a/.claude/skills/adopt-workflow-parts/SKILL.md b/.claude/skills/adopt-workflow-parts/SKILL.md new file mode 100644 index 0000000..a66ebae --- /dev/null +++ b/.claude/skills/adopt-workflow-parts/SKILL.md @@ -0,0 +1,115 @@ +--- +name: adopt-workflow-parts +description: > + Move a repository's GitHub workflows onto the github-workflow-parts catalog, change the catalog when a repository needs a part it lacks, + and bump consumers to a new catalog commit. + Use when adopting, updating or bumping workflow parts in any repository, or when an IstarCI run of a consumer fails. +--- + +# Adopt workflow parts + +## Goal + +The catalog is a high quality set of GitHub parts that pushes best practices into every repository and removes most of their pipeline code. +A consumer workflow is a few `uses:` of reusable workflows with inputs, no hand written tool installs, no action referenced by a version tag. + +## Procedure + +### Catalog change + +1. Start from the closest existing part and verify every claim (a CLI flag, a config file, a release) before stating it. +2. Propose the design: steps, inputs with defaults, and what the change adds or removes. +3. Wait for approval before any edit. +4. Edit, run `test/run_static_checks.sh`, commit, `istarci wait`. +5. The user pushes; consumers reference the pushed SHA only. + +### Consumer adoption + +1. `git rev-parse --abbrev-ref HEAD`: commit on the current branch, unless it is `main`, where a feature branch comes first. +2. Strip any UTF-8 byte order mark from tracked files and set `.editorconfig` `charset = utf-8-bom` to `utf-8`, in a commit of its own. +3. Replace `.github/workflows` jobs with the reusable workflows, every `uses:` and `workflow-parts-version` on the same full SHA. +4. Change one repository first, show its diff, wait, then do the others. +5. Commit, then `istarci wait`, and fix what fails before moving on. + +### Bump pass + +1. Replace the old SHA with the new one in every `uses:` and `workflow-parts-version` of each repository. +2. Commit "Use workflow parts *short sha*, ..." stating what the new commit brings, then `istarci wait` repository by repository. +3. A failure from a new check (a vulnerability found by an image scan) is fixed in its own commit, in the same pass. + +## Conventions + +### Reusable workflows + +- A few broad steps driven by conventions: + npm scripts `lint`, `format:check`, `typecheck`, `build`, `test` run with `--if-present`, a Pages build with `npm ci` then `npm run build`. +- Optional parts behind a flag or an input with a default (`mongodb-enabled`, `mongodb-version`, `severity`), + and a single optional `custom-tests` step. +- Never one step running whatever command a caller gives. +- Every shell input reaches the script through `env:`, never `${{ }}` inside `run:`, which is a script injection. +- Tool configuration files of the repository are respected (`sonar-project.properties`, `.trivyignore.yaml`). +- The checkout of the catalog itself is added to `.git/info/exclude`, so linters and scanners of the consumer skip it. +- No test in a publication workflow (`pkg.yml`): quality runs in CI, publication only builds and ships. +- `reusable-dotnet-quality.yml` is an old example, improved alongside rather than copied. + +### Supply chain + +Safety before convenience, and fewer dependencies over more. + +- First-party actions (`actions/*`, `github/*`) pinned by full SHA with a `# vX.Y.Z` inline comment with two spaces before (yamllint). +- No third-party action: a tool is a release binary checked against its SHA256 in a `setup` action, or the runner's OS packages. +- No third-party container image; `ghcr.io/devpro/*` and Docker Hub `devprofr/*` are first party. +- No Dependabot: versions are bumped by hand after checking the release exists. +- Lock files with hashes: `npm ci`, `pip install --require-hashes`, never an unpinned global install. +- A vulnerable image with no fixed release is replaced by a first-party one (`alpine-node` over `node`, an in-suite mailbox over Mailpit) + rather than ignored. +- A Trivy ignore, with a reason and an expiry under a month, is the last resort, for something run unprivileged in tests only. +- An npm `overrides` entry is a last resort too: + only when no release of the parent package, `latest` or `next`, fixes the dependency, and it is reverted once one does. +- Secrets stay secrets, even a low value one such as an IndexNow key in a public repository. + +### Versions + +- Node.js 24 by default, a library tested on 22, 24 and 26. +- MongoDB 9.0 in Docker Compose files, a single-node replica set `rs0` when the code uses transactions. + +## Boundaries + +- Azure DevOps and GitLab pipelines are never removed or emptied: the same repositories run on several CI systems. + GitLab gets its own session; only a minimal path fix keeping it working is allowed here. +- Deliberately vulnerable images for security workshops (container-images `tinyfilemanager`, `nextportal`) are left untouched, + neither scanned nor published. +- Commit only when asked, never push, never commit on `main`. +- YAML and Markdown linters are never run by an agent: CI runs them. + +## IstarCI + +- The daemon runs every commit by itself: after a commit, `istarci wait` only. +- `istarci run ` retries a run that must run again (daemon restart, change to `~/.istarci/config.yml`), never right after a commit. +- A command over the 600 second tool limit is moved to the background, against the foreground rule: + `timeout 590 istarci wait` then call it again while the run is going. +- `.istarci.yml` of a consumer sets the runner image per job (`ghcr.io/devpro/debian-node-playwright` for browser tests), + the jobs needing Docker, + and `skipJobs` for jobs that must never run on the developer machine (kind's privileged node, a QEMU registration in the kernel). +- The global ignores of image scans live in `~/.istarci/config.yml`, each with a reason and an expiry. + +## Gotchas + +- npm keeps the lock entry of a parent package when an override is added, and may even drop the child: + delete the lock entries of the parent and of the child, then `npm install`, and check `npm ls --all` says `overridden`. +- Use the Node.js of `~/.nvm/versions/node/v24.22.1/bin`: the shell `PATH` may resolve an older, vulnerable one. +- A file starting with a byte order mark breaks some tools, + and `dotnet format` fails every C# file when `.editorconfig` asks for `utf-8-bom` and the files have none. +- `build-push` uses the Docker daemon BuildKit (`--builder default`), with one tag per platform merged by `docker buildx imagetools create`, + as a `docker-container` builder cannot see images of the daemon. +- Trivy scans Java dependencies with `--offline-scan`, or it calls Maven Central for each one. +- CodeQL needs more memory than the default on large code bases, and its .NET extractor needs ICU. +- Python scanners reading non-ASCII sources need `PYTHONUTF8=1` in a container whose locale is not UTF-8. +- Playwright tests are found through `node_modules/.bin/playwright` of the root only, not of a workspace. +- A Sonar scan without `sonar-host-url` targets SonarCloud. + +## Commits + +- One sentence saying why, in the writing style of `AGENTS.md`: no em dash, no second person, the why timeless. +- Script files `snake_case`, committed with the executable bit (`git update-index --chmod=+x`). +- The final report lists each repository, branch, commit and run result, and what is left to the user (pushes, secrets, a first publication run). diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7e4956e..5624dc0 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -19,12 +19,23 @@ jobs: name: Markup uses: ./.github/workflows/reusable-markup-lint.yml + static-checks: + name: "Test / Static checks of the actions and workflows" + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Run actionlint, shellcheck and the conventions of this repository + run: test/run_static_checks.sh + test-terraform-setup: name: "Test / Terraform setup action" runs-on: ubuntu-latest + timeout-minutes: 20 steps: - name: Clone repository - uses: actions/checkout@v7 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - name: Install terraform uses: ./actions/terraform/setup - name: Verify terraform is installed @@ -35,9 +46,10 @@ jobs: test-tflint-setup: name: "Test / TFLint setup action" runs-on: ubuntu-latest + timeout-minutes: 20 steps: - name: Clone repository - uses: actions/checkout@v7 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - name: Setup TFLint uses: ./actions/tflint/setup - name: Verify tflint is installed @@ -45,29 +57,618 @@ jobs: tflint --version command -v tflint + test-setup-versions: + name: "Test / Setup actions install the version asked for" + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Install terraform + uses: ./actions/terraform/setup + with: + terraform-version: 1.9.8 + - name: Setup TFLint + uses: ./actions/tflint/setup + with: + tflint-version: v0.53.0 + - name: Setup Helm + uses: ./actions/helm/setup + with: + helm-version: v3.16.4 + - name: Setup Task + uses: ./actions/task/setup + with: + task-version: v3.52.0 + - name: Setup kubeconform + uses: ./actions/kubeconform/setup + with: + kubeconform-version: v0.7.0 + - name: Setup kind + uses: ./actions/kind/setup + with: + kind-version: v0.32.0 + kubectl-version: v1.36.4 + - name: Verify the versions + run: | + # Compared once captured, since a reader closing the pipe early fails the step with SIGPIPE under pipefail. + [[ "$(terraform version)" == "Terraform v1.9.8"* ]] + [[ "$(tflint --version)" == *"version 0.53.0"* ]] + [[ "$(helm version --short)" == v3.16.4+* ]] + [[ "$(task --version)" == *3.52.0* ]] + [[ "$(kubeconform -v)" == v0.7.0 ]] + [[ "$(kind version)" == "kind v0.32.0 "* ]] + [[ "$(kubectl version --client)" == *"v1.36.4"* ]] + + # A setup action that cannot verify what it downloaded stops the job rather than running it. + test-setup-fails-closed: + name: "Test / Setup actions fail on a missing release or a wrong checksum" + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Install a terraform release that does not exist + id: terraform-missing-release + continue-on-error: true + uses: ./actions/terraform/setup + with: + terraform-version: 0.0.0 + - name: Install a Trivy release that does not exist + id: trivy-missing-release + continue-on-error: true + uses: ./actions/trivy/scan + with: + trivy-version: 0.0.0 + scan-type: config + target: test/fixtures/terraform + - name: Install a Cosign release that does not exist + id: cosign-missing-release + continue-on-error: true + uses: ./actions/cosign/sign + with: + cosign-version: v0.0.0 + image-name: busybox + image-path: docker.io/library + image-tag: "1.37" + readonly: "true" + - name: Install a Task release that does not exist + id: task-missing-release + continue-on-error: true + uses: ./actions/task/setup + with: + task-version: v0.0.0 + - name: Install kube-linter with a checksum that does not match + id: kube-linter-wrong-checksum + continue-on-error: true + uses: ./actions/kube-linter/setup + with: + kube-linter-sha256: "0000000000000000000000000000000000000000000000000000000000000000" + - name: Install Opengrep with a checksum that does not match + id: opengrep-wrong-checksum + continue-on-error: true + uses: ./actions/opengrep/setup + with: + opengrep-sha256: "0000000000000000000000000000000000000000000000000000000000000000" + - name: Install a kind release that does not exist + id: kind-missing-release + continue-on-error: true + uses: ./actions/kind/setup + with: + kind-version: v0.0.0 + - name: Start MongoDB with a mongosh archive whose checksum does not match + id: mongodb-wrong-checksum + continue-on-error: true + uses: ./actions/mongodb/start + with: + mongosh-sha256: "0000000000000000000000000000000000000000000000000000000000000000" + - name: Verify every step failed + env: + STEPS: ${{ toJSON(steps) }} + run: | + passed=$(jq -r 'to_entries[] | select(.value.outcome != "failure") | .key' <<< "$STEPS") + if [ -n "$passed" ]; then + echo "Steps that did not fail: $passed" >&2 + exit 1 + fi + + test-helm-setup: + name: "Test / Helm setup action" + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Setup Helm + uses: ./actions/helm/setup + - name: Verify helm is installed + run: | + helm version + command -v helm + + test-kubernetes-tools-setup: + name: "Test / kube-linter, kubeconform, helm-unittest and kind setup actions" + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Setup Helm + uses: ./actions/helm/setup + - name: Setup kube-linter + uses: ./actions/kube-linter/setup + - name: Setup kubeconform + uses: ./actions/kubeconform/setup + - name: Setup helm-unittest + uses: ./actions/helm-unittest/setup + - name: Setup kind + uses: ./actions/kind/setup + - name: Verify the tools run on a chart + run: | + helm create "${RUNNER_TEMP}/demo" >/dev/null + helm template demo "${RUNNER_TEMP}/demo" | kubeconform -strict -summary + kube-linter lint "${RUNNER_TEMP}/demo" --format json > "${RUNNER_TEMP}/kube-linter.json" || true + jq -e '.Reports != null' "${RUNNER_TEMP}/kube-linter.json" + mkdir -p "${RUNNER_TEMP}/demo/tests" + printf '%s\n' 'suite: deployment' 'templates: [templates/deployment.yaml]' 'tests:' ' - it: renders a deployment' ' asserts:' ' - isKind:' ' of: Deployment' > "${RUNNER_TEMP}/demo/tests/deployment_test.yaml" + untt "${RUNNER_TEMP}/demo" + kind version + kubectl version --client + + test-scanner-setup: + name: "Test / Opengrep, Trivy and CodeQL setup actions" + runs-on: ubuntu-latest + timeout-minutes: 30 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Setup Opengrep + uses: ./actions/opengrep/setup + - name: Setup Trivy + uses: ./actions/trivy/setup + - name: Setup CodeQL + uses: ./actions/codeql/setup + - name: Verify each scanner runs + run: | + mkdir -p "${RUNNER_TEMP}/opengrep-fixture" + printf '%s\n' 'const input = process.argv[2];' 'eval(input);' > "${RUNNER_TEMP}/opengrep-fixture/app.js" + # $X is a metavariable of the Opengrep pattern, which the shell must leave as it is. + # shellcheck disable=SC2016 + [ "$(opengrep scan --quiet --json -e 'eval($X)' --lang js "${RUNNER_TEMP}/opengrep-fixture" | jq '.results | length')" = 1 ] + trivy fs --quiet --scanners vuln --exit-code 0 test/fixtures/node + [[ "$(trivy --version)" == *"Version: "* ]] + codeql resolve languages --format=json | jq -e 'has("javascript") and has("python")' + + test-chart-testing-setup: + name: "Test / chart-testing setup action" + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Setup Helm + uses: ./actions/helm/setup + - name: Setup chart-testing + uses: ./actions/chart-testing/setup + - name: Verify chart-testing is installed + run: | + ct version + command -v ct yamllint yamale + + test-goreleaser-setup: + name: "Test / GoReleaser setup action" + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Setup GoReleaser + uses: ./actions/goreleaser/setup + - name: Verify goreleaser is installed + run: | + goreleaser --version + command -v goreleaser + + test-go-lint-govulncheck-scan: + name: "Test / Go lint and govulncheck scan actions" + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Setup Go + uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0 + with: + go-version-file: test/fixtures/go/module/go.mod + cache: false + - name: Lint a module that passes every check + uses: ./actions/go/lint + with: + working-directory: test/fixtures/go/module + - name: Scan the module with govulncheck + uses: ./actions/govulncheck/scan + with: + working-directory: test/fixtures/go/module + - name: Lint a module gofmt did not format, failing on it + id: unformatted + continue-on-error: true + uses: ./actions/go/lint + with: + working-directory: test/fixtures/go/unformatted + - name: Lint a module whose go.mod requires what it never imports, failing on it + id: untidy + continue-on-error: true + uses: ./actions/go/lint + with: + working-directory: test/fixtures/go/untidy + - name: Verify both failed and govulncheck is on the path + env: + UNFORMATTED: ${{ steps.unformatted.outcome }} + UNTIDY: ${{ steps.untidy.outcome }} + run: | + [ "$UNFORMATTED" = failure ] + [ "$UNTIDY" = failure ] + govulncheck -version + + test-indexnow-submit: + name: "Test / IndexNow submit action, against a local API" + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + # The server outlives this step, which the next ones reach, and goes with the job. + - name: Serve a sitemap index and an IndexNow API that records the submission + run: | + nohup node test/fixtures/indexnow/server.js "${RUNNER_TEMP}/indexnow.json" 18089 > "${RUNNER_TEMP}/indexnow.log" 2>&1 & + for _ in $(seq 1 20); do + curl -fs -o /dev/null http://127.0.0.1:18089/sitemap.xml && exit 0 + sleep 0.5 + done + cat "${RUNNER_TEMP}/indexnow.log" + exit 1 + - name: Submit the pages of the sitemap index + uses: ./actions/indexnow/submit + with: + api-url: http://127.0.0.1:18089/indexnow + key: fixture-key + sitemap-url: http://127.0.0.1:18089/sitemap.xml + - name: Submit to an API refusing the submission, failing on it + id: refused + continue-on-error: true + uses: ./actions/indexnow/submit + with: + api-url: http://127.0.0.1:18089/refuse + key: fixture-key + sitemap-url: http://127.0.0.1:18089/sitemap.xml + - name: Verify the submission and the refusal + env: + REFUSED: ${{ steps.refused.outcome }} + run: | + jq -e ' + .host == "127.0.0.1:18089" + and .key == "fixture-key" + and .keyLocation == "http://127.0.0.1:18089/fixture-key.txt" + and .urlList == ["http://127.0.0.1:18089/a?x=1&y=2", "http://127.0.0.1:18089/b", "http://127.0.0.1:18089/c"] + ' "${RUNNER_TEMP}/indexnow.json" + [ "$REFUSED" = failure ] + + test-task-setup: + name: "Test / Task setup action" + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Setup Task + uses: ./actions/task/setup + - name: Verify task is installed + run: | + task --version + command -v task + + test-cosign-sign: + name: "Test / Cosign sign action, without signing" + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + # Signing needs an OIDC token and a registry to push to, so the run stops short of it. + - name: Install Cosign and read the digest of a public image + id: cosign + uses: ./actions/cosign/sign + with: + image-name: busybox + image-path: docker.io/library + image-tag: "1.37" + readonly: "true" + - name: Read the digest of an image that does not exist + id: missing-image + continue-on-error: true + uses: ./actions/cosign/sign + with: + image-name: busybox + image-path: docker.io/library + image-tag: tag-that-does-not-exist + readonly: "true" + - name: Verify the digest and the failure + env: + DIGEST: ${{ steps.cosign.outputs.digest }} + MISSING_IMAGE_OUTCOME: ${{ steps.missing-image.outcome }} + run: | + command -v cosign + [[ "$DIGEST" =~ ^sha256:[0-9a-f]{64}$ ]] + [ "$MISSING_IMAGE_OUTCOME" = failure ] + + test-dotnet-pack-push: + name: "Test / .NET pack-push action, without pushing" + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Package the library fixture + uses: ./actions/dotnet/pack-push + with: + registry-token: unused + working-directory: test/fixtures/dotnet-library + - name: Verify the package was created + run: test -s test/fixtures/dotnet-library/output/Fixture.Library.1.0.0.nupkg + + test-qemu-setup: + name: "Test / QEMU setup action runs an arm64 image" + runs-on: ubuntu-latest + timeout-minutes: 20 + services: + registry: + image: registry:3.1.2 + ports: + - 5000:5000 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Setup QEMU + uses: ./actions/qemu/setup + with: + platforms: linux/amd64,linux/arm64,linux/arm/v7 + - name: Verify an arm64 image runs + run: | + [ "$(docker run --rm --platform linux/arm64 busybox:1.37 uname -m)" = aarch64 ] + - name: Build an image for two platforms, one emulated + id: emulated + uses: ./actions/container/build-push + with: + image-definition: test/fixtures/container/base/Dockerfile + image-repository: localhost:5000/emulated-test + image-tag: ${{ github.run_id }} + platforms: linux/amd64,linux/arm64 + - name: Verify the tag names a manifest list holding both platforms, each also under a tag of its own + env: + DIGEST: ${{ steps.emulated.outputs.digest }} + run: | + image="localhost:5000/emulated-test:${GITHUB_RUN_ID}" + [ "$(docker buildx imagetools inspect "$image" --format '{{json .Manifest}}' | jq -r .digest)" = "$DIGEST" ] + platforms=$(docker buildx imagetools inspect "$image" --raw | jq -r '[.manifests[].platform | select(.os == "linux") | .architecture] | sort | join(",")') + [ "$platforms" = amd64,arm64 ] + for platform in linux-amd64 linux-arm64; do + docker buildx imagetools inspect "${image}-${platform}" >/dev/null + done + + test-container-build-push: + name: "Test / Container build-push action, a base and an image built on it" + runs-on: ubuntu-latest + timeout-minutes: 20 + services: + registry: + image: registry:3.1.2 + ports: + - 5000:5000 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Build and push the base image under a tag + id: base + uses: ./actions/container/build-push + with: + image-definition: test/fixtures/container/base/Dockerfile + image-repository: localhost:5000/build-test-base + image-tag: ${{ github.run_id }} + # The derived image fails to build unless the build argument names the base pushed above. + - name: Build and push the derived image + id: derived + uses: ./actions/container/build-push + with: + image-definition: test/fixtures/container/derived/Dockerfile + image-repository: localhost:5000/build-test-derived + image-tag: ${{ github.run_id }} + build-arguments: | + BASE_IMAGE=localhost:5000/build-test-base:${{ github.run_id }} + - name: Verify both images are in the registry under their digest, and not left in the daemon + env: + BASE_DIGEST: ${{ steps.base.outputs.digest }} + DERIVED_DIGEST: ${{ steps.derived.outputs.digest }} + run: | + for image in "build-test-base:${GITHUB_RUN_ID}=${BASE_DIGEST}" "build-test-derived:${GITHUB_RUN_ID}=${DERIVED_DIGEST}"; do + ref="localhost:5000/${image%%=*}" + [ "$(docker buildx imagetools inspect "$ref" --format '{{json .Manifest}}' | jq -r .digest)" = "${image#*=}" ] + if docker image inspect "$ref" >/dev/null 2>&1; then + echo "ERROR: ${ref} is left in the daemon" >&2 + exit 1 + fi + done + + test-container-build-push-arm64: + name: "Test / Container build-push action on an arm64 runner" + runs-on: ubuntu-24.04-arm + timeout-minutes: 20 + services: + registry: + image: registry:3.1.2 + ports: + - 5000:5000 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Build and push the base image natively + id: native + uses: ./actions/container/build-push + with: + image-definition: test/fixtures/container/base/Dockerfile + image-repository: localhost:5000/native-test + image-tag: ${{ github.run_id }} + platforms: linux/arm64 + - name: Verify the image is for arm64 + env: + DIGEST: ${{ steps.native.outputs.digest }} + run: | + image="localhost:5000/native-test@${DIGEST}" + architectures=$(docker buildx imagetools inspect "$image" --raw | jq -r '[.manifests[]?.platform | select(.os == "linux") | .architecture] | join(",")') + # A single platform pushed without attestations is a manifest rather than a list, and names its architecture in its configuration. + if [ -z "$architectures" ]; then + architectures=$(docker buildx imagetools inspect "$image" --format '{{json .Image}}' | jq -r .architecture) + fi + [ "$architectures" = arm64 ] + + # Each tool runs once installed, which a binary built for another processor does not. + test-setup-arm64: + name: "Test / Setup actions install arm64 binaries on an arm64 runner" + runs-on: ubuntu-24.04-arm + timeout-minutes: 20 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Setup Terraform + uses: ./actions/terraform/setup + - name: Setup TFLint + uses: ./actions/tflint/setup + - name: Setup Helm + uses: ./actions/helm/setup + - name: Setup chart-testing + uses: ./actions/chart-testing/setup + - name: Setup GoReleaser + uses: ./actions/goreleaser/setup + - name: Setup Task + uses: ./actions/task/setup + - name: Setup kube-linter + uses: ./actions/kube-linter/setup + - name: Setup kubeconform + uses: ./actions/kubeconform/setup + - name: Setup helm-unittest + uses: ./actions/helm-unittest/setup + - name: Setup kind + uses: ./actions/kind/setup + - name: Setup Opengrep + uses: ./actions/opengrep/setup + - name: Setup CodeQL + uses: ./actions/codeql/setup + - name: Start MongoDB + uses: ./actions/mongodb/start + - name: Generate SBOM with Syft + uses: ./actions/syft/generate-sbom + with: + target-path: test/fixtures/container + upload-artifact: "false" + - name: Scan with Trivy + uses: ./actions/trivy/scan + with: + scan-type: config + target: test/fixtures/container + - name: Verify the binaries are for arm64 + run: | + for tool in terraform tflint helm ct goreleaser task kube-linter kubeconform untt kind kubectl opengrep mongod mongosh syft trivy; do + # The ELF machine field, 0xb7 being aarch64, read without the file command a runner image may lack. + [ "$(od -An -tx1 -j18 -N1 "$(command -v "$tool")" | tr -d ' ')" = b7 ] + done + # The codeql command is a launcher script, run here so that its arm64 Java and extractors answer. + codeql version --format=terse + + test-mongodb-start: + name: "Test / MongoDB start action" + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Start MongoDB + uses: ./actions/mongodb/start + - name: Verify MongoDB answers, on the latest release by default + run: | + mongosh --quiet --eval 'db.runCommand({ ping: 1 }).ok' mongodb://127.0.0.1:27017 | grep -qx 1 + mongosh --quiet --eval 'db.version()' mongodb://127.0.0.1:27017 | grep -q '^9\.0\.' + + test-mongodb-start-version: + name: "Test / MongoDB start action, an earlier version" + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Start MongoDB 8.0 + uses: ./actions/mongodb/start + with: + mongodb-version: 8.0.34 + - name: Verify the server is the version asked for + run: | + [ "$(mongosh --quiet --eval 'db.version()' mongodb://127.0.0.1:27017)" = 8.0.34 ] + + test-mongodb-start-replica-set: + name: "Test / MongoDB start action, replica set" + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Start MongoDB + uses: ./actions/mongodb/start + with: + port: "27117" + replica-set: rs0 + - name: Verify MongoDB is the primary of the replica set + run: | + mongosh --quiet --eval 'db.hello().isWritablePrimary' "mongodb://127.0.0.1:27117/?directConnection=true" | grep -qx true + test-syft-generate-sbom: name: "Test / Syft generate-sbom action" runs-on: ubuntu-latest + timeout-minutes: 20 steps: - name: Clone repository - uses: actions/checkout@v7 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - name: Generate SBOM for this repository uses: ./actions/syft/generate-sbom with: target-path: actions output-file: sbom-test.spdx.json upload-artifact: "false" + - name: Generate SBOM for a directory, one folder left out + uses: ./actions/syft/generate-sbom + with: + exclude-paths: node + target-path: test/fixtures + output-file: sbom-exclude-test.spdx.json + upload-artifact: "false" + - name: Generate SBOM for a public image + uses: ./actions/syft/generate-sbom + with: + target-image: busybox:1.37 + output-format: cyclonedx-json + output-file: sbom-image-test.cdx.json + upload-artifact: "false" - name: Verify SBOM was generated run: | test -s sbom-test.spdx.json grep -q '"spdxVersion"' sbom-test.spdx.json + jq -e '.bomFormat == "CycloneDX" and (.components | length > 0)' sbom-image-test.cdx.json + jq -e '[.packages[].name] | index("playwright-fixture") and (index("node-fixture") | not)' sbom-exclude-test.spdx.json test-trivy-scan: name: "Test / Trivy scan action" runs-on: ubuntu-latest + timeout-minutes: 20 steps: - name: Clone repository - uses: actions/checkout@v7 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - name: Scan a real image with Trivy uses: ./actions/trivy/scan with: @@ -83,14 +684,380 @@ jobs: format: sarif output-file: trivy-test.sarif severity: HIGH,CRITICAL - - name: Verify SARIF report was generated - run: test -s trivy-test.sarif + # The base fixture runs as root, which Trivy reports as a HIGH misconfiguration. + - name: Scan a directory with a finding, failing on it + id: trivy-finding + continue-on-error: true + uses: ./actions/trivy/scan + with: + scan-type: config + target: test/fixtures/container/base + severity: HIGH,CRITICAL + exit-code: "1" + - name: Scan a directory with a finding the ignore file leaves out + uses: ./actions/trivy/scan + with: + scan-type: config + target: test/fixtures/container/base + severity: HIGH,CRITICAL + exit-code: "1" + ignore-file: test/fixtures/trivy/ignore-current.yaml + - name: Scan a directory with a finding whose ignore entry expired, failing on it + id: trivy-expired + continue-on-error: true + uses: ./actions/trivy/scan + with: + scan-type: config + target: test/fixtures/container/base + severity: HIGH,CRITICAL + exit-code: "1" + ignore-file: test/fixtures/trivy/ignore-expired.yaml + - name: Scan with an ignore file that does not exist, failing on it + id: trivy-missing-ignore-file + continue-on-error: true + uses: ./actions/trivy/scan + with: + scan-type: config + target: test/fixtures/terraform + ignore-file: test/fixtures/trivy/missing.yaml + - name: Verify SARIF report was generated and the findings failed the scans + env: + EXPIRED_OUTCOME: ${{ steps.trivy-expired.outcome }} + FINDING_OUTCOME: ${{ steps.trivy-finding.outcome }} + MISSING_IGNORE_FILE_OUTCOME: ${{ steps.trivy-missing-ignore-file.outcome }} + run: | + jq -e '.version == "2.1.0"' trivy-test.sarif + [ "$FINDING_OUTCOME" = failure ] + [ "$EXPIRED_OUTCOME" = failure ] + [ "$MISSING_IGNORE_FILE_OUTCOME" = failure ] test-terraform-quality: name: "Test / Terraform quality reusable workflow" uses: ./.github/workflows/reusable-terraform-quality.yml with: working-directory: test/fixtures/terraform - # explicit override: for this same-repo local "./" call, github.workflow_sha's behavior is unverified, - # so pin to a ref we know is fetchable (branch on push, PR merge ref on PRs) - workflow-parts-version: ${{ github.ref }} + + test-dotnet-quality: + name: "Test / .NET quality reusable workflow" + uses: ./.github/workflows/reusable-dotnet-quality.yml + with: + working-directory: test/fixtures/dotnet + # Each line runs as written, quotes included, or the commands fail. + custom-commands: | + words=("two words" three) + [ "${#words[@]}" = 2 ] + [ "$(dotnet --version | cut -d . -f 1)" -ge 10 ] + # The fixture holds a test that fails, which only the filter keeps the job from running, its quotes being read as shell quotes rather than passed on. + dotnet-test-args: --report-xunit-trx --coverage --coverage-output-format cobertura --filter-not-namespace 'Fixture.Tests.LeftOut*' + + # Runs where a Sonar server is set in SONAR_HOST_URL, as IstarCI provides a SonarQube locally, and is skipped otherwise. + test-dotnet-quality-sonar: + name: "Test / .NET quality reusable workflow with Sonar" + if: vars.SONAR_HOST_URL != '' + uses: ./.github/workflows/reusable-dotnet-quality.yml + with: + working-directory: test/fixtures/dotnet + dotnet-test-args: --report-xunit-trx --coverage --coverage-output-format cobertura --filter-not-namespace 'Fixture.Tests.LeftOut*' + sonar-enabled: true + sonar-host-url: ${{ vars.SONAR_HOST_URL }} + sonar-organization: ${{ vars.SONAR_ORGANIZATION }} + sonar-project-key: github-workflow-parts-dotnet-fixture + sonar-project-name: GitHub workflow parts .NET fixture + secrets: + sonar-token: ${{ secrets.SONAR_TOKEN }} + + test-dotnet-build-test: + name: "Test / .NET build-test action writes results, coverage and reports" + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Install .NET & lint + uses: ./actions/dotnet/install-lint-restore + with: + working-directory: test/fixtures/dotnet + - name: Build & test + uses: ./actions/dotnet/build-test + with: + dotnet-test-args: --report-xunit-trx --coverage --coverage-output-format cobertura --filter-not-namespace 'Fixture.Tests.LeftOut*' + working-directory: test/fixtures/dotnet + # The paths are those the .NET quality workflow uploads as its artifact. + - name: Verify the results, the coverage and the reports + working-directory: test/fixtures/dotnet/report + run: | + compgen -G "TestResults/*.trx" >/dev/null + compgen -G "TestResults/*.cobertura.xml" >/dev/null + test -s Summary.txt + test -s index.html + grep -q 'Line coverage' Summary.txt + + test-container-scan-on-base: + name: "Test / Container scan reusable workflow on a base built in the run" + uses: ./.github/workflows/reusable-container-scan.yml + with: + base-image-definition: test/fixtures/container/base/Dockerfile + image-definition: test/fixtures/container/derived/Dockerfile + image-name: scan-test + image-path: local + image-tag: ${{ github.run_id }} + severity: CRITICAL,HIGH,MEDIUM + trivy-ignore-file: test/fixtures/trivy/ignore-current.yaml + + test-git-check-changes: + name: "Test / Git check-changes action" + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + # The empty tree of git stands for a base holding nothing, which needs no history in the clone. + - name: Compare a path with the empty tree + id: added + uses: ./actions/git/check-changes + with: + paths: README.md actions + base: 4b825dc642cb6eb9a060e54bf8d69288fbee4904 + - name: Compare a path that does not exist with the empty tree + id: absent + uses: ./actions/git/check-changes + with: + paths: path-that-does-not-exist + base: 4b825dc642cb6eb9a060e54bf8d69288fbee4904 + - name: Compare a path with the commit itself + id: unchanged + uses: ./actions/git/check-changes + with: + paths: README.md + base: HEAD + - name: Compare a path with a commit the clone does not have + id: unreadable + uses: ./actions/git/check-changes + with: + paths: README.md + base: "0000000000000000000000000000000000000000" + - name: Verify the comparisons + env: + ADDED: ${{ steps.added.outputs.changed }} + ABSENT: ${{ steps.absent.outputs.changed }} + UNCHANGED: ${{ steps.unchanged.outputs.changed }} + UNREADABLE: ${{ steps.unreadable.outputs.changed }} + run: | + [ "$ADDED" = true ] + [ "$ABSENT" = false ] + [ "$UNCHANGED" = false ] + [ "$UNREADABLE" = true ] + + # A local run reads the outputs of a called workflow under the id of the job inside it, which the caller shares. + git-check: + name: "Test / Git check reusable workflow" + uses: ./.github/workflows/reusable-git-check.yml + with: + paths: test/fixtures + version-file: test/fixtures/version/Directory.Build.props + + test-git-check-outputs: + name: "Test / Git check reusable workflow outputs" + needs: git-check + runs-on: ubuntu-latest + timeout-minutes: 5 + steps: + - name: Verify the outputs + env: + BRANCH_SLUG: ${{ needs.git-check.outputs.branch-slug }} + CHANGED: ${{ needs.git-check.outputs.changed }} + VERSION_MAJOR_MINOR: ${{ needs.git-check.outputs.version-major-minor }} + run: | + [ "$VERSION_MAJOR_MINOR" = 1.2 ] + [[ "$BRANCH_SLUG" =~ ^[a-z0-9-]+$ ]] + [[ "$CHANGED" =~ ^(true|false)$ ]] + + test-npm-publish: + name: "Test / npm publish action, without publishing" + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Publish a version the registry already has + id: published + uses: ./actions/npm/publish + with: + publish-command: "false" + working-directory: test/fixtures/npm/published + # A dry run goes through every step of a publication but the upload, which needs no credentials. + - name: Publish a package the registry does not have + id: unpublished + uses: ./actions/npm/publish + with: + publish-command: npm publish --dry-run + working-directory: test/fixtures/npm/unpublished + # GitHub Packages refuses to answer a run with no token about a package it holds, as a local run is, and answers E404 for one it does not. + - name: Publish with no credential + id: no-credential + uses: ./actions/npm/publish + with: + publish-command: touch "${RUNNER_TEMP}/packed" && npm publish + working-directory: test/fixtures/npm/no-credential + - name: Verify only the new package was published + env: + NO_CREDENTIAL: ${{ steps.no-credential.outputs.published }} + PUBLISHED: ${{ steps.published.outputs.published }} + PUBLISHED_VERSION: ${{ steps.published.outputs.version }} + UNPUBLISHED: ${{ steps.unpublished.outputs.published }} + run: | + [ "$PUBLISHED" = false ] + [ "$PUBLISHED_VERSION" = 1.3.0 ] + [ "$UNPUBLISHED" = true ] + [ "$NO_CREDENTIAL" = false ] + [ -f "${RUNNER_TEMP}/packed" ] + + test-playwright-install: + name: "Test / Playwright install action" + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Install the dependencies + working-directory: test/fixtures/playwright + run: npm ci + - name: Install Playwright browsers + uses: ./actions/playwright/install + with: + working-directory: test/fixtures/playwright + - name: Verify the browser starts + working-directory: test/fixtures/playwright + run: node -e "require('playwright').chromium.launch().then((browser) => browser.close())" + + test-pages-deployment: + name: "Test / Pages deployment reusable workflow, without deploying" + uses: ./.github/workflows/reusable-pages-deployment.yml + with: + build-command: | + [ "$FIXTURE_SECRET" = hidden ] + task --version + npm ci + npm run build + deploy: false + node-version: "24" + site-path: _site + task-enabled: true + working-directory: test/fixtures/pages + secrets: + additional-vars: | + FIXTURE_SECRET=hidden + + test-pages-deployment-conventions: + name: "Test / Pages deployment reusable workflow, building by convention" + uses: ./.github/workflows/reusable-pages-deployment.yml + with: + deploy: false + site-path: _site + working-directory: test/fixtures/pages + + test-node-quality-npm: + name: "Test / Node.js quality reusable workflow, with npm" + uses: ./.github/workflows/reusable-node-quality.yml + with: + custom-tests: | + [ "$FIXTURE_VALUE" = extra ] + words=("two words" three) + [ "${#words[@]}" = 2 ] + mongosh --quiet --eval 'db.hello().isWritablePrimary' "mongodb://127.0.0.1:27017/?replicaSet=rs0" | grep -qx true + task --version + terraform version + node -e "require('playwright').chromium.launch().then((browser) => browser.close())" + [ -f coverage/lcov.info ] + # Each script ran once, in that order, the missing ones being skipped. + diff <(printf '%s\n' lint format:check typecheck build test:coverage) report/scripts.log + # The custom tests see the repository alone, without the workflow parts checked out beside it. + [ ! -e "${GITHUB_WORKSPACE}/workflow-parts" ] + extra-vars: | + FIXTURE_VALUE=extra + mongodb-enabled: true + task-enabled: true + terraform-version: 1.17.0 + test-script: test:coverage + working-directory: test/fixtures/node + + test-node-quality-pnpm: + name: "Test / Node.js quality reusable workflow, with pnpm" + uses: ./.github/workflows/reusable-node-quality.yml + with: + custom-tests: | + [[ "$(pnpm --version)" == 12.* ]] + pnpm-version: "12" + working-directory: test/fixtures/pnpm + + # Runs where a Sonar server is set in SONAR_HOST_URL, as IstarCI provides a SonarQube locally, and is skipped otherwise. + test-node-quality-sonar: + name: "Test / Node.js quality reusable workflow with Sonar" + if: vars.SONAR_HOST_URL != '' + uses: ./.github/workflows/reusable-node-quality.yml + with: + sonar-enabled: true + sonar-host-url: ${{ vars.SONAR_HOST_URL }} + sonar-organization: ${{ vars.SONAR_ORGANIZATION }} + sonar-project-key: github-workflow-parts-node-fixture + sonar-project-name: GitHub workflow parts Node.js fixture + test-script: test:coverage + working-directory: test/fixtures/node + secrets: + sonar-token: ${{ secrets.SONAR_TOKEN }} + + # Runs where a Sonar server is set in SONAR_HOST_URL, as IstarCI provides a SonarQube locally, and is skipped otherwise. + test-sonar-scan-properties: + name: "Test / Sonar scan action with a sonar-project.properties" + if: vars.SONAR_HOST_URL != '' + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + # As the Node.js quality workflow does with its workflow parts. + - name: Leave a folder of the fixture out of git + run: | + exclude="$(git rev-parse --git-path info/exclude)" + mkdir -p "$(dirname "$exclude")" + echo "/test/fixtures/sonar-properties/extra/" >> "$exclude" + # No project key nor exclusions given: the file holds them. + - name: Analyze the fixture + uses: ./actions/sonar/scan + env: + # The report the scanner sends is kept, for the step below to read which files were indexed. + SONAR_SCANNER_JSON_PARAMS: '{"sonar.scanner.keepReport": "true"}' + with: + host-url: ${{ vars.SONAR_HOST_URL }} + organization: ${{ vars.SONAR_ORGANIZATION }} + token: ${{ secrets.SONAR_TOKEN }} + working-directory: test/fixtures/sonar-properties + # The token of the run may only send an analysis, so the report the scanner kept tells what it indexed. + - name: Verify the analysis holds only the file neither the exclusions of the file nor git leave out + working-directory: test/fixtures/sonar-properties/.scannerwork + run: | + grep -q '^projectKey=github-workflow-parts-sonar-properties$' report-task.txt + indexed=$(cat scanner-report/component-*.pb | grep -aoE '[a-z/]+\.js' | sort -u | paste -sd ,) + [ "$indexed" = kept.js ] + + test-npm-publication: + name: "Test / npm publication reusable workflow, without publishing" + uses: ./.github/workflows/reusable-npm-publication.yml + with: + # A dry run goes through every step of a publication but the upload, which needs no credentials. + publish-command: npm publish --dry-run + working-directory: test/fixtures/npm/unpublished + + test-npm-publication-outputs: + name: "Test / npm publication reusable workflow outputs" + needs: test-npm-publication + runs-on: ubuntu-latest + timeout-minutes: 5 + steps: + - name: Verify the package was published + env: + PUBLISHED: ${{ needs.test-npm-publication.outputs.published }} + VERSION: ${{ needs.test-npm-publication.outputs.version }} + run: | + [ "$PUBLISHED" = true ] + [ "$VERSION" = 0.0.0 ] diff --git a/.github/workflows/reusable-container-publication.yml b/.github/workflows/reusable-container-publication.yml index 396388d..e88d581 100644 --- a/.github/workflows/reusable-container-publication.yml +++ b/.github/workflows/reusable-container-publication.yml @@ -43,22 +43,32 @@ on: type: string required: true image-platform: - description: Image platform + description: Platforms to build, emulated with QEMU on operating-system, unless arm64-runner is set type: string required: false default: "linux/amd64,linux/arm64" + arm64-runner: + description: Runner label building linux/arm64 natively (for example "ubuntu-24.04-arm"), beside linux/amd64 on operating-system, rather than emulating every platform with QEMU on operating-system, image-platform being then not read + type: string + required: false + default: "" job-name: description: Job name type: string required: false default: Publication + timeout-minutes: + description: Minutes the job may run before it is cancelled + type: number + required: false + default: 60 operating-system: description: Operating system executing the runner type: string required: false default: ubuntu-latest workflow-parts-version: - description: GitHub workflow parts version (branch/tag/SHA). Should match the ref/tag/SHA the caller pinned for this reusable workflow itself. + description: GitHub workflow parts version (branch/tag/SHA). Should match the ref/tag/SHA the caller pinned for this reusable workflow itself. Ignored when the caller is this repository, whose checkout already holds the actions under test. type: string required: false default: main @@ -79,12 +89,11 @@ on: required: false jobs: - container-publication: - name: ${{ inputs.job-name }} + build-emulated: + name: ${{ inputs.job-name }} build (emulated) + if: ${{ inputs.arm64-runner == '' }} runs-on: ${{ inputs.operating-system }} - defaults: - run: - working-directory: ${{ inputs.working-directory }} + timeout-minutes: ${{ inputs.timeout-minutes }} steps: - name: Set secret variables shell: bash @@ -92,7 +101,7 @@ jobs: SECRET_VARS: ${{ secrets.secret-vars }} run: | if [[ -n "$SECRET_VARS" ]]; then - echo "$SECRET_VARS" | while IFS='=' read -r key val; do + echo "$SECRET_VARS" | while IFS='=' read -r _ val; do if [[ -n "$val" ]]; then echo "::add-mask::$val" fi @@ -100,14 +109,14 @@ jobs: echo "$SECRET_VARS" >> "$GITHUB_ENV" fi - name: Clone repository - uses: actions/checkout@v7 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - name: Checkout workflow parts - uses: actions/checkout@v7 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: repository: devpro/github-workflow-parts - ref: ${{ inputs.workflow-parts-version }} + # The caller being this repository, the actions under test are those of the commit under test. + ref: ${{ github.repository == 'devpro/github-workflow-parts' && github.sha || inputs.workflow-parts-version }} path: workflow-parts - # The Docker CLI of the runner does what the docker/* actions wrapped, with no third party action in between. - name: Login to container registry env: REGISTRY: ${{ inputs.container-registry }} @@ -117,21 +126,146 @@ jobs: - name: Set up QEMU # Emulation is only needed for a platform other than the runner's own. if: ${{ inputs.image-platform != 'linux/amd64' }} - run: docker run --privileged --rm tonistiigi/binfmt --install all - - name: Set up Buildx - run: docker buildx create --use --driver docker-container + uses: ./workflow-parts/actions/qemu/setup + with: + platforms: ${{ inputs.image-platform }} + # The action puts the platforms together under the tag once each is pushed. - name: Build and push container image - # The gha cache backend is reachable from JavaScript actions only, so the build runs without a cache. + uses: ./workflow-parts/actions/container/build-push + with: + image-definition: ${{ inputs.image-definition }} + image-repository: ${{ env.IMAGE_REPOSITORY }} + image-tag: ${{ inputs.image-tag }} + platforms: ${{ inputs.image-platform }} + build-arguments: ${{ inputs.extra-build-arguments }} + working-directory: ${{ inputs.working-directory }} + + build-amd64: + name: ${{ inputs.job-name }} build (linux/amd64) + if: ${{ inputs.arm64-runner != '' }} + runs-on: ${{ inputs.operating-system }} + timeout-minutes: ${{ inputs.timeout-minutes }} + outputs: + digest: ${{ steps.build.outputs.digest }} + steps: + - name: Set secret variables + shell: bash + env: + SECRET_VARS: ${{ secrets.secret-vars }} + run: | + if [[ -n "$SECRET_VARS" ]]; then + echo "$SECRET_VARS" | while IFS='=' read -r _ val; do + if [[ -n "$val" ]]; then + echo "::add-mask::$val" + fi + done + echo "$SECRET_VARS" >> "$GITHUB_ENV" + fi + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Checkout workflow parts + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: devpro/github-workflow-parts + # The caller being this repository, the actions under test are those of the commit under test. + ref: ${{ github.repository == 'devpro/github-workflow-parts' && github.sha || inputs.workflow-parts-version }} + path: workflow-parts + - name: Login to container registry + env: + REGISTRY: ${{ inputs.container-registry }} + REGISTRY_USERNAME: ${{ secrets.container-registry-username }} + REGISTRY_PASSWORD: ${{ secrets.container-registry-password }} + run: echo "$REGISTRY_PASSWORD" | docker login "$REGISTRY" --username "$REGISTRY_USERNAME" --password-stdin + - name: Build and push container image under the tag of its platform + id: build + uses: ./workflow-parts/actions/container/build-push + with: + image-definition: ${{ inputs.image-definition }} + image-repository: ${{ env.IMAGE_REPOSITORY }} + image-tag: ${{ inputs.image-tag }}-linux-amd64 + platforms: linux/amd64 + build-arguments: ${{ inputs.extra-build-arguments }} + working-directory: ${{ inputs.working-directory }} + + build-arm64: + name: ${{ inputs.job-name }} build (linux/arm64) + if: ${{ inputs.arm64-runner != '' }} + runs-on: ${{ inputs.arm64-runner }} + timeout-minutes: ${{ inputs.timeout-minutes }} + outputs: + digest: ${{ steps.build.outputs.digest }} + steps: + - name: Set secret variables + shell: bash + env: + SECRET_VARS: ${{ secrets.secret-vars }} + run: | + if [[ -n "$SECRET_VARS" ]]; then + echo "$SECRET_VARS" | while IFS='=' read -r _ val; do + if [[ -n "$val" ]]; then + echo "::add-mask::$val" + fi + done + echo "$SECRET_VARS" >> "$GITHUB_ENV" + fi + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Checkout workflow parts + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: devpro/github-workflow-parts + # The caller being this repository, the actions under test are those of the commit under test. + ref: ${{ github.repository == 'devpro/github-workflow-parts' && github.sha || inputs.workflow-parts-version }} + path: workflow-parts + - name: Login to container registry + env: + REGISTRY: ${{ inputs.container-registry }} + REGISTRY_USERNAME: ${{ secrets.container-registry-username }} + REGISTRY_PASSWORD: ${{ secrets.container-registry-password }} + run: echo "$REGISTRY_PASSWORD" | docker login "$REGISTRY" --username "$REGISTRY_USERNAME" --password-stdin + - name: Build and push container image under the tag of its platform + id: build + uses: ./workflow-parts/actions/container/build-push + with: + image-definition: ${{ inputs.image-definition }} + image-repository: ${{ env.IMAGE_REPOSITORY }} + image-tag: ${{ inputs.image-tag }}-linux-arm64 + platforms: linux/arm64 + build-arguments: ${{ inputs.extra-build-arguments }} + working-directory: ${{ inputs.working-directory }} + + # The platforms built on runners of their own are only put under the tag here, so that the image a tag names is complete. + publication: + name: ${{ inputs.job-name }} + needs: [build-emulated, build-amd64, build-arm64] + # One build path is skipped by design, which an implicit success() would take for a failure. + if: ${{ !cancelled() && !contains(needs.*.result, 'failure') }} + runs-on: ${{ inputs.operating-system }} + timeout-minutes: ${{ inputs.timeout-minutes }} + steps: + - name: Checkout workflow parts + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: devpro/github-workflow-parts + # The caller being this repository, the actions under test are those of the commit under test. + ref: ${{ github.repository == 'devpro/github-workflow-parts' && github.sha || inputs.workflow-parts-version }} + path: workflow-parts + - name: Login to container registry env: - IMAGE_DEFINITION: ${{ inputs.image-definition }} - IMAGE_PLATFORM: ${{ inputs.image-platform }} - BUILD_ARGUMENTS: ${{ inputs.extra-build-arguments }} + REGISTRY: ${{ inputs.container-registry }} + REGISTRY_USERNAME: ${{ secrets.container-registry-username }} + REGISTRY_PASSWORD: ${{ secrets.container-registry-password }} + run: echo "$REGISTRY_PASSWORD" | docker login "$REGISTRY" --username "$REGISTRY_USERNAME" --password-stdin + - name: Tag the image of every platform + if: ${{ inputs.arm64-runner != '' }} + env: + DIGESTS: ${{ needs.build-amd64.outputs.digest }} ${{ needs.build-arm64.outputs.digest }} run: | - args=(--file "$IMAGE_DEFINITION" --platform "$IMAGE_PLATFORM" --tag "$IMAGE_REF" --push) - while IFS= read -r argument; do - [ -n "$argument" ] && args+=(--build-arg "$argument") - done <<< "$BUILD_ARGUMENTS" - docker buildx build "${args[@]}" . + sources=() + for digest in $DIGESTS; do + sources+=("${IMAGE_REPOSITORY}@${digest}") + done + docker buildx imagetools create --tag "$IMAGE_REF" "${sources[@]}" - name: Generate SBOM with Syft uses: ./workflow-parts/actions/syft/generate-sbom continue-on-error: true @@ -140,15 +274,15 @@ jobs: - name: Push latest tag if: ${{ inputs.create-latest }} run: | - docker buildx imagetools create \ - --tag ${{ env.IMAGE_REF_LATEST }} \ - ${{ env.IMAGE_REF }} + docker buildx imagetools create --tag "$IMAGE_REF_LATEST" "$IMAGE_REF" - name: Sign container image with Cosign uses: ./workflow-parts/actions/cosign/sign with: image-name: ${{ inputs.image-name }} image-path: ${{ inputs.image-path }} image-tag: ${{ inputs.image-tag }} - env: - IMAGE_REF: ${{ inputs.image-path }}/${{ inputs.image-name }}:${{ inputs.image-tag }} - IMAGE_REF_LATEST: ${{ inputs.image-path }}/${{ inputs.image-name }}:latest + +env: + IMAGE_REPOSITORY: ${{ inputs.image-path }}/${{ inputs.image-name }} + IMAGE_REF: ${{ inputs.image-path }}/${{ inputs.image-name }}:${{ inputs.image-tag }} + IMAGE_REF_LATEST: ${{ inputs.image-path }}/${{ inputs.image-name }}:latest diff --git a/.github/workflows/reusable-container-scan.yml b/.github/workflows/reusable-container-scan.yml index 33405b3..69a0998 100644 --- a/.github/workflows/reusable-container-scan.yml +++ b/.github/workflows/reusable-container-scan.yml @@ -3,6 +3,16 @@ name: Reusable - Container scan on: workflow_call: inputs: + base-image-argument: + description: Build argument the image definition reads the base image from + type: string + required: false + default: BASE_IMAGE + base-image-definition: + description: Path to the definition of the image this one is built on, built first so that the scan reads what this commit produces rather than a published base + type: string + required: false + default: "" image-definition: description: Path to the container definition file (Dockerfile, Containerfile) type: string @@ -24,33 +34,28 @@ on: type: string required: false default: Scan - max-high-cves: - description: Maximum number of high CVEs authorized - type: number - required: false - default: 0 - max-medium-cves: - description: Maximum number of medium CVEs authorized + timeout-minutes: + description: Minutes the job may run before it is cancelled type: number required: false - default: 0 - neuvector-enabled: - description: "Use NeuVector to scan the image?" - type: boolean - required: false - default: false + default: 30 operating-system: description: Operating system executing the runner type: string required: false default: ubuntu-latest - trivy-enabled: - description: "Use Trivy to scan the image?" - type: boolean + severity: + description: Severities of a vulnerability with a fix that fail the scan, "CRITICAL,HIGH,MEDIUM" to hold an image to MEDIUM as well + type: string required: false - default: true + default: CRITICAL,HIGH + trivy-ignore-file: + description: Trivy ignore file, relative to the root of the repository, naming the vulnerabilities left out, each with the reason and the day it counts again (expired_at), read when it exists + type: string + required: false + default: .trivyignore.yaml workflow-parts-version: - description: "GitHub workflow parts version (branch/tag/SHA). Should match the ref/tag/SHA the caller pinned for this reusable workflow itself." + description: "GitHub workflow parts version (branch/tag/SHA). Should match the ref/tag/SHA the caller pinned for this reusable workflow itself. Ignored when the caller is this repository, whose checkout already holds the actions under test." type: string required: false default: "main" @@ -64,33 +69,45 @@ jobs: container-scan: name: ${{ inputs.job-name }} runs-on: ${{ inputs.operating-system }} + timeout-minutes: ${{ inputs.timeout-minutes }} defaults: run: working-directory: ${{ inputs.working-directory }} steps: - name: Clone repository - uses: actions/checkout@v7 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - name: Checkout workflow parts - uses: actions/checkout@v7 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: repository: devpro/github-workflow-parts - ref: ${{ inputs.workflow-parts-version }} + # The caller being this repository, the actions under test are those of the commit under test. + ref: ${{ github.repository == 'devpro/github-workflow-parts' && github.sha || inputs.workflow-parts-version }} path: workflow-parts + # A scan judges what a clean build produces today. + # A cached `apt-get upgrade` layer or a stale base image keeps an already fixed package version, on a runner whose Docker daemon outlives the job. + - name: Build base container image + if: ${{ inputs.base-image-definition != '' }} + run: docker build . --pull --no-cache --file "$BASE_DEFINITION" --tag "$BASE_REF" + shell: bash + env: + BASE_DEFINITION: ${{ inputs.base-image-definition }} + # An image built on the base above reads it from the local daemon, which a pull would look for in a registry that has never seen it. - name: Build container image - run: docker build . --file ${{ inputs.image-definition }} --tag ${{ env.IMAGE_REF }} + run: | + args=(--no-cache --file "$IMAGE_DEFINITION" --tag "$IMAGE_REF") + if [ -n "$BASE_DEFINITION" ]; then + args+=(--build-arg "${BASE_ARGUMENT}=${BASE_REF}") + else + args+=(--pull) + fi + docker build . "${args[@]}" shell: bash - # deactivated 2026-02-22: "Build container for action use... process "/bin/sh -c zypper in -y jq docker && zypper clean" did not complete successfully: exit code: 8" - # - name: Scan container image with NeuVector - # if: ${{ inputs.neuvector-enabled }} - # uses: neuvector/scan-action@main - # with: - # image-repository: ${{ inputs.image-path }}/${{ inputs.image-name }} - # image-tag: ${{ inputs.image-tag }} - # min-high-cves-to-fail: '${{ inputs.max-high-cves }}' - # min-medium-cves-to-fail: '${{ inputs.max-medium-cves }}' - # nv-scanner-image: neuvector/scanner:5 + env: + BASE_ARGUMENT: ${{ inputs.base-image-argument }} + BASE_DEFINITION: ${{ inputs.base-image-definition }} + IMAGE_DEFINITION: ${{ inputs.image-definition }} + # The only judge of the image a repository builds: any vulnerability of those severities with a fix fails the job, unless the ignore file names it until a day. - name: Scan container image with Trivy - if: ${{ inputs.trivy-enabled }} uses: ./workflow-parts/actions/trivy/scan with: scan-type: image @@ -98,8 +115,11 @@ jobs: format: table exit-code: "1" ignore-unfixed: "true" + # The default file is optional, and any other one named must exist. + ignore-file: ${{ (inputs.trivy-ignore-file != '.trivyignore.yaml' || hashFiles('.trivyignore.yaml') != '') && inputs.trivy-ignore-file || '' }} pkg-types: os,library - severity: CRITICAL,HIGH + severity: ${{ inputs.severity }} env: GITHUB_TOKEN: ${{ github.token }} IMAGE_REF: ${{ inputs.image-path }}/${{ inputs.image-name }}:${{ inputs.image-tag }} + BASE_REF: ${{ inputs.image-path }}/${{ inputs.image-name }}-base:${{ inputs.image-tag }} diff --git a/.github/workflows/reusable-dotnet-quality.yml b/.github/workflows/reusable-dotnet-quality.yml index 786b27d..14cec26 100644 --- a/.github/workflows/reusable-dotnet-quality.yml +++ b/.github/workflows/reusable-dotnet-quality.yml @@ -33,11 +33,26 @@ on: type: boolean required: false default: false + mongodb-enabled: + description: Start a local MongoDB server on 127.0.0.1:27017, with mongosh on the path, before build & test + type: boolean + required: false + default: false + mongodb-version: + description: Version of the MongoDB server mongodb-enabled starts, an 8.0 one for instance, the latest release the MongoDB action names when empty + type: string + required: false + default: "" job-name: description: Job name type: string required: false default: Quality + timeout-minutes: + description: Minutes the job may run before it is cancelled + type: number + required: false + default: 60 operating-system: description: Operating system executing the runner type: string @@ -79,7 +94,7 @@ on: required: false default: "" workflow-parts-version: - description: GitHub workflow parts version (branch/tag/SHA). Should match the ref/tag/SHA the caller pinned for this reusable workflow itself. + description: GitHub workflow parts version (branch/tag/SHA). Should match the ref/tag/SHA the caller pinned for this reusable workflow itself. Ignored when the caller is this repository, whose checkout already holds the actions under test. type: string required: false default: main @@ -103,12 +118,15 @@ jobs: dotnet-quality: name: ${{ inputs.job-name }} runs-on: ${{ inputs.operating-system }} + timeout-minutes: ${{ inputs.timeout-minutes }} defaults: run: working-directory: ${{ inputs.working-directory }} steps: - name: Set additional variables shell: bash + # Run before the clone, which creates the working directory. + working-directory: . env: EXTRA_VARS: ${{ inputs.extra-vars }} run: | @@ -117,11 +135,13 @@ jobs: fi - name: Set additional secrets shell: bash + # Run before the clone, which creates the working directory. + working-directory: . env: ADDITIONAL_VARS: ${{ secrets.additional-vars }} run: | if [[ -n "$ADDITIONAL_VARS" ]]; then - echo "$ADDITIONAL_VARS" | while IFS='=' read -r key val; do + echo "$ADDITIONAL_VARS" | while IFS='=' read -r _ val; do if [[ -n "$val" ]]; then echo "::add-mask::$val" fi @@ -133,33 +153,44 @@ jobs: run: git config --global core.autocrlf false shell: bash - name: Clone repository - uses: actions/checkout@v7 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: fetch-depth: 0 - name: Checkout workflow parts - uses: actions/checkout@v7 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: repository: devpro/github-workflow-parts - ref: ${{ inputs.workflow-parts-version }} + # The caller being this repository, the actions under test are those of the commit under test. + ref: ${{ github.repository == 'devpro/github-workflow-parts' && github.sha || inputs.workflow-parts-version }} path: workflow-parts - name: Install .NET & lint uses: ./workflow-parts/actions/dotnet/install-lint-restore with: dotnet-version: ${{ inputs.dotnet-version }} + working-directory: ${{ inputs.working-directory }} + - name: Start MongoDB + if: ${{ inputs.mongodb-enabled }} + uses: ./workflow-parts/actions/mongodb/start + with: + mongodb-version: ${{ inputs.mongodb-version }} - name: Run optional custom commands if: ${{ inputs.custom-commands != '' }} - run: | - ${{ inputs.custom-commands }} + env: + CUSTOM_COMMANDS: ${{ inputs.custom-commands }} + # Shell code by contract, run from a variable rather than pasted into the script by GitHub. + run: eval "$CUSTOM_COMMANDS" - name: Build & test if: ${{ ! inputs.sonar-enabled }} uses: ./workflow-parts/actions/dotnet/build-test with: dotnet-test-args: ${{ inputs.dotnet-test-args }} + working-directory: ${{ inputs.working-directory }} - name: Build, test & analyze if: ${{ inputs.sonar-enabled }} uses: ./workflow-parts/actions/dotnet/build-test-sonar with: dotnet-test-args: ${{ inputs.dotnet-test-args }} + working-directory: ${{ inputs.working-directory }} sonar-cpd-exclusions: ${{ inputs.sonar-cpd-exclusions }} sonar-exclusions: ${{ inputs.sonar-exclusions }} sonar-host-url: ${{ inputs.sonar-host-url }} @@ -168,6 +199,7 @@ jobs: sonar-project-name: ${{ inputs.sonar-project-name }} sonar-token: ${{ secrets.sonar-token }} - name: Check license compliance with FOSSA + # A repository turns it off where FOSSA cannot be reached, a local run holding no API key. if: ${{ inputs.fossa-enabled }} uses: ./workflow-parts/actions/fossa/analyze # https://status.fossa.com/ @@ -175,24 +207,28 @@ jobs: timeout-minutes: 3 with: api-key: "${{ secrets.fossa-api-key }}" + # The workflow parts are checked out beside the code, and are no part of it. + exclude-paths: workflow-parts run-tests: ${{ inputs.fossa-test && github.event_name == 'pull_request' }} test-diff-revision: ${{ github.event.pull_request.base.sha }} report-file: report/fossa.html + working-directory: ${{ inputs.working-directory }} - name: Generate SBOM with Syft uses: ./workflow-parts/actions/syft/generate-sbom continue-on-error: true + with: + exclude-paths: workflow-parts + target-path: ${{ inputs.working-directory }} - name: Archive test results if: always() - uses: actions/upload-artifact@v7 + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 with: name: dotnet-test-results + # The build, test and FOSSA actions write the results, the coverage and the reports in the report folder of the working directory. path: | + ${{ inputs.working-directory }}/report/ ./**/*test-result.xml - ./test/*/TestResults/*/coverage.cobertura.xml ./**/failure_*.png* - ./**/SonarQube.xml - ./**/Summary.txt - ./**/fossa.html env: DOTNET_CLI_TELEMETRY_OPTOUT: 1 DOTNET_NOLOGO: 1 diff --git a/.github/workflows/reusable-git-check.yml b/.github/workflows/reusable-git-check.yml new file mode 100644 index 0000000..4f69d10 --- /dev/null +++ b/.github/workflows/reusable-git-check.yml @@ -0,0 +1,94 @@ +name: Reusable - Git check + +on: + workflow_call: + inputs: + job-name: + description: Job name + type: string + required: false + default: Check + timeout-minutes: + description: Minutes the job may run before it is cancelled + type: number + required: false + default: 10 + operating-system: + description: Operating system executing the runner + type: string + required: false + default: ubuntu-latest + paths: + description: Paths whose change the changed output tells, separated by spaces + type: string + required: false + default: "src test" + version-file: + description: MSBuild file whose VersionPrefix gives the version-major-minor output (Directory.Build.props for instance), none when empty + type: string + required: false + default: "" + workflow-parts-version: + description: "GitHub workflow parts version (branch/tag/SHA). Should match the ref/tag/SHA the caller pinned for this reusable workflow itself. Ignored when the caller is this repository, whose checkout already holds the actions under test." + type: string + required: false + default: "main" + outputs: + branch-slug: + description: Branch name without its first segment (feature/, release/), lower case, with only letters, digits and dashes, to name an environment + value: ${{ jobs.git-check.outputs.branch-slug }} + changed: + description: "\"true\" when the commit changes one of the paths, or when there is no base to compare with" + value: ${{ jobs.git-check.outputs.changed }} + version-major-minor: + description: Major and minor parts of the VersionPrefix of the version file, to tag an image + value: ${{ jobs.git-check.outputs.version-major-minor }} + +jobs: + git-check: + name: ${{ inputs.job-name }} + runs-on: ${{ inputs.operating-system }} + timeout-minutes: ${{ inputs.timeout-minutes }} + outputs: + branch-slug: ${{ steps.slug.outputs.branch-slug }} + changed: ${{ steps.changes.outputs.changed }} + version-major-minor: ${{ steps.version.outputs.version-major-minor }} + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + # The base of the comparison is in the history. + fetch-depth: 0 + - name: Checkout workflow parts + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: devpro/github-workflow-parts + # The caller being this repository, the actions under test are those of the commit under test. + ref: ${{ github.repository == 'devpro/github-workflow-parts' && github.sha || inputs.workflow-parts-version }} + path: workflow-parts + - name: Check changes + id: changes + uses: ./workflow-parts/actions/git/check-changes + with: + paths: ${{ inputs.paths }} + - name: Read the version + id: version + if: inputs.version-file != '' + env: + VERSION_FILE: ${{ inputs.version-file }} + run: | + version=$(grep -oP '(?<=)[^<]+' "$VERSION_FILE" | head -n 1 | tr -d '[:space:]') + if [ -z "$version" ]; then + echo "ERROR: no VersionPrefix in ${VERSION_FILE}" >&2 + exit 1 + fi + echo "Version ${version}" + echo "version-major-minor=$(cut -d . -f 1-2 <<< "$version")" >> "$GITHUB_OUTPUT" + # A pull request names its branch in GITHUB_HEAD_REF, GITHUB_REF_NAME holding its merge ref. + - name: Name the branch + id: slug + run: | + branch="${GITHUB_HEAD_REF:-$GITHUB_REF_NAME}" + slug=$(tr '/' '-' <<< "${branch#*/}" | tr '[:upper:]' '[:lower:]' | tr -cd 'a-z0-9-') + echo "Branch ${branch}, slug ${slug}" + echo "branch-slug=${slug}" >> "$GITHUB_OUTPUT" diff --git a/.github/workflows/reusable-markup-lint.yml b/.github/workflows/reusable-markup-lint.yml index 54e8dd1..8839e3a 100644 --- a/.github/workflows/reusable-markup-lint.yml +++ b/.github/workflows/reusable-markup-lint.yml @@ -8,6 +8,11 @@ on: type: string required: false default: "Lint" + timeout-minutes: + description: Minutes the job may run before it is cancelled + type: number + required: false + default: 10 operating-system: description: "Operating system executing the runner" type: string @@ -23,13 +28,15 @@ jobs: markup-lint: name: ${{ inputs.job-name }} runs-on: ${{ inputs.operating-system }} + timeout-minutes: ${{ inputs.timeout-minutes }} defaults: run: working-directory: ${{ inputs.working-directory }} steps: - name: Check out repository - uses: actions/checkout@v7 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + # The linters are pinned, so that a new release changes the verdict on a repository only when this one moves to it. - name: Lint Markdown files - run: npx markdownlint-cli2 "**/*.md" + run: npx --yes markdownlint-cli2@0.24.0 "**/*.md" - name: Lint YAML files - run: pipx run yamllint . + run: pipx run --spec yamllint==1.38.0 yamllint . diff --git a/.github/workflows/reusable-node-quality.yml b/.github/workflows/reusable-node-quality.yml new file mode 100644 index 0000000..d360e7c --- /dev/null +++ b/.github/workflows/reusable-node-quality.yml @@ -0,0 +1,323 @@ +name: Reusable - Node.js quality + +on: + workflow_call: + inputs: + custom-tests: + description: Optional shell commands testing what build & test produced, such as end-to-end tests against the built application + type: string + required: false + default: "" + extra-vars: + description: "Additional environment variables at the start of the pipeline" + type: string + required: false + default: "" + fossa-enabled: + description: Enable license compliance with FOSSA + type: boolean + required: false + default: false + fossa-test: + description: Run FOSSA test on PR + type: boolean + required: false + default: false + job-name: + description: Job name + type: string + required: false + default: Quality + mongodb-enabled: + description: Start a local MongoDB server on 127.0.0.1:27017, as the single-node replica set rs0 transactions need, with mongosh on the path, before lint, build & test + type: boolean + required: false + default: false + mongodb-version: + description: Version of the MongoDB server mongodb-enabled starts, an 8.0 one for instance, the latest release the MongoDB action names when empty + type: string + required: false + default: "" + node-version: + description: Node.js version + type: string + required: false + default: "24" + operating-system: + description: Operating system executing the runner + type: string + required: false + default: ubuntu-latest + pnpm-version: + description: Version of pnpm installing the dependencies from pnpm-lock.yaml, npm installing them from package-lock.json when empty + type: string + required: false + default: "" + sonar-enabled: + description: Enable check done by Sonar (SonarQube or SonarCloud) + type: boolean + required: false + default: false + sonar-coverage-paths: + description: lcov reports of the coverage the test script writes, relative to the working directory, separated by commas, coverage/lcov.info when empty and the project has no sonar-project.properties + type: string + required: false + default: "" + sonar-cpd-exclusions: + description: Files or directories that should be excluded from Sonar code duplication analysis + type: string + required: false + default: "" + sonar-exclusions: + description: Files or directories that should be excluded from Sonar analysis, **/node_modules/** when empty and the project has no sonar-project.properties + type: string + required: false + default: "" + sonar-host-url: + description: Sonar host URL + type: string + required: false + default: "https://sonarcloud.io" + sonar-organization: + description: Sonar organization + type: string + required: false + default: "" + sonar-project-key: + description: Sonar project key, which sonar-project.properties gives otherwise + type: string + required: false + default: "" + sonar-project-name: + description: Sonar project name + type: string + required: false + default: "" + task-enabled: + description: Install Task (taskfile.dev) for the scripts and the custom tests + type: boolean + required: false + default: false + terraform-version: + description: Version of Terraform to install for the scripts and the custom tests, none when empty + type: string + required: false + default: "" + test-script: + description: Script of package.json running the tests, for example "test:coverage" to write the coverage Sonar reads + type: string + required: false + default: test + timeout-minutes: + description: Minutes the job may run before it is cancelled + type: number + required: false + default: 60 + workflow-parts-version: + description: GitHub workflow parts version (branch/tag/SHA). Should match the ref/tag/SHA the caller pinned for this reusable workflow itself. Ignored when the caller is this repository, whose checkout already holds the actions under test. + type: string + required: false + default: main + working-directory: + description: Directory holding the package.json and the lock file + type: string + required: false + default: "." + secrets: + fossa-api-key: + description: FOSSA API KEY + required: false + sonar-token: + description: Sonar token for login + required: false + additional-vars: + description: "Additional variables" + required: false + +jobs: + node-quality: + name: ${{ inputs.job-name }} + runs-on: ${{ inputs.operating-system }} + timeout-minutes: ${{ inputs.timeout-minutes }} + defaults: + run: + shell: bash + working-directory: ${{ inputs.working-directory }} + steps: + - name: Set additional variables + # Run before the clone, which creates the working directory. + working-directory: . + env: + EXTRA_VARS: ${{ inputs.extra-vars }} + run: | + if [[ -n "$EXTRA_VARS" ]]; then + echo "$EXTRA_VARS" >> "$GITHUB_ENV" + fi + - name: Set additional secrets + # Run before the clone, which creates the working directory. + working-directory: . + env: + ADDITIONAL_VARS: ${{ secrets.additional-vars }} + run: | + if [[ -n "$ADDITIONAL_VARS" ]]; then + echo "$ADDITIONAL_VARS" | while IFS='=' read -r _ val; do + if [[ -n "$val" ]]; then + echo "::add-mask::$val" + fi + done + echo "$ADDITIONAL_VARS" >> "$GITHUB_ENV" + fi + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: 0 + - name: Checkout workflow parts + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: devpro/github-workflow-parts + # The caller being this repository, the actions under test are those of the commit under test. + ref: ${{ github.repository == 'devpro/github-workflow-parts' && github.sha || inputs.workflow-parts-version }} + path: workflow-parts + persist-credentials: false + # Git ignores them, and so does Sonar, which leaves out what git ignores, without overriding the exclusions of the project. + - name: Leave workflow parts out of git + working-directory: . + run: | + exclude="$(git rev-parse --git-path info/exclude)" + mkdir -p "$(dirname "$exclude")" + echo "/workflow-parts/" >> "$exclude" + - name: Setup Node.js + uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + with: + node-version: ${{ inputs.node-version }} + # The npm cache is keyed on the lock file, which a package without dependencies does not have. + cache: ${{ inputs.pnpm-version == '' && hashFiles(format('{0}/package-lock.json', inputs.working-directory)) != '' && 'npm' || '' }} + cache-dependency-path: ${{ inputs.working-directory }}/package-lock.json + # pnpm installed with the npm of the Node set up above, rather than through a third party action. + - name: Setup pnpm + id: pnpm + if: inputs.pnpm-version != '' + env: + PNPM_VERSION: ${{ inputs.pnpm-version }} + run: | + # An image already carrying that version, as a container job's may, needs no install. + current=$(pnpm --version 2>/dev/null || true) + if [[ "$current" != "$PNPM_VERSION" && "$current" != "$PNPM_VERSION".* ]]; then + npm install --global "pnpm@${PNPM_VERSION}" + fi + echo "store-path=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT" + - name: Cache pnpm store + if: inputs.pnpm-version != '' + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: ${{ steps.pnpm.outputs.store-path }} + key: pnpm-${{ runner.os }}-${{ hashFiles(format('{0}/pnpm-lock.yaml', inputs.working-directory)) }} + restore-keys: pnpm-${{ runner.os }}- + - name: Install dependencies + id: install + env: + PNPM_VERSION: ${{ inputs.pnpm-version }} + run: | + if [[ -n "$PNPM_VERSION" ]]; then + pnpm install --frozen-lockfile + elif [[ -f package-lock.json ]]; then + npm ci + else + echo "No package-lock.json, nothing to install" + fi + # The browsers have to match the Playwright version installed, so a project depending on it gets them without asking. + if [[ -x node_modules/.bin/playwright ]]; then + echo "playwright=true" >> "$GITHUB_OUTPUT" + fi + - name: Install Playwright browsers + if: steps.install.outputs.playwright == 'true' + uses: ./workflow-parts/actions/playwright/install + with: + working-directory: ${{ inputs.working-directory }} + - name: Setup Task + if: ${{ inputs.task-enabled }} + uses: ./workflow-parts/actions/task/setup + - name: Setup Terraform + if: inputs.terraform-version != '' + uses: ./workflow-parts/actions/terraform/setup + with: + terraform-version: ${{ inputs.terraform-version }} + - name: Start MongoDB + if: ${{ inputs.mongodb-enabled }} + uses: ./workflow-parts/actions/mongodb/start + with: + replica-set: rs0 + mongodb-version: ${{ inputs.mongodb-version }} + # The scripts lint, format and test the repository, which the workflow parts are no part of. + - name: Set aside workflow parts + working-directory: . + run: mv workflow-parts "${RUNNER_TEMP}/workflow-parts" + - name: Lint + env: + PACKAGE_MANAGER: ${{ inputs.pnpm-version != '' && 'pnpm' || 'npm' }} + run: | + for script in lint format:check typecheck; do + "$PACKAGE_MANAGER" run --if-present "$script" + done + - name: Build & test + env: + PACKAGE_MANAGER: ${{ inputs.pnpm-version != '' && 'pnpm' || 'npm' }} + TEST_SCRIPT: ${{ inputs.test-script }} + run: | + "$PACKAGE_MANAGER" run --if-present build + "$PACKAGE_MANAGER" run --if-present "$TEST_SCRIPT" + - name: Run optional custom tests + if: ${{ inputs.custom-tests != '' }} + env: + CUSTOM_TESTS: ${{ inputs.custom-tests }} + # Shell code by contract, run from a variable rather than pasted into the script by GitHub. + run: eval "$CUSTOM_TESTS" + - name: Bring back workflow parts + working-directory: . + run: mv "${RUNNER_TEMP}/workflow-parts" workflow-parts + - name: Analyze with Sonar + if: ${{ inputs.sonar-enabled }} + uses: ./workflow-parts/actions/sonar/scan + with: + coverage-paths: ${{ inputs.sonar-coverage-paths }} + cpd-exclusions: ${{ inputs.sonar-cpd-exclusions }} + exclusions: ${{ inputs.sonar-exclusions }} + host-url: ${{ inputs.sonar-host-url }} + organization: ${{ inputs.sonar-organization }} + project-key: ${{ inputs.sonar-project-key }} + project-name: ${{ inputs.sonar-project-name }} + token: ${{ secrets.sonar-token }} + working-directory: ${{ inputs.working-directory }} + - name: Check license compliance with FOSSA + # A repository turns it off where FOSSA cannot be reached, a local run holding no API key. + if: ${{ inputs.fossa-enabled }} + uses: ./workflow-parts/actions/fossa/analyze + # https://status.fossa.com/ + continue-on-error: true + timeout-minutes: 3 + with: + api-key: "${{ secrets.fossa-api-key }}" + # The workflow parts are checked out beside the code, and are no part of it. + exclude-paths: workflow-parts + run-tests: ${{ inputs.fossa-test && github.event_name == 'pull_request' }} + test-diff-revision: ${{ github.event.pull_request.base.sha }} + report-file: report/fossa.html + working-directory: ${{ inputs.working-directory }} + - name: Generate SBOM with Syft + uses: ./workflow-parts/actions/syft/generate-sbom + continue-on-error: true + with: + exclude-paths: workflow-parts + target-path: ${{ inputs.working-directory }} + - name: Archive test results + if: always() + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: node-test-results + # The usual folders of the coverage and of Playwright, those of a workspace included, and report for anything else, a custom test's server log for instance. + path: | + ${{ inputs.working-directory }}/report/ + ${{ inputs.working-directory }}/coverage/ + ${{ inputs.working-directory }}/**/playwright-report/ + ${{ inputs.working-directory }}/**/test-results/ + !${{ inputs.working-directory }}/**/node_modules/** diff --git a/.github/workflows/reusable-npm-publication.yml b/.github/workflows/reusable-npm-publication.yml new file mode 100644 index 0000000..d356f99 --- /dev/null +++ b/.github/workflows/reusable-npm-publication.yml @@ -0,0 +1,135 @@ +name: Reusable - npm publication + +# The caller grants "packages: write" to publish to GitHub Packages, or "id-token: write" for npm trusted publishing to npmjs.org. +on: + workflow_call: + inputs: + build-command: + description: Commands building and checking the package before it is published, run from the root of the repository, the build script of its package.json by default, none when empty + type: string + required: false + # A repository whose root holds no package.json has nothing npm could build there. + default: "if [ -f package.json ]; then npm run --if-present build; fi" + job-name: + description: Job name + type: string + required: false + default: Publication + node-version: + description: Version of Node.js to install + type: string + required: false + default: "24" + operating-system: + description: Operating system executing the runner + type: string + required: false + default: ubuntu-latest + pnpm-version: + description: Version of pnpm installing the dependencies from pnpm-lock.yaml, npm installing them from package-lock.json when empty + type: string + required: false + default: "" + publish-command: + description: Command publishing the package, "pnpm publish --no-git-checks" for a pnpm workspace for instance + type: string + required: false + default: "npm publish" + registry-url: + description: Registry to publish to, GitHub Packages (https://npm.pkg.github.com) being authenticated with the token of the run, npmjs.org with the OIDC token of trusted publishing + type: string + required: false + default: "https://registry.npmjs.org" + scope: + description: Scope of the package, "@devpro" for instance, which GitHub Packages requires + type: string + required: false + default: "" + timeout-minutes: + description: Minutes the job may run before it is cancelled + type: number + required: false + default: 20 + workflow-parts-version: + description: "GitHub workflow parts version (branch/tag/SHA). Should match the ref/tag/SHA the caller pinned for this reusable workflow itself. Ignored when the caller is this repository, whose checkout already holds the actions under test." + type: string + required: false + default: "main" + working-directory: + description: Directory holding the package.json of the package, relative to the root of the repository + type: string + required: false + default: "." + outputs: + published: + description: "\"true\" when the package was published, \"false\" when its version already was" + value: ${{ jobs.npm-publication.outputs.published }} + version: + description: Version of the package, read from its package.json + value: ${{ jobs.npm-publication.outputs.version }} + +jobs: + npm-publication: + name: ${{ inputs.job-name }} + runs-on: ${{ inputs.operating-system }} + timeout-minutes: ${{ inputs.timeout-minutes }} + outputs: + published: ${{ steps.publish.outputs.published }} + version: ${{ steps.publish.outputs.version }} + steps: + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Checkout workflow parts + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: devpro/github-workflow-parts + # The caller being this repository, the actions under test are those of the commit under test. + ref: ${{ github.repository == 'devpro/github-workflow-parts' && github.sha || inputs.workflow-parts-version }} + path: workflow-parts + persist-credentials: false + - name: Setup Node.js + uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + with: + node-version: ${{ inputs.node-version }} + # Writes the .npmrc that npm publish and npm view authenticate with. + registry-url: ${{ inputs.registry-url }} + scope: ${{ inputs.scope }} + # A cache restored into a publication could carry what another branch put in it. + package-manager-cache: false + # pnpm installed with the npm of the Node set up above, rather than through a third party action. + - name: Setup pnpm + if: inputs.pnpm-version != '' + env: + PNPM_VERSION: ${{ inputs.pnpm-version }} + run: | + # An image already carrying that version, as a container job's may, needs no install. + current=$(pnpm --version 2>/dev/null || true) + if [[ "$current" != "$PNPM_VERSION" && "$current" != "$PNPM_VERSION".* ]]; then + npm install --global "pnpm@${PNPM_VERSION}" + fi + - name: Install dependencies + env: + PNPM_VERSION: ${{ inputs.pnpm-version }} + run: | + if [[ -n "$PNPM_VERSION" ]]; then + pnpm install --frozen-lockfile + elif [[ -f package-lock.json ]]; then + npm ci + else + echo "No package-lock.json, nothing to install" + fi + - name: Build the package + if: inputs.build-command != '' + env: + BUILD_COMMAND: ${{ inputs.build-command }} + # Shell code by contract, run from a variable rather than pasted into the script by GitHub. + run: eval "$BUILD_COMMAND" + # GitHub Packages takes the token of the run, and npmjs.org the short-lived OIDC token of trusted publishing, which npm asks for itself. + - name: Publish the package + id: publish + uses: ./workflow-parts/actions/npm/publish + with: + publish-command: ${{ inputs.publish-command }} + working-directory: ${{ inputs.working-directory }} + env: + NODE_AUTH_TOKEN: ${{ contains(inputs.registry-url, 'npm.pkg.github.com') && github.token || '' }} diff --git a/.github/workflows/reusable-pages-deployment.yml b/.github/workflows/reusable-pages-deployment.yml new file mode 100644 index 0000000..85b2dd4 --- /dev/null +++ b/.github/workflows/reusable-pages-deployment.yml @@ -0,0 +1,155 @@ +name: Reusable - Pages deployment + +# The caller grants the deployment job "pages: write" and "id-token: write", and the repository publishes its Pages from GitHub Actions. +on: + workflow_call: + inputs: + build-command: + description: Commands building the site into site-path, run from the working directory, npm ci (npm install without a lock file) then npm run build when empty + type: string + required: false + default: "" + deploy: + description: Deploy the site once built, false building it only, to check a pull request for instance + type: boolean + required: false + default: true + fetch-depth: + description: Number of commits to clone, 0 for the whole history a site reading the date of each page needs + type: number + required: false + default: 1 + job-name: + description: Job name + type: string + required: false + default: Build + node-version: + description: Version of Node.js to install, with the npm cache of the lock file when there is one, none when empty + type: string + required: false + default: "24" + operating-system: + description: Operating system executing the runner + type: string + required: false + default: ubuntu-latest + site-path: + description: Directory the build writes the site to, relative to the working directory + type: string + required: true + task-enabled: + description: Install Task (taskfile.dev) for the build command + type: boolean + required: false + default: false + timeout-minutes: + description: Minutes the job may run before it is cancelled + type: number + required: false + default: 30 + workflow-parts-version: + description: "GitHub workflow parts version (branch/tag/SHA). Should match the ref/tag/SHA the caller pinned for this reusable workflow itself. Ignored when the caller is this repository, whose checkout already holds the actions under test." + type: string + required: false + default: "main" + working-directory: + description: Working directory + type: string + required: false + default: "." + outputs: + page-url: + description: URL of the deployed site + value: ${{ jobs.pages-deploy.outputs.page-url }} + secrets: + additional-vars: + description: Secret environment variables of the build command, one KEY=value per line, masked in the log + required: false + +jobs: + pages-build: + name: ${{ inputs.job-name }} + runs-on: ${{ inputs.operating-system }} + timeout-minutes: ${{ inputs.timeout-minutes }} + defaults: + run: + working-directory: ${{ inputs.working-directory }} + steps: + - name: Set additional secrets + # Run before the clone, which creates the working directory. + working-directory: . + env: + ADDITIONAL_VARS: ${{ secrets.additional-vars }} + run: | + if [[ -n "$ADDITIONAL_VARS" ]]; then + echo "$ADDITIONAL_VARS" | while IFS='=' read -r _ val; do + if [[ -n "$val" ]]; then + echo "::add-mask::$val" + fi + done + echo "$ADDITIONAL_VARS" >> "$GITHUB_ENV" + fi + - name: Clone repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: ${{ inputs.fetch-depth }} + - name: Checkout workflow parts + if: inputs.task-enabled + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: devpro/github-workflow-parts + # The caller being this repository, the actions under test are those of the commit under test. + ref: ${{ github.repository == 'devpro/github-workflow-parts' && github.sha || inputs.workflow-parts-version }} + path: workflow-parts + - name: Setup Task + if: inputs.task-enabled + uses: ./workflow-parts/actions/task/setup + - name: Setup Node.js + if: inputs.node-version != '' + uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + with: + node-version: ${{ inputs.node-version }} + # The npm cache is keyed on the lock file, which a site installing its tools globally does not have. + cache: ${{ hashFiles(format('{0}/package-lock.json', inputs.working-directory)) != '' && 'npm' || '' }} + cache-dependency-path: ${{ inputs.working-directory }}/package-lock.json + - name: Build the site + env: + BUILD_COMMAND: ${{ inputs.build-command }} + # Shell code by contract, run from a variable rather than pasted into the script by GitHub. + run: | + if [[ -n "$BUILD_COMMAND" ]]; then + eval "$BUILD_COMMAND" + else + if [[ -f package-lock.json ]]; then + npm ci + else + npm install + fi + npm run build + fi + - name: Verify the site was built + env: + SITE_PATH: ${{ inputs.site-path }} + run: test -s "${SITE_PATH}/index.html" + - name: Upload the site + if: inputs.deploy + uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5.0.0 + with: + path: ${{ inputs.working-directory }}/${{ inputs.site-path }} + + pages-deploy: + name: Deploy + if: inputs.deploy + needs: pages-build + runs-on: ${{ inputs.operating-system }} + timeout-minutes: ${{ inputs.timeout-minutes }} + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + outputs: + page-url: ${{ steps.deployment.outputs.page_url }} + steps: + - name: Deploy to GitHub Pages + id: deployment + uses: actions/deploy-pages@368f82528645a54fb793d4d04e342629a3f51346 # v5.0.1 diff --git a/.github/workflows/reusable-terraform-deployment.yml b/.github/workflows/reusable-terraform-deployment.yml index c2bc2ed..727cc6e 100644 --- a/.github/workflows/reusable-terraform-deployment.yml +++ b/.github/workflows/reusable-terraform-deployment.yml @@ -1,4 +1,4 @@ -name: Reusable - Terraform apply +name: Reusable - Terraform apply on: workflow_call: @@ -8,6 +8,11 @@ on: type: string required: false default: "Deploy" + timeout-minutes: + description: Minutes the job may run before it is cancelled + type: number + required: false + default: 60 working-directory: description: "Working directory" type: string @@ -18,6 +23,11 @@ on: type: string required: false default: "ubuntu-latest" + destroy: + description: "Destroy the resources of the environment rather than apply the configuration" + type: boolean + required: false + default: false environment: description: "GitHub environment" type: string @@ -37,7 +47,7 @@ on: required: false default: "" workflow-parts-version: - description: "GitHub workflow parts version (branch/tag/SHA). Should match the ref/tag/SHA the caller pinned for this reusable workflow itself." + description: "GitHub workflow parts version (branch/tag/SHA). Should match the ref/tag/SHA the caller pinned for this reusable workflow itself. Ignored when the caller is this repository, whose checkout already holds the actions under test." type: string required: false default: "main" @@ -49,7 +59,7 @@ on: description: "Atlas private key" required: true atlas-groupid: - description: "Atlas group IP" + description: "Atlas group ID" required: true tfbackend-connstring: description: "Terraform backend connection string" @@ -74,6 +84,7 @@ jobs: terraform-apply: name: ${{ inputs.job-name }} runs-on: ${{ inputs.operating-system }} + timeout-minutes: ${{ inputs.timeout-minutes }} environment: ${{ inputs.environment }} defaults: run: @@ -90,12 +101,13 @@ jobs: - 8080:8080 steps: - name: Clone repository - uses: actions/checkout@v7 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - name: Checkout workflow parts - uses: actions/checkout@v7 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: repository: devpro/github-workflow-parts - ref: ${{ inputs.workflow-parts-version }} + # The caller being this repository, the actions under test are those of the commit under test. + ref: ${{ github.repository == 'devpro/github-workflow-parts' && github.sha || inputs.workflow-parts-version }} path: workflow-parts - name: Add runner ID to MongoDB Atlas uses: ./workflow-parts/actions/mongodb-atlas/add-runner-ip @@ -104,23 +116,25 @@ jobs: atlas-privatekey: ${{ secrets.atlas-privatekey }} atlas-groupid: ${{ secrets.atlas-groupid }} - name: Set additional variables + env: + ADDITIONAL_VARS: ${{ secrets.additional-vars }} run: | - if [[ -z "${{ secrets.additional-vars }}" ]]; then - echo "No additional-vars bundle provided - skipping." - else - echo "${{ secrets.additional-vars }}" | while IFS='=' read -r key val; do + if [[ -n "$ADDITIONAL_VARS" ]]; then + echo "$ADDITIONAL_VARS" | while IFS='=' read -r _ val; do if [[ -n "$val" ]]; then echo "::add-mask::$val" fi done - echo "${{ secrets.additional-vars }}" >> "$GITHUB_ENV" + echo "$ADDITIONAL_VARS" >> "$GITHUB_ENV" fi - name: Run optional custom commands if: ${{ inputs.custom-commands != '' }} - run: | - ${{ inputs.custom-commands }} + env: + CUSTOM_COMMANDS: ${{ inputs.custom-commands }} + # Shell code by contract, run from a variable rather than pasted into the script by GitHub. + run: eval "$CUSTOM_COMMANDS" - name: Cache Terraform plugins - uses: actions/cache@v6 + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: | ~/.terraform.d/plugin-cache @@ -132,7 +146,15 @@ jobs: - name: Terraform validate run: terraform validate - name: Terraform plan - run: terraform plan -input=false -out=plan.tfplan ${{ inputs.terraform-var-flags }} + # Shell words by contract, quotes included, parsed from a variable rather than pasted into the script by GitHub. + run: | + var_flags=() + eval "var_flags=(${TERRAFORM_VAR_FLAGS})" + terraform plan -input=false -out=plan.tfplan "${var_flags[@]}" + env: + TERRAFORM_VAR_FLAGS: ${{ inputs.terraform-var-flags }} + # Read by Terraform as arguments of the plan command, the plan of a destruction being applied as any other. + TF_CLI_ARGS_plan: ${{ inputs.destroy && '-destroy' || '' }} - name: Terraform apply run: terraform apply -auto-approve plan.tfplan env: diff --git a/.github/workflows/reusable-terraform-quality.yml b/.github/workflows/reusable-terraform-quality.yml index 4e41fbc..d16b654 100644 --- a/.github/workflows/reusable-terraform-quality.yml +++ b/.github/workflows/reusable-terraform-quality.yml @@ -8,6 +8,11 @@ on: type: string required: false default: "Validate" + timeout-minutes: + description: Minutes the job may run before it is cancelled + type: number + required: false + default: 30 working-directory: description: "Working directory" type: string @@ -19,7 +24,7 @@ on: required: false default: "ubuntu-latest" workflow-parts-version: - description: "GitHub workflow parts version (branch/tag/SHA). Should match the ref/tag/SHA the caller pinned for this reusable workflow itself." + description: "GitHub workflow parts version (branch/tag/SHA). Should match the ref/tag/SHA the caller pinned for this reusable workflow itself. Ignored when the caller is this repository, whose checkout already holds the actions under test." type: string required: false default: "main" @@ -28,20 +33,22 @@ jobs: terraform-validate: name: ${{ inputs.job-name }} runs-on: ${{ inputs.operating-system }} + timeout-minutes: ${{ inputs.timeout-minutes }} defaults: run: working-directory: ${{ inputs.working-directory }} steps: - name: Clone repository - uses: actions/checkout@v7 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - name: Checkout workflow parts - uses: actions/checkout@v7 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: repository: devpro/github-workflow-parts - ref: ${{ inputs.workflow-parts-version }} + # The caller being this repository, the actions under test are those of the commit under test. + ref: ${{ github.repository == 'devpro/github-workflow-parts' && github.sha || inputs.workflow-parts-version }} path: workflow-parts - name: Cache Terraform plugins - uses: actions/cache@v6 + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: | ~/.terraform.d/plugin-cache @@ -57,20 +64,20 @@ jobs: # Checkov is a static code analysis tool for infrastructure as code (IaC) and also a software composition analysis (SCA) tool for images and open source packages - name: Run Checkov run: | - pipx run checkov -d . \ + pipx run --spec checkov==3.3.30 checkov -d . \ --soft-fail \ --output cli --output sarif \ --output-file-path console,results.sarif continue-on-error: true # needs GitHub code security > code scanning, not available on private repos # - name: Upload SARIF file - # uses: github/codeql-action/upload-sarif@v3 + # uses: github/codeql-action/upload-sarif@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1 # if: success() || failure() # with: # sarif_file: results.sarif # TFLint is a pluggable terraform linter (ref. https://github.com/terraform-linters/tflint) - name: Cache TFLint plugins - uses: actions/cache@v6 + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: ~/.tflint.d/plugins key: tflint-${{ hashFiles('**/.tflint.hcl') }} @@ -78,7 +85,7 @@ jobs: uses: ./workflow-parts/actions/tflint/setup with: # ref. https://github.com/terraform-linters/tflint/releases - tflint-version: v0.60.0 + tflint-version: v0.65.0 - name: Initialize TFLint run: tflint --init --recursive env: @@ -95,7 +102,7 @@ jobs: output-file: trivy-results.sarif severity: HIGH,CRITICAL - name: Upload SARIF as artifact - uses: actions/upload-artifact@v7 + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 if: always() with: name: security-results diff --git a/.istarci.yml b/.istarci.yml new file mode 100644 index 0000000..0df94be --- /dev/null +++ b/.istarci.yml @@ -0,0 +1,27 @@ +# IstarCI configuration, read from the root of the repository. + +runner: + # The .NET jobs need the SDK, which a hosted runner has and the default image does not. + images: + "test-dotnet-*": ghcr.io/devpro/ubuntu-dotnet:latest + # The npm test runs npm, and the Playwright one needs the system packages of the browser, which a local run cannot install without root. + test-npm-publish: ghcr.io/devpro/debian-node:latest + test-playwright-install: ghcr.io/devpro/debian-node-playwright:latest + test-node-quality-npm: ghcr.io/devpro/debian-node-playwright:latest + test-node-quality-pnpm: ghcr.io/devpro/debian-node:latest + test-node-quality-sonar: ghcr.io/devpro/debian-node-playwright:latest + test-npm-publication: ghcr.io/devpro/debian-node:latest + # The Go actions need the Go toolchain, which setup-go installs on GitHub and a local run takes from the SUSE image, rebuilt with the Debian fixes the official one still lacks at times. + test-go-lint-govulncheck-scan: registry.suse.com/bci/golang:1.27 + # The container scan workflow builds an image through the host Docker daemon, as a hosted runner does, and Cosign reads a digest with the Docker CLI. + docker: ["test-container-scan-on-base*", "test-cosign-sign*", "test-container-build-push*"] + # The Sonar paths of the .NET and Node.js quality workflows and of the Sonar action run against a SonarQube of this machine, which GitHub does not have. + sonarqube: true + +workflow: + # Registering an emulator changes the kernel of the machine, which a local run leaves alone. + skipJobs: [test-qemu-setup] + +vars: + # The local SonarQube has no organization, which only SonarCloud needs. + SONAR_ORGANIZATION: "" diff --git a/AGENTS.md b/AGENTS.md index 93e9423..8f1a556 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -10,6 +10,8 @@ Linux or WSL2, Docker and `bash`. No background commands, no subagents, no forks, no parallel tasks, even for a long commands. - Headers stay: short documentation still has sections. - Linters for YAML and Markdown are never run by an agent. +- The IstarCI daemon runs the pipeline on every commit by itself, so an agent never starts a run with `istarci run`. + `istarci wait` waits for the run of `HEAD` and prints the log of each job that failed, and the `pre-push` hook gates a push on it. - Commit only when asked, and never push. Shell scripts are `snake_case` and committed with the executable bit (`git update-index --chmod=+x`). - Documentation is as short as possible. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..e45afeb --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,17 @@ +# Contributing + +## Run the pipeline locally + +[IstarCI](https://github.com/devpro/istarci) runs the GitHub Actions pipeline in Docker before the push. +`istarci add` and `istarci install-hook` set it up for this repository. + +### Jobs left to GitHub + +`test-qemu-setup` registers an emulator in the kernel, so `.istarci.yml` skips it, as a local run leaves the machine unchanged. +The jobs on `ubuntu-24.04-arm` run arm64 binaries, the tools installed and the `RUN` steps of the images built, which only an arm64 processor executes natively. +IstarCI runs them on an arm64 machine, emulated on an amd64 one with a `qemu-aarch64` registration holding the F flag (from `qemu-user-static`), and skips them with a warning otherwise. + +## Run the static checks + +`test/run_static_checks.sh` runs actionlint, shellcheck on the scripts of the composite actions, and the conventions of `test/check_workflow_parts.py`. +A part no CI job can exercise is listed in `UNTESTED_PARTS` of that file, with the reason. diff --git a/README.md b/README.md index cf9a308..282eea4 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ GitHub workflow components for code repositories. > [!IMPORTANT] -> Keep your pipelines **safe** and **DRY**! +> Keep pipelines **safe** and **DRY**! ## Reusable workflows @@ -15,21 +15,34 @@ GitHub workflow components for code repositories. Containers: -- [Publish your image](.github/workflows/reusable-container-publication.yml) -- [Scan your image](.github/workflows/reusable-container-scan.yml) +- [Publish an image](.github/workflows/reusable-container-publication.yml) +- [Scan an image](.github/workflows/reusable-container-scan.yml) .NET: -- [Ensure the quality of your code](.github/workflows/reusable-dotnet-quality.yml) +- [Ensure the quality of the code](.github/workflows/reusable-dotnet-quality.yml) + +Git: + +- [Tell what changed, the version and the branch slug, for the jobs that follow](.github/workflows/reusable-git-check.yml) + +GitHub Pages: + +- [Build a site and deploy it](.github/workflows/reusable-pages-deployment.yml) Markup (Markdown, YAML): -- [Ensure the quality of your content](.github/workflows/reusable-markup-lint.yml) +- [Ensure the quality of the content](.github/workflows/reusable-markup-lint.yml) + +Node.js: + +- [Lint, build, test and analyze the code](.github/workflows/reusable-node-quality.yml) +- [Publish a package to npmjs.org or GitHub Packages](.github/workflows/reusable-npm-publication.yml) Terraform: -- [Deploy your environment](.github/workflows/reusable-terraform-deployment.yml) -- [Ensure the quality of your infrastructure code](.github/workflows/reusable-terraform-quality.yml) +- [Deploy or destroy an environment](.github/workflows/reusable-terraform-deployment.yml) +- [Ensure the quality of the infrastructure code](.github/workflows/reusable-terraform-quality.yml) ## Composite actions @@ -39,23 +52,76 @@ Terraform: Containers: -- [Sign your image with Cosign](actions/cosign/sign/action.yml) +- [Build and push an image for one or more platforms](actions/container/build-push/action.yml) +- [Sign an image with Cosign](actions/cosign/sign/action.yml) +- [Setup QEMU to build images for other platforms](actions/qemu/setup/action.yml) +- [Scan an image or a directory with Trivy](actions/trivy/scan/action.yml) +- [Setup Trivy](actions/trivy/setup/action.yml) .NET: -- [Build, test your code and analyze it with Sonar](actions/dotnet/build-test-sonar/action.yml) -- [Build and test your code](actions/dotnet/build-test/action.yml) -- [Install the SDK, lint the code and restore NuGet packages](actions/dotnet/install-lint-restore/) +- [Build, test the code and analyze it with Sonar](actions/dotnet/build-test-sonar/action.yml) +- [Build and test the code](actions/dotnet/build-test/action.yml) +- [Install the SDK, lint the code and restore NuGet packages](actions/dotnet/install-lint-restore/action.yml) +- [Package the projects and push them to a NuGet registry](actions/dotnet/pack-push/action.yml) + +Git: + +- [Tell whether a commit changes some paths](actions/git/check-changes/action.yml) + +Go: + +- [Lint the code with the Go toolchain](actions/go/lint/action.yml) +- [Scan the code with govulncheck](actions/govulncheck/scan/action.yml) +- [Setup GoReleaser](actions/goreleaser/setup/action.yml) + +Helm: + +- [Setup Helm](actions/helm/setup/action.yml) +- [Setup chart-testing](actions/chart-testing/setup/action.yml) +- [Setup helm-unittest](actions/helm-unittest/setup/action.yml) +- [Setup kube-linter](actions/kube-linter/setup/action.yml) +- [Setup kubeconform](actions/kubeconform/setup/action.yml) + +Kubernetes: + +- [Setup kind and kubectl](actions/kind/setup/action.yml) + +License compliance: + +- [Analyze the dependencies with FOSSA](actions/fossa/analyze/action.yml) MongoDB: -- [Add your runner IP address to Atlas access list](actions/mongodb-atlas/add-runner-ip/action.yml) -- [Start a server in your pipeline](actions/mongodb/start/action.yml) +- [Add the runner IP address to the Atlas access list](actions/mongodb-atlas/add-runner-ip/action.yml) +- [Start a server in the pipeline](actions/mongodb/start/action.yml) + +Node.js: + +- [Publish a package unless its version already is](actions/npm/publish/action.yml) +- [Install the Playwright browsers](actions/playwright/install/action.yml) + +Search engines: + +- [Submit the pages of a sitemap to IndexNow](actions/indexnow/submit/action.yml) + +Code scanning: + +- [Setup CodeQL](actions/codeql/setup/action.yml) +- [Setup Opengrep](actions/opengrep/setup/action.yml) + +Sonar: + +- [Analyze the code of a project other than .NET](actions/sonar/scan/action.yml) SBOM: - [Generate a SBOM with Syft](actions/syft/generate-sbom/action.yml) +Task: + +- [Setup Task](actions/task/setup/action.yml) + Terraform: - [Setup Terraform](actions/terraform/setup/action.yml) diff --git a/actions/chart-testing/setup/action.yml b/actions/chart-testing/setup/action.yml new file mode 100644 index 0000000..c680e7f --- /dev/null +++ b/actions/chart-testing/setup/action.yml @@ -0,0 +1,64 @@ +name: Setup chart-testing +description: | + Installs the chart-testing CLI (ct) by downloading the official release and verifying its SHA256 checksum, with the yamllint and yamale it calls + (replacement for helm/chart-testing-action, to reduce third-party GitHub Action supply-chain risk). + Helm and git must be installed beforehand, for example with the Setup Helm action. + Linux x64 or arm64 runners only (for example ubuntu-latest or ubuntu-24.04-arm). + +inputs: + chart-testing-version: + description: Version of chart-testing to install (check latest from https://github.com/helm/chart-testing/releases) + required: false + default: "v3.15.0" + yamllint-version: + description: Version of yamllint to install + required: false + default: "1.38.0" + yamale-version: + description: Version of yamale to install + required: false + default: "6.1.0" + +runs: + using: "composite" + steps: + - name: Install chart-testing + shell: bash + env: + CT_VERSION: ${{ inputs.chart-testing-version }} + CT_ARCH: ${{ runner.arch == 'ARM64' && 'arm64' || 'amd64' }} + YAMLLINT_VERSION: ${{ inputs.yamllint-version }} + YAMALE_VERSION: ${{ inputs.yamale-version }} + run: | + # Downloaded into a directory of its own, since the repository may hold a file or a directory named like the binary. + cd "$(mktemp -d)" + CT_TAR="chart-testing_${CT_VERSION#v}_linux_${CT_ARCH}.tar.gz" + BASE_URL="https://github.com/helm/chart-testing/releases/download/${CT_VERSION}" + + echo "Downloading chart-testing binary and checksums..." + curl -fsSL -O "${BASE_URL}/${CT_TAR}" + curl -fsSL -O "${BASE_URL}/checksums.txt" + + echo "Verifying checksum..." + if grep " ${CT_TAR}$" checksums.txt | sha256sum --check --status; then + echo "Verification successful!" + tar -xzf "${CT_TAR}" ct etc + # Installed where the job owns the directory, so neither root nor sudo is needed, on a hosted runner or in a container. + mkdir -p "${RUNNER_TEMP}/bin" + mv ct "${RUNNER_TEMP}/bin/ct" + # The schema and the lint rules are looked up by ct in $HOME/.ct when no flag names them. + mkdir -p "${HOME}/.ct" + cp etc/chart_schema.yaml etc/lintconf.yaml "${HOME}/.ct/" + # Kept in a virtual environment of their own, since the system Python may refuse a global install. + python3 -m venv "${RUNNER_TEMP}/ct-venv" + "${RUNNER_TEMP}/ct-venv/bin/pip" install --quiet "yamllint==${YAMLLINT_VERSION}" "yamale==${YAMALE_VERSION}" + ln -sf "${RUNNER_TEMP}/ct-venv/bin/yamllint" "${RUNNER_TEMP}/bin/yamllint" + ln -sf "${RUNNER_TEMP}/ct-venv/bin/yamale" "${RUNNER_TEMP}/bin/yamale" + echo "${RUNNER_TEMP}/bin" >> "$GITHUB_PATH" + export PATH="${RUNNER_TEMP}/bin:$PATH" + rm -rf etc "${CT_TAR}" checksums.txt + ct version + else + echo "ERROR: Checksum verification failed!" >&2 + exit 1 + fi diff --git a/actions/codeql/setup/action.yml b/actions/codeql/setup/action.yml new file mode 100644 index 0000000..13d63cf --- /dev/null +++ b/actions/codeql/setup/action.yml @@ -0,0 +1,33 @@ +name: Setup CodeQL +description: | + Installs the CodeQL CLI with its standard query packs, from the CodeQL bundle the codeql-action project releases, + verifying its SHA256 checksum, for a job running CodeQL itself rather than through github/codeql-action. + Linux x64 or arm64 runners only (for example ubuntu-latest or ubuntu-24.04-arm). + +inputs: + codeql-version: + description: Version of the CodeQL bundle to install, without its leading v (check latest from https://github.com/github/codeql-action/releases) + required: false + default: "2.27.3" + +runs: + using: "composite" + steps: + - name: Install CodeQL + shell: bash + env: + CODEQL_VERSION: ${{ inputs.codeql-version }} + CODEQL_BUNDLE: codeql-bundle-${{ runner.arch == 'ARM64' && 'linux-arm64' || 'linux64' }}.tar.gz + run: | + # Downloaded into a directory of its own, since the repository may hold a file or a directory named like the bundle. + cd "$(mktemp -d)" + BASE_URL="https://github.com/github/codeql-action/releases/download/codeql-bundle-v${CODEQL_VERSION}" + curl -fsSL -O "${BASE_URL}/${CODEQL_BUNDLE}" + curl -fsSL -O "${BASE_URL}/${CODEQL_BUNDLE}.checksum.txt" + sha256sum --check --status "${CODEQL_BUNDLE}.checksum.txt" + # The bundle is a codeql directory holding the CLI and the query packs it resolves beside itself. + rm -rf "${RUNNER_TEMP}/codeql" + tar -xzf "${CODEQL_BUNDLE}" -C "${RUNNER_TEMP}" + rm -f "${CODEQL_BUNDLE}" "${CODEQL_BUNDLE}.checksum.txt" + echo "${RUNNER_TEMP}/codeql" >> "$GITHUB_PATH" + "${RUNNER_TEMP}/codeql/codeql" version --format=terse diff --git a/actions/container/build-push/action.yml b/actions/container/build-push/action.yml new file mode 100644 index 0000000..ce0772a --- /dev/null +++ b/actions/container/build-push/action.yml @@ -0,0 +1,88 @@ +name: Build and push a container image +description: | + Builds a container image with the BuildKit of the Docker daemon for one or more platforms and pushes it under a tag, + each platform of several being pushed under a tag of its own first, then put together under the tag. + The registry login is left to the job, which holds the credentials. + Linux runners only (for example ubuntu-latest or ubuntu-24.04-arm). + +inputs: + image-definition: + description: Path to the container definition file (Dockerfile, Containerfile), relative to the working directory + required: true + image-repository: + description: Repository of the image, with its registry (for example docker.io/devprofr/app) + required: true + image-tag: + description: Tag to push, the tag of each platform of several being this one followed by the platform (1.0.1-linux-arm64 for instance) + required: true + platforms: + description: Comma-separated platforms to build (a platform other than the runner's own needs the Setup QEMU action first) + required: false + default: "linux/amd64" + build-arguments: + description: Build arguments, one KEY=value per line + required: false + default: "" + working-directory: + description: Directory the build runs from, relative to the workspace, which a composite action does not take from the defaults of the calling job + required: false + default: "." + +outputs: + digest: + description: Digest of the pushed image, a manifest list when several platforms were built + value: ${{ steps.build.outputs.digest }} + +runs: + using: "composite" + steps: + # The Docker CLI of the runner does what the docker/* actions wrapped, with no third party action in between. + # The builder of the daemon runs no BuildKit image, which a builder of its own would pull, and keeps no state of its own a later job would find. + # It pushes no image by digest alone and builds one platform at a time on the classic image store, so each platform gets a tag. + - name: Build and push container image + id: build + shell: bash + working-directory: ${{ inputs.working-directory }} + env: + IMAGE_DEFINITION: ${{ inputs.image-definition }} + IMAGE_REPOSITORY: ${{ inputs.image-repository }} + IMAGE_TAG: ${{ inputs.image-tag }} + PLATFORMS: ${{ inputs.platforms }} + BUILD_ARGUMENTS: ${{ inputs.build-arguments }} + run: | + build_args=() + while IFS= read -r argument; do + [ -n "$argument" ] && build_args+=(--build-arg "$argument") + done <<< "$BUILD_ARGUMENTS" + + # Builds and pushes one platform under a tag, and prints the digest of what was pushed. + build_push() { + local platform="$1" ref="$2" metadata digest + metadata="$(mktemp)" + docker buildx build --builder default --file "$IMAGE_DEFINITION" --platform "$platform" \ + --tag "$ref" --push --metadata-file "$metadata" "${build_args[@]}" . >&2 + digest="$(jq -r '."containerimage.digest" // empty' "$metadata")" + # The daemon keeps the image it built, which the registry now holds. + docker image rm "$ref" >/dev/null 2>&1 || true + if [[ "$digest" != sha256:* ]]; then + echo "ERROR: The build reported no digest for ${ref}" >&2 + return 1 + fi + echo "$digest" + } + + IFS=',' read -ra platform_list <<< "$PLATFORMS" + if [ "${#platform_list[@]}" -eq 1 ]; then + digest="$(build_push "${platform_list[0]}" "${IMAGE_REPOSITORY}:${IMAGE_TAG}")" + else + sources=() + for platform in "${platform_list[@]}"; do + platform_digest="$(build_push "$platform" "${IMAGE_REPOSITORY}:${IMAGE_TAG}-${platform//\//-}")" + sources+=("${IMAGE_REPOSITORY}@${platform_digest}") + done + # The tag names a complete image only once every platform is pushed. + docker buildx imagetools create --tag "${IMAGE_REPOSITORY}:${IMAGE_TAG}" "${sources[@]}" + digest="$(docker buildx imagetools inspect "${IMAGE_REPOSITORY}:${IMAGE_TAG}" --format '{{json .Manifest}}' | jq -r .digest)" + fi + echo "Pushed ${IMAGE_REPOSITORY}:${IMAGE_TAG}@${digest}" + echo "digest=$digest" >> "$GITHUB_OUTPUT" diff --git a/actions/cosign/sign/action.yml b/actions/cosign/sign/action.yml index f754b0e..c411d22 100644 --- a/actions/cosign/sign/action.yml +++ b/actions/cosign/sign/action.yml @@ -11,7 +11,7 @@ inputs: cosign-version: description: Version of cosign (check latest from https://github.com/sigstore/cosign/releases) required: false - default: 'v3.0.6' + default: 'v3.1.4' image-name: description: Image name required: true @@ -26,6 +26,11 @@ inputs: required: false default: 'false' +outputs: + digest: + description: Digest of the image, which the signature is attached to + value: ${{ steps.digest.outputs.DIGEST }} + runs: using: "composite" steps: @@ -34,19 +39,17 @@ runs: # Downloaded into a directory of its own, since the repository may hold a file or a directory named like the binary. cd "$(mktemp -d)" echo "Downloading Cosign binary and checksums..." - curl -sL -O https://github.com/sigstore/cosign/releases/download/${COSIGN_VERSION}/${COSIGN_BINARY} - curl -sL -O "https://github.com/sigstore/cosign/releases/download/${COSIGN_VERSION}/${COSIGN_CHECKSUM_FILE}" + curl -fsSL -O "https://github.com/sigstore/cosign/releases/download/${COSIGN_VERSION}/${COSIGN_BINARY}" + curl -fsSL -O "https://github.com/sigstore/cosign/releases/download/${COSIGN_VERSION}/${COSIGN_CHECKSUM_FILE}" echo "Verifying checksum..." - grep "${COSIGN_BINARY}$" "${COSIGN_CHECKSUM_FILE}" | sha256sum --check --status - - if [ $? -eq 0 ]; then + if grep " ${COSIGN_BINARY}$" "${COSIGN_CHECKSUM_FILE}" | sha256sum --check --status; then echo "Verification successful!" - rm $COSIGN_CHECKSUM_FILE - chmod +x $COSIGN_BINARY + rm "$COSIGN_CHECKSUM_FILE" + chmod +x "$COSIGN_BINARY" # Installed where the job owns the directory, so neither root nor sudo is needed, on a hosted runner or in a container. mkdir -p "${RUNNER_TEMP}/bin" - mv $COSIGN_BINARY "${RUNNER_TEMP}/bin/cosign" + mv "$COSIGN_BINARY" "${RUNNER_TEMP}/bin/cosign" echo "${RUNNER_TEMP}/bin" >> "$GITHUB_PATH" export PATH="${RUNNER_TEMP}/bin:$PATH" cosign version @@ -56,21 +59,27 @@ runs: fi shell: bash env: - COSIGN_BINARY: cosign-linux-amd64 + COSIGN_BINARY: cosign-linux-${{ runner.arch == 'ARM64' && 'arm64' || 'amd64' }} COSIGN_CHECKSUM_FILE: cosign_checksums.txt COSIGN_VERSION: ${{ inputs.cosign-version }} - name: Get image digest id: digest run: | - DIGEST=$(docker buildx imagetools inspect \ - ${{ inputs.image-path }}/${{ inputs.image-name }}:${{ inputs.image-tag }} \ - --format '{{json .Manifest}}' | jq -r '.digest') - echo "DIGEST=$DIGEST" >> $GITHUB_OUTPUT + DIGEST=$(docker buildx imagetools inspect "$IMAGE_REF" --format '{{json .Manifest}}' | jq -r '.digest') + # A digest missing from the registry answer would otherwise be signed as the literal text "null". + if [[ "$DIGEST" != sha256:* ]]; then + echo "ERROR: No digest found for ${IMAGE_REF}" >&2 + exit 1 + fi + echo "DIGEST=$DIGEST" >> "$GITHUB_OUTPUT" shell: bash + env: + IMAGE_REF: ${{ inputs.image-path }}/${{ inputs.image-name }}:${{ inputs.image-tag }} - name: Sign image with Cosign if: inputs.readonly != 'true' env: COSIGN_EXPERIMENTAL: 1 - run: | - cosign sign --yes ${{ inputs.image-path }}/${{ inputs.image-name }}@${{ steps.digest.outputs.DIGEST }} + IMAGE_REPOSITORY: ${{ inputs.image-path }}/${{ inputs.image-name }} + DIGEST: ${{ steps.digest.outputs.DIGEST }} + run: cosign sign --yes "${IMAGE_REPOSITORY}@${DIGEST}" shell: bash diff --git a/actions/dotnet/build-test-sonar/action.yml b/actions/dotnet/build-test-sonar/action.yml index 5cadcc9..0c90527 100644 --- a/actions/dotnet/build-test-sonar/action.yml +++ b/actions/dotnet/build-test-sonar/action.yml @@ -38,72 +38,114 @@ inputs: description: Sonar project name required: false default: "" + sonarscanner-version: + description: Version of the Sonar scanner for .NET (check latest from https://www.nuget.org/packages/dotnet-sonarscanner) + required: false + default: "12.0.0" sonar-token: description: Sonar token for login required: false default: "" + working-directory: + description: Directory of the .NET code, relative to the workspace, which a composite action does not take from the defaults of the calling job + required: false + default: "." runs: using: "composite" steps: - name: Set up JDK for Sonar - uses: actions/setup-java@v5 + uses: actions/setup-java@b843ebf9b913146b38da2eaa7aeddf76074ae1ad # v6.1.0 with: java-version: ${{ inputs.java-version }} distribution: "zulu" + # The analyzers Sonar downloads land in its user home. - name: Cache Sonar packages - uses: actions/cache@v6 + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: - path: ~/sonar/cache + path: ~/.sonar/cache key: ${{ runner.os }}-sonar restore-keys: ${{ runner.os }}-sonar + # Keyed by version with no fallback, since a scanner restored from another key would be another version. - name: Cache Sonar scanner id: cache-sonar-scanner - uses: actions/cache@v6 + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: ./.sonar/scanner - key: ${{ runner.os }}-sonar-scanner - restore-keys: ${{ runner.os }}-sonar-scanner + key: ${{ runner.os }}-sonar-scanner-${{ inputs.sonarscanner-version }} - name: Install Sonar scanner if: ${{ steps.cache-sonar-scanner.outputs.cache-hit != 'true' }} run: | mkdir -p ./.sonar/scanner - dotnet tool update dotnet-sonarscanner --tool-path ./.sonar/scanner + dotnet tool install dotnet-sonarscanner --tool-path ./.sonar/scanner --version "$SONARSCANNER_VERSION" shell: bash + env: + SONARSCANNER_VERSION: ${{ inputs.sonarscanner-version }} - name: Start code analysis run: | - ./.sonar/scanner/dotnet-sonarscanner begin \ - /k:"${{ inputs.sonar-project-key }}" \ - /o:"${{ inputs.sonar-organization }}" \ - /n:"${{ inputs.sonar-project-name }}" \ - /d:sonar.token="${{ inputs.sonar-token}}" \ - /d:sonar.host.url="${{ inputs.sonar-host-url }}" \ - /d:sonar.cpd.exclusions="${{ inputs.report-folder }}/**,${{ inputs.sonar-cpd-exclusions }}" \ - /d:sonar.exclusions="${{ inputs.report-folder }}/**,${{ inputs.sonar-exclusions }}" \ - /d:sonar.coverageReportPaths="${{ inputs.report-folder }}/SonarQube.xml" \ - /d:sonar.cs.vstest.reportsPaths="${{ inputs.report-folder }}/TestResults/*.trx" + args=( + /k:"$SONAR_PROJECT_KEY" + /d:sonar.token="$SONAR_TOKEN" + /d:sonar.host.url="$SONAR_HOST_URL" + /d:sonar.cpd.exclusions="$REPORT_FOLDER/**,$SONAR_CPD_EXCLUSIONS" + /d:sonar.exclusions="$REPORT_FOLDER/**,$SONAR_EXCLUSIONS" + /d:sonar.coverageReportPaths="$REPORT_FOLDER/SonarQube.xml" + /d:sonar.cs.vstest.reportsPaths="$REPORT_FOLDER/TestResults/*.trx" + ) + # SonarCloud needs an organization, which a SonarQube server has no notion of. + if [ -n "$SONAR_ORGANIZATION" ]; then + args+=(/o:"$SONAR_ORGANIZATION") + fi + if [ -n "$SONAR_PROJECT_NAME" ]; then + args+=(/n:"$SONAR_PROJECT_NAME") + fi + "$GITHUB_WORKSPACE/.sonar/scanner/dotnet-sonarscanner" begin "${args[@]}" shell: bash + working-directory: ${{ inputs.working-directory }} + env: + REPORT_FOLDER: ${{ inputs.report-folder }} + SONAR_CPD_EXCLUSIONS: ${{ inputs.sonar-cpd-exclusions }} + SONAR_EXCLUSIONS: ${{ inputs.sonar-exclusions }} + SONAR_HOST_URL: ${{ inputs.sonar-host-url }} + SONAR_ORGANIZATION: ${{ inputs.sonar-organization }} + SONAR_PROJECT_KEY: ${{ inputs.sonar-project-key }} + SONAR_PROJECT_NAME: ${{ inputs.sonar-project-name }} + SONAR_TOKEN: ${{ inputs.sonar-token }} - name: Build .NET solution run: dotnet build --no-restore --configuration Debug shell: bash + working-directory: ${{ inputs.working-directory }} env: BUILD_SOURCEBRANCHNAME: ${{ github.ref_name }} BUILD_BUILDID: ${{ github.run_id }} - name: Run tests + # Shell words by contract, quotes included, parsed from a variable rather than pasted into the script by GitHub. run: | + test_args=() + eval "test_args=(${DOTNET_TEST_ARGS})" dotnet test --no-build --verbosity normal --configuration Debug \ - --results-directory "${{ inputs.report-folder }}/TestResults" \ - ${{ inputs.dotnet-test-args }} + --results-directory "$REPORT_FOLDER/TestResults" \ + "${test_args[@]}" shell: bash + working-directory: ${{ inputs.working-directory }} env: ASPNETCORE_ENVIRONMENT: Development + DOTNET_TEST_ARGS: ${{ inputs.dotnet-test-args }} + REPORT_FOLDER: ${{ inputs.report-folder }} + # Microsoft.Testing.Platform writes the coverage beside the results, and VSTest in a folder of its own for each run. - name: Generate test report - if: ${{ always() && hashFiles(format('{0}/TestResults/*.cobertura.xml', inputs.report-folder)) != '' }} + if: ${{ always() && hashFiles(format('{0}/{1}/TestResults/**/*.cobertura.xml', inputs.working-directory, inputs.report-folder)) != '' }} run: | - reportgenerator "-reports:${{ inputs.report-folder }}/TestResults/*.cobertura.xml" \ - "-targetdir:${{ inputs.report-folder }}" \ + reportgenerator "-reports:$REPORT_FOLDER/TestResults/**/*.cobertura.xml" \ + "-targetdir:$REPORT_FOLDER" \ "-reporttypes:Cobertura;Html;TextSummary;SonarQube" shell: bash + working-directory: ${{ inputs.working-directory }} + env: + REPORT_FOLDER: ${{ inputs.report-folder }} - name: Complete code analysis - run: ./.sonar/scanner/dotnet-sonarscanner end /d:sonar.token="${{ inputs.sonar-token }}" + run: '"$GITHUB_WORKSPACE/.sonar/scanner/dotnet-sonarscanner" end /d:sonar.token="$SONAR_TOKEN"' shell: bash + working-directory: ${{ inputs.working-directory }} + env: + SONAR_TOKEN: ${{ inputs.sonar-token }} diff --git a/actions/dotnet/build-test/action.yml b/actions/dotnet/build-test/action.yml index d13591e..33030a1 100644 --- a/actions/dotnet/build-test/action.yml +++ b/actions/dotnet/build-test/action.yml @@ -10,6 +10,10 @@ inputs: description: Folder where report files will be generated required: false default: report + working-directory: + description: Directory of the .NET code, relative to the workspace, which a composite action does not take from the defaults of the calling job + required: false + default: "." runs: using: "composite" @@ -17,21 +21,32 @@ runs: - name: Build .NET code run: dotnet build --no-restore --configuration Debug shell: bash + working-directory: ${{ inputs.working-directory }} env: BUILD_SOURCEBRANCHNAME: ${{ github.ref_name }} BUILD_BUILDID: ${{ github.run_id }} - name: Run tests + # Shell words by contract, quotes included, parsed from a variable rather than pasted into the script by GitHub. run: | + test_args=() + eval "test_args=(${DOTNET_TEST_ARGS})" dotnet test --no-build --verbosity normal --configuration Debug \ - --results-directory "${{ inputs.report-folder }}/TestResults" \ - ${{ inputs.dotnet-test-args }} + --results-directory "$REPORT_FOLDER/TestResults" \ + "${test_args[@]}" shell: bash + working-directory: ${{ inputs.working-directory }} env: ASPNETCORE_ENVIRONMENT: Development + DOTNET_TEST_ARGS: ${{ inputs.dotnet-test-args }} + REPORT_FOLDER: ${{ inputs.report-folder }} + # Microsoft.Testing.Platform writes the coverage beside the results, and VSTest in a folder of its own for each run. - name: Generate test report - if: ${{ always() && hashFiles(format('{0}/TestResults/*.cobertura.xml', inputs.report-folder)) != '' }} + if: ${{ always() && hashFiles(format('{0}/{1}/TestResults/**/*.cobertura.xml', inputs.working-directory, inputs.report-folder)) != '' }} run: | - reportgenerator "-reports:${{ inputs.report-folder }}/TestResults/*.cobertura.xml" \ - "-targetdir:${{ inputs.report-folder }}" \ + reportgenerator "-reports:$REPORT_FOLDER/TestResults/**/*.cobertura.xml" \ + "-targetdir:$REPORT_FOLDER" \ "-reporttypes:Cobertura;Html;TextSummary" shell: bash + working-directory: ${{ inputs.working-directory }} + env: + REPORT_FOLDER: ${{ inputs.report-folder }} diff --git a/actions/dotnet/install-lint-restore/action.yml b/actions/dotnet/install-lint-restore/action.yml index c1fa6ee..9daf962 100644 --- a/actions/dotnet/install-lint-restore/action.yml +++ b/actions/dotnet/install-lint-restore/action.yml @@ -6,12 +6,20 @@ inputs: description: .NET SDK version to be used required: false default: "10.0" + reportgenerator-version: + description: Version of ReportGenerator (check latest from https://www.nuget.org/packages/dotnet-reportgenerator-globaltool) + required: false + default: "5.5.11" + working-directory: + description: Directory of the .NET code, relative to the workspace, which a composite action does not take from the defaults of the calling job + required: false + default: "." runs: using: "composite" steps: - name: Install .NET - uses: actions/setup-dotnet@v6 + uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0 with: dotnet-version: ${{ inputs.dotnet-version }} - name: Install .NET linters @@ -20,12 +28,16 @@ runs: shell: bash - name: Install .NET tools run: | - dotnet tool install --global dotnet-reportgenerator-globaltool + dotnet tool install --global dotnet-reportgenerator-globaltool --version "$REPORTGENERATOR_VERSION" echo "$HOME/.dotnet/tools" >> "$GITHUB_PATH" shell: bash + env: + REPORTGENERATOR_VERSION: ${{ inputs.reportgenerator-version }} - name: Restore .NET packages run: dotnet restore shell: bash + working-directory: ${{ inputs.working-directory }} - name: Lint .NET code run: dotnet format --verify-no-changes --severity warn --verbosity:diagnostic shell: bash + working-directory: ${{ inputs.working-directory }} diff --git a/actions/dotnet/pack-push/action.yml b/actions/dotnet/pack-push/action.yml index 324e291..f73c725 100644 --- a/actions/dotnet/pack-push/action.yml +++ b/actions/dotnet/pack-push/action.yml @@ -7,28 +7,32 @@ inputs: required: false default: "10.0" registry-token: - description: Registry + description: Registry API key required: true registry-address: description: Registry address (NuGet.org by default) required: false default: "https://api.nuget.org/v3/index.json" push-enabled: - description: Push + description: Push the packages to the registry required: false default: "false" + working-directory: + description: Directory of the .NET code, relative to the workspace, which a composite action does not take from the defaults of the calling job + required: false + default: "." runs: using: "composite" steps: - name: Install .NET - uses: actions/setup-dotnet@v6 + uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0 with: dotnet-version: ${{ inputs.dotnet-version }} - name: Package - # "**/*.csproj;!**/*Tests.csproj" run: dotnet pack --configuration Release -o output shell: bash + working-directory: ${{ inputs.working-directory }} env: DOTNET_CLI_TELEMETRY_OPTOUT: 1 DOTNET_NOLOGO: 1 @@ -36,6 +40,12 @@ runs: BUILD_BUILDID: ${{ github.run_id }} - name: Push to registry if: ${{ inputs.push-enabled == 'true' }} - # "$(Build.ArtifactStagingDirectory)/**/*.nupkg;!$(Build.ArtifactStagingDirectory)/**/*.symbols.nupkg" - run: for f in output/*.nupkg; do dotnet nuget push "$f" --api-key ${{ inputs.registry-token }} --source ${{ inputs.registry-address }} ; done + run: | + for f in output/*.nupkg; do + dotnet nuget push "$f" --api-key "$REGISTRY_TOKEN" --source "$REGISTRY_ADDRESS" + done shell: bash + working-directory: ${{ inputs.working-directory }} + env: + REGISTRY_ADDRESS: ${{ inputs.registry-address }} + REGISTRY_TOKEN: ${{ inputs.registry-token }} diff --git a/actions/fossa/analyze/action.yml b/actions/fossa/analyze/action.yml index 4fa96c2..2e0c85e 100644 --- a/actions/fossa/analyze/action.yml +++ b/actions/fossa/analyze/action.yml @@ -3,16 +3,20 @@ description: | Installs the FOSSA CLI by downloading the official binary from its GitHub release and verifying its SHA256 checksum, then analyzes the dependencies for license compliance, optionally tests them against the policy, and writes an HTML report (replacement for fossas/fossa-action, to reduce third-party GitHub Action supply-chain risk). - Linux runners only (for example ubuntu-latest). + Linux x64 or arm64 runners only (for example ubuntu-latest or ubuntu-24.04-arm). inputs: fossa-version: description: Version of the FOSSA CLI to install (check latest from https://github.com/fossas/fossa-cli/releases) required: false - default: "3.19.3" + default: "3.20.7" api-key: description: FOSSA API key required: true + exclude-paths: + description: Paths left out of the analysis, relative to working-directory, one per line + required: false + default: "" run-tests: description: Test the analyzed dependencies against the FOSSA policy required: false @@ -22,9 +26,13 @@ inputs: required: false default: "" report-file: - description: Path of the generated HTML report (empty writes none) + description: Path of the generated HTML report, relative to the working directory (empty writes none) required: false default: "report/fossa.html" + working-directory: + description: Directory of the project to analyze, relative to the workspace, which a composite action does not take from the defaults of the calling job + required: false + default: "." runs: using: "composite" @@ -33,21 +41,19 @@ runs: shell: bash env: FOSSA_VERSION: ${{ inputs.fossa-version }} - FOSSA_TARBALL: fossa_${{ inputs.fossa-version }}_linux_amd64.tar.gz + FOSSA_TARBALL: fossa_${{ inputs.fossa-version }}_linux_${{ runner.arch == 'ARM64' && 'arm64' || 'amd64' }}.tar.gz run: | # Downloaded into a directory of its own, since the repository may hold a file or a directory named like the binary. cd "$(mktemp -d)" BASE_URL="https://github.com/fossas/fossa-cli/releases/download/v${FOSSA_VERSION}" echo "Downloading FOSSA CLI binary and checksum..." - curl -sSL -O "${BASE_URL}/${FOSSA_TARBALL}" - curl -sSL -O "${BASE_URL}/${FOSSA_TARBALL}.sha256" + curl -fsSL -O "${BASE_URL}/${FOSSA_TARBALL}" + curl -fsSL -O "${BASE_URL}/${FOSSA_TARBALL}.sha256" echo "Verifying checksum..." # The checksum file holds the hash, with or without the file name after it. - echo "$(cut -d ' ' -f 1 "${FOSSA_TARBALL}.sha256") ${FOSSA_TARBALL}" | sha256sum --check --status - - if [ $? -eq 0 ]; then + if echo "$(cut -d ' ' -f 1 "${FOSSA_TARBALL}.sha256") ${FOSSA_TARBALL}" | sha256sum --check --status; then echo "Verification successful!" tar -xzf "${FOSSA_TARBALL}" fossa chmod +x fossa @@ -65,13 +71,23 @@ runs: - name: Analyze dependencies shell: bash + working-directory: ${{ inputs.working-directory }} env: FOSSA_API_KEY: ${{ inputs.api-key }} - run: fossa analyze + EXCLUDE_PATHS: ${{ inputs.exclude-paths }} + run: | + args=() + while IFS= read -r path; do + if [ -n "$path" ]; then + args+=(--exclude-path "$path") + fi + done <<< "$EXCLUDE_PATHS" + fossa analyze "${args[@]}" - name: Test dependencies against the policy if: ${{ inputs.run-tests == 'true' }} shell: bash + working-directory: ${{ inputs.working-directory }} env: FOSSA_API_KEY: ${{ inputs.api-key }} DIFF_REVISION: ${{ inputs.test-diff-revision }} @@ -85,6 +101,7 @@ runs: - name: Write HTML report if: ${{ inputs.report-file != '' }} shell: bash + working-directory: ${{ inputs.working-directory }} env: FOSSA_API_KEY: ${{ inputs.api-key }} REPORT_FILE: ${{ inputs.report-file }} diff --git a/actions/git/check-changes/action.yml b/actions/git/check-changes/action.yml new file mode 100644 index 0000000..6e795fe --- /dev/null +++ b/actions/git/check-changes/action.yml @@ -0,0 +1,51 @@ +name: Check changes +description: | + Tells whether the checked out commit changes any of the given paths, compared with a base: + the one given, or else the base of the pull request, or else the commit before the push, or else the parent commit. + A base the clone cannot read counts as a change, as on a new branch or in a shallow clone, so nothing is skipped by mistake. + The repository is checked out with its history (fetch-depth 0) for the base to be readable. + +inputs: + paths: + description: Paths to compare, separated by spaces (for example "src test") + required: true + base: + description: Commit or tree to compare with, the base of the pull request, the commit before the push or the parent commit when empty + required: false + default: "" + +outputs: + changed: + description: "\"true\" when a path changed, \"false\" otherwise" + value: ${{ steps.compare.outputs.changed }} + +runs: + using: "composite" + steps: + - name: Compare with the base + id: compare + shell: bash + env: + BASE: ${{ inputs.base || github.event.pull_request.base.sha || github.event.before }} + PATHS: ${{ inputs.paths }} + run: | + read -ra path_list <<< "$PATHS" + # An event naming no base, a manual run or a local one, is compared with the parent commit. + BASE="${BASE:-HEAD~1}" + if ! git cat-file -e "${BASE}^{tree}" 2>/dev/null; then + echo "No readable base to compare with, counted as a change" + echo "changed=true" >> "$GITHUB_OUTPUT" + exit 0 + fi + echo "Comparing ${PATHS} between ${BASE} and HEAD" + # The exit status tells a change (1) from a failure of git (above 1), which stops the step. + status=0 + git diff --quiet "$BASE" HEAD -- "${path_list[@]}" || status=$? + case "$status" in + 0) echo "changed=false" >> "$GITHUB_OUTPUT" ;; + 1) + git diff --name-status "$BASE" HEAD -- "${path_list[@]}" + echo "changed=true" >> "$GITHUB_OUTPUT" + ;; + *) exit "$status" ;; + esac diff --git a/actions/go/lint/action.yml b/actions/go/lint/action.yml new file mode 100644 index 0000000..0a3ff25 --- /dev/null +++ b/actions/go/lint/action.yml @@ -0,0 +1,34 @@ +name: Lint Go code +description: | + Checks a Go module with the Go toolchain alone, no third party linter: its files formatted by gofmt, + its go.mod and go.sum tidy, and go vet finding nothing. + Needs Go 1.23 or later on the path, from actions/setup-go or the image of the job. + +inputs: + working-directory: + description: Directory of the Go module (go.mod), relative to the workspace, which a composite action does not take from the defaults of the calling job + required: false + default: "." + +runs: + using: "composite" + steps: + - name: Check formatting + shell: bash + working-directory: ${{ inputs.working-directory }} + run: | + unformatted="$(gofmt -l .)" + if [ -n "$unformatted" ]; then + echo "ERROR: Files not formatted by gofmt, which gofmt -w fixes:" >&2 + echo "$unformatted" >&2 + exit 1 + fi + # A requirement nothing imports, or one missing, shows here rather than as a surprise in the next build. + - name: Check go.mod and go.sum are tidy + shell: bash + working-directory: ${{ inputs.working-directory }} + run: go mod tidy -diff + - name: Vet + shell: bash + working-directory: ${{ inputs.working-directory }} + run: go vet ./... diff --git a/actions/goreleaser/setup/action.yml b/actions/goreleaser/setup/action.yml index 4da2257..8e8b4b1 100644 --- a/actions/goreleaser/setup/action.yml +++ b/actions/goreleaser/setup/action.yml @@ -2,13 +2,13 @@ name: Setup GoReleaser description: | Installs GoReleaser by downloading the official binary from its GitHub release and verifying its SHA256 checksum (replacement for goreleaser/goreleaser-action, to reduce third-party GitHub Action supply-chain risk). - Linux runners only (for example ubuntu-latest). + Linux x64 or arm64 runners only (for example ubuntu-latest or ubuntu-24.04-arm). inputs: goreleaser-version: description: Version of GoReleaser to install (check latest from https://github.com/goreleaser/goreleaser/releases) required: false - default: "2.18.2" + default: "2.19.0" runs: using: "composite" @@ -17,20 +17,18 @@ runs: shell: bash env: GORELEASER_VERSION: ${{ inputs.goreleaser-version }} - GORELEASER_TARBALL: goreleaser_Linux_x86_64.tar.gz + GORELEASER_TARBALL: goreleaser_Linux_${{ runner.arch == 'ARM64' && 'arm64' || 'x86_64' }}.tar.gz run: | # Downloaded into a directory of its own, since the repository may hold a file or a directory named like the binary. cd "$(mktemp -d)" BASE_URL="https://github.com/goreleaser/goreleaser/releases/download/v${GORELEASER_VERSION}" echo "Downloading GoReleaser binary and checksums..." - curl -sSL -O "${BASE_URL}/${GORELEASER_TARBALL}" - curl -sSL -O "${BASE_URL}/checksums.txt" + curl -fsSL -O "${BASE_URL}/${GORELEASER_TARBALL}" + curl -fsSL -O "${BASE_URL}/checksums.txt" echo "Verifying checksum..." - grep " ${GORELEASER_TARBALL}$" checksums.txt | sha256sum --check --status - - if [ $? -eq 0 ]; then + if grep " ${GORELEASER_TARBALL}$" checksums.txt | sha256sum --check --status; then echo "Verification successful!" tar -xzf "${GORELEASER_TARBALL}" goreleaser chmod +x goreleaser diff --git a/actions/govulncheck/scan/action.yml b/actions/govulncheck/scan/action.yml new file mode 100644 index 0000000..aea90e3 --- /dev/null +++ b/actions/govulncheck/scan/action.yml @@ -0,0 +1,33 @@ +name: Scan Go code with govulncheck +description: | + Installs govulncheck, the vulnerability checker of the Go team, with go install, which checks the module against the Go checksum database, + then reports the known vulnerabilities of the code the module actually calls, and fails on any. + Needs Go on the path, from actions/setup-go or the image of the job. + +inputs: + govulncheck-version: + description: Version of govulncheck to install (check latest from https://pkg.go.dev/golang.org/x/vuln/cmd/govulncheck?tab=versions) + required: false + default: "v1.9.0" + working-directory: + description: Directory of the Go module (go.mod), relative to the workspace, which a composite action does not take from the defaults of the calling job + required: false + default: "." + +runs: + using: "composite" + steps: + - name: Install govulncheck + shell: bash + env: + GOVULNCHECK_VERSION: ${{ inputs.govulncheck-version }} + run: | + # Installed where the job owns the directory, so neither root nor sudo is needed, on a hosted runner or in a container. + mkdir -p "${RUNNER_TEMP}/bin" + GOBIN="${RUNNER_TEMP}/bin" go install "golang.org/x/vuln/cmd/govulncheck@${GOVULNCHECK_VERSION}" + echo "${RUNNER_TEMP}/bin" >> "$GITHUB_PATH" + "${RUNNER_TEMP}/bin/govulncheck" -version + - name: Scan with govulncheck + shell: bash + working-directory: ${{ inputs.working-directory }} + run: govulncheck ./... diff --git a/actions/helm-unittest/setup/action.yml b/actions/helm-unittest/setup/action.yml new file mode 100644 index 0000000..7d48aa2 --- /dev/null +++ b/actions/helm-unittest/setup/action.yml @@ -0,0 +1,37 @@ +name: Setup helm-unittest +description: | + Installs untt, the standalone binary of the helm-unittest plugin, so no plugin is installed into Helm, + by downloading the official release and verifying its SHA256 checksum. + Linux x64 or arm64 runners only (for example ubuntu-latest or ubuntu-24.04-arm). + +inputs: + helm-unittest-version: + description: Version of helm-unittest to install, without its leading v (check latest from https://github.com/helm-unittest/helm-unittest/releases) + required: false + default: "1.2.1" + +runs: + using: "composite" + steps: + - name: Install helm-unittest + shell: bash + env: + HELM_UNITTEST_VERSION: ${{ inputs.helm-unittest-version }} + HELM_UNITTEST_PLATFORM: linux-${{ runner.arch == 'ARM64' && 'arm64' || 'amd64' }} + run: | + # Downloaded into a directory of its own, since the repository may hold a file or a directory named like the binary. + cd "$(mktemp -d)" + BASE_URL="https://github.com/helm-unittest/helm-unittest/releases/download/v${HELM_UNITTEST_VERSION}" + archive="helm-unittest-${HELM_UNITTEST_PLATFORM}-${HELM_UNITTEST_VERSION}.tgz" + curl -fsSL -O "${BASE_URL}/${archive}" + curl -fsSL -O "${BASE_URL}/helm-unittest-checksum.sha" + # The checksum file names each archive by the path it was built at, ./_dist/. + checksum="$(grep "/${archive}$" helm-unittest-checksum.sha | cut -d ' ' -f 1)" + echo "${checksum} ${archive}" | sha256sum --check --status + tar -xzf "${archive}" "untt-${HELM_UNITTEST_PLATFORM}" + # Installed where the job owns the directory, so neither root nor sudo is needed, on a hosted runner or in a container. + mkdir -p "${RUNNER_TEMP}/bin" + mv "untt-${HELM_UNITTEST_PLATFORM}" "${RUNNER_TEMP}/bin/untt" + echo "${RUNNER_TEMP}/bin" >> "$GITHUB_PATH" + # untt has no version flag, so it is only checked to run. + "${RUNNER_TEMP}/bin/untt" --help > /dev/null diff --git a/actions/helm/setup/action.yml b/actions/helm/setup/action.yml new file mode 100644 index 0000000..f128769 --- /dev/null +++ b/actions/helm/setup/action.yml @@ -0,0 +1,45 @@ +name: Setup Helm +description: | + Installs the Helm CLI by downloading the official binary from get.helm.sh and verifying its SHA256 checksum + (replacement for azure/setup-helm, to reduce third-party GitHub Action supply-chain risk). + Linux x64 or arm64 runners only (for example ubuntu-latest or ubuntu-24.04-arm). + +inputs: + helm-version: + description: Version of Helm to install (check latest from https://github.com/helm/helm/releases) + required: false + default: "v4.3.1" + +runs: + using: "composite" + steps: + - name: Install Helm + shell: bash + env: + HELM_VERSION: ${{ inputs.helm-version }} + HELM_PLATFORM: linux-${{ runner.arch == 'ARM64' && 'arm64' || 'amd64' }} + run: | + # Downloaded into a directory of its own, since the repository may hold a file or a directory named like the binary. + cd "$(mktemp -d)" + HELM_TAR="helm-${HELM_VERSION}-${HELM_PLATFORM}.tar.gz" + BASE_URL="https://get.helm.sh" + + echo "Downloading Helm binary and checksum..." + curl -fsSL -O "${BASE_URL}/${HELM_TAR}" + curl -fsSL -O "${BASE_URL}/${HELM_TAR}.sha256sum" + + echo "Verifying checksum..." + if sha256sum --check --status "${HELM_TAR}.sha256sum"; then + echo "Verification successful!" + tar -xzf "${HELM_TAR}" "${HELM_PLATFORM}/helm" + # Installed where the job owns the directory, so neither root nor sudo is needed, on a hosted runner or in a container. + mkdir -p "${RUNNER_TEMP}/bin" + mv "${HELM_PLATFORM}/helm" "${RUNNER_TEMP}/bin/helm" + echo "${RUNNER_TEMP}/bin" >> "$GITHUB_PATH" + export PATH="${RUNNER_TEMP}/bin:$PATH" + rm -rf "${HELM_PLATFORM}" "${HELM_TAR}" "${HELM_TAR}.sha256sum" + helm version + else + echo "ERROR: Checksum verification failed!" >&2 + exit 1 + fi diff --git a/actions/indexnow/submit/action.yml b/actions/indexnow/submit/action.yml new file mode 100644 index 0000000..31a2d67 --- /dev/null +++ b/actions/indexnow/submit/action.yml @@ -0,0 +1,82 @@ +name: Submit pages to IndexNow +description: | + Reads the pages of a site from its sitemap, a sitemap index included, and submits them to IndexNow, + which passes them on to the search engines taking part (Bing, Yandex, Naver, Seznam and others), + with curl and jq rather than a third party action. + The site serves the key at key-location, which proves it owns the key. + +inputs: + api-url: + description: Address of the IndexNow API the pages are submitted to + required: false + default: "https://api.indexnow.org/indexnow" + key: + description: IndexNow key of the site, kept in a secret, since it stands for the site with IndexNow + required: true + key-location: + description: Address of the file holding the key, https:///.txt of the host of the sitemap when empty + required: false + default: "" + sitemap-url: + description: Address of the sitemap (sitemap.xml) of the site, or of its sitemap index + required: true + +runs: + using: "composite" + steps: + - name: Submit the pages of the sitemap + shell: bash + env: + API_URL: ${{ inputs.api-url }} + INDEXNOW_KEY: ${{ inputs.key }} + KEY_LOCATION: ${{ inputs.key-location }} + SITEMAP_URL: ${{ inputs.sitemap-url }} + run: | + echo "::add-mask::$INDEXNOW_KEY" + origin="$(echo "$SITEMAP_URL" | sed -E 's#^(https?://[^/]+).*#\1#')" + host="${origin#*://}" + KEY_LOCATION="${KEY_LOCATION:-${origin}/${INDEXNOW_KEY}.txt}" + + # The addresses of the elements, which XML writes with & as &. + locations() { + curl -fsSL "$1" | tr -d '\r\n' | grep -o '[^<]*' | sed -e 's###g' -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' \ + -e 's/<//g' -e "s/'/'/g" -e 's/"/"/g' -e 's/&/\&/g' + } + + urls=() + # Read whole, since grep -q leaving a pipe early fails the curl writing to it, under pipefail. + sitemap_content="$(curl -fsSL "$SITEMAP_URL")" + if [[ "$sitemap_content" == *"&2 + exit 1 + fi + echo "Submitting ${#urls[@]} page(s) of ${host} to ${API_URL}" + + # IndexNow takes at most 10,000 addresses a request. + for ((start = 0; start < ${#urls[@]}; start += 10000)); do + payload="$(mktemp)" + jq -n --arg host "$host" --arg key "$INDEXNOW_KEY" --arg keyLocation "$KEY_LOCATION" \ + '{host: $host, key: $key, keyLocation: $keyLocation, urlList: $ARGS.positional}' \ + --args "${urls[@]:start:10000}" > "$payload" + response="$(mktemp)" + status="$(curl -sS -o "$response" -w '%{http_code}' -H 'Content-Type: application/json; charset=utf-8' --data-binary "@$payload" "$API_URL")" + # 200 is a submission taken, 202 one taken before the key is checked. + if [[ "$status" != 200 && "$status" != 202 ]]; then + echo "ERROR: IndexNow answered ${status}" >&2 + cat "$response" >&2 + echo >&2 + exit 1 + fi + echo "IndexNow answered ${status}" + done diff --git a/actions/kind/setup/action.yml b/actions/kind/setup/action.yml new file mode 100644 index 0000000..d9ed5ff --- /dev/null +++ b/actions/kind/setup/action.yml @@ -0,0 +1,44 @@ +name: Setup kind +description: | + Installs kind, which runs a Kubernetes cluster in containers, and kubectl, + by downloading the official binaries and verifying their SHA256 checksums + (replacement for helm/kind-action, a JavaScript action, which a container job cannot run without Node.js). + Linux x64 or arm64 runners only (for example ubuntu-latest or ubuntu-24.04-arm). + +inputs: + kind-version: + description: Version of kind to install (check latest from https://github.com/kubernetes-sigs/kind/releases) + required: false + default: "v0.33.0" + kubectl-version: + description: Version of kubectl to install, within one minor version of the clusters it talks to (check latest from https://dl.k8s.io/release/stable.txt) + required: false + default: "v1.37.1" + +runs: + using: "composite" + steps: + - name: Install kind and kubectl + shell: bash + env: + ARCH: ${{ runner.arch == 'ARM64' && 'arm64' || 'amd64' }} + KIND_VERSION: ${{ inputs.kind-version }} + KUBECTL_VERSION: ${{ inputs.kubectl-version }} + run: | + # Downloaded into a directory of its own, since the repository may hold a file or a directory named like the binary. + cd "$(mktemp -d)" + KIND_BINARY="kind-linux-${ARCH}" + curl -fsSL -O "https://github.com/kubernetes-sigs/kind/releases/download/${KIND_VERSION}/${KIND_BINARY}" + curl -fsSL -O "https://github.com/kubernetes-sigs/kind/releases/download/${KIND_VERSION}/${KIND_BINARY}.sha256sum" + sha256sum --check --status "${KIND_BINARY}.sha256sum" + KUBECTL_URL="https://dl.k8s.io/release/${KUBECTL_VERSION}/bin/linux/${ARCH}/kubectl" + curl -fsSL -O "$KUBECTL_URL" + # The checksum file of kubectl holds the hash alone. + echo "$(curl -fsSL "${KUBECTL_URL}.sha256") kubectl" | sha256sum --check --status + # Installed where the job owns the directory, so neither root nor sudo is needed, on a hosted runner or in a container. + mkdir -p "${RUNNER_TEMP}/bin" + install -m 0755 "${KIND_BINARY}" "${RUNNER_TEMP}/bin/kind" + install -m 0755 kubectl "${RUNNER_TEMP}/bin/kubectl" + echo "${RUNNER_TEMP}/bin" >> "$GITHUB_PATH" + "${RUNNER_TEMP}/bin/kind" version + "${RUNNER_TEMP}/bin/kubectl" version --client diff --git a/actions/kube-linter/setup/action.yml b/actions/kube-linter/setup/action.yml new file mode 100644 index 0000000..f14db84 --- /dev/null +++ b/actions/kube-linter/setup/action.yml @@ -0,0 +1,40 @@ +name: Setup kube-linter +description: | + Installs kube-linter by downloading the official binary from its GitHub release and verifying its SHA256 checksum, pinned here since the release publishes none + (replacement for stackrox/kube-linter-action, to reduce third-party GitHub Action supply-chain risk). + Linux x64 or arm64 runners only (for example ubuntu-latest or ubuntu-24.04-arm). + +inputs: + kube-linter-version: + description: Version of kube-linter to install (check latest from https://github.com/stackrox/kube-linter/releases) + required: false + default: "v0.8.3" + kube-linter-sha256: + description: SHA256 of the kube-linter archive for x64, given with any other kube-linter-version, since its release publishes no checksum file + required: false + default: "1a6d8419b11971372971fdbc22682b684ebfb7cf1c39591662d1b6ca736c41df" + kube-linter-sha256-arm64: + description: SHA256 of the kube-linter archive for arm64, given with any other kube-linter-version + required: false + default: "802e1b09eabd08f6f0a060a6b8ab2bf7bc7e6bf4f673bb2692303704c84b3e22" + +runs: + using: "composite" + steps: + - name: Install kube-linter + shell: bash + env: + KUBE_LINTER_VERSION: ${{ inputs.kube-linter-version }} + KUBE_LINTER_TARBALL: kube-linter-linux${{ runner.arch == 'ARM64' && '_arm64' || '' }}.tar.gz + KUBE_LINTER_SHA256: ${{ runner.arch == 'ARM64' && inputs.kube-linter-sha256-arm64 || inputs.kube-linter-sha256 }} + run: | + # Downloaded into a directory of its own, since the repository may hold a file or a directory named like the binary. + cd "$(mktemp -d)" + curl -fsSL -O "https://github.com/stackrox/kube-linter/releases/download/${KUBE_LINTER_VERSION}/${KUBE_LINTER_TARBALL}" + echo "${KUBE_LINTER_SHA256} ${KUBE_LINTER_TARBALL}" | sha256sum --check --status + tar -xzf "${KUBE_LINTER_TARBALL}" kube-linter + # Installed where the job owns the directory, so neither root nor sudo is needed, on a hosted runner or in a container. + mkdir -p "${RUNNER_TEMP}/bin" + mv kube-linter "${RUNNER_TEMP}/bin/kube-linter" + echo "${RUNNER_TEMP}/bin" >> "$GITHUB_PATH" + "${RUNNER_TEMP}/bin/kube-linter" version diff --git a/actions/kubeconform/setup/action.yml b/actions/kubeconform/setup/action.yml new file mode 100644 index 0000000..8491279 --- /dev/null +++ b/actions/kubeconform/setup/action.yml @@ -0,0 +1,33 @@ +name: Setup kubeconform +description: | + Installs kubeconform, which validates Kubernetes manifests against the schemas of the API, + by downloading the official binary from its GitHub release and verifying its SHA256 checksum. + Linux x64 or arm64 runners only (for example ubuntu-latest or ubuntu-24.04-arm). + +inputs: + kubeconform-version: + description: Version of kubeconform to install (check latest from https://github.com/yannh/kubeconform/releases) + required: false + default: "v0.8.0" + +runs: + using: "composite" + steps: + - name: Install kubeconform + shell: bash + env: + KUBECONFORM_VERSION: ${{ inputs.kubeconform-version }} + KUBECONFORM_TARBALL: kubeconform-linux-${{ runner.arch == 'ARM64' && 'arm64' || 'amd64' }}.tar.gz + run: | + # Downloaded into a directory of its own, since the repository may hold a file or a directory named like the binary. + cd "$(mktemp -d)" + BASE_URL="https://github.com/yannh/kubeconform/releases/download/${KUBECONFORM_VERSION}" + curl -fsSL -O "${BASE_URL}/${KUBECONFORM_TARBALL}" + curl -fsSL -O "${BASE_URL}/CHECKSUMS" + grep " ${KUBECONFORM_TARBALL}$" CHECKSUMS | sha256sum --check --status + tar -xzf "${KUBECONFORM_TARBALL}" kubeconform + # Installed where the job owns the directory, so neither root nor sudo is needed, on a hosted runner or in a container. + mkdir -p "${RUNNER_TEMP}/bin" + mv kubeconform "${RUNNER_TEMP}/bin/kubeconform" + echo "${RUNNER_TEMP}/bin" >> "$GITHUB_PATH" + "${RUNNER_TEMP}/bin/kubeconform" -v diff --git a/actions/mongodb-atlas/add-runner-ip/action.yml b/actions/mongodb-atlas/add-runner-ip/action.yml index 15c321f..3b1dbf9 100644 --- a/actions/mongodb-atlas/add-runner-ip/action.yml +++ b/actions/mongodb-atlas/add-runner-ip/action.yml @@ -17,9 +17,13 @@ runs: steps: - name: Add runner IP to MongoDB Atlas shell: bash + env: + ATLAS_GROUP_ID: ${{ inputs.atlas-groupid }} + ATLAS_PRIVATE_KEY: ${{ inputs.atlas-privatekey }} + ATLAS_PUBLIC_KEY: ${{ inputs.atlas-publickey }} run: | # gets current outbound IP of this runner - RUNNER_IP=$(curl -s https://api.ipify.org || curl -s https://ifconfig.me) + RUNNER_IP=$(curl -fsS https://api.ipify.org || curl -fsS https://ifconfig.me) if [ -z "$RUNNER_IP" ]; then echo "Failed to detect runner IP" exit 1 @@ -33,11 +37,11 @@ runs: PAYLOAD="[{\"ipAddress\": \"$RUNNER_IP\", \"comment\": \"GH Actions temp - run ${{ github.run_id }}\", \"deleteAfterDate\": \"$DELETE_AFTER\"}]" # calls Atlas API v2 to add the entry (uses digest Auth via curl) - RESPONSE=$(curl -s -w "%{http_code}" --user "${{ inputs.atlas-publickey }}:${{ inputs.atlas-privatekey }}" --digest \ + RESPONSE=$(curl -s -w "%{http_code}" --user "${ATLAS_PUBLIC_KEY}:${ATLAS_PRIVATE_KEY}" --digest \ -H "Accept: application/vnd.atlas.2023-01-01+json" \ -H "Content-Type: application/json" \ --data "$PAYLOAD" \ - "https://cloud.mongodb.com/api/atlas/v2/groups/${{ inputs.atlas-groupid }}/accessList") + "https://cloud.mongodb.com/api/atlas/v2/groups/${ATLAS_GROUP_ID}/accessList") HTTP_CODE=${RESPONSE: -3} diff --git a/actions/mongodb/start/action.yml b/actions/mongodb/start/action.yml index 78d995d..7fd0688 100644 --- a/actions/mongodb/start/action.yml +++ b/actions/mongodb/start/action.yml @@ -1,24 +1,92 @@ name: Start MongoDB -description: Starts a local MongoDB database +description: | + Starts a local MongoDB server on 127.0.0.1, with mongosh on the path, + from the official archives with their SHA256 checksums verified rather than the apt packages, + so neither root nor sudo is needed, on a hosted runner or in a container. + Linux x64 or arm64 runners on Ubuntu 24.04 only (for example ubuntu-latest or ubuntu-24.04-arm). + +inputs: + mongodb-version: + description: Version of the MongoDB server, an 8.0 one for instance, the latest release (9.0.4) when empty (check latest from https://www.mongodb.com/try/download/community) + required: false + default: "" + mongosh-version: + description: Version of mongosh (check latest from https://github.com/mongodb-js/mongosh/releases) + required: false + default: "2.14.0" + mongosh-sha256: + description: SHA256 of the mongosh archive for x64, given with any other mongosh-version, since its release publishes no checksum file + required: false + default: "c1c5f1a9709e91c5eb1c6861a99c305c165fa564afb465fabfd2b2334c08a23c" + mongosh-sha256-arm64: + description: SHA256 of the mongosh archive for arm64, given with any other mongosh-version + required: false + default: "3c6a3631667f9caf0a7b5dfe87e433f973f0a82f2cf04130cd52115076e945e4" + port: + description: Port the server listens on + required: false + default: "27017" + replica-set: + description: Name of a single-node replica set the server starts as, for an application writing in transactions, which a standalone server refuses, standalone when empty + required: false + default: "" runs: using: "composite" steps: - # ref. https://www.mongodb.com/docs/manual/tutorial/install-mongodb-on-ubuntu/ - name: Install & start MongoDB - run: | - curl -fsSL https://www.mongodb.org/static/pgp/server-8.0.asc | \ - sudo gpg -o /usr/share/keyrings/mongodb-server-8.0.gpg \ - --dearmor - echo "deb [ arch=amd64,arm64 signed-by=/usr/share/keyrings/mongodb-server-8.0.gpg ] https://repo.mongodb.org/apt/ubuntu noble/mongodb-org/8.2 multiverse" | - sudo tee /etc/apt/sources.list.d/mongodb-org-8.2.list - sudo apt-get update - sudo apt-get install -y mongodb-org - sudo systemctl start mongod - timeout 30 bash -c " - until mongosh --host localhost:27017 --eval 'db.runCommand({ ping: 1 })' >/dev/null 2>&1; do - echo 'MongoDB not ready yet, retrying in 1 second...' - sleep 1 - done - " shell: bash + env: + # The one place naming the default version, which the workflows calling this action leave empty. + MONGODB_ARCHIVE: mongodb-linux-${{ runner.arch == 'ARM64' && 'aarch64' || 'x86_64' }}-ubuntu2404-${{ inputs.mongodb-version || '9.0.4' }}.tgz + MONGOSH_VERSION: ${{ inputs.mongosh-version }} + MONGOSH_ARCHIVE: mongosh-${{ inputs.mongosh-version }}-linux-${{ runner.arch == 'ARM64' && 'arm64' || 'x64' }}.tgz + MONGOSH_SHA256: ${{ runner.arch == 'ARM64' && inputs.mongosh-sha256-arm64 || inputs.mongosh-sha256 }} + MONGODB_PORT: ${{ inputs.port }} + REPLICA_SET: ${{ inputs.replica-set }} + run: | + # Downloaded into a directory of its own, since the repository may hold a file or a directory named like an archive. + cd "$(mktemp -d)" + + echo "Downloading MongoDB server and mongosh..." + curl -fsSL -O "https://fastdl.mongodb.org/linux/${MONGODB_ARCHIVE}" + curl -fsSL -O "https://fastdl.mongodb.org/linux/${MONGODB_ARCHIVE}.sha256" + curl -fsSL -O "https://github.com/mongodb-js/mongosh/releases/download/v${MONGOSH_VERSION}/${MONGOSH_ARCHIVE}" + + echo "Verifying checksums..." + sha256sum --check --status "${MONGODB_ARCHIVE}.sha256" + echo "${MONGOSH_SHA256} ${MONGOSH_ARCHIVE}" | sha256sum --check --status + + # Installed where the job owns the directory, so neither root nor sudo is needed. + mkdir -p "${RUNNER_TEMP}/bin" "${RUNNER_TEMP}/mongodb/data" + tar xzf "${MONGODB_ARCHIVE}" --strip-components=2 -C "${RUNNER_TEMP}/bin" --wildcards "*/bin/mongod" + tar xzf "${MONGOSH_ARCHIVE}" --strip-components=2 -C "${RUNNER_TEMP}/bin" --wildcards "*/bin/mongosh" + echo "${RUNNER_TEMP}/bin" >> "$GITHUB_PATH" + export PATH="${RUNNER_TEMP}/bin:$PATH" + rm -f "${MONGODB_ARCHIVE}" "${MONGODB_ARCHIVE}.sha256" "${MONGOSH_ARCHIVE}" + + # Forked rather than started by systemd, which a container has no instance of. + log="${RUNNER_TEMP}/mongodb/mongod.log" + args=(--dbpath "${RUNNER_TEMP}/mongodb/data" --bind_ip 127.0.0.1 --port "$MONGODB_PORT" --fork --logpath "$log") + check='db.runCommand({ ping: 1 })' + if [[ -n "$REPLICA_SET" ]]; then + args+=(--replSet "$REPLICA_SET") + # Ready once the node is primary, the set being initiated by the first check the server answers. + check="try { rs.status() } catch (e) { rs.initiate({ _id: '${REPLICA_SET}', members: [{ _id: 0, host: '127.0.0.1:${MONGODB_PORT}' }] }) } quit(db.hello().isWritablePrimary ? 0 : 1)" + fi + # The log is the only place saying why a server did not start, and it is gone with the runner. + if ! mongod "${args[@]}"; then + cat "$log" + exit 1 + fi + for _ in $(seq 1 30); do + if mongosh --host "127.0.0.1:${MONGODB_PORT}" --quiet --eval "$check" >/dev/null 2>&1; then + mongod --version | head -1 + exit 0 + fi + echo "MongoDB not ready yet, retrying in 1 second..." + sleep 1 + done + echo "::error::MongoDB was not ready within 30 seconds" + cat "$log" + exit 1 diff --git a/actions/npm/publish/action.yml b/actions/npm/publish/action.yml new file mode 100644 index 0000000..f29e2ed --- /dev/null +++ b/actions/npm/publish/action.yml @@ -0,0 +1,67 @@ +name: Publish an npm package +description: | + Publishes the package of the working directory unless its version is already on the registry, + which refuses to publish over it, so a push that did not bump the version is skipped rather than failed. + The registry and the credentials are those of the .npmrc that actions/setup-node writes with registry-url: + a token given as NODE_AUTH_TOKEN, or the OIDC token of npm trusted publishing (permission id-token write). + A run with neither, as a local one is, packs and checks the package with --dry-run added to the command, the registry refusing to answer it. + +inputs: + publish-command: + description: Command publishing the package, "pnpm publish --no-git-checks" for a pnpm workspace for instance + required: false + default: "npm publish" + working-directory: + description: Directory holding the package.json of the package + required: false + default: "." + +outputs: + published: + description: "\"true\" when the package was published, \"false\" when its version already was" + value: ${{ steps.registry.outputs.exists == 'false' && 'true' || 'false' }} + version: + description: Version of the package, read from its package.json + value: ${{ steps.registry.outputs.version }} + +runs: + using: "composite" + steps: + - name: Check whether the version is already published + id: registry + shell: bash + working-directory: ${{ inputs.working-directory }} + run: | + name=$(node -p "require('./package.json').name") + version=$(node -p "require('./package.json').version") + echo "version=${version}" >> "$GITHUB_OUTPUT" + # npm answers E404 for a package or a version the registry does not have, and anything else is a failure to read it. + errors=$(mktemp) + if npm view "${name}@${version}" version > /dev/null 2> "$errors"; then + echo "${name}@${version} is already published, nothing to do" + echo "exists=true" >> "$GITHUB_OUTPUT" + elif grep -q E404 "$errors"; then + echo "${name}@${version} is not published yet" + echo "exists=false" >> "$GITHUB_OUTPUT" + elif grep -qE 'E401|ENEEDAUTH' "$errors" && [[ -z "${NODE_AUTH_TOKEN:-}" && -z "${ACTIONS_ID_TOKEN_REQUEST_URL:-}" ]]; then + echo "No credential for the registry, so ${name}@${version} is packed and checked with --dry-run instead" + echo "exists=dry-run" >> "$GITHUB_OUTPUT" + else + cat "$errors" >&2 + exit 1 + fi + - name: Publish the package + if: steps.registry.outputs.exists == 'false' + shell: bash + working-directory: ${{ inputs.working-directory }} + env: + PUBLISH_COMMAND: ${{ inputs.publish-command }} + # Shell code by contract, run from a variable rather than pasted into the script by GitHub. + run: eval "$PUBLISH_COMMAND" + - name: Check the package without publishing it + if: steps.registry.outputs.exists == 'dry-run' + shell: bash + working-directory: ${{ inputs.working-directory }} + env: + PUBLISH_COMMAND: ${{ inputs.publish-command }} + run: eval "$PUBLISH_COMMAND --dry-run" diff --git a/actions/opengrep/setup/action.yml b/actions/opengrep/setup/action.yml new file mode 100644 index 0000000..3294030 --- /dev/null +++ b/actions/opengrep/setup/action.yml @@ -0,0 +1,39 @@ +name: Setup Opengrep +description: | + Installs Opengrep by downloading the official binary from its GitHub release and verifying its SHA256 checksum, pinned here since the release publishes signatures but no checksum file + (replacement for its install.sh, which a pipeline would run from the main branch of the project, unpinned). + Linux x64 or arm64 runners only (for example ubuntu-latest or ubuntu-24.04-arm). + +inputs: + opengrep-version: + description: Version of Opengrep to install (check latest from https://github.com/opengrep/opengrep/releases) + required: false + default: "v1.32.0" + opengrep-sha256: + description: SHA256 of the Opengrep binary for x64, given with any other opengrep-version, since its release publishes no checksum file + required: false + default: "7360d60174880bd9214f2dfbe439b7bce82b7a3d44cfe945ba833c85cc8bda75" + opengrep-sha256-arm64: + description: SHA256 of the Opengrep binary for arm64, given with any other opengrep-version + required: false + default: "dc6229319c0d68948733053786a5614873947e5ec1d4669cc0e94ae9038cb99d" + +runs: + using: "composite" + steps: + - name: Install Opengrep + shell: bash + env: + OPENGREP_VERSION: ${{ inputs.opengrep-version }} + OPENGREP_BINARY: opengrep_manylinux_${{ runner.arch == 'ARM64' && 'aarch64' || 'x86' }} + OPENGREP_SHA256: ${{ runner.arch == 'ARM64' && inputs.opengrep-sha256-arm64 || inputs.opengrep-sha256 }} + run: | + # Downloaded into a directory of its own, since the repository may hold a file or a directory named like the binary. + cd "$(mktemp -d)" + curl -fsSL -O "https://github.com/opengrep/opengrep/releases/download/${OPENGREP_VERSION}/${OPENGREP_BINARY}" + echo "${OPENGREP_SHA256} ${OPENGREP_BINARY}" | sha256sum --check --status + # Installed where the job owns the directory, so neither root nor sudo is needed, on a hosted runner or in a container. + mkdir -p "${RUNNER_TEMP}/bin" + install -m 0755 "${OPENGREP_BINARY}" "${RUNNER_TEMP}/bin/opengrep" + echo "${RUNNER_TEMP}/bin" >> "$GITHUB_PATH" + "${RUNNER_TEMP}/bin/opengrep" --version diff --git a/actions/playwright/install/action.yml b/actions/playwright/install/action.yml new file mode 100644 index 0000000..7e84d18 --- /dev/null +++ b/actions/playwright/install/action.yml @@ -0,0 +1,36 @@ +name: Install Playwright browsers +description: | + Installs the browsers of the Playwright version the project depends on, with their system packages on a hosted runner. + The browser build has to match that version, so it is installed after the dependencies (npm ci) rather than baked into an image. + System packages need root, which a hosted runner gets through sudo, and a container job's image already installed them. + +inputs: + browsers: + description: Browsers to install, separated by spaces + required: false + default: chromium + working-directory: + description: Directory of the project depending on Playwright + required: false + default: "." + +runs: + using: "composite" + steps: + - name: Install the system packages of the browsers + if: runner.environment == 'github-hosted' + shell: bash + working-directory: ${{ inputs.working-directory }} + env: + BROWSERS: ${{ inputs.browsers }} + run: | + read -ra browser_list <<< "$BROWSERS" + npx --no-install playwright install-deps "${browser_list[@]}" + - name: Install the browsers + shell: bash + working-directory: ${{ inputs.working-directory }} + env: + BROWSERS: ${{ inputs.browsers }} + run: | + read -ra browser_list <<< "$BROWSERS" + npx --no-install playwright install "${browser_list[@]}" diff --git a/actions/qemu/setup/action.yml b/actions/qemu/setup/action.yml new file mode 100644 index 0000000..5fdc892 --- /dev/null +++ b/actions/qemu/setup/action.yml @@ -0,0 +1,53 @@ +name: Setup QEMU +description: | + Registers QEMU user emulation for other platforms than the runner's own, so that Docker builds and runs their images, + from the qemu-user-static package of the Ubuntu archive (qemu-user-binfmt from Ubuntu 26.04), which apt verifies against the keys of the runner's own distribution + (replacement for docker/setup-qemu-action and its tonistiigi/binfmt image, to run no third party image with privileges). + Linux runners on Ubuntu only (for example ubuntu-latest), with sudo, since the registration is made in the kernel. + +inputs: + platforms: + description: Comma-separated platforms to check the emulation of, as Docker names them (for example "linux/arm64,linux/arm/v7") + required: false + default: "linux/arm64" + +runs: + using: "composite" + steps: + - name: Install QEMU + shell: bash + run: | + sudo apt-get update -qq + # From QEMU 9.1 (Ubuntu 26.04), qemu-user ships the static binaries and qemu-user-static is only a virtual package, which apt cannot install. + package=qemu-user-static + if apt-cache show --no-all-versions qemu-user-binfmt | grep -q '^Provides:.*\bqemu-user-static\b'; then + package=qemu-user-binfmt + fi + sudo DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends "$package" + # The package ships its registrations in binfmt.d, which systemd-binfmt loads into the kernel. + sudo /usr/lib/systemd/systemd-binfmt + - name: Check the emulation of each platform + shell: bash + env: + PLATFORMS: ${{ inputs.platforms }} + run: | + IFS=',' read -ra requested <<< "$PLATFORMS" + for platform in "${requested[@]}"; do + platform="${platform// /}" + case "$platform" in + linux/amd64|linux/386) continue ;; + linux/arm64*) name=aarch64 ;; + linux/arm/*) name=arm ;; + linux/ppc64le) name=ppc64le ;; + linux/riscv64) name=riscv64 ;; + linux/s390x) name=s390x ;; + *) echo "ERROR: No QEMU emulation known for ${platform}" >&2; exit 1 ;; + esac + registration="/proc/sys/fs/binfmt_misc/qemu-${name}" + # The F flag opens the interpreter at registration, which is how a container without it in its file system runs the binaries. + if ! grep -qx enabled "$registration" 2>/dev/null || ! grep -q '^flags: .*F' "$registration"; then + echo "ERROR: ${platform} is not emulated: ${registration} is missing, disabled or without the F flag" >&2 + exit 1 + fi + echo "${platform} is emulated by $(grep '^interpreter' "$registration")" + done diff --git a/actions/sonar/scan/action.yml b/actions/sonar/scan/action.yml new file mode 100644 index 0000000..7eb3f4c --- /dev/null +++ b/actions/sonar/scan/action.yml @@ -0,0 +1,119 @@ +name: Analyze with Sonar +description: | + Installs the Sonar scanner CLI from the official archive, its SHA256 checksum verified, with the Java runtime it bundles, + and analyzes the code of a project built by another tool than .NET, JavaScript and TypeScript first, with the coverage of its lcov reports + (replacement for SonarSource/sonarqube-scan-action, to reduce third-party GitHub Action supply-chain risk). + Linux x64 or arm64 runners only (for example ubuntu-latest or ubuntu-24.04-arm). + +inputs: + scanner-version: + description: Version of the Sonar scanner CLI (check latest from https://github.com/SonarSource/sonar-scanner-cli/releases) + required: false + default: "8.2.0.6438" + scanner-sha256: + description: SHA256 of the scanner archive for x64, given with any other scanner-version + required: false + default: "83257bf97a475d626b8ba2b214aeeb37334ca84db1b3e3be5105783e4facb053" + scanner-sha256-arm64: + description: SHA256 of the scanner archive for arm64, given with any other scanner-version + required: false + default: "3add94b481f1d541897b8d667f4c0e86bc7855bd3eaeb1a7c9ee5f162b909e6e" + coverage-paths: + description: lcov reports of the coverage, relative to the working directory, separated by commas, coverage/lcov.info when empty and the project has no sonar-project.properties + required: false + default: "" + cpd-exclusions: + description: Files or directories that should be excluded from Sonar code duplication analysis + required: false + default: "" + exclusions: + description: Files or directories that should be excluded from Sonar analysis, **/node_modules/** when empty and the project has no sonar-project.properties + required: false + default: "" + host-url: + description: Sonar host URL + required: true + organization: + description: Sonar organization, which SonarCloud needs and a SonarQube server has no notion of + required: false + default: "" + project-key: + description: Sonar project key, which sonar-project.properties gives otherwise + required: false + default: "" + project-name: + description: Sonar project name + required: false + default: "" + token: + description: Sonar token for login + required: true + working-directory: + description: Directory of the project to analyze, relative to the workspace, which a composite action does not take from the defaults of the calling job + required: false + default: "." + +runs: + using: "composite" + steps: + - name: Install Sonar scanner + shell: bash + env: + SCANNER_VERSION: ${{ inputs.scanner-version }} + SCANNER_PLATFORM: linux-${{ runner.arch == 'ARM64' && 'aarch64' || 'x64' }} + SCANNER_SHA256: ${{ runner.arch == 'ARM64' && inputs.scanner-sha256-arm64 || inputs.scanner-sha256 }} + run: | + # Downloaded into a directory of its own, since the repository may hold a file or a directory named like the archive. + cd "$(mktemp -d)" + archive="sonar-scanner-cli-${SCANNER_VERSION}-${SCANNER_PLATFORM}.zip" + curl -fsSL -O "https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/${archive}" + echo "${SCANNER_SHA256} ${archive}" | sha256sum --check --status + unzip -q "$archive" + # Installed where the job owns the directory, so neither root nor sudo is needed, on a hosted runner or in a container. + rm -rf "${RUNNER_TEMP}/sonar-scanner" + mv "sonar-scanner-${SCANNER_VERSION}-${SCANNER_PLATFORM}" "${RUNNER_TEMP}/sonar-scanner" + rm -f "$archive" + echo "${RUNNER_TEMP}/sonar-scanner/bin" >> "$GITHUB_PATH" + # The analyzers Sonar downloads land in its user home. + - name: Cache Sonar packages + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: ~/.sonar/cache + key: ${{ runner.os }}-sonar + restore-keys: ${{ runner.os }}-sonar + # A sonar-project.properties is the configuration of the project, which an input only overrides when it is given. + - name: Analyze the code + shell: bash + working-directory: ${{ inputs.working-directory }} + env: + COVERAGE_PATHS: ${{ inputs.coverage-paths }} + SONAR_CPD_EXCLUSIONS: ${{ inputs.cpd-exclusions }} + SONAR_EXCLUSIONS: ${{ inputs.exclusions }} + SONAR_HOST_URL: ${{ inputs.host-url }} + SONAR_ORGANIZATION: ${{ inputs.organization }} + SONAR_PROJECT_KEY: ${{ inputs.project-key }} + SONAR_PROJECT_NAME: ${{ inputs.project-name }} + SONAR_TOKEN: ${{ inputs.token }} + run: | + if [ ! -f sonar-project.properties ]; then + if [ -z "$SONAR_PROJECT_KEY" ]; then + echo "ERROR: No project-key, and no sonar-project.properties to read it from" >&2 + exit 1 + fi + COVERAGE_PATHS="${COVERAGE_PATHS:-coverage/lcov.info}" + SONAR_EXCLUSIONS="${SONAR_EXCLUSIONS:-**/node_modules/**}" + fi + # The token is read from SONAR_TOKEN, rather than given on the command line any process can list. + args=(-Dsonar.host.url="$SONAR_HOST_URL") + add() { + if [ -n "$2" ]; then + args+=("-D$1=$2") + fi + } + add sonar.cpd.exclusions "$SONAR_CPD_EXCLUSIONS" + add sonar.exclusions "$SONAR_EXCLUSIONS" + add sonar.javascript.lcov.reportPaths "$COVERAGE_PATHS" + add sonar.organization "$SONAR_ORGANIZATION" + add sonar.projectKey "$SONAR_PROJECT_KEY" + add sonar.projectName "$SONAR_PROJECT_NAME" + sonar-scanner "${args[@]}" diff --git a/actions/syft/generate-sbom/action.yml b/actions/syft/generate-sbom/action.yml index b06e833..c8cfc4c 100644 --- a/actions/syft/generate-sbom/action.yml +++ b/actions/syft/generate-sbom/action.yml @@ -3,13 +3,13 @@ description: | Installs Syft by downloading the official binary from its GitHub release, verifying its SHA256 checksum, and uses it to generate a Software Bill of Materials for a container image or a source directory (replacement for anchore/sbom-action, to reduce third-party GitHub Action supply-chain risk). - Linux runners only (for example ubuntu-latest). + Linux x64 or arm64 runners only (for example ubuntu-latest or ubuntu-24.04-arm). inputs: syft-version: description: Version of Syft to install (check latest from https://github.com/anchore/syft/releases) required: false - default: "1.49.0" + default: "1.55.0" target-image: description: Container image reference to scan (mutually exclusive with target-path) required: false @@ -18,6 +18,10 @@ inputs: description: Source directory to scan, used only when target-image is not set required: false default: "." + exclude-paths: + description: Paths left out of a source directory's SBOM, relative to target-path, one per line + required: false + default: "" output-format: description: SBOM format (see https://github.com/anchore/syft#supported-output-formats) required: false @@ -42,7 +46,7 @@ runs: shell: bash env: SYFT_VERSION: ${{ inputs.syft-version }} - SYFT_TARBALL: syft_${{ inputs.syft-version }}_linux_amd64.tar.gz + SYFT_TARBALL: syft_${{ inputs.syft-version }}_linux_${{ runner.arch == 'ARM64' && 'arm64' || 'amd64' }}.tar.gz SYFT_CHECKSUMS: syft_${{ inputs.syft-version }}_checksums.txt run: | # Downloaded into a directory of its own, since the repository may hold a file or a directory named like the binary. @@ -50,13 +54,11 @@ runs: BASE_URL="https://github.com/anchore/syft/releases/download/v${SYFT_VERSION}" echo "Downloading Syft binary and checksums..." - curl -sSL -O "${BASE_URL}/${SYFT_TARBALL}" - curl -sSL -O "${BASE_URL}/${SYFT_CHECKSUMS}" + curl -fsSL -O "${BASE_URL}/${SYFT_TARBALL}" + curl -fsSL -O "${BASE_URL}/${SYFT_CHECKSUMS}" echo "Verifying checksum..." - grep " ${SYFT_TARBALL}$" "${SYFT_CHECKSUMS}" | sha256sum --check --status - - if [ $? -eq 0 ]; then + if grep " ${SYFT_TARBALL}$" "${SYFT_CHECKSUMS}" | sha256sum --check --status; then echo "Verification successful!" tar -xzf "${SYFT_TARBALL}" syft chmod +x syft @@ -76,6 +78,7 @@ runs: env: TARGET_IMAGE: ${{ inputs.target-image }} TARGET_PATH: ${{ inputs.target-path }} + EXCLUDE_PATHS: ${{ inputs.exclude-paths }} OUTPUT_FORMAT: ${{ inputs.output-format }} OUTPUT_FILE: ${{ inputs.output-file }} run: | @@ -85,11 +88,19 @@ runs: SOURCE="dir:${TARGET_PATH}" fi + # Syft matches the exclusions of a directory against paths starting with ./ + args=() + while IFS= read -r path; do + if [ -n "$path" ]; then + args+=(--exclude "./${path#./}") + fi + done <<< "$EXCLUDE_PATHS" + echo "Scanning ${SOURCE} with Syft..." - syft "$SOURCE" -o "${OUTPUT_FORMAT}=${OUTPUT_FILE}" + syft "$SOURCE" "${args[@]}" -o "${OUTPUT_FORMAT}=${OUTPUT_FILE}" - name: Upload SBOM as artifact if: ${{ inputs.upload-artifact == 'true' }} - uses: actions/upload-artifact@v7 + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 with: name: ${{ inputs.artifact-name }} path: ${{ inputs.output-file }} diff --git a/actions/task/setup/action.yml b/actions/task/setup/action.yml new file mode 100644 index 0000000..1d5a75a --- /dev/null +++ b/actions/task/setup/action.yml @@ -0,0 +1,44 @@ +name: Setup Task +description: | + Installs Task (taskfile.dev) by downloading the official binary from its GitHub release and verifying its SHA256 checksum + (replacement for go-task/setup-task and for an install through npm, to reduce supply-chain risk). + Linux x64 or arm64 runners only (for example ubuntu-latest or ubuntu-24.04-arm). + +inputs: + task-version: + description: Version of Task to install (check latest from https://github.com/go-task/task/releases) + required: false + default: "v3.54.0" + +runs: + using: "composite" + steps: + - name: Install Task + shell: bash + env: + TASK_VERSION: ${{ inputs.task-version }} + TASK_TARBALL: task_linux_${{ runner.arch == 'ARM64' && 'arm64' || 'amd64' }}.tar.gz + run: | + # Downloaded into a directory of its own, since the repository may hold a file or a directory named like the binary. + cd "$(mktemp -d)" + BASE_URL="https://github.com/go-task/task/releases/download/${TASK_VERSION}" + + echo "Downloading Task binary and checksums..." + curl -fsSL -O "${BASE_URL}/${TASK_TARBALL}" + curl -fsSL -O "${BASE_URL}/task_checksums.txt" + + echo "Verifying checksum..." + if grep " ${TASK_TARBALL}$" task_checksums.txt | sha256sum --check --status; then + echo "Verification successful!" + tar -xzf "${TASK_TARBALL}" task + # Installed where the job owns the directory, so neither root nor sudo is needed, on a hosted runner or in a container. + mkdir -p "${RUNNER_TEMP}/bin" + mv task "${RUNNER_TEMP}/bin/task" + echo "${RUNNER_TEMP}/bin" >> "$GITHUB_PATH" + export PATH="${RUNNER_TEMP}/bin:$PATH" + rm -f "${TASK_TARBALL}" task_checksums.txt + task --version + else + echo "ERROR: Checksum verification failed!" >&2 + exit 1 + fi diff --git a/actions/terraform/setup/action.yml b/actions/terraform/setup/action.yml index 8fd1dab..79ae72a 100644 --- a/actions/terraform/setup/action.yml +++ b/actions/terraform/setup/action.yml @@ -2,13 +2,13 @@ name: Setup Terraform description: | Installs the Terraform CLI by downloading the official binary from HashiCorp's release server and verifying its SHA256 checksum (replacement for hashicorp/setup-terraform, to reduce third-party GitHub Action supply-chain risk). - Linux runners only (for example ubuntu-latest). + Linux x64 or arm64 runners only (for example ubuntu-latest or ubuntu-24.04-arm). inputs: terraform-version: description: Version of Terraform to install (check latest from https://releases.hashicorp.com/terraform/) required: false - default: "1.15.8" + default: "1.17.0" runs: using: "composite" @@ -17,7 +17,7 @@ runs: shell: bash env: TERRAFORM_VERSION: ${{ inputs.terraform-version }} - TERRAFORM_ZIP: terraform_${{ inputs.terraform-version }}_linux_amd64.zip + TERRAFORM_ZIP: terraform_${{ inputs.terraform-version }}_linux_${{ runner.arch == 'ARM64' && 'arm64' || 'amd64' }}.zip TERRAFORM_SHA256SUMS: terraform_${{ inputs.terraform-version }}_SHA256SUMS run: | # Downloaded into a directory of its own, since the repository may hold a file or a directory named like the binary. @@ -25,13 +25,11 @@ runs: BASE_URL="https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}" echo "Downloading Terraform binary and checksums..." - curl -sSL -O "${BASE_URL}/${TERRAFORM_ZIP}" - curl -sSL -O "${BASE_URL}/${TERRAFORM_SHA256SUMS}" + curl -fsSL -O "${BASE_URL}/${TERRAFORM_ZIP}" + curl -fsSL -O "${BASE_URL}/${TERRAFORM_SHA256SUMS}" echo "Verifying checksum..." - grep " ${TERRAFORM_ZIP}$" "${TERRAFORM_SHA256SUMS}" | sha256sum --check --status - - if [ $? -eq 0 ]; then + if grep " ${TERRAFORM_ZIP}$" "${TERRAFORM_SHA256SUMS}" | sha256sum --check --status; then echo "Verification successful!" unzip -o "${TERRAFORM_ZIP}" chmod +x terraform diff --git a/actions/tflint/setup/action.yml b/actions/tflint/setup/action.yml index c60b0aa..bf2d218 100644 --- a/actions/tflint/setup/action.yml +++ b/actions/tflint/setup/action.yml @@ -2,13 +2,13 @@ name: Setup TFLint description: | Installs TFLint by downloading the official binary from its GitHub release and verifying its SHA256 checksum (replacement for terraform-linters/setup-tflint, to reduce third-party GitHub Action supply-chain risk). - Linux runners only (for example ubuntu-latest). + Linux x64 or arm64 runners only (for example ubuntu-latest or ubuntu-24.04-arm). inputs: tflint-version: description: Version of TFLint to install (check latest from https://github.com/terraform-linters/tflint/releases) required: false - default: "v0.60.0" + default: "v0.65.0" runs: using: "composite" @@ -17,20 +17,18 @@ runs: shell: bash env: TFLINT_VERSION: ${{ inputs.tflint-version }} - TFLINT_ZIP: tflint_linux_amd64.zip + TFLINT_ZIP: tflint_linux_${{ runner.arch == 'ARM64' && 'arm64' || 'amd64' }}.zip run: | # Downloaded into a directory of its own, since the repository may hold a file or a directory named like the binary. cd "$(mktemp -d)" BASE_URL="https://github.com/terraform-linters/tflint/releases/download/${TFLINT_VERSION}" echo "Downloading TFLint binary and checksums..." - curl -sSL -O "${BASE_URL}/${TFLINT_ZIP}" - curl -sSL -O "${BASE_URL}/checksums.txt" + curl -fsSL -O "${BASE_URL}/${TFLINT_ZIP}" + curl -fsSL -O "${BASE_URL}/checksums.txt" echo "Verifying checksum..." - grep " ${TFLINT_ZIP}$" checksums.txt | sha256sum --check --status - - if [ $? -eq 0 ]; then + if grep " ${TFLINT_ZIP}$" checksums.txt | sha256sum --check --status; then echo "Verification successful!" unzip -o "${TFLINT_ZIP}" chmod +x tflint diff --git a/actions/trivy/scan/action.yml b/actions/trivy/scan/action.yml index fd0eb3d..5a36c52 100644 --- a/actions/trivy/scan/action.yml +++ b/actions/trivy/scan/action.yml @@ -3,13 +3,13 @@ description: | Installs Trivy by downloading the official binary from its GitHub release and verifying its SHA256 checksum, then scans a container image or a source directory for vulnerabilities or misconfigurations (replacement for aquasecurity/trivy-action, to reduce third-party GitHub Action supply-chain risk). - Linux runners only (for example ubuntu-latest). + Linux x64 or arm64 runners only (for example ubuntu-latest or ubuntu-24.04-arm). inputs: trivy-version: description: Version of Trivy to install (check latest from https://github.com/aquasecurity/trivy/releases) required: false - default: "0.72.0" + default: "0.76.1" scan-type: description: What to scan, "image" (container image reference) or "config" (misconfiguration/IaC scan of a directory) required: false @@ -41,6 +41,10 @@ inputs: description: Comma-separated package types to scan, os and/or library (scan-type "image" only) required: false default: "os,library" + ignore-file: + description: Trivy ignore file (.trivyignore.yaml) naming the findings left out, each with the reason and the day it counts again (expired_at), none when empty + required: false + default: "" runs: using: "composite" @@ -49,7 +53,7 @@ runs: shell: bash env: TRIVY_VERSION: ${{ inputs.trivy-version }} - TRIVY_TARBALL: trivy_${{ inputs.trivy-version }}_Linux-64bit.tar.gz + TRIVY_TARBALL: trivy_${{ inputs.trivy-version }}_Linux-${{ runner.arch == 'ARM64' && 'ARM64' || '64bit' }}.tar.gz TRIVY_CHECKSUMS: trivy_${{ inputs.trivy-version }}_checksums.txt run: | # Downloaded into a directory of its own, since the repository may hold a file or a directory named like the binary. @@ -57,13 +61,11 @@ runs: BASE_URL="https://github.com/aquasecurity/trivy/releases/download/v${TRIVY_VERSION}" echo "Downloading Trivy binary and checksums..." - curl -sSL -O "${BASE_URL}/${TRIVY_TARBALL}" - curl -sSL -O "${BASE_URL}/${TRIVY_CHECKSUMS}" + curl -fsSL -O "${BASE_URL}/${TRIVY_TARBALL}" + curl -fsSL -O "${BASE_URL}/${TRIVY_CHECKSUMS}" echo "Verifying checksum..." - grep " ${TRIVY_TARBALL}$" "${TRIVY_CHECKSUMS}" | sha256sum --check --status - - if [ $? -eq 0 ]; then + if grep " ${TRIVY_TARBALL}$" "${TRIVY_CHECKSUMS}" | sha256sum --check --status; then echo "Verification successful!" tar -xzf "${TRIVY_TARBALL}" trivy chmod +x trivy @@ -89,15 +91,27 @@ runs: EXIT_CODE: ${{ inputs.exit-code }} IGNORE_UNFIXED: ${{ inputs.ignore-unfixed }} PKG_TYPES: ${{ inputs.pkg-types }} + IGNORE_FILE: ${{ inputs.ignore-file }} run: | args=("$SCAN_TYPE" --format "$FORMAT" --severity "$SEVERITY" --exit-code "$EXIT_CODE") + # Trivy goes on without an ignore file it cannot find, which would hide a path given wrong. + if [ -n "$IGNORE_FILE" ]; then + if [ ! -f "$IGNORE_FILE" ]; then + echo "ERROR: Ignore file ${IGNORE_FILE} not found" >&2 + exit 1 + fi + args+=(--ignorefile "$IGNORE_FILE") + fi + if [ -n "$OUTPUT_FILE" ]; then args+=(--output "$OUTPUT_FILE") fi - # ignore-unfixed and pkg-types only make sense for vulnerability (image) scans + # no-progress, ignore-unfixed and pkg-types only exist for vulnerability (image) scans if [ "$SCAN_TYPE" = "image" ]; then + # A CI log is not a terminal, and every redraw of the database download bar becomes log text. + args+=(--no-progress) if [ "$IGNORE_UNFIXED" = "true" ]; then args+=(--ignore-unfixed) fi diff --git a/actions/trivy/setup/action.yml b/actions/trivy/setup/action.yml new file mode 100644 index 0000000..64fac75 --- /dev/null +++ b/actions/trivy/setup/action.yml @@ -0,0 +1,47 @@ +name: Setup Trivy +description: | + Installs Trivy by downloading the official binary from its GitHub release and verifying its SHA256 checksum, + for a job running Trivy itself, where the Trivy scan action installs and runs it in one step. + Linux x64 or arm64 runners only (for example ubuntu-latest or ubuntu-24.04-arm). + +inputs: + trivy-version: + description: Version of Trivy to install (check latest from https://github.com/aquasecurity/trivy/releases) + required: false + default: "0.76.1" + +runs: + using: "composite" + steps: + # The same install as the Trivy scan action, which a composite action of this repository cannot call from a caller's workspace. + - name: Install Trivy + shell: bash + env: + TRIVY_VERSION: ${{ inputs.trivy-version }} + TRIVY_TARBALL: trivy_${{ inputs.trivy-version }}_Linux-${{ runner.arch == 'ARM64' && 'ARM64' || '64bit' }}.tar.gz + TRIVY_CHECKSUMS: trivy_${{ inputs.trivy-version }}_checksums.txt + run: | + # Downloaded into a directory of its own, since the repository may hold a file or a directory named like the binary. + cd "$(mktemp -d)" + BASE_URL="https://github.com/aquasecurity/trivy/releases/download/v${TRIVY_VERSION}" + + echo "Downloading Trivy binary and checksums..." + curl -fsSL -O "${BASE_URL}/${TRIVY_TARBALL}" + curl -fsSL -O "${BASE_URL}/${TRIVY_CHECKSUMS}" + + echo "Verifying checksum..." + if grep " ${TRIVY_TARBALL}$" "${TRIVY_CHECKSUMS}" | sha256sum --check --status; then + echo "Verification successful!" + tar -xzf "${TRIVY_TARBALL}" trivy + chmod +x trivy + # Installed where the job owns the directory, so neither root nor sudo is needed, on a hosted runner or in a container. + mkdir -p "${RUNNER_TEMP}/bin" + mv trivy "${RUNNER_TEMP}/bin/trivy" + echo "${RUNNER_TEMP}/bin" >> "$GITHUB_PATH" + export PATH="${RUNNER_TEMP}/bin:$PATH" + rm -f "${TRIVY_TARBALL}" "${TRIVY_CHECKSUMS}" + trivy version + else + echo "ERROR: Checksum verification failed!" >&2 + exit 1 + fi diff --git a/test/check_workflow_parts.py b/test/check_workflow_parts.py new file mode 100755 index 0000000..168c3b2 --- /dev/null +++ b/test/check_workflow_parts.py @@ -0,0 +1,416 @@ +#!/usr/bin/env python3 +"""Checks the composite actions and reusable workflows against the conventions of this repository. + +actionlint reads the workflows only, and knows nothing of what this repository promises its callers, +so each check here guards one of those promises against a regression. +Run by test/run_static_checks.sh, which provides PyYAML and shellcheck. +""" + +import fnmatch +import pathlib +import re +import subprocess +import sys +import tempfile + +import yaml + +ROOT = pathlib.Path(__file__).resolve().parent.parent +SELF_REPOSITORY = "devpro/github-workflow-parts" + +# Parts no CI job can exercise, each with the reason, which the coverage check takes as known. +UNTESTED_PARTS = { + "actions/fossa/analyze": "needs a FOSSA API key", + "actions/mongodb-atlas/add-runner-ip": "needs MongoDB Atlas credentials", + ".github/workflows/reusable-container-publication.yml": "needs registry credentials and an OIDC token to sign", + ".github/workflows/reusable-terraform-deployment.yml": "needs MongoDB Atlas credentials and a Terraform backend", +} + +# Inputs nothing reads, kept since a caller passing an input a reusable workflow does not declare fails. +RESERVED_INPUTS = {} + +TEXT_SUFFIXES = {".cs", ".csproj", ".hcl", ".json", ".md", ".py", ".sh", ".tf", ".yaml", ".yml", ""} +DASHES = (chr(0x2013), chr(0x2014)) +SECOND_PERSON = re.compile(r"\b(you|your|yours|yourself)\b", re.IGNORECASE) +EXPRESSION = re.compile(r"\$\{\{\s*(.*?)\s*\}\}") +UNSAFE_IN_RUN = re.compile(r"^(inputs|secrets|env|steps|vars|matrix|needs|github\.event|github\.head_ref)\b") +INPUT_REFERENCE = re.compile(r"\binputs\.([A-Za-z0-9_-]+)") +SECRET_REFERENCE = re.compile(r"\bsecrets\.([A-Za-z0-9_-]+)") +ACTION_REFERENCE = re.compile(r"^actions/[a-z0-9-]+@[0-9a-f]{40}$") +# A tag can be moved to another commit, a commit cannot, and the comment tells which release it is. +PINNED_LINE = re.compile(r"uses: actions/[a-z0-9-]+@[0-9a-f]{40} # v\d+\.\d+\.\d+$") + +failures = [] + + +def fail(path, message): + failures.append(f"{path}: {message}") + + +def relative(path): + return path.relative_to(ROOT).as_posix() + + +def load(path): + # The base loader keeps every scalar a string, so the "on" key of a workflow is not read as true. + return yaml.load(path.read_text(encoding="utf-8"), Loader=yaml.BaseLoader) + + +def git(*arguments): + return subprocess.run(["git", *arguments], cwd=ROOT, check=True, capture_output=True, text=True).stdout + + +def tracked_files(): + """Maps the files of the working tree git does not ignore to their mode, the one of the index when they are staged.""" + modes = {} + for line in git("ls-files", "-s").splitlines(): + meta, name = line.split("\t", 1) + modes[name] = meta.split()[0] + files = {} + for name in git("ls-files", "--cached", "--others", "--exclude-standard").splitlines(): + path = ROOT / name + if path.is_file(): + files[name] = modes.get(name, "100755" if path.stat().st_mode & 0o100 else "100644") + return files + + +def action_paths(): + return sorted(path.parent for path in (ROOT / "actions").glob("*/*/action.yml")) + + +def workflow_paths(): + return sorted((ROOT / ".github/workflows").glob("*.yml")) + + +def steps_of(document): + """Yields every step of a composite action or of the jobs of a workflow.""" + if "runs" in document: + yield from document["runs"].get("steps", []) + for job in (document.get("jobs") or {}).values(): + yield from job.get("steps", []) + + +def check_text_hygiene(files): + for name, mode in files.items(): + path = ROOT / name + if path.suffix not in TEXT_SUFFIXES or path.name == "LICENSE": + continue + raw = path.read_bytes() + if raw.startswith(b"\xef\xbb\xbf"): + fail(name, "starts with a byte order mark") + if b"\r\n" in raw: + fail(name, "has CRLF line endings, where .editorconfig asks for LF") + if raw and not raw.endswith(b"\n"): + fail(name, "does not end with a new line") + text = raw.decode("utf-8") + for number, line in enumerate(text.splitlines(), 1): + if any(dash in line for dash in DASHES): + fail(f"{name}:{number}", "holds an en or em dash, which the writing style replaces with a colon, a comma or a full stop") + # A quoted source keeps its own words. + if path.suffix in {".md", ".yml", ".yaml"} and name not in {"AGENTS.md", "CLAUDE.md"} and not line.lstrip().startswith(">"): + if SECOND_PERSON.search(line): + fail(f"{name}:{number}", "speaks in the second person") + if path.suffix in {".sh", ".py"}: + if not re.fullmatch(r"[a-z0-9_]+", path.stem): + fail(name, "script name is not snake_case") + if mode != "100755": + fail(name, "script is committed without the executable bit") + if path.suffix in {".yml", ".yaml"}: + try: + load(path) + except yaml.YAMLError as error: + fail(name, f"is not valid YAML: {error}") + + +def check_inputs_are_used(name, text, declared, kind, pattern): + # A commented out step reads nothing. + text = "\n".join(line for line in text.splitlines() if not line.lstrip().startswith("#")) + used = set(pattern.findall(text)) + reserved = RESERVED_INPUTS.get(name, set()) if kind == "input" else set() + for missing in sorted(used - set(declared)): + fail(name, f"references the {kind} '{missing}', which is not declared") + for unused in sorted(set(declared) - used - reserved): + fail(name, f"declares the {kind} '{unused}', which nothing reads") + for stale in sorted(reserved & used): + fail(name, f"lists the input '{stale}' in RESERVED_INPUTS, but reads it") + + +def check_action(directory): + path = directory / "action.yml" + name = relative(path) + document = load(path) + for key in ("name", "description"): + if not str(document.get(key, "")).strip(): + fail(name, f"has no {key}") + runs = document.get("runs") or {} + if runs.get("using") != "composite": + fail(name, "is not a composite action") + inputs = document.get("inputs") or {} + for input_name, spec in inputs.items(): + if not str(spec.get("description", "")).strip(): + fail(name, f"input '{input_name}' has no description") + if spec.get("required") == "true" and "default" in spec: + fail(name, f"input '{input_name}' is required and has a default, which GitHub never uses") + check_inputs_are_used(name, path.read_text(encoding="utf-8"), inputs, "input", INPUT_REFERENCE) + for index, step in enumerate(runs.get("steps", []), 1): + if not step.get("name"): + fail(name, f"step {index} has no name") + if "run" in step and step.get("shell") != "bash": + fail(name, f"step '{step.get('name', index)}' runs a script without 'shell: bash', which a composite action requires and the actions assume") + + +def check_reusable_workflow(path): + name = relative(path) + document = load(path) + if not str(document.get("name", "")).startswith("Reusable - "): + fail(name, "name does not start with 'Reusable - '") + call = (document.get("on") or {}).get("workflow_call") + if call is None: + fail(name, "is not triggered by workflow_call") + return + call = call or {} + text = path.read_text(encoding="utf-8") + inputs = call.get("inputs") or {} + for input_name, spec in inputs.items(): + if not str(spec.get("description", "")).strip(): + fail(name, f"input '{input_name}' has no description") + if not spec.get("type"): + fail(name, f"input '{input_name}' has no type") + check_inputs_are_used(name, text, inputs, "input", INPUT_REFERENCE) + check_inputs_are_used(name, text, call.get("secrets") or {}, "secret", SECRET_REFERENCE) + for step in steps_of(document): + if step.get("with", {}).get("repository") == SELF_REPOSITORY: + # Without it, the CI of this repository would test the workflow parts of the default branch, not those of the commit. + expected = f"github.repository == '{SELF_REPOSITORY}' && github.sha || inputs.workflow-parts-version" + if expected not in step["with"].get("ref", ""): + fail(name, f"checks out the workflow parts without '{expected}' as ref") + + +def check_run_scripts(path, document): + name = relative(path) + for step in steps_of(document): + script = step.get("run") + if script is None: + continue + label = f"step '{step.get('name', '?')}'" + for expression in EXPRESSION.findall(script): + # Shell code included, which a script runs from a variable with eval. + if UNSAFE_IN_RUN.match(expression): + fail(name, f"{label} writes '${{{{ {expression} }}}}' into its script, where an env variable keeps the value from being run as code") + if re.search(r"\$\?\s+-(eq|ne)", script): + fail(name, f"{label} tests $? after a command, which bash -e never reaches when the command fails") + # A comment runs nothing, even when it names a command. + code = "\n".join(line for line in script.splitlines() if not line.lstrip().startswith("#")) + joined = code.replace("\\\n", " ") + for command in re.findall(r"\bcurl\b[^\n|;&)]*", joined): + flags = " ".join(token for token in command.split() if token.startswith("-")) + if not re.search(r"(^|\s)(-[a-zA-Z]*[fw][a-zA-Z]*|--fail|--write-out)\b", flags): + fail(name, f"{label} calls curl without --fail nor --write-out, which saves an HTTP error page as the download") + if "curl" in code and " -O " in joined and "sha256sum --check" not in script: + fail(name, f"{label} downloads a file without verifying its SHA256 checksum") + + +def shellcheck_actions(): + with tempfile.TemporaryDirectory() as directory: + scripts = {} + for action in action_paths(): + document = load(action / "action.yml") + for index, step in enumerate(document["runs"].get("steps", []), 1): + if "run" not in step: + continue + script = pathlib.Path(directory) / f"{action.relative_to(ROOT).as_posix().replace('/', '_')}_{index}.sh" + # An expression becomes a word shellcheck reads as a plain value, as the runner writes it before bash runs. + body = EXPRESSION.sub("expression_value", step["run"]) + script.write_text(f"#!/usr/bin/env bash\nset -eo pipefail\n{body}\n", encoding="utf-8") + scripts[script.name] = f"{relative(action / 'action.yml')} step '{step.get('name', index)}'" + if not scripts: + return + result = subprocess.run( + ["shellcheck", "--shell=bash", "--format=gcc", *sorted(scripts)], + cwd=directory, + capture_output=True, + text=True, + ) + for line in result.stdout.splitlines(): + script, _, message = line.partition(":") + fail(scripts.get(script, script), f"shellcheck line {message.strip()}") + if result.returncode not in (0, 1): + fail("shellcheck", result.stderr.strip()) + + +def local_target(uses): + """Maps a local reference to the action or workflow of this repository it names.""" + for prefix in ("./workflow-parts/", "./"): + if uses.startswith(prefix): + return uses[len(prefix):].rstrip("/") + return None + + +def check_references(path, document): + name = relative(path) + for number, line in enumerate(path.read_text(encoding="utf-8").splitlines(), 1): + if "uses: actions/" in line and not PINNED_LINE.search(line): + fail(f"{name}:{number}", "pins a first party action otherwise than to a full commit SHA followed by '# vX.Y.Z'") + references = [step for step in steps_of(document) if "uses" in step] + references += [job for job in (document.get("jobs") or {}).values() if "uses" in job] + for item in references: + uses = item["uses"] + target = local_target(uses) + if target is None: + # First party actions only, the rest being downloaded with a checksum by the actions of this repository. + if not ACTION_REFERENCE.match(uses): + fail(name, f"uses '{uses}', which is neither local nor a first party action pinned to a commit SHA") + continue + target_path = ROOT / target + given = set((item.get("with") or {}).keys()) + if target.endswith(".yml"): + if not target_path.is_file(): + fail(name, f"calls '{uses}', which does not exist") + continue + declared = (load(target_path)["on"]["workflow_call"] or {}).get("inputs") or {} + else: + if not (target_path / "action.yml").is_file(): + fail(name, f"uses '{uses}', which does not exist") + continue + declared = load(target_path / "action.yml").get("inputs") or {} + for unknown in sorted(given - set(declared)): + fail(name, f"passes '{unknown}' to '{uses}', which does not declare it") + for input_name, spec in declared.items(): + if spec.get("required") == "true" and "default" not in spec and input_name not in given: + fail(name, f"does not pass the required input '{input_name}' to '{uses}'") + + +def check_timeouts(path, document): + # A hung job otherwise holds a runner for the six hours GitHub allows. + for job_id, job in (document.get("jobs") or {}).items(): + if "steps" in job and "timeout-minutes" not in job: + fail(relative(path), f"job '{job_id}' runs steps without timeout-minutes") + + +def check_action_versions(): + versions = {} + for path in [*workflow_paths(), *(action / "action.yml" for action in action_paths())]: + for step in steps_of(load(path)): + uses = step.get("uses", "") + if ACTION_REFERENCE.match(uses): + action, version = uses.split("@") + versions.setdefault(action, {}).setdefault(version, []).append(relative(path)) + for action, by_version in versions.items(): + if len(by_version) > 1: + fail(action, f"is pinned to several versions: {', '.join(f'{v} in {sorted(set(f))}' for v, f in sorted(by_version.items()))}") + + +def check_readme(): + readme = (ROOT / "README.md").read_text(encoding="utf-8") + links = set(re.findall(r"\]\(([^)#]+)\)", readme)) + for link in sorted(links): + if not link.startswith("http") and not (ROOT / link).exists(): + fail("README.md", f"links to '{link}', which does not exist") + linked = {link.rstrip("/").removesuffix("/action.yml") for link in links} + parts = [relative(action) for action in action_paths()] + parts += [relative(path) for path in workflow_paths() if path.name.startswith("reusable-")] + for part in parts: + if part not in linked: + fail("README.md", f"does not list '{part}'") + + +def runs_with(condition, values): + """Tells whether a condition holds for the given inputs, None when it reads anything else than an input.""" + condition = condition.strip() + if condition.startswith("${{") and condition.endswith("}}"): + condition = condition[3:-2].strip() + match = re.fullmatch(r"(!\s*)?inputs\.([A-Za-z0-9_-]+)(?:\s*(==|!=)\s*'([^']*)')?", condition) + if not match or match.group(2) not in values or "${{" in values[match.group(2)]: + return None + negation, name, operator, expected = match.groups() + value = values[name] + if operator: + return (value == expected) == (operator == "==") + return (value not in ("", "false", "0")) != bool(negation) + + +def declared_inputs(path): + document = load(path) + if "runs" in document: + return document.get("inputs") or {} + return (document["on"]["workflow_call"] or {}).get("inputs") or {} + + +def exercised_parts(): + """Collects what the CI workflow runs, directly or through the steps of what it calls that its inputs let run.""" + exercised = set() + pending = [(ROOT / ".github/workflows/ci.yml", None)] + while pending: + path, values = pending.pop() + document = load(path) + items = [job for job in (document.get("jobs") or {}).values() if "uses" in job] + items += [step for step in steps_of(document) if "uses" in step] + for item in items: + target = local_target(item["uses"]) + if target is None: + continue + # A step of the CI itself runs as written, and a step of a called part only when the inputs it is called with let it. + if values is not None and "if" in item and not runs_with(item["if"], values): + continue + target_path = ROOT / target if target.endswith(".yml") else ROOT / target / "action.yml" + called_with = {name: spec.get("default", "") for name, spec in declared_inputs(target_path).items()} + called_with.update(item.get("with") or {}) + exercised.add(target) + pending.append((target_path, called_with)) + return exercised + + +def check_coverage(): + exercised = exercised_parts() + parts = [relative(action) for action in action_paths()] + parts += [relative(path) for path in workflow_paths() if path.name.startswith("reusable-")] + for part in parts: + if part in exercised and part in UNTESTED_PARTS: + fail(part, "is exercised by the CI, so it no longer belongs to UNTESTED_PARTS") + elif part not in exercised and part not in UNTESTED_PARTS: + fail(part, "is exercised by no CI job, nor listed in UNTESTED_PARTS with the reason") + for part in sorted(set(UNTESTED_PARTS) - set(parts)): + fail(part, "is listed in UNTESTED_PARTS, but does not exist") + + +def check_istarci(): + path = ROOT / ".istarci.yml" + if not path.exists(): + return + config = load(path) + runner = config.get("runner") or {} + jobs = list((load(ROOT / ".github/workflows/ci.yml").get("jobs") or {}).keys()) + patterns = list((runner.get("images") or {}).keys()) + list(runner.get("docker") or []) + patterns += list((config.get("workflow") or {}).get("skipJobs") or []) + for pattern in patterns: + if not fnmatch.filter(jobs, pattern): + fail(".istarci.yml", f"pattern '{pattern}' matches no job of the CI workflow") + + +def main(): + check_text_hygiene(tracked_files()) + for action in action_paths(): + check_action(action) + document = load(action / "action.yml") + check_run_scripts(action / "action.yml", document) + check_references(action / "action.yml", document) + for path in workflow_paths(): + document = load(path) + if path.name.startswith("reusable-"): + check_reusable_workflow(path) + check_run_scripts(path, document) + check_references(path, document) + check_timeouts(path, document) + shellcheck_actions() + check_action_versions() + check_readme() + check_coverage() + check_istarci() + for failure in failures: + print(f"FAIL {failure}") + print(f"{len(failures)} failure(s)") + return 1 if failures else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/test/fixtures/container/base/Dockerfile b/test/fixtures/container/base/Dockerfile new file mode 100644 index 0000000..dd3ca4f --- /dev/null +++ b/test/fixtures/container/base/Dockerfile @@ -0,0 +1,3 @@ +# A base image marked as built by this run, which the derived image checks it was built on. +FROM busybox:1.37 +RUN echo built-in-this-run > /base-marker diff --git a/test/fixtures/container/derived/Dockerfile b/test/fixtures/container/derived/Dockerfile new file mode 100644 index 0000000..dab611b --- /dev/null +++ b/test/fixtures/container/derived/Dockerfile @@ -0,0 +1,4 @@ +# The default names a base no registry has, so a build ignoring the base of the run fails rather than passing on a published one. +ARG BASE_IMAGE=registry.invalid/never-published:latest +FROM ${BASE_IMAGE} +RUN test "$(cat /base-marker)" = built-in-this-run diff --git a/test/fixtures/dotnet-library/.editorconfig b/test/fixtures/dotnet-library/.editorconfig new file mode 100644 index 0000000..78de46e --- /dev/null +++ b/test/fixtures/dotnet-library/.editorconfig @@ -0,0 +1,11 @@ +# C# is indented by four, where the rest of this repository is indented by two. +root = true + +[*] +charset = utf-8 +end_of_line = lf +insert_final_newline = true + +[*.cs] +indent_style = space +indent_size = 4 diff --git a/test/fixtures/dotnet-library/.gitignore b/test/fixtures/dotnet-library/.gitignore new file mode 100644 index 0000000..3e4ab67 --- /dev/null +++ b/test/fixtures/dotnet-library/.gitignore @@ -0,0 +1,3 @@ +bin/ +obj/ +output/ diff --git a/test/fixtures/dotnet-library/Fixture.Library.csproj b/test/fixtures/dotnet-library/Fixture.Library.csproj new file mode 100644 index 0000000..64bdfe4 --- /dev/null +++ b/test/fixtures/dotnet-library/Fixture.Library.csproj @@ -0,0 +1,11 @@ + + + + net10.0 + enable + enable + Fixture.Library + 1.0.0 + + + diff --git a/test/fixtures/dotnet-library/Greeting.cs b/test/fixtures/dotnet-library/Greeting.cs new file mode 100644 index 0000000..5aaecdf --- /dev/null +++ b/test/fixtures/dotnet-library/Greeting.cs @@ -0,0 +1,7 @@ +namespace Fixture.Library; + +// Packaged by the CI, which checks that the pack action finds the project in its working directory. +public static class Greeting +{ + public static string Hello() => "Hello"; +} diff --git a/test/fixtures/dotnet/.editorconfig b/test/fixtures/dotnet/.editorconfig new file mode 100644 index 0000000..78de46e --- /dev/null +++ b/test/fixtures/dotnet/.editorconfig @@ -0,0 +1,11 @@ +# C# is indented by four, where the rest of this repository is indented by two. +root = true + +[*] +charset = utf-8 +end_of_line = lf +insert_final_newline = true + +[*.cs] +indent_style = space +indent_size = 4 diff --git a/test/fixtures/dotnet/.gitignore b/test/fixtures/dotnet/.gitignore new file mode 100644 index 0000000..8ee3b66 --- /dev/null +++ b/test/fixtures/dotnet/.gitignore @@ -0,0 +1,3 @@ +bin/ +obj/ +report/ diff --git a/test/fixtures/dotnet/Fixture.Tests.csproj b/test/fixtures/dotnet/Fixture.Tests.csproj new file mode 100644 index 0000000..0c676b0 --- /dev/null +++ b/test/fixtures/dotnet/Fixture.Tests.csproj @@ -0,0 +1,16 @@ + + + + net10.0 + Exe + enable + enable + false + + + + + + + + diff --git a/test/fixtures/dotnet/KeptTests.cs b/test/fixtures/dotnet/KeptTests.cs new file mode 100644 index 0000000..57abe55 --- /dev/null +++ b/test/fixtures/dotnet/KeptTests.cs @@ -0,0 +1,9 @@ +using Xunit; + +namespace Fixture.Tests.Kept; + +public class KeptTests +{ + [Fact] + public void Passes() => Assert.True(true); +} diff --git a/test/fixtures/dotnet/LeftOutTests.cs b/test/fixtures/dotnet/LeftOutTests.cs new file mode 100644 index 0000000..d84a124 --- /dev/null +++ b/test/fixtures/dotnet/LeftOutTests.cs @@ -0,0 +1,10 @@ +using Xunit; + +namespace Fixture.Tests.LeftOut; + +// Fails on purpose: the CI job passes only when the test filter leaves this namespace out. +public class LeftOutTests +{ + [Fact] + public void FailsUnlessLeftOut() => Assert.Fail("the test filter did not leave this test out"); +} diff --git a/test/fixtures/dotnet/global.json b/test/fixtures/dotnet/global.json new file mode 100644 index 0000000..3140116 --- /dev/null +++ b/test/fixtures/dotnet/global.json @@ -0,0 +1,5 @@ +{ + "test": { + "runner": "Microsoft.Testing.Platform" + } +} diff --git a/test/fixtures/go/module/fixture.go b/test/fixtures/go/module/fixture.go new file mode 100644 index 0000000..4cbfe8f --- /dev/null +++ b/test/fixtures/go/module/fixture.go @@ -0,0 +1,7 @@ +// Package fixture is a Go module the Go actions check, formatted, tidy and with nothing for go vet nor govulncheck to report. +package fixture + +// Add returns the sum of two numbers. +func Add(a, b int) int { + return a + b +} diff --git a/test/fixtures/go/module/fixture_test.go b/test/fixtures/go/module/fixture_test.go new file mode 100644 index 0000000..194561f --- /dev/null +++ b/test/fixtures/go/module/fixture_test.go @@ -0,0 +1,9 @@ +package fixture + +import "testing" + +func TestAdd(t *testing.T) { + if Add(1, 2) != 3 { + t.Fatal("1 + 2 is not 3") + } +} diff --git a/test/fixtures/go/module/go.mod b/test/fixtures/go/module/go.mod new file mode 100644 index 0000000..50a0545 --- /dev/null +++ b/test/fixtures/go/module/go.mod @@ -0,0 +1,3 @@ +module example.com/fixture + +go 1.27 diff --git a/test/fixtures/go/unformatted/go.mod b/test/fixtures/go/unformatted/go.mod new file mode 100644 index 0000000..25588eb --- /dev/null +++ b/test/fixtures/go/unformatted/go.mod @@ -0,0 +1,3 @@ +module example.com/unformatted + +go 1.27 diff --git a/test/fixtures/go/unformatted/unformatted.go b/test/fixtures/go/unformatted/unformatted.go new file mode 100644 index 0000000..6475340 --- /dev/null +++ b/test/fixtures/go/unformatted/unformatted.go @@ -0,0 +1,4 @@ +// Package unformatted is not formatted by gofmt, which the lint action fails on. +package unformatted + +func Value( ) int {return 1} diff --git a/test/fixtures/go/untidy/go.mod b/test/fixtures/go/untidy/go.mod new file mode 100644 index 0000000..b662c07 --- /dev/null +++ b/test/fixtures/go/untidy/go.mod @@ -0,0 +1,5 @@ +module example.com/untidy + +go 1.27 + +require golang.org/x/text v0.43.0 diff --git a/test/fixtures/go/untidy/untidy.go b/test/fixtures/go/untidy/untidy.go new file mode 100644 index 0000000..97c68c0 --- /dev/null +++ b/test/fixtures/go/untidy/untidy.go @@ -0,0 +1,7 @@ +// Package untidy requires a module it never imports, which go mod tidy would take out and the lint action fails on. +package untidy + +// Value returns one. +func Value() int { + return 1 +} diff --git a/test/fixtures/indexnow/server.js b/test/fixtures/indexnow/server.js new file mode 100644 index 0000000..1cfe1e2 --- /dev/null +++ b/test/fixtures/indexnow/server.js @@ -0,0 +1,35 @@ +// Serves a sitemap index of two sitemaps, and an IndexNow API that records what it is sent, for the IndexNow action to be tested without reaching IndexNow. +// Usage: node server.js +const fs = require("node:fs"); +const http = require("node:http"); + +const [recordPath, port] = process.argv.slice(2); +const origin = `http://127.0.0.1:${port}`; +const urlset = (...locs) => + `\n\n${locs.map((loc) => ` \n \n ${loc}\n \n `).join("\n")}\n\n`; +const pages = { + "/sitemap.xml": `\n\n ${origin}/pages.xml\n ${origin}/posts.xml\n\n`, + "/pages.xml": urlset(`${origin}/a?x=1&y=2`, `${origin}/b`), + "/posts.xml": urlset(`${origin}/c`), +}; + +http + .createServer((req, res) => { + if (req.method === "GET" && pages[req.url]) { + res.writeHead(200, { "content-type": "application/xml" }).end(pages[req.url]); + return; + } + let body = ""; + req.on("data", (chunk) => (body += chunk)); + req.on("end", () => { + if (req.method === "POST" && req.url === "/indexnow") { + fs.writeFileSync(recordPath, body); + res.writeHead(202).end(); + } else if (req.method === "POST" && req.url === "/refuse") { + res.writeHead(422).end("Unprocessable Entity: the URLs do not belong to the host, or the key does not match the schema"); + } else { + res.writeHead(404).end(); + } + }); + }) + .listen(Number(port), "127.0.0.1"); diff --git a/test/fixtures/node/package-lock.json b/test/fixtures/node/package-lock.json new file mode 100644 index 0000000..04becae --- /dev/null +++ b/test/fixtures/node/package-lock.json @@ -0,0 +1,40 @@ +{ + "name": "node-fixture", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "node-fixture", + "dependencies": { + "playwright": "1.64.1" + } + }, + "node_modules/playwright": { + "version": "1.64.1", + "resolved": "https://registry.npmjs.org/playwright/-/playwright-1.64.1.tgz", + "integrity": "sha512-HCrVhHr5QQuwYoswEUGTb7gGp522yflbv/ST8nF4sFWGuLjQCt17Q/EubpGGSNKYn1tsTCPZxqHhHCfVURu7UA==", + "license": "Apache-2.0", + "dependencies": { + "playwright-core": "1.64.1" + }, + "bin": { + "playwright": "cli.js" + }, + "engines": { + "node": ">=20" + } + }, + "node_modules/playwright-core": { + "version": "1.64.1", + "resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.64.1.tgz", + "integrity": "sha512-BC4s8y6SEVA6Q1rSxZTto8hI1KUI3xXlHGZgvGqP4IqPdI/a6FX9H29IDNz7vKo9l21b6LfYosScPL6M3xxyDQ==", + "license": "Apache-2.0", + "bin": { + "playwright-core": "cli.js" + }, + "engines": { + "node": ">=20" + } + } + } +} diff --git a/test/fixtures/node/package.json b/test/fixtures/node/package.json new file mode 100644 index 0000000..f20561e --- /dev/null +++ b/test/fixtures/node/package.json @@ -0,0 +1,15 @@ +{ + "name": "node-fixture", + "private": true, + "scripts": { + "lint": "node record.js lint", + "format:check": "node record.js format:check", + "typecheck": "node record.js typecheck", + "build": "node record.js build", + "test": "node record.js test", + "test:coverage": "node record.js test:coverage && mkdir -p coverage && node --test --experimental-test-coverage --test-reporter=spec --test-reporter-destination=stdout --test-reporter=lcov --test-reporter-destination=coverage/lcov.info" + }, + "dependencies": { + "playwright": "1.64.1" + } +} diff --git a/test/fixtures/node/record.js b/test/fixtures/node/record.js new file mode 100644 index 0000000..513525c --- /dev/null +++ b/test/fixtures/node/record.js @@ -0,0 +1,10 @@ +// Writes the name of the script run into report/scripts.log, for the custom tests of the CI to tell which ran and in which order. +const fs = require("node:fs"); +const path = require("node:path"); + +// The scripts see the repository alone, without the workflow parts checked out beside it. +if (process.env.GITHUB_WORKSPACE && fs.existsSync(path.join(process.env.GITHUB_WORKSPACE, "workflow-parts"))) { + throw new Error("The workflow parts are beside the repository"); +} +fs.mkdirSync("report", { recursive: true }); +fs.appendFileSync("report/scripts.log", `${process.argv[2]}\n`); diff --git a/test/fixtures/node/record.test.js b/test/fixtures/node/record.test.js new file mode 100644 index 0000000..716a484 --- /dev/null +++ b/test/fixtures/node/record.test.js @@ -0,0 +1,6 @@ +const assert = require("node:assert"); +const test = require("node:test"); + +test("the fixture has a test, whose coverage Sonar reads", () => { + assert.strictEqual(1 + 1, 2); +}); diff --git a/test/fixtures/npm/no-credential/.npmrc b/test/fixtures/npm/no-credential/.npmrc new file mode 100644 index 0000000..6c871c2 --- /dev/null +++ b/test/fixtures/npm/no-credential/.npmrc @@ -0,0 +1 @@ +@devpro:registry=https://npm.pkg.github.com diff --git a/test/fixtures/npm/no-credential/package.json b/test/fixtures/npm/no-credential/package.json new file mode 100644 index 0000000..dc03bf4 --- /dev/null +++ b/test/fixtures/npm/no-credential/package.json @@ -0,0 +1,5 @@ +{ + "name": "@devpro/istarci", + "version": "0.0.0", + "files": ["package.json"] +} diff --git a/test/fixtures/npm/published/package.json b/test/fixtures/npm/published/package.json new file mode 100644 index 0000000..52edf7f --- /dev/null +++ b/test/fixtures/npm/published/package.json @@ -0,0 +1,4 @@ +{ + "name": "left-pad", + "version": "1.3.0" +} diff --git a/test/fixtures/npm/unpublished/index.js b/test/fixtures/npm/unpublished/index.js new file mode 100644 index 0000000..65f9b92 --- /dev/null +++ b/test/fixtures/npm/unpublished/index.js @@ -0,0 +1 @@ +module.exports = "fixture"; diff --git a/test/fixtures/npm/unpublished/package.json b/test/fixtures/npm/unpublished/package.json new file mode 100644 index 0000000..9096535 --- /dev/null +++ b/test/fixtures/npm/unpublished/package.json @@ -0,0 +1,5 @@ +{ + "name": "@devpro/github-workflow-parts-fixture", + "version": "0.0.0", + "files": ["index.js"] +} diff --git a/test/fixtures/pages/.gitignore b/test/fixtures/pages/.gitignore new file mode 100644 index 0000000..57510a2 --- /dev/null +++ b/test/fixtures/pages/.gitignore @@ -0,0 +1 @@ +_site/ diff --git a/test/fixtures/pages/build.js b/test/fixtures/pages/build.js new file mode 100644 index 0000000..38db323 --- /dev/null +++ b/test/fixtures/pages/build.js @@ -0,0 +1,4 @@ +// Writes the site the Pages deployment workflow checks, as a site generator would. +const fs = require("node:fs"); +fs.mkdirSync("_site", { recursive: true }); +fs.writeFileSync("_site/index.html", "Fixture\n"); diff --git a/test/fixtures/pages/package-lock.json b/test/fixtures/pages/package-lock.json new file mode 100644 index 0000000..a05fe2d --- /dev/null +++ b/test/fixtures/pages/package-lock.json @@ -0,0 +1,10 @@ +{ + "name": "pages-fixture", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "pages-fixture" + } + } +} diff --git a/test/fixtures/pages/package.json b/test/fixtures/pages/package.json new file mode 100644 index 0000000..fe0260c --- /dev/null +++ b/test/fixtures/pages/package.json @@ -0,0 +1,7 @@ +{ + "name": "pages-fixture", + "private": true, + "scripts": { + "build": "node build.js" + } +} diff --git a/test/fixtures/playwright/package-lock.json b/test/fixtures/playwright/package-lock.json new file mode 100644 index 0000000..b9ff5f0 --- /dev/null +++ b/test/fixtures/playwright/package-lock.json @@ -0,0 +1,40 @@ +{ + "name": "playwright-fixture", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "playwright-fixture", + "dependencies": { + "playwright": "1.64.1" + } + }, + "node_modules/playwright": { + "version": "1.64.1", + "resolved": "https://registry.npmjs.org/playwright/-/playwright-1.64.1.tgz", + "integrity": "sha512-HCrVhHr5QQuwYoswEUGTb7gGp522yflbv/ST8nF4sFWGuLjQCt17Q/EubpGGSNKYn1tsTCPZxqHhHCfVURu7UA==", + "license": "Apache-2.0", + "dependencies": { + "playwright-core": "1.64.1" + }, + "bin": { + "playwright": "cli.js" + }, + "engines": { + "node": ">=20" + } + }, + "node_modules/playwright-core": { + "version": "1.64.1", + "resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.64.1.tgz", + "integrity": "sha512-BC4s8y6SEVA6Q1rSxZTto8hI1KUI3xXlHGZgvGqP4IqPdI/a6FX9H29IDNz7vKo9l21b6LfYosScPL6M3xxyDQ==", + "license": "Apache-2.0", + "bin": { + "playwright-core": "cli.js" + }, + "engines": { + "node": ">=20" + } + } + } +} diff --git a/test/fixtures/playwright/package.json b/test/fixtures/playwright/package.json new file mode 100644 index 0000000..eb96274 --- /dev/null +++ b/test/fixtures/playwright/package.json @@ -0,0 +1,7 @@ +{ + "name": "playwright-fixture", + "private": true, + "dependencies": { + "playwright": "1.64.1" + } +} diff --git a/test/fixtures/pnpm/package.json b/test/fixtures/pnpm/package.json new file mode 100644 index 0000000..448bd21 --- /dev/null +++ b/test/fixtures/pnpm/package.json @@ -0,0 +1,4 @@ +{ + "name": "pnpm-fixture", + "private": true +} diff --git a/test/fixtures/pnpm/pnpm-lock.yaml b/test/fixtures/pnpm/pnpm-lock.yaml new file mode 100644 index 0000000..9b60ae1 --- /dev/null +++ b/test/fixtures/pnpm/pnpm-lock.yaml @@ -0,0 +1,9 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: {} diff --git a/test/fixtures/sonar-properties/excluded/skipped.js b/test/fixtures/sonar-properties/excluded/skipped.js new file mode 100644 index 0000000..89cfee4 --- /dev/null +++ b/test/fixtures/sonar-properties/excluded/skipped.js @@ -0,0 +1,2 @@ +// Analyzed only when neither sonar.exclusions nor what git ignores names it. +export const name = "excluded/skipped"; diff --git a/test/fixtures/sonar-properties/extra/skipped.js b/test/fixtures/sonar-properties/extra/skipped.js new file mode 100644 index 0000000..6443303 --- /dev/null +++ b/test/fixtures/sonar-properties/extra/skipped.js @@ -0,0 +1,2 @@ +// Analyzed only when neither sonar.exclusions nor what git ignores names it. +export const name = "extra/skipped"; diff --git a/test/fixtures/sonar-properties/kept.js b/test/fixtures/sonar-properties/kept.js new file mode 100644 index 0000000..cc02e07 --- /dev/null +++ b/test/fixtures/sonar-properties/kept.js @@ -0,0 +1,2 @@ +// Analyzed only when neither sonar.exclusions nor what git ignores names it. +export const name = "kept"; diff --git a/test/fixtures/sonar-properties/sonar-project.properties b/test/fixtures/sonar-properties/sonar-project.properties new file mode 100644 index 0000000..6ec063a --- /dev/null +++ b/test/fixtures/sonar-properties/sonar-project.properties @@ -0,0 +1,5 @@ +# The configuration the Sonar action reads, with no input but the connection. +sonar.projectKey=github-workflow-parts-sonar-properties +sonar.projectName=GitHub workflow parts properties file +sonar.sources=. +sonar.exclusions=excluded/** diff --git a/test/fixtures/terraform/.terraform.lock.hcl b/test/fixtures/terraform/.terraform.lock.hcl index be7a6ca..4f0ac51 100644 --- a/test/fixtures/terraform/.terraform.lock.hcl +++ b/test/fixtures/terraform/.terraform.lock.hcl @@ -2,22 +2,25 @@ # Manual edits may be lost in future updates. provider "registry.terraform.io/hashicorp/local" { - version = "2.9.0" + version = "2.9.1" constraints = "~> 2.5" hashes = [ - "h1:9rBZCMNpxKwMlRbWH2QpwD3kqUCAejdOZQ/aiiDObXQ=", - "zh:0baa4566cf77f1ff52f4293d1c8536202dd23edc197c3196413a28343c3ac3a0", - "zh:16b5559c3c07088ddad11a9bb9e9c0799999363c2958e9a5be2bcbbf2cd9ca64", - "zh:197c79015a10d1cce904a8ea722cbc750c42aeae2da53f44a6a0751d9fd1aa90", - "zh:29d0b03e5343a80677ebfeb2e2c31cbe4b1f65e736e53417454a4277fec2544c", - "zh:4896bfa6cf1d2fd562b47ef2e87f47862ae92a04f8ad5d764380f0c6653473b8", - "zh:531f8529cbca49f681883e57761a05a8398afaef6d1ab0d205d26bf12f4428e8", - "zh:6aaf5011d83161c86d2bfb80c0923ec934e578288758da2f37acb7aec129004b", - "zh:7430275253d3d3c40aa6179e0ec0d63212874dbbc06c5a51b9d07ec590f9756c", + "h1:OZGJN0LSSat5QIxZPxDXtVr4XpHb2oG7cVKJhSqBFIE=", + "h1:bek0XHcE8X713/1PR+sMKZFwLnamznJdYbORghbFRn0=", + "h1:qGLHCuYSus+uHNnoEL4SuJqOs5yrNOcB7gnuHoVqizo=", + "h1:tX/GU6iR6pP5XMj3zo22mIsZEIW2GUZ9GxfVdo2KjoI=", + "zh:25606c7a5e308144fb627f6e31611bb52ff72bb9ae2d27af39673ab1a6b3c1bf", + "zh:2568c4ef4dab31821f6f7040af0d1a2aa2b9455b8d9cc546598b791b8ada34cd", + "zh:25ac210f136042047975896e5e11fe6012425301c826d6b7ce22a49f96a1e0e8", + "zh:55d3a7bf01eced8e1f259b548020ef1221c5687e0fe6b5b30f81ad0b4121ff12", + "zh:6168d6934777b853c57815baa18a54101c604a6f1124ca47cd23d016c6f2f1e5", + "zh:63c0aa17c8373f761123376226debffd1f2259901ff259d4547c520def9a6787", + "zh:6d1f1572db6c85bd6d6b1b1deef247084e189816875f67e6bcc5d4736794e0a9", "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", - "zh:be17dc611e95e26cdf6cad79dfccf1064f0e32032a2efeb939a9bbe7fb1cbfe9", - "zh:f0e3b0aa644202e1d79d2000dca91f6019425da71e9800fa23f27e51c034f195", - "zh:f62bae4519e4ead49182ddc8afe8cf61e2a4c3ba3973b0fbba967736a2696aa3", - "zh:fcafa360a5b0b96244f26f4e3a6d642b716a376557142c2442ff2fb12d11da18", + "zh:a8ea1c155bd5e0d695a28d3c0e2f542de8844e1791ea021f015a3f08e1c059e1", + "zh:ccd790f97269509ade0574ef63c955d3943cb1d812c59177bd85ce4c1d17ca0f", + "zh:dcfc17bb666e659a9daa9ff52fd8f6ec9ea519868726015e645955c372296332", + "zh:eb83abbfa7f52cba609b48cbb9a6a5cab07780a672b2020fc2ba190e699a2516", + "zh:eedffca70074ae153790fb23c4ace8d1637a60ceba4e3c42c8eaeda7c1f9d10f", ] } diff --git a/test/fixtures/trivy/ignore-current.yaml b/test/fixtures/trivy/ignore-current.yaml new file mode 100644 index 0000000..c06b00c --- /dev/null +++ b/test/fixtures/trivy/ignore-current.yaml @@ -0,0 +1,5 @@ +# Leaves out the root user of the base container fixture until a day that has not come. +misconfigurations: + - id: DS-0002 + statement: The base container fixture runs as root on purpose + expired_at: 2099-12-31 diff --git a/test/fixtures/trivy/ignore-expired.yaml b/test/fixtures/trivy/ignore-expired.yaml new file mode 100644 index 0000000..260f642 --- /dev/null +++ b/test/fixtures/trivy/ignore-expired.yaml @@ -0,0 +1,5 @@ +# Leaves out the root user of the base container fixture until a day that has passed, so it counts again. +misconfigurations: + - id: DS-0002 + statement: The base container fixture runs as root on purpose + expired_at: 2020-01-01 diff --git a/test/fixtures/version/Directory.Build.props b/test/fixtures/version/Directory.Build.props new file mode 100644 index 0000000..e62eab6 --- /dev/null +++ b/test/fixtures/version/Directory.Build.props @@ -0,0 +1,5 @@ + + + 1.2.3 + + diff --git a/test/requirements.txt b/test/requirements.txt new file mode 100644 index 0000000..8f635d9 --- /dev/null +++ b/test/requirements.txt @@ -0,0 +1,81 @@ +# Tools of test/run_static_checks.sh, pinned with the hashes of every published file, as pip --require-hashes asks. +pyyaml==6.0.3 \ + --hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \ + --hash=sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a \ + --hash=sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3 \ + --hash=sha256:02ea2dfa234451bbb8772601d7b8e426c2bfa197136796224e50e35a78777956 \ + --hash=sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6 \ + --hash=sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c \ + --hash=sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65 \ + --hash=sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a \ + --hash=sha256:1ebe39cb5fc479422b83de611d14e2c0d3bb2a18bbcb01f229ab3cfbd8fee7a0 \ + --hash=sha256:214ed4befebe12df36bcc8bc2b64b396ca31be9304b8f59e25c11cf94a4c033b \ + --hash=sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1 \ + --hash=sha256:22ba7cfcad58ef3ecddc7ed1db3409af68d023b7f940da23c6c2a1890976eda6 \ + --hash=sha256:27c0abcb4a5dac13684a37f76e701e054692a9b2d3064b70f5e4eb54810553d7 \ + --hash=sha256:28c8d926f98f432f88adc23edf2e6d4921ac26fb084b028c733d01868d19007e \ + --hash=sha256:2e71d11abed7344e42a8849600193d15b6def118602c4c176f748e4583246007 \ + --hash=sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310 \ + --hash=sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4 \ + --hash=sha256:3c5677e12444c15717b902a5798264fa7909e41153cdf9ef7ad571b704a63dd9 \ + --hash=sha256:3ff07ec89bae51176c0549bc4c63aa6202991da2d9a6129d7aef7f1407d3f295 \ + --hash=sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea \ + --hash=sha256:418cf3f2111bc80e0933b2cd8cd04f286338bb88bdc7bc8e6dd775ebde60b5e0 \ + --hash=sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e \ + --hash=sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac \ + --hash=sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9 \ + --hash=sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7 \ + --hash=sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35 \ + --hash=sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb \ + --hash=sha256:5cf4e27da7e3fbed4d6c3d8e797387aaad68102272f8f9752883bc32d61cb87b \ + --hash=sha256:5e0b74767e5f8c593e8c9b5912019159ed0533c70051e9cce3e8b6aa699fcd69 \ + --hash=sha256:5ed875a24292240029e4483f9d4a4b8a1ae08843b9c54f43fcc11e404532a8a5 \ + --hash=sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b \ + --hash=sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c \ + --hash=sha256:6344df0d5755a2c9a276d4473ae6b90647e216ab4757f8426893b5dd2ac3f369 \ + --hash=sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd \ + --hash=sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824 \ + --hash=sha256:66291b10affd76d76f54fad28e22e51719ef9ba22b29e1d7d03d6777a9174198 \ + --hash=sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065 \ + --hash=sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c \ + --hash=sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c \ + --hash=sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764 \ + --hash=sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196 \ + --hash=sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b \ + --hash=sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00 \ + --hash=sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac \ + --hash=sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8 \ + --hash=sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e \ + --hash=sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28 \ + --hash=sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3 \ + --hash=sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5 \ + --hash=sha256:9c57bb8c96f6d1808c030b1687b9b5fb476abaa47f0db9c0101f5e9f394e97f4 \ + --hash=sha256:9c7708761fccb9397fe64bbc0395abcae8c4bf7b0eac081e12b809bf47700d0b \ + --hash=sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf \ + --hash=sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5 \ + --hash=sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702 \ + --hash=sha256:b30236e45cf30d2b8e7b3e85881719e98507abed1011bf463a8fa23e9c3e98a8 \ + --hash=sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788 \ + --hash=sha256:b865addae83924361678b652338317d1bd7e79b1f4596f96b96c77a5a34b34da \ + --hash=sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d \ + --hash=sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc \ + --hash=sha256:bdb2c67c6c1390b63c6ff89f210c8fd09d9a1217a465701eac7316313c915e4c \ + --hash=sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba \ + --hash=sha256:c2514fceb77bc5e7a2f7adfaa1feb2fb311607c9cb518dbc378688ec73d8292f \ + --hash=sha256:c3355370a2c156cffb25e876646f149d5d68f5e0a3ce86a5084dd0b64a994917 \ + --hash=sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5 \ + --hash=sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26 \ + --hash=sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f \ + --hash=sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b \ + --hash=sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be \ + --hash=sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c \ + --hash=sha256:efd7b85f94a6f21e4932043973a7ba2613b059c4a000551892ac9f1d11f5baf3 \ + --hash=sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6 \ + --hash=sha256:fa160448684b4e94d80416c0fa4aac48967a969efe22931448d853ada8baf926 \ + --hash=sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0 +shellcheck-py==0.11.0.1 \ + --hash=sha256:1b274df81de5b000ff78db433e7328b87e52e3c38481c60f8e488c3095beef05 \ + --hash=sha256:5c620c88901e8f1d3be5934b31ea99e3310065e1245253741eafd0a275c8c9cc \ + --hash=sha256:6b88d0a244c82ed07e06a53e444da841f69330ca59ae15d4a66c391655dae7a0 \ + --hash=sha256:784156289ecb17e91c692cd783ab5152333309588cabb10032a047331c63e759 \ + --hash=sha256:b6a3fee28efda2e16e38d6e6d59faf7224300256456639727370d404730849e8 diff --git a/test/run_static_checks.sh b/test/run_static_checks.sh new file mode 100755 index 0000000..b2add31 --- /dev/null +++ b/test/run_static_checks.sh @@ -0,0 +1,50 @@ +#!/usr/bin/env bash +# Runs the static checks of the workflow parts, the same way on a workstation and in the CI. +# The tools are pinned and verified like the binaries the actions download, and kept between two local runs. +# Linux x64 only. + +set -euo pipefail + +ACTIONLINT_VERSION=1.7.12 + +repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +tools_dir="${RUNNER_TEMP:-${XDG_CACHE_HOME:-$HOME/.cache}}/github-workflow-parts-checks" +requirements="${repo_root}/test/requirements.txt" +requirements_hash="$(sha256sum "$requirements" | cut -d ' ' -f 1)" + +# The virtual environment is built again whenever the requirements change. +if [ ! -f "${tools_dir}/venv/${requirements_hash}" ]; then + echo "Installing PyYAML and shellcheck..." + rm -rf "${tools_dir}/venv" + python3 -m venv "${tools_dir}/venv" + "${tools_dir}/venv/bin/pip" install --quiet --disable-pip-version-check --require-hashes --only-binary :all: -r "$requirements" + touch "${tools_dir}/venv/${requirements_hash}" +fi + +actionlint="${tools_dir}/actionlint-${ACTIONLINT_VERSION}" +if [ ! -x "$actionlint" ]; then + echo "Installing actionlint ${ACTIONLINT_VERSION}..." + download_dir="$(mktemp -d)" + tarball="actionlint_${ACTIONLINT_VERSION}_linux_amd64.tar.gz" + base_url="https://github.com/rhysd/actionlint/releases/download/v${ACTIONLINT_VERSION}" + curl -fsSL -o "${download_dir}/${tarball}" "${base_url}/${tarball}" + curl -fsSL -o "${download_dir}/checksums.txt" "${base_url}/actionlint_${ACTIONLINT_VERSION}_checksums.txt" + if ! (cd "$download_dir" && grep " ${tarball}$" checksums.txt | sha256sum --check --status); then + echo "ERROR: Checksum verification failed for ${tarball}" >&2 + exit 1 + fi + tar -xzf "${download_dir}/${tarball}" -C "$download_dir" actionlint + mkdir -p "$tools_dir" + mv "${download_dir}/actionlint" "$actionlint" + rm -rf "$download_dir" +fi + +export PATH="${tools_dir}/venv/bin:$PATH" +cd "$repo_root" + +status=0 +echo "Running actionlint on the workflows..." +"$actionlint" -shellcheck "$(command -v shellcheck)" || status=1 +echo "Checking the conventions of the actions and workflows..." +python3 test/check_workflow_parts.py || status=1 +exit "$status"