Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -145,13 +145,23 @@ jobs:
body.append(line[indent:])
io.open('action-step.sh', 'w', encoding='utf-8').write('\n'.join(body))
PY
# The stamp is read from action.yml rather than hard-coded, so this asserts the line
# the action really renders. The `${{ github.action_ref }}` plumbing that fills these
# in can only be exercised by a real `uses:`, but the rendering can be pinned here.
stamp="$(sed -n "s/^ *CIFAIL_ACTION_VERSION: *'\(.*\)'.*/\1/p" action.yml | head -1)"
test -n "$stamp"
INPUT_LOG=samples/nuget-nu1101.log INPUT_MODE=analyze INPUT_ARGS= \
INPUT_IMAGE=cifail:ci INPUT_SUMMARY=false INPUT_FAIL=true INPUT_COMMENT=false \
INPUT_SARIF=action.sarif INPUT_RULES= \
CIFAIL_ACTION_VERSION="$stamp" CIFAIL_ACTION_REF=v1 \
CIFAIL_ACTION_REPO=SebHenn/ci-failure-intelligence \
bash --noprofile --norc -e -o pipefail action-step.sh | tee action-out.txt
# The script deletes the markdown file after rendering it, so assert on what it printed
# — that text is the job summary and the PR comment.
grep -q 'cifail analysis' action-out.txt
# Issue #21: the log must name the build that produced that analysis. An unstamped or
# half-rendered line ("v unknown", a missing pin) is the state this guards against.
grep -q "cifail action v$stamp (SebHenn/ci-failure-intelligence@v1) | cifail " action-out.txt
test -s action.sarif

# Real external databases (Postgres/MySQL/SQL Server/MongoDB) via Testcontainers. Kept in
Expand Down
26 changes: 26 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -285,10 +285,36 @@ jobs:

# The composite action's major version is tracked independently of the CLI version,
# per the GitHub Actions convention. Prereleases (v0.2.0-rc1) must not move it.
#
# `@v1` is the ONLY pin the README documents, so it has one job: resolve to the newest
# release. Two ways that quietly stops being true, both guarded here — re-releasing an
# older patch line would drag it backwards, and a failed push would leave it behind
# while this job still went green.
- name: Move the v1 major tag
if: ${{ !contains(github.ref_name, '-') }}
run: |
set -euo pipefail
# `|| true`: grep exits 1 on an all-prerelease list, and under pipefail that would
# read as "not the newest" — i.e. it would skip the move for the wrong reason.
newest="$(git tag -l 'v[0-9]*.[0-9]*.[0-9]*' | { grep -v -- '-' || true; } | sort -V | tail -1)"
if [ -z "$newest" ]; then
echo "error: this release's own tag is not visible; refusing to guess about v1." >&2
exit 1
fi
if [ "$newest" != "${{ github.ref_name }}" ]; then
echo "::notice::${{ github.ref_name }} is not the newest release ($newest); leaving v1 where it is."
exit 0
fi

git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git tag -f v1 "${{ github.sha }}"
git push --force origin v1

landed="$(git ls-remote origin refs/tags/v1 | cut -f1)"
if [ "$landed" != "${{ github.sha }}" ]; then
echo "error: v1 is at ${landed:-nothing} but this release is ${{ github.sha }}." >&2
echo "Every documented 'uses: ...@v1' would still resolve to the previous build." >&2
exit 1
fi
echo "v1 -> ${{ github.ref_name }} (${{ github.sha }})"
21 changes: 21 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,25 @@ after 1.0.

## [Unreleased]

### Added

- **The GitHub Action names the build that ran.** One line, printed before the analysis and
repeated at the foot of the step summary and the PR comment:
`cifail action v0.3.1 (SebHenn/ci-failure-intelligence@v1) | cifail 0.3.1 | image
ghcr.io/sebhenn/cifail:latest@sha256:…`. Both documented pins are *moving* references —
`@v1` follows the newest release and `:latest` moves with it, and a runner may serve either
from cache — so until now a log could not answer "which cifail was this?", and a fix that
shipped weeks ago was indistinguishable from a fix that does not work ([#21]). Also exposed
as the `action-version` and `image-digest` outputs. The version is stamped in `action.yml`
and `scripts/check-versions.sh` fails the build if it drifts from `Directory.Build.props`.

### Fixed

- **The release workflow now verifies the `v1` tag actually moved,** and refuses to move it
backwards when an older patch line is re-released. `@v1` is the only pin the README
documents, so "resolves to the newest release" is the one thing it must never quietly stop
doing; previously the push was fire-and-forget and the job went green either way.

## [0.3.1] - 2026-08-21

A fix release for the reporting path: the GitHub Action could not explain a failure, because
Expand Down Expand Up @@ -478,6 +497,8 @@ First tagged release. **Superseded — do not install it**: its assets are named
documented install instructions can find them. Fixed in 0.2.0.

[Unreleased]: https://github.com/SebHenn/ci-failure-intelligence/compare/v0.3.1...HEAD

[#21]: https://github.com/SebHenn/ci-failure-intelligence/issues/21
[0.3.1]: https://github.com/SebHenn/ci-failure-intelligence/compare/v0.3.0...v0.3.1
[0.3.0]: https://github.com/SebHenn/ci-failure-intelligence/compare/v0.2.0...v0.3.0
[0.2.0]: https://github.com/SebHenn/ci-failure-intelligence/compare/v0.1.0...v0.2.0
Expand Down
19 changes: 18 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,14 @@ docker compose -f docker-compose.test.yml up -d # manual DBs for --db-* runs
**Release chain (`release.yml`): `verify → build → release(draft) → smoke → finalize`,** with
`nuget` and `docker` hanging off `verify`. The GitHub Release stays a **draft** until the real
`install.sh` has installed the real assets on Linux *and* macOS; `finalize` un-drafts it and
force-moves the `v1` tag that `uses: SebHenn/ci-failure-intelligence@v1` resolves to.
force-moves the `v1` tag that `uses: SebHenn/ci-failure-intelligence@v1` resolves to. That move
is now **guarded on both sides** (issue #21): it is skipped when the tag being released isn't the
newest version tag (re-releasing an older patch line must not drag `v1` backwards), and the push
is verified against `git ls-remote` afterwards, because a fire-and-forget force-push left every
documented `@v1` on the previous build while the job went green. Note the mirror-image trap when
*reading* that tag locally: `git fetch` does **not** move an existing tag, so a clone that has
ever seen `v1` keeps reporting the old commit until `git fetch --tags --force` — which is what
made #21 look like a workflow bug when the tag had in fact moved.

**`main` is a protected branch.** PRs require the three `ci.yml` checks (`build-test`,
`docker-smoke`, `db-integration`) and must be **up to date with `main`** before merging; force
Expand Down Expand Up @@ -706,6 +713,16 @@ is a normal outcome here — either one ended the step before `code=$?`, before
echoed, and before the `fail:` input could apply, making `fail: false` inoperative. Hence the
explicit `set +e` and `if` blocks rather than `&&`. Keep both when editing that script.

**`action.yml` stamps its own version (`CIFAIL_ACTION_VERSION`) and prints a provenance line
first thing** — `cifail action v0.3.1 (owner/repo@v1) | cifail 0.3.1 | image name:tag@sha256:…` —
into the log, the step summary and the PR comment, plus the `action-version`/`image-digest`
outputs. Both documented pins are *moving* references (`@v1` and `:latest`), so nothing else in
the output identifies the build, and a shipped fix is indistinguishable from a broken one. The
stamp is checked against `Directory.Build.props` by `scripts/check-versions.sh` (check 3) — bump
it with the version, or the release fails at `verify`. `ci.yml`'s "Run the action's own step
script" step asserts the rendered line; the `${{ github.action_ref }}` plumbing that fills it can
only be exercised by a real `uses:`, so that test passes the vars in explicitly.

## Conventions / gotchas

- Spectre.Console.Cli 0.55: `Command<T>.Execute` and `.Validate` overrides are
Expand Down
17 changes: 17 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -340,6 +340,21 @@ the job's **step summary**. No install needed; it runs the Docker image for you.
> `windows-*` / `macos-*` runners have no Docker daemon. On those, install the binary and call
> it directly — see [Anywhere else](#anywhere-else-plain-pipe).

**`@v1` is a moving tag**: it follows the newest release, so you get fixes without editing
your workflow. The flip side is that the ref you wrote doesn't identify a build — so the step
prints one line naming what actually ran, first thing in its log and at the foot of the step
summary:

```
cifail action v0.3.1 (SebHenn/ci-failure-intelligence@v1) | cifail 0.3.1 | image ghcr.io/sebhenn/cifail:latest@sha256:…
```

If a fix listed in the [changelog](CHANGELOG.md) doesn't seem to be there, read that line
before anything else — it names the action build, the CLI inside it, and the exact image
digest your runner used (`:latest` moves too, and runners cache). Pin either one by version
if you'd rather upgrade deliberately: `uses: SebHenn/ci-failure-intelligence@v0.3.1`, and
`image: ghcr.io/sebhenn/cifail:0.3.1`.

Inputs: `log` (file, **directory or glob** to analyze), `mode` (`analyze`, the default, or
`gate`), `args` (extra flags, e.g. `--type node`), `image` (pin a version instead of
`:latest`), `summary` (write to the step summary, default `true`), `fail` (propagate cifail's
Expand All @@ -358,6 +373,8 @@ For shared history, set `CIFAIL_DB_PROVIDER` / `CIFAIL_DB_CONNECTION` in the job
| `count` | how many things were analyzed |
| `new-failures` | `gate` mode: how many failures weren't in the baseline |
| `exit-code` | cifail's own exit code, whatever `fail` was set to |
| `action-version` | version of the action itself — what `@v1` resolved to on this run |
| `image-digest` | immutable digest of the image that ran (`name@sha256:…`) |

```yaml
- name: Explain failures
Expand Down
48 changes: 47 additions & 1 deletion action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,12 @@ outputs:
exit-code:
description: "cifail's own exit code, whatever `fail` was set to. 0 ok, 1 negative result, 2 usage."
value: ${{ steps.cifail.outputs.exit-code }}
action-version:
description: 'Version of this action itself — what `@v1` actually resolved to on this run.'
value: ${{ steps.cifail.outputs.action-version }}
image-digest:
description: 'Immutable digest of the image that ran (`name@sha256:…`), for pinning or for reproducing a run.'
value: ${{ steps.cifail.outputs.image-digest }}

runs:
using: 'composite'
Expand All @@ -84,6 +90,13 @@ runs:
INPUT_COMMENT: ${{ inputs.comment }}
INPUT_SARIF: ${{ inputs.sarif }}
INPUT_RULES: ${{ inputs.rules }}
# Which build of this action is running. `@v1` is a moving tag, so the ref a consumer
# wrote does not identify a build — this stamp does. Bumped with <Version> in
# Directory.Build.props and enforced by scripts/check-versions.sh, so it cannot drift.
CIFAIL_ACTION_VERSION: '0.3.1'
# Empty for a local `uses: ./` checkout; that is reported as such rather than guessed.
CIFAIL_ACTION_REF: ${{ github.action_ref }}
CIFAIL_ACTION_REPO: ${{ github.action_repository }}
run: |
set -uo pipefail
# The runner invokes this as `bash --noprofile --norc -e -o pipefail {0}`, so errexit is
Expand All @@ -102,6 +115,32 @@ runs:

docker pull "$INPUT_IMAGE" >/dev/null 2>&1 || true

# ---- provenance ----------------------------------------------------------------
# Name the build that is running, before it runs anything (issue #21). Both things a
# consumer pins are MOVING references: `@v1` follows the newest release, and the default
# `:latest` image moves with it — and a runner can serve either from cache. So a log
# that shows only the analysis cannot answer "which cifail was this?", and a fix that
# shipped weeks ago looks like a fix that does not work. Printing this first also means
# it survives a `docker run` that dies.
if [ -n "${CIFAIL_ACTION_REPO:-}" ] && [ -n "${CIFAIL_ACTION_REF:-}" ]; then
pin="$CIFAIL_ACTION_REPO@$CIFAIL_ACTION_REF"
else
pin="local checkout"
fi

# RepoDigests is the immutable identity of what was actually pulled; the tag in
# $INPUT_IMAGE is not. Empty for a locally-built image that was never pushed.
image_digest="$(docker image inspect \
--format '{{ if .RepoDigests }}{{ index .RepoDigests 0 }}{{ end }}' \
"$INPUT_IMAGE" 2>/dev/null | tr -d '\r')"
image_desc="$INPUT_IMAGE"
if [ -n "$image_digest" ]; then image_desc="$INPUT_IMAGE@${image_digest#*@}"; fi

cli_version="$(docker run --rm "$INPUT_IMAGE" --version 2>/dev/null | tr -d '\r' | head -1)"
provenance="cifail action ${CIFAIL_ACTION_VERSION:+v}${CIFAIL_ACTION_VERSION:-of unknown version} ($pin)"
provenance="$provenance | cifail ${cli_version:-unknown} | image $image_desc"
echo "$provenance"

run_args=(--rm -v "$PWD:/work" -w /work)
# Pass shared-database config through to the container when configured.
if [ -n "${CIFAIL_DB_PROVIDER:-}" ]; then run_args+=(-e CIFAIL_DB_PROVIDER); fi
Expand Down Expand Up @@ -144,6 +183,8 @@ runs:
>> "${GITHUB_OUTPUT:-/dev/null}"
}
emit "exit-code" "$code"
emit "action-version" "${CIFAIL_ACTION_VERSION:-}"
emit "image-digest" "$image_digest"

if [ "$INPUT_MODE" = "gate" ]; then
emit "matched" "$(printf '%s' "$json" | jq -r '.Passed // false' 2>/dev/null || echo false)"
Expand Down Expand Up @@ -176,11 +217,16 @@ runs:

echo "$body"

# The provenance line goes wherever the analysis goes — the summary and the PR comment
# are where someone reads this, and "which build ran?" is asked precisely when the
# analysis looks wrong.
if [ "$INPUT_SUMMARY" = "true" ] && [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
{
echo "## cifail"
echo ""
echo "$body"
echo ""
echo "<sub>$provenance</sub>"
} >> "$GITHUB_STEP_SUMMARY"
fi

Expand All @@ -193,7 +239,7 @@ runs:
echo "cifail: comment=true but no pull request in this event — skipping PR comment." >&2
else
marker="<!-- cifail -->"
comment_body="$(printf '%s\n## cifail\n\n%s\n' "$marker" "$body")"
comment_body="$(printf '%s\n## cifail\n\n%s\n\n<sub>%s</sub>\n' "$marker" "$body" "$provenance")"
repo="$GITHUB_REPOSITORY"
# Find our existing comment (by marker) and update it; otherwise create one.
id="$(gh api "repos/$repo/issues/$pr/comments" --paginate \
Expand Down
14 changes: 13 additions & 1 deletion scripts/check-versions.sh
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,19 @@ if [ "$chart_app" != "$version" ]; then
note "$chart_file appVersion is '$chart_app' but the version is '$version'"
fi

# 3. On a release, the tag must match the version. Accepts v-prefixed and bare tags.
# 3. The composite action stamps its own version so its output can name the build that ran
# (issue #21 — `@v1` and `:latest` are both moving references, so neither identifies one).
# A stamp nobody checks is a stamp that lies, and it would lie in exactly the situation it
# exists for: someone trying to work out which build they are on.
action_file="action.yml"
action_stamp="$(sed -n "s/^[[:space:]]*CIFAIL_ACTION_VERSION:[[:space:]]*['\"]\{0,1\}\([^'\"]*\)['\"]\{0,1\}[[:space:]]*$/\1/p" "$action_file" | head -1)"
if [ -z "$action_stamp" ]; then
note "$action_file has no CIFAIL_ACTION_VERSION stamp; the action could not report its version"
elif [ "$action_stamp" != "$version" ]; then
note "$action_file CIFAIL_ACTION_VERSION is '$action_stamp' but the version is '$version'"
fi

# 4. On a release, the tag must match the version. Accepts v-prefixed and bare tags.
# An empty argument means "not a tag build" (the release workflow passes "" on a manual
# dispatch), so it must be treated as no-tag rather than as a tag that matches nothing.
if [ "${1:-}" != "" ]; then
Expand Down