ci: run the unit suite on every PR and push to main - #53
Merged
Merged
Conversation
There was no CI at all — `.github/` held only FUNDING.yml — so nothing ran the tests until someone ran them by hand. PR #48 reached review carrying an ImportError against `agent.config.USE_BATCH_RPC` and `agent/services/headers.py`, both deleted by a PR that merged while #48 was open. The code could not import on main, and the only reason that surfaced is that someone happened to try. This job is what catches that class of thing. Matrix is 3.10 and 3.13: the floor the README badge advertises and the version development runs on. 3.10 was verified rather than assumed — the full suite runs there, 259 of 259 non-ffmpeg tests pass. Two things the run needs that are easy to get wrong: `fonts-dejavu-core` is installed alongside ffmpeg. The drawtext call in video_reviewer.py passes no `fontfile=`, so it asks fontconfig for a default family; with --no-install-recommends a runner can end up holding an ffmpeg that has drawtext and no font for it to use. The ffmpeg check renders a frame instead of grepping `ffmpeg -filters`. Grep proves the filter is compiled in, not that a font exists, and a stock Homebrew ffmpeg already demonstrates the first failure mode: it ships without --enable-libfreetype, and 13 tests then fail with "Filter not found", which reads like a test bug rather than a missing codec. Rendering one frame through the same filter the tests use catches both modes in the step whose name says what is wrong. Verified in both directions against a good and a drawtext-less ffmpeg, running the script as PyYAML parses it so the escaping is checked too. timeout-minutes is 10 against a ~7s suite. That ceiling only trips on a hang, which this suite has had: an aiosqlite connection left open by a missing mock keeps a non-daemon worker thread alive and the interpreter never exits. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E5BJVhNgerhFTUgmfnKTSj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the repo's first CI.
.github/held onlyFUNDING.yml, so nothing ran the tests until someone ran them by hand.Why now
PR #48 reached review carrying an
ImportError:agent.config.USE_BATCH_RPCandagent/services/headers.pywere both deleted by #49, which merged while #48 was open. The code could not import onmainat all, and the only reason that surfaced is that someone happened to try it. This job is what catches that.What it runs
python -m pytest tests/unitonpull_requestand onpushtomain, across Python 3.10 and 3.13 — the floor the README badge advertises, and the version development runs on.3.10 was verified, not assumed: the full suite runs there and all 259 non-ffmpeg tests pass. Had it not, this matrix would have arrived red.
Two things the run needs that are easy to get wrong
fonts-dejavu-coreis installed alongside ffmpeg. Thedrawtextcall invideo_reviewer.py:204passes nofontfile=, so it asks fontconfig for a default family. With--no-install-recommends, a runner can end up with an ffmpeg that has drawtext and no font for it to use.The ffmpeg check renders a frame rather than grepping
ffmpeg -filters. Grep proves the filter is compiled in, not that a font exists. And the first failure mode is not hypothetical — a stock Homebrew ffmpeg ships without--enable-libfreetype, and 13 tests then fail withError opening output files: Filter not found, which reads like a test bug rather than a missing codec. Rendering one frame through the same filter the tests use catches both modes in a step whose name says what is wrong:Verified in both directions locally — against a good ffmpeg (exit 0) and a drawtext-less one (exit 1) — running the script exactly as PyYAML parses it, so the
%{pts\:hms}escaping through YAML into bash is checked too.Other choices
timeout-minutes: 10against a ~7s suite. That ceiling only trips on a hang, which this suite has had: an aiosqlite connection left open by a missing mock keeps a non-daemon worker thread alive and the interpreter never exits.concurrencywithcancel-in-progress, so a new push supersedes the run in flight.fail-fast: false, so a 3.10-only break still shows the 3.13 result.Note
This PR is the first thing that will exercise the workflow — a workflow file only really runs once it is on a branch GitHub evaluates. If the runner's ffmpeg surprises us, the assert step is the thing that will say so plainly.
🤖 Generated with Claude Code