Skip to content

test: run test=pytest fences from any docs page (#835) - #840

Merged
0x054 merged 2 commits into
mainfrom
sdd/835-docs-harness-pytest-fences
Oct 30, 2026
Merged

0x054 merged 2 commits into
mainfrom
sdd/835-docs-harness-pytest-fences

Conversation

@0x054

@0x054 0x054 commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Closes #835

What

Any docs page can own {test=pytest path="tests/<file>.py"} fences. The harness builds one scratch project per page: myapp/models.py is docs/examples/<page-stem-with-underscores>.py (the testing how-to keeps testing_models.py), with the shared conftest and pytest block. The Inspecting SQL caplog example is now test=pytest and runs under the db fixture.

Verify

Ran the ticket's Verify block on 1aa4327 (plus Postgres 18 via --db-backends=sqlite,postgres for the fixtures and examples files): passed.
TDD: RED pytest tests/test_docs_testing_fixtures.py failed 3 (the inspecting-sql page had no pytest fence yet); GREEN passed 237/237 with test_docs_examples, lint exit 0, test_docs_output.py -k inspecting 12 passed.
Verify block corrected: no

Interfaces

Interfaces changed: none (pages_with_pytest_fences() and models_module_for(page) are added in tests/test_docs_testing_fixtures.py; the missing-module error text is as specified, raised as FileNotFoundError).

Laws

  • AGENTS.md#I-8: n/a: the caplog fence already uses a lambda named after the model
  • AGENTS.md#I-5: the slug rule is recorded in docs/solutions/patterns/docs-page-conventions.md
  • AGENTS.md#I-10: n/a: no CHANGELOG edit

Notes for the reviewer

  • The ticket says a path outside tests/ is "refused as today", but fence_settings has no such check. _page_test_modules now raises ValueError ("path=... is outside tests/") and a test pins it. tests/_docs_harness.py changed only in one docstring sentence.
  • test_page_models_import_without_running_the_example imports myapp.models with ferro.connect replaced by a raiser, in a subprocess.

🤖 Generated with Claude Code

Each page with a pytest fence gets its own scratch project whose
myapp/models.py is the page's example module by slug. The Inspecting SQL
caplog example now runs.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

@0x054 0x054 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spec: PASS

  • Minor · [ticket#Test scenarios] · tests/test_docs_testing_fixtures.py:168 at d37a39a · Misunderstood · the missing-slug error path calls models_module_for(page) on a page path that is never written and never goes through _write_project, so the test pins the message but not that building a project for such a page fails · the scenario says "a temporary page (written to tmp_path, fed to the builder) with a pytest fence and no slug module → the harness error"
  • Minor · [ticket#Test scenarios] · tests/test_docs_testing_fixtures.py:151 at d37a39a · Missing · test_page_models_import_without_running_the_example asserts only that no connect() ran; nothing asserts that importing myapp.models registers Account · the edge-case scenario says the import "registers Account and runs no main()"
    Cannot verify: none

Checked, no finding:

  • The missing-module error text in models_module_for (lines 82-85) matches the ticket's Interfaces line byte for byte, and the test asserts the full string.
  • The path refusal: fence_settings (tests/_docs_harness.py:317-364) has no path check, so the ticket's "refused as today" is wrong. The new ValueError in _page_test_modules is the refusal the acceptance criterion asks for. It sits in the test module because the ticket keeps _docs_harness.py unchanged, and the PR body declares it. Accepted as scope.
  • The no-side-effects subprocess: ferro.connect is replaced before import myapp.models, and inspecting_sql.py imports connect inside main() (line 34), so an unguarded main() at import would hit the raiser and exit non-zero.
  • pages_with_pytest_fences() is pinned to EXPECTED_MODULES, so a new page with a pytest fence fails the suite until it is listed.

Quality: PASS

No Critical or Important findings.

Checked, no finding:

  • inspecting-sql.md lead-in: the db/Testing link moved into the sentence before the fence, and the sentence after it was shortened to match. The page's captured output did not change (tests/test_docs_output.py -k inspecting: 12 passed). The prose is plain and points back to the Account model at the top of the page.
  • docs-page-conventions.md: the table row and the new paragraph state the per-page project, the slug rule, the how-to's exception and the import guard (I-5).
  • Laws lines (I-8, I-5, I-10) agree with the diff. Verify block corrected: no holds: the worker ran the ticket's block, plus Postgres.
  • Depth: models_module_for holds the slug rule and its one exception behind one call, and pages_with_pytest_fences is used only to pin discovery. No check fired.

Verify: PR head d37a39a passed (pytest tests/test_docs_testing_fixtures.py tests/test_docs_examples.py -q: 224 passed, 12 skipped; the same with --db-backends=sqlite,postgres on Postgres 18: 236 passed; python -m tests._docs_checks docs/pages/guide/inspecting-sql.md docs/pages/howto/testing.md: exit 0; pytest tests/test_docs_output.py -k inspecting -q: 12 passed)
Trial merge: not run (base unchanged)

…odel and tests/.. paths (#835)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

@0x054 0x054 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review: d37a39a → 1aa4327

  • ADDRESSED · tests/test_docs_testing_fixtures.py:173-188 at 1aa4327 · the test now writes no-such-example.md with a pytest fence to tmp_path, passes it to _write_project, and asserts the full FileNotFoundError text
  • ADDRESSED · tests/test_docs_testing_fixtures.py:157-166 at 1aa4327 · the subprocess checks that User (testing how-to) or Account (inspecting-sql) is among the names in REGISTRY.models(), as well as checking that connect() never ran

New breakage in the fix diff: none. The path check in _page_test_modules (lines 57-58) now compares PurePosixPath parts and rejects ... The refusal test runs over src/x.py and tests/../x.py, and both raise.

Spec: PASS

Cannot verify: none

Quality: PASS

No Critical or Important findings.

Verify: PR head 1aa4327 passed (pytest tests/test_docs_testing_fixtures.py tests/test_docs_examples.py -q: 225 passed, 12 skipped; the same with --db-backends=sqlite,postgres on Postgres 18: 237 passed; python -m tests._docs_checks docs/pages/guide/inspecting-sql.md docs/pages/howto/testing.md: exit 0; pytest tests/test_docs_output.py -k inspecting -q: 12 passed)
Trial merge: not run (base unchanged)

@0x054
0x054 merged commit e4a87aa into main Oct 30, 2026
11 checks passed
@0x054
0x054 deleted the sdd/835-docs-harness-pytest-fences branch October 30, 2026 21:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Docs harness: run test=pytest fences from pages other than the testing how-to

1 participant