Add reboot.bdd: write Reboot tests in Gherkin with pytest-bdd - #149
Draft
benh wants to merge 5 commits into
Draft
Conversation
Current Aviator status
This pull request is currently open (not queued). How to mergeTo merge this PR, comment
See the real-time status of this PR on the
Aviator webapp.
Use the Aviator Chrome Extension
to see the status of your PR within GitHub.
|
`start()` creates a `monitor_event_loop()` task but `stop()` never cancelled it. Tests based on `IsolatedAsyncioTestCase` hide the leak because each test's event loop closes right after `stop()`, but on a long-lived event loop (as `reboot.bdd` uses) every harness left a pending task behind that warned at garbage collection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015whh5EmdU8PAoxSevMRn5m
benh
force-pushed
the
pytest-bdd
branch
25 times, most recently
from
September 2, 2026 03:29
9aa02e5 to
28ca315
Compare
Developers write Gherkin scenarios against built-in steps, e.g.:
Given the application is up
And an `Account` for "alice" gets created via `open` with
`initial_balance=100`
When the `Account` for "alice" gets a `deposit` with `amount=50`
Then `balance` on the `Account` for "alice" has
`balance=150`
A test module brings in the built-in steps and the fixtures they run
on with `from reboot.bdd.steps import *` and defines an `application`
fixture returning the `Application` under test. Each scenario runs
against a fresh started `Reboot` harness on its own event loop, one
loop per scenario the way one application runs on one event loop
under `rbt dev run` and `rbt serve`. Each step's call runs on a fresh
`ExternalContext`, the way each external call in production arrives
with its own, unless the scenario creates one to share via
'Given a shared context'.
Custom steps may be `async def`: the `reboot.bdd`
`given`/`when`/`then` decorators run them on the scenario's event
loop, the same loop the harness and the built-in steps run on, which
is what lets `reboot.bdd` work under plain pytest without
`pytest-asyncio`.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015whh5EmdU8PAoxSevMRn5m
`reboot.bdd` resolves state types and calls methods the same way for proto and pydantic codegen, since both come from the same template; this pins the pydantic path with its own `Account` mirroring `tests/reboot/bdd/accounts.feature`, plus a custom `async def` step that calls through `World.call()` instead of the generated code. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015whh5EmdU8PAoxSevMRn5m
benh
force-pushed
the
pytest-bdd
branch
11 times, most recently
from
September 2, 2026 19:32
ce87dc2 to
22bfb0b
Compare
Six of the example test suites take a value out of one response, e.g.
an account or order ID, and use it in the next call. In Gherkin, a
'has' or 'where' list is a list of clauses: a comparing clause
asserts, and a saving clause saves a property under a name:
When `get_owner` on the `Account` for "frank" has
`owner.name` saved as "$owner_name"
A Then 'has' or 'where' asserts and refuses saving clauses; a Given
or When 'has' saves and refuses comparing clauses. Later steps say
`$name` to use a saved value, in a state's ID or as a property
value; a quoted "$name" stays the literal string, and saved values
are used as-is, so a saved message can be passed straight into a
later call's properties.
Readers are only read via '`reader` on ...', and a reader's abort
gets its own assertion, 'aborts with `SomeError` where ...';
'gets a' and 'attempts a' refuse readers, pointing at those steps,
the way they refuse writers. Whether a method is a reader comes from
the generated `reactively()` surface, which serves exactly the unary
readers. Reading records the result, so 'the result' always
means the most recent call any step made.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015whh5EmdU8PAoxSevMRn5m
benh
force-pushed
the
pytest-bdd
branch
10 times, most recently
from
September 3, 2026 03:42
d560abf to
a0f3238
Compare
Properties now read as the members of one JSON object, `name: value`
each in backticks, and an object or array value goes through the
named method's request type, looked up via the generated client
class's `<Method>Request` alias: pydantic requests validate with
`model_validate` and proto requests parse with
`json_format.ParseDict`, so nested messages, enums by name, and
64-bit integers all follow the JSON semantics those types define:
When the `Account` for "frank" gets a `set_owner` with
`owner.name: "Frankie"` and `owner.tags: ["pro"]`
A dotted name nests when calling, and a name that collides with
another, e.g. both `owner` and `owner.name`, raises. On assertions,
an object compares as the complete message the actual value's type
parses it as, and an array compares elementwise; a saved property
whose value is already a message is merged into the request as-is.
Also guards pydantic request validation against its default of
ignoring unknown keys, which made a mistyped property a silent
no-op.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015whh5EmdU8PAoxSevMRn5m
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
reboot.bdd, a pytest-bdd layer for testing Reboot applications with Gherkin, plus its test suites.What a test looks like
The test module does
from reboot.bdd.steps import *and defines anapplicationfixture returning theApplicationunder test. Each scenario runs against a fresh startedRebootharness, and each step's call runs on a freshExternalContext(the way each external call in production arrives with its own) unless the scenario creates one to share viaGiven a shared context.Built-in steps
Given the application is upGiven a shared contextGiven an `Account` for "alice" gets created via `open` with `initial_balance: 100`(usable asWhentoo)When the `Account` for "alice" gets a `deposit` with `amount: 50`(usable asGiventoo)When the `Account` for "bob" attempts a `withdraw` with `amount: 50`Then the attempt aborts with `OverdraftError` where `amount: 20`Then `balance` on the `Account` for "alice" has `balance: 150`Then `balance` on the `Account` for "ghost" aborts with `StateNotConstructed`When `get_owner` on the `Account` for "frank" has `owner.name` saved as "$owner_name"(later steps say$owner_namein a state's ID or as a property value)Then the result has `updated_balance: 150`(asGiven/When, saves instead)The property grammar
A
has/where/withlist holds JSON object members,`name: value`each in backticks, separated by commas orand. Values are JSON (an object or array calls through the method's request type; on assertions it compares as the complete message); a dotted name nests when calling and reaches into the response when asserting; a malformed or unknown property raises instead of being skipped. In ahasorwherelist a clause can instead be`name` saved as "$name": a Thenhas/whereasserts and refuses saving clauses, a Given/Whenhassaves and refuses comparing clauses.Readers are only read via
`reader` on ...;gets a/attempts arefuse them with a pointing error, and the reader steps refuse writers likewise (whether a method is a reader comes from the generatedreactively()surface, which serves exactly the unary readers). Reading records the result, so "the result" always means the most recent call any step made.State types are resolved from the
Application's servicers by class name, or by full state type name (e.g.bank.v1.Account) when more than one state type goes by the class name; both proto and pydantic codegen are supported.How it works
pytest and pytest-bdd are synchronous while everything Reboot is
async, and we deliberately don't usepytest-asyncio(pytest-bdd never awaits step functions, so it wouldn't help anyway). Instead, each scenario runs one event loop on a background thread, created before its first step and closed after its last: one loop per scenario, matching bothunittest.IsolatedAsyncioTestCase(one loop per test) and production (one application runs on one event loop underrbt dev runandrbt serve), so nothing a scenario leaks can run on into later scenarios. The loop's teardown mirrorsIsolatedAsyncioTestCase: cancel pending tasks, shut down async generators, close.The
reboot.bddgiven/when/then/stepdecorators work like pytest-bdd's except that the decorated step function may beasync def, so custom developer steps get the same treatment as the built-ins (exercised by the custom steps intests/reboot/bdd/bdd_tests.pyandtests/reboot/bdd/pydantic/bdd_tests.py).Also in this PR
Reboot.stop()now cancels (viawait_for_tasks) themonitor_event_loop()task thatstart()creates.IsolatedAsyncioTestCasehid this leak by closing its per-test loop; on a longer-lived loop every harness left a pending task behind. Ran//tests/reboot:external_context_tests_pyto check for regressions, but this touches every harness user, so please look closely.pytest==8.4.2andpytest-bdd==8.1.0added toreboot/requirements.in(andmypy.iniignore sections for both).Testing
bazel test //tests/reboot/bdd:bdd_tests_py //tests/reboot/bdd/pydantic:bdd_tests_py(proto and pydantic suites, plus unit tests for name collisions, unknown properties, and reader detection).Not yet done (follow-ups)
pytest-bddas areboot[bdd]extra rather than a hard dependency; only the Bazel side is wired here.testing-bdd.mdskill reference), state assertions via a test-only raw-state read (e.g.Then the state of the `Account` for "alice" has `balance: 70`), reactiveeventuallyvariants of the assertion steps, identity/auth steps, task steps, and failure/recovery steps.🤖 Generated with Claude Code
https://claude.ai/code/session_015whh5EmdU8PAoxSevMRn5m