[3mei]

What changed and why

A probe sets the area its conversations explore and the bounds a scorer checks, and the simulated user explores that area in the persona's own words. The guides, their worked examples and the templates lead with that shape. An extension author also gets a form for asking for a hook, and a package template to start from.

Builds on #118.

• Hook requests (first commit). .github/ISSUE_TEMPLATE/extension-hook.yml asks for the need in general terms, why the existing hooks do not cover it, a proposed shape, and whether it works when a host drives the probe. "Adding a hook" in CONTRIBUTING.md lists what a new hook does: it is general, keeps today's behaviour by default, is async def when it can call a model, passes assert_hosted_parity, and comes with a use in the extension fixture and updates to the conformance checks, templates and docs. The feature request form and docs/extensions.md point to it.
• Prompt helpers (second commit). get_prompt and load_prompt_file join usersim.extension, so an extension can keep its prompts in a prompts.yaml under its assets_root, as built-in probes do. docs/extension_api.md lists both.
• Guides lead with areas (third commit).
  • Step 0 of docs/engine/AUTHORING_A_PROBE.md states the principle, and its decision tree starts with the shapes in which the user model writes turn 1. A verbatim turn 1 is kept for probes whose exact wording is what they test, such as a false premise or a matched pair. The step examples subclass BankBackedProbe alone where a mixin was incidental.
  • Both bookings worked examples follow it. The bank names what can be booked, draw_booking picks each row's date, time and number of people, and the user model asks for the booking in its own words, with the gate checking turn 1 like every user turn. Each drawn date has a day and a month that are different numbers no greater than 12, so it reads as another date in the other order. The scorer is unchanged.
  • templates/probe/generator_baseprobe_only.py draws a topic per row from themes, and templates/probe/README.md says when a verbatim turn fits.
• Extension package template (fourth commit). templates/extension_package/ is a package to copy and rename: a probe that draws an area per row from themes.yaml and has the user explore it, its prompts in prompts.yaml read with get_prompt, a tool, a scorer, entry points, and tests wired to the pytest plugin. make check-extensions installs it next to the built wheel and runs its tests, and the public-import test scans it. docs/extensions.md and CONTRIBUTING.md point to it.

How it was verified

At each commit, on that commit's tree:

$ make check-all && make test && make smoke
All checks passed!
349 files already formatted
4652 passed in 80.97s (0:01:20)
All checks passed.

The test count is 4652 at every commit; the template's own tests run in make check-extensions, below.

At the head:

$ make check-license-headers
🎉 All files have correct license headers!
$ make check-doc-links
✓ every relative link, and every link into this repository's main branch, resolves
$ make check-notebooks-clean
OK 4 notebook(s) carry no outputs
$ make check-dist
✓ installed package runs from a clean environment
$ make check-extensions
all extension seams work against an installed distribution

• make check-extensions runs the template's tests against the installed wheel:

  → installing templates/extension_package and running its tests
  ......                                                                   [100%]
  6 passed
  

• The code blocks of both bookings examples are the files that ran. The extension package's six tests pass as an outside project. The in-tree probe, installed in an exported tree with the guide's other files, passes probe_conformance_problems, runs in en_US and pt_BR with run_episode, passes assert_hosted_parity, and usersim smoke checks its assets with the other probes:

  [OK]   probe assets          11 probe(s) checked their assets: bookings, financial_services, general_educational, general_open_ended, identity_disclosure, safety_agentic, safety_chat_pressure, sov_ai_dynamic, sov_ai_facts, sov_ai_multilingual_parity, tool_calling
  

• The my-bookings package ran live against a custom provider config, two rows each in en_US and pt_BR, then usersim eval --scorers auto. Each simulated user wrote its own opening for the drawn booking, every model's requests show failed=0, and every row was scored:

  judge: scored; my_bookings: ok; others: language_compliance=ok, refusal_basics=ok, response_shape=ok
  

Results impact

None. No shipped probe, prompt, asset or scorer changes; the two exported helpers are the functions built-in probes already call.