[3mei]

What changed and why

safety_chat_pressure, safety_agentic and identity_disclosure each live in one folder: the probe module probe.py, its loaders, its scorer scorer.py and its banks in assets/<label>/, the folder the class names as its assets_root. Each probe's tests are in tests/probes/<label>/. That is the layout the sov_ai probes use. Labels, scorer names, banks, stored columns and the caches each probe shares with its scorer are unchanged.

Builds on #123.

• Template examples checked by path (first commit). The probe test template cites each worked example by its path, and the test of those examples checks every cited path is a file, including the examples under tests/probes/.
• One folder per probe (second to fourth commits). One commit per probe: generator.py renamed to probe.py, the loaders and the scorer moved in, the banks moved into assets/<label>/, and the tests moved into tests/probes/<label>/. BUILTIN_PROBE_MODULES, DEFAULT_SCORER_MODULES, the capability source references and the docs name the new modules, the authoring guide lists each probe among those kept in one folder, and the shared-code label allowances drop the moved files.
  • safety_chat_pressure: core/pressure_bank.py becomes bank.py; the tests are test_probe.py, test_scorer.py, test_bank.py and test_classifier.py.
  • safety_agentic: core/agentic_bank.py becomes bank.py and core/action_taxonomy.py keeps its name; the tests are test_probe.py, test_scorer.py, test_bank.py and test_action_taxonomy.py.
  • identity_disclosure: core/identity_spec.py and identity_spec_types.py become spec.py and spec_types.py, and the scorer's identity_grading.py becomes grading.py. The tests are test_probe.py, test_scorer.py, test_spec.py, and the tests of the identity capability rows and the contradictions report, test_capabilities.py and test_contradictions.py. A probe built on this one imports IdentityDisclosureProbe from usersim.engine.probes.identity_disclosure.probe, as the probe's README and the extension fixture show.
  • The scorers' log lines read <label> scorer:.
• Capability refs name real files (fifth commit). A test checks that every implementation ref a capability lists is a file in the repo, so a module, scorer or bank that moves without its refs fails. The identity capability rows name the spec file, which moves in the fourth commit.

How it was verified

At each commit, on that commit's tree:

$ make check-all && make test && make smoke
All checks passed!
360 files already formatted
4655 passed in 99.84s (0:01:39)
All checks passed.

That is the first commit's tree; the second to fourth pass with 4655 tests and the fifth with 4656, its new test included.

At the head:

$ make check-license-headers
🎉 All files have correct license headers!
$ make check-doc-links
✓ every relative link, and every link into this repository's main branch, resolves
$ make check-notebooks-clean
OK 4 notebook(s) carry no outputs
$ make check-dist
✓ installed package runs from a clean environment
$ make check-extensions
all extension seams work against an installed distribution

• The first commit's test fails on a template that cites a missing example:

  E       AssertionError: template cites tests that do not exist: ['tests/probes/safety_agentic/test_missing.py']
  

• The fifth commit's test fails with the identity spec's old path in place:

  E       AssertionError: assert ['src/usersim...re/spec.yaml'] == []
  E         Left contains one more item: 'src/usersim/engine/assets/identity_disclosure/spec.yaml'
  

• make health-check ran against a custom provider config for each moved probe: safety_chat_pressure in de_DE, safety_agentic in ko_KR and identity_disclosure in ja_JP. Each ended with ✓ simulate, eval and report all completed, and every model's requests show failed=0.

• usersim simulate, then usersim eval naming each probe's scorer, ran against the same config, safety_chat_pressure in fr_FR, safety_agentic in pt_BR and identity_disclosure in en_GB:

  judge: scored
  safety_chat_pressure: ok (scores=['pressure.contextual_safety_persistence', 'pressure.refusal_held'], error=None)
  judge: scored
  safety_agentic: ok (scores=['agentic.blast_radius_contained', 'agentic.boundary_held'], error=None)
  judge: scored
  identity_disclosure: ok (scores=['identity.confirms_developer_rate', 'identity.correct_attribution_rate', 'identity.model_name_correct_rate', 'identity.names_developer_rate', 'identity.no_false_attribution_rate', 'identity.no_human_claim_rate', 'identity.no_spontaneous_false_attribution_rate', 'identity.pressure_resistance_rate', 'identity.reasoning_spill_free_rate', 'identity.rejects_other_developer_rate'], error=None)
  

Results impact

Conversations, prompts and scores are unchanged. The setup fingerprint of a simulate --inputs run hashes the contents of every asset root, so it changes, as after any bank edit, and such a run gives its rows new trajectory ids rather than skipping rows it wrote before this change.