[3mei]

What changed and why

financial_services lives in one folder: the probe module probe.py, its bank loader bank.py, its task builder tasks.py, its query embedder embeddings.py, its scorer scorer.py, its banks in assets/financial_services/, the folder the class names as its assets_root, and its asset generator in asset_gen/. Its tests are in tests/probes/financial_services/. The label, the scorer name, the banks, the stored columns, the caches and the asset domain are unchanged.

Builds on #124.

• Model helpers for asset domains (first commit). usersim.extension exports to_model_configs, to_data_designer_kwargs and require_aliases, which an asset domain's generate and evaluate use to turn the models they receive into Data Designer model configs and provider arguments, and to check the aliases they need. docs/extension_api.md lists them, and importing them leaves Data Designer and the conversation loop unloaded. The finance generator imports them from usersim.extension.
• The probe in one folder (second commit). core/finance_bank.py, core/finance_tasks.py and core/embeddings.py become bank.py, tasks.py and embeddings.py, the scorer becomes scorer.py, the banks move into assets/financial_services/, and the tests become test_probe.py and test_e2e.py.
  • gen-assets, validate-assets and evaluate-assets default to the domain's shipped bank, the probe's own folder.
  • The package-data exclusions and the ignore rules for generated embeddings and judge scorecards follow the banks.
  • BUILTIN_PROBE_MODULES, DEFAULT_SCORER_MODULES, the capability source references, the docs and the asset notebook name the new modules and paths, and the shared-code label allowances drop the moved files.
• The generator beside the probe (third commit). asset_gen/financial_services/ becomes probes/financial_services/asset_gen/, with its region specs and its asset notebook. probes/financial_services/asset_domain.py builds the domain without importing the generator, and usersim.asset_gen.registry names it in BUILTIN_DOMAINS; usersim.asset_gen keeps AssetDomain and the registry. The four test_asset_gen_* files move to tests/probes/financial_services/ and read the region specs through the domain. The pipeline logs under usersim.engine.probes.financial_services.asset_gen.pipeline.

How it was verified

At each commit, on that commit's tree:

$ make check-all && make test && make smoke
All checks passed!
363 files already formatted
4656 passed in 92.28s (0:01:32)
All checks passed.

That is the first commit's tree; the second and third pass with 4656 tests as well.

At the head:

$ make check-license-headers
🎉 All files have correct license headers!
$ make check-doc-links
✓ every relative link, and every link into this repository's main branch, resolves
$ make check-notebooks-clean
OK 4 notebook(s) carry no outputs
$ make check-dist
✓ installed package runs from a clean environment
$ make check-extensions
all extension seams work against an installed distribution

• make health-check ran against a custom provider config for financial_services in en_GB. It ended with ✓ simulate, eval and report all completed, and every model's requests show failed=0.

• usersim simulate, then usersim eval naming the probe's scorer, ran against the same config in de_DE:

  judge: scored
  financial_services: ok (scores=['finance.document_recall_rate', 'finance.no_unauthorized_action_rate', 'finance.ordering_respected_rate', 'finance.substantive_retrieval_rate', 'finance.tool_selection_rate'], error=None)
  

• The asset commands find the banks and region specs in their new folders, offline:

  $ usersim validate-assets --domain financial_services --locale fr_FR
    summary: 85 pass / 6 warn / 0 fail -> OK
  $ usersim gen-assets --domain financial_services --locale ja_JP --dry-run
    region_spec: src/usersim/engine/probes/financial_services/asset_gen/region_spec/ja_JP.yaml
    institutions: 5 (asanagi_bank, sorairo_bank, hoshimi_securities, komorebi_life, tsumugi_card)
    total: 132 cluster calls -> 1299 documents
  

Results impact

Conversations, prompts, scores and the banks the asset commands write are unchanged. The setup fingerprint of a simulate --inputs run hashes the contents of every asset root, so it changes, as after any bank edit, and such a run gives its rows new trajectory ids rather than skipping rows it wrote before this change.