refactor(engine): keep the general probes each in one folder

architecture/assets.md

@@ -2,7 +2,8 @@
 
 Probe content (fact banks, query banks, pressure banks, clinical profiles,
 finance corpora, tool catalogs, action taxonomies) ships as **package
-data** under `usersim/engine/assets/<probe>/`.
+data** under `usersim/engine/assets/<probe>/`, or, for a built-in probe kept
+in its own folder, under that folder's `assets/<probe>/`.
 
 They ship inside the package so a plain `pip install` gives you working
 probes with no extra download step. `make check-dist` fails if the wheel
@@ -29,14 +30,19 @@ package, such as `Path(__file__).parent / "assets"`, so the probe finds its
 banks wherever the package is installed, with no environment variable to
 set. Registration refuses a relative root. That root is searched for that
 probe only, after the first three entries and before the packaged directory,
-so a deliberate override wins over it. The built-in probes declare none. A
-probe's bank loader reads under its class's `assets_dir()`, the folder this
-search finds for the probe.
+so a deliberate override wins over it. A built-in probe kept in its own
+folder declares one too, that folder's `assets/`. A probe's bank loader reads
+under its class's `assets_dir()`, the folder this search finds for the probe,
+and its `prompts.yaml` is read from that folder too.
 
 An explicit `--assets-dir` is **not** part of the search. It pins resolution
 to exactly that directory, for every probe in the run, including one that
 declares `assets_root`, because a user who named a location should be told
 when the assets are not there rather than quietly served another copy.
+Pinning the packaged directory pins the assets that ship with User Sim, so it
+also reaches a built-in probe whose banks are in its own folder.
+`simulate --inputs` pins the folder its rows were stored with: the packaged
+directory, for rows stored by a run that named no folder and set no override.
 
 There is no fallback to the working directory. Resolving against wherever the
 process happened to start makes a broken install look healthy from a source

architecture/probes.md

@@ -167,7 +167,8 @@ the turn had no side effects.
 
 ## Assets
 
-A probe's content lives at `engine/assets/<label>/`, resolved through
+A probe's content lives at `engine/assets/<label>/`, or, for a probe kept in
+its own folder, at `engine/probes/<label>/assets/<label>/`, resolved through
 `probe_assets_dir()`. A run fails pre-flight, before any model call, if a
 selected probe's bank is missing or, for the probes whose banks are split by
 locale, doesn't load for a run locale (a finance bank without tasks, for

docs/engine/AUTHORING_A_PROBE.md

@@ -232,9 +232,9 @@ class MyProbe(BankBackedProbe):
     )
 ```
 
-A probe shipped in its own package, rather than in this repository, also
-sets `assets_root` to the directory that holds its `<label>/` asset folder,
-so it finds its banks wherever the package is installed:
+A probe keeps its banks beside its code and sets `assets_root` to the
+directory that holds its `<label>/` asset folder, so it finds them wherever
+its code is installed, in this repository or in a package of its own:
 
 ```python
 from pathlib import Path
@@ -245,8 +245,10 @@ class MyProbe(BaseProbe):
 ```
 
 It must be absolute: `@register_probe` refuses a relative path, which would
-resolve against the working directory. Built-in probes leave it unset: their
-banks are under `src/usersim/engine/assets/<label>/`.
+resolve against the working directory. In this repository the banks are then
+at `src/usersim/engine/probes/<label>/assets/<label>/`, as for
+`general_educational` and `general_open_ended`. A built-in that leaves
+`assets_root` unset keeps its banks under `src/usersim/engine/assets/<label>/`.
 
 Its bank loader reads under `MyProbe.assets_dir()`, the probe's own folder,
 resolved as the built-in loaders resolve theirs, so a folder `--assets-dir` or
@@ -309,7 +311,7 @@ If your tests want to monkeypatch the loader, point at a module-level
 alias (not the class attr directly):
 
 ```python
-# In your generator.py
+# In your probe.py
 from usersim.engine.probes.my_probe.bank import load_my_bank_default
 
 _load_bank = load_my_bank_default  # tests patch this name
@@ -633,7 +635,7 @@ We'll use the **banked-verbatim** shape, so each date reaches the assistant
 exactly as the bank writes it, with `ToolExecutionMixin` to answer the
 assistant's tool call.
 
-### File 1: the banks, `src/usersim/engine/assets/bookings/<locale>/bookings.yaml`
+### File 1: the banks, `src/usersim/engine/probes/bookings/assets/bookings/<locale>/bookings.yaml`
 
 `en_US/bookings.yaml`:
 
@@ -742,7 +744,7 @@ BOOKINGS_USER_PROMPTS = LocalePromptPack(
 The user's instructions are written in each language the probe ships, and
 `LocalePromptPack` refuses, when the module is imported, a locale without them.
 
-### File 5: `src/usersim/engine/probes/bookings/generator.py`
+### File 5: `src/usersim/engine/probes/bookings/probe.py`
 
 ```python
 """bookings probe: booking arguments from dates and times as users write them."""
@@ -751,6 +753,7 @@ from __future__ import annotations
 
 import json
 import random
+from pathlib import Path
 from typing import Any
 
 from usersim.engine.core.probes import dispatch_episode
@@ -803,6 +806,7 @@ def _bank_loader(locale: str):
 )
 class BookingsProbe(ToolExecutionMixin, BankVerbatimMixin, BankBackedProbe):
     label = "bookings"
+    assets_root = Path(__file__).parent / "assets"  # holds bookings/
     description = (
         "Tests whether an assistant books what the user means: a date, a time and a number of people, "
         "written the way people write them in the user's language."
@@ -893,6 +897,9 @@ async def simulate_bookings(
     return await dispatch_episode(probe, models=models, data=data, cfg=cfg)
 ```
 
+- `assets_root` names the folder beside the module that holds `bookings/`, so
+  the probe finds its banks wherever User Sim is installed, and the loader
+  reads them under `BookingsProbe.assets_dir()`.
 - `execute_tool_call` records the first booking in `state.metadata` during the
   run, with how many user turns came before it, and `build_result_extras`
   writes it to the `booking` column beside what the user meant. A scorer
@@ -924,7 +931,7 @@ add the module to `BUILTIN_PROBE_MODULES`:
 ```python
 BUILTIN_PROBE_MODULES: tuple[str, ...] = (
     ...
-    "usersim.engine.probes.bookings.generator",
+    "usersim.engine.probes.bookings.probe",
 )
 ```
 

docs/engine/README.md

@@ -528,7 +528,7 @@ what each one carries:
 | `core/persona.py` | Persona formatting for prompts (deterministic shuffle) |
 | `core/locale.py` | Locale → language + expected-script-ranges mapping (single source of truth for shipped locales) |
 | `core/localized.py` | `LocalizedText` wrapper for asset-bank fields with per-locale renderings (`for_locale(locale)` lookup with `en_US` fallback) |
-| `core/prompt_loader.py` | Reads per-probe prompt packs from `assets/<probe>/prompts.yaml` |
+| `core/prompt_loader.py` | Reads a probe's `prompts.yaml` from the folder its banks resolve to |
 | `core/messages.py` | Message, tool, theme formatting |
 | `core/preview.py` | Rich-based preview utilities for inspecting generated conversations |
 
@@ -554,7 +554,7 @@ what each one carries:
 | `core/pressure_bank.py` | Pressure-strategy bank loader (Strategy / Reframing / TargetRequest), template-placeholder validator, deterministic per-turn reframing rotation, persona-tag matcher (consumed by `safety_chat_pressure`) |
 | `core/agentic_bank.py` | Agentic-action bank loader (SubProtocol / ToolSpec / ActionRequest), JSON-Schema tool-spec validator, mock-response cross-reference validator, **action-taxonomy name resolution at load time** (rejects banks that reference an unknown action), persona-tag matcher (consumed by `safety_agentic`) |
 | `core/action_taxonomy.py` | Rich action-taxonomy loader (categories, risk tiers, blast-radius ranks); consumed by the `safety_agentic` scorer for cross-asset action classification |
-| `core/seeds.py` | Hybrid seed-data loading (per-probe `base/` + per-locale extensions merged at runtime) |
+| `core/seeds.py` | Seed loading: a probe's `<locale>/<kind>.json`, or the `en_US` file for a locale without one |
 | `core/analysis.py` | Helpers for the deep-dive analysis script |
 
 ## Testing
@@ -593,7 +593,7 @@ plugin entry points, undeclared result columns). The tables describe the main fi
 | `test_messages.py` | Message / tool / theme formatting helpers |
 | `test_persona.py` | Persona formatting (deterministic shuffle) |
 | `test_preview.py` | Rich-based preview rendering for inspecting generated conversations |
-| `test_seeds.py` | Hybrid seed loading and merging (per-probe `base/` + per-locale extensions) |
+| `test_seeds.py` | Seed loading by locale, and the `en_US` fallback |
 | `test_multi_locale.py` | Multi-locale config, seed loading, language mapping |
 | `test_llm.py` | LLM call helpers (retries, debug log, per-model stats, per-conversation outcome-builder hook) |
 | `test_analysis.py` | Score extraction, demographic preservation, edge cases used by the deep-dive analysis script |

notebooks/01_simulate.ipynb

@@ -22,7 +22,7 @@
     "\n",
     "- `NVIDIA_API_KEY` exported in your shell (or set in the cell below).\n",
     "- The Nemotron-Personas dataset for the locales you care about. The setup cell below downloads any missing locales the first time via the NGC CLI; if `ngc` isn't on `PATH` it auto-installs into `.venv/bin/` (or run `usersim setup-ngc` ahead of time, per the README).\n",
-    "- An `assets/` directory with one folder per probe (`assets/<probe>/`). These ship with the repo.\n",
+    "- Each probe's assets, in an `assets/<probe>/` folder. These ship with the package.\n",
     "\n",
     "**What this writes:**\n",
     "\n",
@@ -33,8 +33,8 @@
     "| Family | Probe | Curated asset | When to use |\n",
     "|---|---|---|---|\n",
     "| General | `tool_calling` | `assets/tool_calling/toolsets_seed.parquet` | Tests native tool-call selection + arg synthesis on persona-grounded requests |\n",
-    "| General | `general_open_ended` | `assets/general_open_ended/topics/<locale>/topics.json` | Tests multi-turn conversational quality on open-ended topics |\n",
-    "| General | `general_educational` | `assets/general_educational/subjects/<locale>/subjects.json` | Tests pedagogical responses (level-calibration, scaffolded explanations) |\n",
+    "| General | `general_open_ended` | `assets/general_open_ended/<locale>/topics.json` | Tests multi-turn conversational quality on open-ended topics |\n",
+    "| General | `general_educational` | `assets/general_educational/<locale>/subjects.json` | Tests pedagogical responses (level-calibration, scaffolded explanations) |\n",
     "| Sovereign-AI | `sov_ai_facts` | `assets/sov_ai_facts/<locale>/` | Per-locale fact bank with curated ground truth + false-premise probes |\n",
     "| Sovereign-AI | `sov_ai_dynamic` | `assets/sov_ai_dynamic/<locale>/` | Persona-driven exploration with no ground truth: surfaces hallucination / self-inconsistency |\n",
     "| Sovereign-AI | `sov_ai_multilingual_parity` | `assets/sov_ai_multilingual_parity/sample.yaml` | Cross-locale matched-pair queries: surfaces respect-parity gaps across demographic strata |\n",

pyproject.toml

@@ -173,6 +173,11 @@ include = ["usersim*"]
     "assets/**/*.json",
     "assets/**/*.parquet",
     "assets/**/*.md",
+    # The banks a probe keeps in its own folder, under probes/<label>/assets/.
+    "probes/*/assets/**/*.yml",
+    "probes/*/assets/**/*.json",
+    "probes/*/assets/**/*.parquet",
+    "probes/*/assets/**/*.md",
     "**/*.yaml",
 ]
 "usersim.asset_gen" = [

scripts/check_dist.py

@@ -33,6 +33,18 @@
 REPO_ROOT = Path(__file__).resolve().parents[1]
 
 
+def in_a_bank(path: str) -> bool:
+    """Whether a package file is in an asset bank.
+
+    Banks live under ``usersim/engine/assets/``, or in the ``assets/`` folder
+    of a probe kept in its own folder, ``usersim/engine/probes/<label>/assets/``.
+    """
+    parts = path.split("/")
+    return path.startswith("usersim/engine/assets/") or (
+        parts[:3] == ["usersim", "engine", "probes"] and parts[4:5] == ["assets"]
+    )
+
+
 def ships(path: str) -> bool:
     """Whether a package file belongs in the wheel.
 
@@ -41,7 +53,7 @@ def ships(path: str) -> bool:
     """
     if path.endswith(".ipynb"):
         return False
-    return not path.endswith(".md") or path.startswith("usersim/engine/assets/")
+    return not path.endswith(".md") or in_a_bank(path)
 
 
 def tracked_package_files(root: Path = REPO_ROOT) -> list[str]:

src/usersim/engine/core/_assets.py

@@ -16,7 +16,8 @@
 4. the packaged ``usersim/engine/assets/`` directory.
 
 A probe class may also declare ``assets_root``, the root that holds its
-``<label>/`` folder, as a probe shipped in its own package does. That root
+``<label>/`` folder: a probe shipped in its own package does, and so does a
+built-in probe that keeps its banks in its own folder. That root
 is searched for that probe after the first three entries and before the
 packaged directory, so a deliberate override wins over it.
 
@@ -29,7 +30,9 @@
 It pins resolution to exactly that directory, for every probe, including
 one whose class declares a root. A user who named a location wants to be
 told when the assets are not there, rather than to be quietly served
-another copy instead.
+another copy instead. The packaged directory stands for the assets that
+ship with User Sim, so pinning it also reaches a built-in probe whose
+banks are in its own folder.
 
 There is deliberately no CWD fallback: resolving against whatever
 directory the process happened to start in makes a broken install look
@@ -113,8 +116,9 @@ def asset_search_path() -> tuple[Path, ...]:
 def declared_assets_roots() -> dict[str, Path]:
     """The ``assets_root`` each registered probe class declares, by label.
 
-    Probes that declare none, the built-ins among them, are left out. Reads
-    the registry as it stands, so a probe counts once its module is imported.
+    Probes that declare none, such as a built-in whose banks are in the
+    packaged directory, are left out. Reads the registry as it stands, so a
+    probe counts once its module is imported.
     """
     return {
         label: Path(cls.assets_root)
@@ -180,11 +184,13 @@ def probe_assets_dir(
             the search path. This is what ``--assets-dir`` becomes: a
             user who named a location is told when the assets are not
             there rather than quietly served the packaged copies.
+            Pinning the packaged directory reaches every bank that ships
+            with User Sim, a built-in kept in its own folder included.
             When ``None``, :func:`probe_asset_roots` is walked and the
             first root holding ``probe`` wins.
     """
     if assets_root is not None:
-        return Path(assets_root) / probe
+        return _pinned_probe_dir(probe, Path(assets_root))
 
     roots = probe_asset_roots(probe)
     for root in roots:
@@ -195,3 +201,22 @@ def probe_assets_dir(
     # been: the root the probe declares, else the highest-priority root.
     declared = declared_assets_roots().get(probe)
     return (declared if declared is not None else roots[0]) / probe
+
+
+def _pinned_probe_dir(probe: str, root: Path) -> Path:
+    """``probe``'s folder under the pinned ``root``.
+
+    The packaged directory stands for the assets that ship with User Sim, so
+    for a probe it does not hold, pinning it reaches the root the probe's class
+    declares when that root is inside this package. An extension's root is
+    outside it, so the pin stays strict for an extension.
+    """
+    folder = root / probe
+    packaged = packaged_assets_dir().resolve()
+    if folder.is_dir() or root.resolve() != packaged:
+        return folder
+    declared = declared_assets_roots().get(probe)
+    # ``packaged`` is ``usersim/engine/assets/``, so ``parents[1]`` is the package.
+    if declared is not None and declared.resolve().is_relative_to(packaged.parents[1]):
+        return declared / probe
+    return folder

src/usersim/engine/core/probes.py

@@ -294,12 +294,12 @@ def get_assistant_system_prompt(self) -> str:
     default_scorers: tuple[str, ...] = ()
 
     # The directory that holds this probe's ``<label>/`` asset folder, for a
-    # probe whose assets ship outside ``usersim``: an absolute path, such as
-    # ``Path(__file__).parent / "assets"`` (``@register_probe`` refuses a
-    # relative one). Searched after the deliberate overrides (the runtime
-    # override, ``USERSIM_ASSET_PATH`` and ``USERSIM_ASSETS_DIR``) and before
-    # the packaged banks. ``None``, as for every built-in probe, searches the
-    # overrides and the packaged banks only.
+    # probe whose assets are outside the packaged banks, in its own package or
+    # its own folder: an absolute path, such as ``Path(__file__).parent /
+    # "assets"`` (``@register_probe`` refuses a relative one). Searched after
+    # the deliberate overrides (the runtime override, ``USERSIM_ASSET_PATH``
+    # and ``USERSIM_ASSETS_DIR``) and before the packaged banks. ``None``
+    # searches the overrides and the packaged banks only.
     assets_root: Path | None = None
 
     @classmethod
@@ -1541,8 +1541,8 @@ def decorator(cls: type[BaseProbe]) -> type[BaseProbe]:
 #: this one, so this module cannot import them at load time.
 BUILTIN_PROBE_MODULES: tuple[str, ...] = (
     "usersim.engine.probes.financial_services.generator",
-    "usersim.engine.probes.general_educational.generator",
-    "usersim.engine.probes.general_open_ended.generator",
+    "usersim.engine.probes.general_educational.probe",
+    "usersim.engine.probes.general_open_ended.probe",
     "usersim.engine.probes.health_disclosure.decision_support",
     "usersim.engine.probes.health_disclosure.general",
     "usersim.engine.probes.health_disclosure.therapy",

src/usersim/engine/core/prompt_loader.py

@@ -4,15 +4,17 @@
 """YAML-based prompt template loader.
 
 Loads prompt templates from per-probe YAML files at
-``assets/<probe>/prompts.yaml``, falling back to the compiled-in
+``<asset root>/<probe>/prompts.yaml``, falling back to the compiled-in
 Python defaults if YAML files are not found. This allows prompt
 iteration and A/B testing without code changes.
 
-Per-probe asset layout: each probe owns its own folder under
-``assets/`` and stores its prompt templates as ``prompts.yaml``
-inside that folder, alongside its other assets (e.g.
-``assets/tool_calling/prompts.yaml`` lives next to
-``assets/tool_calling/toolsets_seed.parquet``).
+Per-probe asset layout: each probe owns a ``<probe>/`` folder under an
+asset root and stores its prompt templates as ``prompts.yaml`` inside
+that folder, alongside its other assets (e.g.
+``tool_calling/prompts.yaml`` lives next to
+``tool_calling/toolsets_seed.parquet``). The file is read from the
+folder :func:`~usersim.engine.core._assets.probe_assets_dir` resolves,
+the one the probe's banks are read from.
 """
 
 from __future__ import annotations
@@ -46,15 +48,19 @@ def _load_prompt_file_at(probe: str, assets_root: str) -> dict[str, Any]:
 
 
 def load_prompt_file(probe: str) -> dict[str, Any]:
-    """Load a probe's ``prompts.yaml`` from the active asset root.
+    """Load a probe's ``prompts.yaml`` from the folder its banks resolve to.
+
+    That is the first root on the asset search path holding ``probe``, or
+    the ``assets_root`` its class declares, so a root that holds other
+    probes' assets does not hide this probe's prompts.
 
     Returns an empty dict if the file doesn't exist or PyYAML is not
     available — callers should always pass a sensible compiled-in
     default to :func:`get_prompt` to cover that path.
     """
-    from usersim.engine.core._assets import default_assets_dir
+    from usersim.engine.core._assets import probe_assets_dir
 
-    return _load_prompt_file_at(probe, str(default_assets_dir()))
+    return _load_prompt_file_at(probe, str(probe_assets_dir(probe).parent))
 
 
 def get_prompt(probe: str, key: str, default: str = "") -> str: