diff --git a/.gitignore b/.gitignore index a8328023..9d839e8d 100644 --- a/.gitignore +++ b/.gitignore @@ -43,10 +43,10 @@ data/ artifacts/ output/ # Dense-retrieval vectors `gen-assets` writes beside a bank stay local. -src/usersim/engine/assets/financial_services/**/embeddings.parquet +src/usersim/engine/probes/financial_services/assets/financial_services/**/embeddings.parquet # So do the judge scorecards `evaluate-assets` writes there. -src/usersim/engine/assets/financial_services/**/_eval_scorecard.parquet -src/usersim/engine/assets/financial_services/**/_scorecards/ +src/usersim/engine/probes/financial_services/assets/financial_services/**/_eval_scorecard.parquet +src/usersim/engine/probes/financial_services/assets/financial_services/**/_scorecards/ # IDE .vscode/ diff --git a/architecture/assets.md b/architecture/assets.md index 6587c443..79dbada0 100644 --- a/architecture/assets.md +++ b/architecture/assets.md @@ -92,9 +92,11 @@ Each domain's region specs (`.yaml`, one per locale it can generate) ship as package data, so all three commands find them from an installed package without `--region-spec`. -Domains are discovered through the `usersim.asset_domains` entry point, so a -new domain does not require modifying this project. See -[plugins.md](plugins.md). +A built-in domain's generator lives beside the probe whose banks it builds: +`financial_services` keeps its generator, region specs and asset notebook in +its folder's `asset_gen/`. Other domains are discovered through the +`usersim.asset_domains` entry point, so a new domain does not require +modifying this project. See [plugins.md](plugins.md). ## Every entry declares its provenance diff --git a/architecture/overview.md b/architecture/overview.md index 80abdbb0..d0f929f0 100644 --- a/architecture/overview.md +++ b/architecture/overview.md @@ -52,7 +52,7 @@ the arrow pointing one direction, and a test enforces it. | `usersim.reporting` | Turning parquets into a capability dashboard | [reporting.md](reporting.md) | | `usersim.selection` | Curating trajectories into exportable datasets | [selection.md](selection.md) | | `usersim.taxonomy` | The shared vocabulary of capabilities and evidence | above | -| `usersim.asset_gen` | Generating new asset banks for a domain | [assets.md](assets.md) | +| `usersim.asset_gen` | Dispatching asset-bank generation to a domain's generator | [assets.md](assets.md) | | `usersim.testing` | Conformance checks and the test harness for extensions you write | [plugins.md](plugins.md) | Probes get their own page: [probes.md](probes.md) for the substrate, @@ -84,16 +84,17 @@ throughout, so a name tells you which one you are reading about. src/usersim/ ├── cli/ console driver; models_*.toml catalogs ├── engine/ -│ ├── core/ the conversation loop, probe base classes, banks, -│ │ identity, outcomes, storage +│ ├── core/ the conversation loop, probe base classes, asset lookup, +│ │ shared bank loaders, identity, outcomes, storage │ ├── evaluator/ judges, axes, and the scorer registry -│ ├── probes/ the shipped probes -│ ├── assets/ per-probe YAML and parquet banks +│ ├── probes/ the shipped probes, most with their banks, loaders and +│ │ scorer in their own folder +│ ├── assets/ the banks of the probes that keep none of their own │ └── plugin.py the two Data Designer entry points ├── reporting/ capability dashboard and model comparison ├── selection/ trajectory curation and export ├── taxonomy/ capability definitions, evaluation-cell decoding -├── asset_gen/ asset-bank generation +├── asset_gen/ the asset-domain registry the asset commands dispatch through └── testing/ conformance suites for extension authors architecture/ this directory diff --git a/docs/engine/AUTHORING_A_PROBE.md b/docs/engine/AUTHORING_A_PROBE.md index bb6d88e0..fa76a9de 100644 --- a/docs/engine/AUTHORING_A_PROBE.md +++ b/docs/engine/AUTHORING_A_PROBE.md @@ -258,10 +258,11 @@ It must be absolute: `@register_probe` refuses a relative path, which would resolve against the working directory. In this repository the banks are then at `src/usersim/engine/probes/