[emerardd]

Description

Preserve provenance when loading converted pretrained J++ lenses and add optional filtered vocabulary readouts to JacobianLens, following the scope approved in #1991.

• Retain top-level artifact provenance under metadata["provenance"] across save/load, without adding model-name constraints or new merge rules.
• Add exclude_token_ids to readout(), applied before lens top-k and recorded on the result. Full logits and model top-k remain unchanged; the same filtering works for logit-lens and final-layer readouts.
• Add readout_token_exclusions() in utilities/tokenize_utils.py, with Unicode-aware decoding, padded output-vocabulary coverage and explicit retained ids.
• Extend the existing Jacobian Lens demo with a pinned Qwen3.5-0.8B pretrained J++ example and an identically filtered logit-lens baseline.

Fixes #1991

Type of change

• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

Validation

• JacobianLens unit surface and new tests: 193 passed, including 20 new cases covering provenance round trips, existing merge behavior, pre-top-k ranking, unchanged raw outputs, invalid exclusions, Unicode decoding and negative-infinity ties.
• Tokenizer and selected tiny-model integration surface: 22 passed, 17 deselected.
• mypy .: 391 source files passed.
• pycln check passed; isort and Black passed over Git-tracked and newly added Python files. New notebook cells also pass Black and notebook-schema validation.
• Docs build completed with 265 warnings.
• Both added notebook code cells executed against the real pinned Qwen3.5-0.8B/J++ artifacts; their output is included. The full existing notebook was not executed.

The full PR-review test invocation was attempted offline but stopped during collection because the SoLU model dependencies were not cached. Running the wider JacobianLens integration surface also exposed missing GPT-2 weights and published lens artifacts (218 passed, 2 failed and 12 setup errors, all resource-loading failures). These runs are not reported as passing; full-suite verification remains for CI.

Checklist

• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have not rewritten tests relating to key interfaces which would affect backward compatibility

[jlarson4]

Looks good, merging as is!