Skip to content

Updating documentation and closing some small gaps from 4.x - #2016

Merged
jlarson4 merged 1 commit into
devfrom
4-4-accuracy-review
Nov 11, 2026
Merged

jlarson4 merged 1 commit into
devfrom
4-4-accuracy-review

Conversation

@jlarson4

Copy link
Copy Markdown
Collaborator

Description

  • Updating our adapter and contributor documentation to reflect lessons learned from new bug fixes and to reflect changes in how we approach things due to those fixes
  • Added additional tests to cover gaps that were not yet being tested
  • Updated ActivationCache to cover edge cases and improve DRY state
  • Updated SVDInterpreter with a lazy load to reduce base transformer_lens` load
  • Refactored bridge_core.py to be able to determine _registered_block_count with a method, to reduce the repeated calculations around that value
  • Adjusted the stop_layer system. Now processed as part of the bridge, where the indexes are already known, so they do not need to be reconstructed by the block components
  • Created an _eps helper for normalization
  • Fixed incorrect configurations for Gemma3n and MuseGlimmer
  • Added validation to patching
  • Removed the static _HYBRID_VARIANT_NAMES set and replaced with dynamic hybrid detection
  • Improved warning/error accuracy for DLA & other analysis tools
  • Refactored statistics to use sub-functions instead of mulitlayer if and variable declarations

Changes in behavior

  • Right-padded string-batch loss now includes the transition into a real trailing pad-id token, those loss values change and now equal the mean of the per-prompt token losses.
  • input_to_embed on a padded string batch returns the tokenizer's mask for either padding side. Before, a right-padding tokenizer got None and only a left-padding tokenizer got a mask (rebuilt from the ids).
  • input_to_embed(..., padding_side="right") on a left-padding tokenizer used to ignore the argument when building the mask, so it returned a left-side mask read off right-padded ids. It now returns the mask for the side used.
  • Statistics on GQA-padded cells become NaN; to_dataframe() row counts change on GQA results.
  • DLA no longer raises NotImplementedError for hybrids, it succeeds or raises ValueError.
  • Gemma 3n SVD w_in outputs change for every checkpoint
  • Muse Glimmer gains blocks.{i}.ln1/ln1_post/ln2/ln2_post.hook_scale/hook_normalized, and its post-norm head/neuron decompositions now return output.
  • ModernBERT-decoder lenses are refused at validate_model/fit.
  • run_with_cache(stop_at_layer=k) raises NotImplementedError on bridges without a blocks stack.
  • start_at_layer resumes correctly on Bloom and opaque/SSM stacks, and raises NotImplementedError on AltUp blocks.
  • ±inf effects give NaN p-values (was p = 0 or 1/(n_perm+1)). DLA unit="component" refuses an incomplete decomposition before the forward (was after a full forward).

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • Breaking change
  • This change requires a documentation update

Checklist:

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have not rewritten tests relating to key interfaces which would affect backward compatibility

@jlarson4
jlarson4 merged commit baf0e02 into dev Nov 11, 2026
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant