Skip to content

Refine per-feature smoothing_rounds handling - #700

Merged
paulbkoch merged 3 commits into
mainfrom
paulbkoch-refine-per-feature-smoothing-rounds
Oct 27, 2026
Merged

paulbkoch merged 3 commits into
mainfrom
paulbkoch-refine-per-feature-smoothing-rounds

Conversation

@paulbkoch

@paulbkoch paulbkoch commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Refinements to the sequence forms of smoothing_rounds and interaction_smoothing_rounds.

Changes

  • Per-bag counters: boost() decrements the per-term smoothing counters in place, so it now works on its own copy. Every outer bag starts with its full budget, including when bags run in-process (n_jobs=1).
  • Constant-time smoothing checks: whether smoothing is still active is cached in a flag that is refreshed only when the counters change, instead of scanning every counter on each boosting step. Models fit with a scalar smoothing_rounds are unchanged.
  • Validation before boosting: a sequence interaction_smoothing_rounds is checked against the cleaned interactions list before any boosting starts. One-shot iterables such as itertools.combinations(...) work too.
  • Input types: any integer-valued scalar or 1-D array-like is accepted, including range, pandas.Series, and NumPy scalars. Whole-number checks use each value's own precision, including extended-precision longdouble. Booleans, strings, and values beyond the int64 range are rejected.
  • JSON export: to_json writes the smoothing parameters as built-in integers, or lists of them, so NumPy inputs such as longdouble serialize.
  • Docs: the smoothing_rounds and interaction_smoothing_rounds docstrings describe the sequence forms consistently across estimators: entries for excluded features, or for duplicate or excluded interactions, are ignored, and features that finish smoothing early are not boosted until smoothing ends.

Tests

New cases in tests/glassbox/ebm/test_smoothing_rounds.py cover scalar/sequence equivalence with several outer bags, exclusion versus dropping the column, duplicate and excluded interaction entries, validation before boosting, array-like and NumPy scalar inputs (including longdouble), booleans, values beyond the int64 range, and JSON export.

Validation

  • pytest tests/glassbox/ebm/test_smoothing_rounds.py: 51 passed
  • pytest -m "not slow and not selenium" tests/glassbox/ebm/: 383 passed, 42 skipped
  • ruff format --check / ruff check on the changed files: no new findings
  • mypy: no new errors

- Copy the per-term smoothing counters in boost() so every outer bag
  starts with its full budget, including when bags run in-process.
- Cache whether smoothing is active instead of scanning all counters
  on each boosting step.
- Validate a sequence interaction_smoothing_rounds against the cleaned
  interactions list before boosting starts.
- Accept any integer-valued scalar or 1-D array-like; reject booleans
  and strings.
- Write array-valued smoothing parameters as lists in to_json.
- Describe the sequence forms consistently in the estimator docs.
- Add tests for multiple outer bags, exclusion alignment, validation
  before boosting, array-like inputs, and JSON export.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Out-of-range sequence values can become negative counters, leaving smoothing active and silently changing training behavior.

1 open issue
What changed in this PR

This PR refines per-feature and per-interaction smoothing in EBM training, improving bag isolation, validation, and serialization.

Changes:

  • Copy smoothing counters per outer bag and cache whether smoothing remains active.
  • Validate interaction sequences before boosting and broaden accepted numeric inputs.
  • Export array-valued smoothing parameters as JSON lists, clarify docstrings, and add regression tests.
File Description
python/​interpret-core/​tests/​glassbox/​ebm/​test_smoothing_rounds.py Adds regression coverage for smoothing budgets, validation, input types, and JSON export.
python/​interpret-core/​interpret/​glassbox/​_ebm.py Refines normalization, early interaction validation, and parameter documentation.
python/​interpret-core/​interpret/​glassbox/​_ebm_core/​_json.py Converts smoothing parameters to JSON-compatible values.
python/​interpret-core/​interpret/​glassbox/​_ebm_core/​_boost.py Isolates per-bag counters and caches smoothing-state checks.

Review effort: Balanced

Comment thread python/interpret-core/interpret/glassbox/_ebm.py Outdated
Values of 2**63 or more, such as a list containing 2**63 or floats like
1e19, now raise a ValueError instead of wrapping to negative counters.
Scalars that large raise the same error up front instead of an
OverflowError inside boost().

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Extended-precision NumPy inputs can bypass integer validation and fail JSON serialization.

No open issues 🎉

2 previously missed issues

In code that hasn't changed since last review

Medium severity Preserve longdouble precision when validating integrality

python/​interpret-core/​interpret/​glassbox/​_ebm.py:245

Converting to Python float before checking integrality loses precision for extended-precision NumPy scalars. On platforms with wider longdouble, np.longdouble("1.0000000000000001") becomes 1.0 here, so this fractional input is accepted and silently truncated to 1. Check integrality in the original dtype, as the sequence branch does, before converting to int.

Medium severity Convert longdouble smoothing parameters to serializable integers

python/​interpret-core/​interpret/​glassbox/​_ebm_core/​_json.py:172

tolist() does not always produce built-in scalars: for longdouble, it retains NumPy scalar objects. A newly supported value such as smoothing_rounds=np.longdouble(5) therefore fits successfully but still makes to_json() raise TypeError; the same applies to interaction_smoothing_rounds. Convert these integer-valued parameters to int64 or built-in integers before serialization, preserving None.

1 resolved since last review

Review effort: Balanced

- Check a scalar smoothing_rounds for a fractional part in the input's
  own dtype, as sequences already were. Converting to float first could
  drop the fraction of an extended-precision longdouble.
- Convert sequence entries to exact Python ints before the int64 range
  check. Comparing longdouble values with Python ints can round, or
  raise TypeError or OverflowError, depending on the platform and NumPy
  version.
- Write the smoothing parameters to JSON as built-in ints so longdouble
  values serialize. None is still written as null.
- Skip the RuntimeWarning that inf entries triggered during the check.
- Add longdouble cases to the smoothing_rounds tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused changes are consistent with their callers, regression tests cover the principal behavior, and no blocking issues remain.

No open issues 🎉

Review effort: Balanced

@paulbkoch
paulbkoch merged commit 4a492be into main Oct 27, 2026
63 checks passed
@codecov

codecov Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 67.78%. Comparing base (26780a6) to head (49e0b91).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #700      +/-   ##
==========================================
+ Coverage   67.76%   67.78%   +0.02%     
==========================================
  Files          77       77              
  Lines       11890    11893       +3     
==========================================
+ Hits         8057     8062       +5     
+ Misses       3833     3831       -2     
Flag Coverage Δ
bdist_linux_311_python 67.52% <100.00%> (+<0.01%) ⬆️
bdist_linux_314_python 67.45% <100.00%> (+<0.01%) ⬆️
bdist_linuxarm_311_python 67.56% <100.00%> (+0.01%) ⬆️
bdist_linuxarm_314_python 67.45% <100.00%> (-0.01%) ⬇️
bdist_mac_311_python 67.70% <100.00%> (+0.02%) ⬆️
bdist_mac_314_python 67.63% <100.00%> (+0.04%) ⬆️
bdist_win_311_python 67.72% <100.00%> (+<0.01%) ⬆️
bdist_win_314_python 67.63% <100.00%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants