Skip to content

lib: bound the generated source map cache - #65761

Open
lazerg wants to merge 1 commit into
nodejs:mainfrom
lazerg:fix-source-map-cache
Open

lib: bound the generated source map cache#65761
lazerg wants to merge 1 commit into
nodejs:mainfrom
lazerg:fix-source-map-cache

Conversation

@lazerg

@lazerg lazerg commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

generatedSourceMapCache grows for every evaluated source that carries a unique //# sourceURL, and nothing ever removes an entry. Unlike module sources, generated sources have no reachable script instance to key the cache weakly against, so code that re-evaluates the same file under a changing source url, which is what hot module replacement does, keeps every payload it ever mapped alive.

Hold the least recently used generated source maps within a byte budget instead. Bytes rather than entries, because one map can carry hundreds of kilobytes of sourcesContent while the next carries a handful. With the repro from the issue, retained heap settles at 40 MB rather than growing past 500 MB over 1200 evaluations.

Fixes: #65760

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. source maps Issues and PRs related to source map support. labels Sep 3, 2026
@lazerg
lazerg force-pushed the fix-source-map-cache branch from 096455f to 0022ed5 Compare September 3, 2026 12:49
@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.06%. Comparing base (4e207b1) to head (30748a2).
⚠️ Report is 17 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #65761      +/-   ##
==========================================
+ Coverage   89.95%   90.06%   +0.10%     
==========================================
  Files         759      769      +10     
  Lines      258637   261316    +2679     
  Branches    49015    49625     +610     
==========================================
+ Hits       232665   235361    +2696     
+ Misses      17018    16981      -37     
- Partials     8954     8974      +20     

see 53 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@arunanshub

Copy link
Copy Markdown

Thanks for the fast fix. Two points.

Recency updates only on write.

maybeCacheSourceMap deletes and sets the key. findSourceMap does not change the order. Thus the cache is FIFO, not LRU. This is correct for HMR, because each key is unique. But a long-lived generated function loses its map after 256 other evals. The function is still live. Its stack traces become unmapped. Refresh the entry on read in findSourceMap to prevent this.

The limit counts entries not bytes.

Entry size follows the size of the mapped file. In the reported MDX case each entry is approximately 612 KB. 256 entries is then approximately 157 MB. In the synthetic repro each entry is approximately 415 KB, or approximately 106 MB. The ceiling changes with the user's largest generated file. Did you consider a byte budget?

Minor

the test uses 300 against a limit of 256. Export the constant to keep the test correct.

@lazerg
lazerg force-pushed the fix-source-map-cache branch from 0022ed5 to 40865c5 Compare September 3, 2026 16:15
Signed-off-by: Lazizbek Ergashev <lazerg2@gmail.com>
@lazerg
lazerg force-pushed the fix-source-map-cache branch from 40865c5 to 30748a2 Compare September 3, 2026 16:59
@lazerg

lazerg commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

All three applied in 30748a2.

The cache now keeps a byte budget of 32 MiB instead of an entry count. Each entry is charged its source map url plus its line lengths, and the payload is added when it is resolved. The oldest entries go first, and the newest entry is always kept, so one large map can still be mapped on its own.

findSourceMap now moves the entry it reads back to the newest end, so a generated source that is still in use no longer ages out.

The limit is exported, and the test reads it to work out how many sources to evaluate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-ci PRs that need a full CI run. source maps Issues and PRs related to source map support.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

generatedSourceMapCache is an unbounded strong Map: eval'd code with unique //# sourceURL grows the heap without limit under --enable-source-maps

3 participants