Skip to content

src: warn when heap would exceed pointer compression limit - #67468

Open
jasnell wants to merge 1 commit into
nodejs:mainfrom
jasnell:heap-pointer-compression-warning
Open

jasnell wants to merge 1 commit into
nodejs:mainfrom
jasnell:heap-pointer-compression-warning

Conversation

@jasnell

@jasnell jasnell commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

In builds without pointer compression, emit a process warning with the code HEAP_EXCEEDS_POINTER_COMPRESSION_LIMIT the first time the heap that survives a full garbage collection reaches 6.5 GB / 8 GB threshold. With pointer compression enabled, the heap of each isolate is limited to its 4 GB cage, so applications that currently use larger heaps may run out of memory in builds that enable it.

The check runs in a GC epilogue callback for mark-compact GCs only, is done at most once per Environment, and unregisters itself after the warning has been scheduled. The warning can be silenced with --disable-warning=HEAP_EXCEEDS_POINTER_COMPRESSION_LIMIT.

With any luck, the hope is that we can work towards enabling pointer compression by default in releases.

Note: Node.js builds in V8's multi-cage mode, where the main thread and every worker thread is created in it's own Isolate Group. Every Isolate Group has a max 4 GB heap limit. It used to be that the 4 GB limit was process wide, but that's not the case any longer. However, with pointer compression, no single thread can exceed the 4 GB limit, so applications running larger heaps would crash.

Currently, pointer compression has to be turned on at build, with the default build having it off. I'd like to work towards flipping that. Applications running larger-than-4GB heaps should be the exception but we're forcing every application to use more memory than they need to in order to support those exceptions.

For consideration: the warning here may not be entirely accurate yet. An application that hits 4 GB limit today won't actually fail under pointer compression since it will be using less heap... I'm working on refining the heuristic. May be worth holding this PR open until we refine that.

Update: Ok, I've updated it so that the warning uses a more accurate threshold... 6.5 GB / 8 GB.

@jasnell
jasnell requested review from a team, joyeecheung and mcollina November 14, 2026 15:46
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/startup

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. v8 engine Issues and PRs related to the V8 dependency. labels Nov 14, 2026

@mcollina mcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@jasnell
jasnell force-pushed the heap-pointer-compression-warning branch from b8b33ac to 139e03c Compare November 14, 2026 16:00

@mcollina mcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@jasnell jasnell added request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. author ready PRs with CI started, the required approvals, and no outstanding review comments. labels Nov 14, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Nov 14, 2026
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

@codecov

codecov Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.70968% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.67%. Comparing base (47d6da0) to head (ae315f7).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
src/node_v8.cc 73.33% 0 Missing and 4 partials ⚠️
src/env.cc 92.50% 0 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #67468      +/-   ##
==========================================
- Coverage   90.68%   90.67%   -0.02%     
==========================================
  Files         799      799              
  Lines      280243   280305      +62     
  Branches    54121    54142      +21     
==========================================
+ Hits       254136   254157      +21     
- Misses      16221    16274      +53     
+ Partials     9886     9874      -12     
Files with missing lines Coverage Δ
src/env-inl.h 94.20% <100.00%> (+0.04%) ⬆️
src/env.h 98.70% <ø> (ø)
src/node.cc 79.27% <100.00%> (+0.06%) ⬆️
src/env.cc 84.53% <92.50%> (+0.34%) ⬆️
src/node_v8.cc 87.00% <73.33%> (-0.37%) ⬇️

... and 35 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

@jasnell

This comment was marked as resolved.

In builds without pointer compression, emit a process warning with
the code HEAP_EXCEEDS_POINTER_COMPRESSION_LIMIT when the heap that
survives a full garbage collection suggests that it would not fit in
the 4 GiB cage that limits the heap of each isolate when pointer
compression is enabled.

Pointer compression shrinks a heap by at most half, and by about 40%
in practice, so the warning says the heap is likely to exceed the
limit from 6.5 GiB, and that it will exceed the limit from 8 GiB.

The check runs in a GC epilogue callback for mark-compact GCs only,
is done at most once per Environment, and unregisters itself after
the warning has been scheduled. The warning can be silenced with
--disable-warning=HEAP_EXCEEDS_POINTER_COMPRESSION_LIMIT.

Assisted-by: Claude
Signed-off-by: James M Snell <jasnell@gmail.com>
@jasnell
jasnell force-pushed the heap-pointer-compression-warning branch from 139e03c to ae315f7 Compare November 15, 2026 21:29
@nodejs-github-bot

nodejs-github-bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Collaborator

@gurgunday gurgunday left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@jasnell jasnell added the commit-queue PRs queued for automated landing through the Commit Queue. label Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author ready PRs with CI started, the required approvals, and no outstanding review comments. c++ Issues and PRs that require attention from people who are familiar with C++. commit-queue PRs queued for automated landing through the Commit Queue. needs-ci PRs that need a full CI run. v8 engine Issues and PRs related to the V8 dependency.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants