[savetheclocktower]

This PR delivers a series of performance improvements that are meant to allow text-buffer to do smaller increments of work at a time — by changing the unit of work from a buffer row to a screen row.

This has a huge performance upside for Pulsar’s longtime nemesis: a large file with very long buffer lines and few (or zero) newline characters. There are further optimizations we could make, but the changes in this PR are enough to vastly improve the experience of interacting with such a file in Pulsar.

The fact that we’re now operating in increments of screen rows prompts a question: what about when soft-wrapping is disabled? Do we lose these optimizations altogether? No! Because soft-wrapping is never truly disabled — only minimized. By default, even when editor.softWrap is false, Pulsar will force a soft wrap on a line at or before the 500-character mark for performance reasons. (The threshold is configurable, but this constraint cannot be disabled.)

So these optimizations will make plenty of difference no matter which mode the user is in simply by no longer committing to a unit of work that is theoretically unbounded (the length of a single buffer line).

Each optimization is its own commit, so I’ll walk through them one at a time.

Commit 1: Stop the screen-line builder from doing throwaway work

The ScreenLineBuilder instance is responsible for taking buffer text and turning it into text as it's presented on screen. That includes soft-wrapping long lines, hiding folded text, and expanding hard tab characters into the appropriate number of spaces.

As the name implies, screen rows are its currency; it is often asked to build/rebuild only certain screen rows rather than a whole buffer line. But on master, it still does unnecessary work related to the screen rows that occurred on the same buffer row before the screen row(s) it's asked to render. It rebuilds each of those screen rows and caches them, which is polite… but too eager.

ScreenLineBuilder’s screen-line cache is a poor choice for cache “warming” in this manner because of how often it gets invalidated. It makes sense to memoize work you’ve already done in case you can reuse it later, but it does not make sense to pre-fill the cache. Every keystroke ends up clearing some of that cache, so pre-filling will more often than not result in unused work.

The performance boost delivered by this change is proof enough of that; skipping this eager caching is responsible for almost all of the gains. I had Claude run some benchmarks on a buffer with one row consisting of 1,000,000 characters:

Measurement	15.1.2	Commit 1
Render the first 50 screen rows (cold)	5.9ms	6.0ms
Render the last 50 screen rows (cold)	262ms	6.7ms
Edit (spatial index update)	21ms	22ms
Render after the edit	138ms	6.3ms

On master we end up doing more and more work as the edits move farther and farther from the beginning of the line, but after this commit the render time is no longer dependent on how far into the row the edit is.

Commit 2: Re-wrap only affected screen rows after an edit

We keep something called a spatial index in order to help us convert between buffer coordinates and screen coordinates. This is an important cache that prevents us from doing redundant work during these conversions.

That’s why it was bad for us that, when an edit touched a certain buffer row, we re-wrapped more text than we needed to and cleared the cache for the entire buffer row instead of just the part of the buffer row affected by the edit.

It is tricky to predict exactly what will rewrap once you make a given edit. It’s obvious how adding text can reflow the current screen row and possibly all lines below it on the same buffer row. But it’s also possible for an edit to reflow the row before the current screen row (if it’s a deletion that makes it possible for some of the current screen row’s content to fit on the row above) and even, in rarer scenarios, any arbitrary screen row above the current screen row (though only in scenarios involving hard tab characters).

So another commit in this PR reduces the amount of extra work we do by identifying which screen rows can stay the same and which must be re-wrapped. For screen rows after the edited row, it figures out where (if anywhere) screen rows converge — i.e., the point at which the edit stopped a cascade of re-wraps on subsequent screen lines. The first screen row that does not get re-wrapped after an edit is what tells us we can stop doing work, since all screen rows after that row must also have stayed the same.

This commit also introduces the BufferRowReader class. Its purpose is to optimize the reading of the superstring buffer in JS code. We don’t want to ask for more text than we need (making superstring do unnecessary work), but we also don’t want to waste time asking for text character-by-character and incurring the same overhead multiple times. So BufferRowReader does what it says on the tin: it asks for fixed-size chunks of a buffer row as needed, splitting the difference between the two approaches.

As with the first commit, the performance gains in this commit mostly come from skipping throwaway work (in this case the unnecessary invalidation and repopulation of the spatial index). The actual improvement depends on where the edit falls in a long buffer row, but with our 1M-character test buffer, a middle-of-the-row edit improved performance from 130ms to 0.16ms (about 130x faster!).

Sadly, there is less of a performance benefit when hard tab characters are present above us in the buffer row, because they defeat our strategy to detect which screen rows will re-wrap in response to an edit. We still enjoy the performance benefit of skipping work on screen rows after the convergence point.

The challenge of hard tabs is as follows: suppose an edit happens on screen row 10 of a buffer row. Any hard tabs that appear on screen rows 2–10 mean that we can’t rule out the possibility of needing to re-wrap screen rows above the edit; so we are forced to re-wrap from the start of the buffer row as before (though, again, we are still able to stop before the end of the buffer row in most cases).

This is a conservative approach and does not rule out further optimizations in the future to better target the edge cases that defeat our strategy.

This commit changes layout behavior slightly, but in a way that is more consistent and predictable. The behavior change is incredibly obscure and only comes into play when the buffer row contains hard tab characters, but here’s a contrived example.

Contrived example

This example uses the following settings:

• soft wrap column 9;
• tab length 4;
• no hanging indent.

The text is a aa→ aa, where → is a hard tab. The tab is followed by two spaces.

Before (15.1.2):

a
aa→
aa

After:

a
aa→   aa

Claude’s description:

This changes layout slightly: after inserting a soft wrap, the scanner now
rewinds to the wrap's buffer column and rescans in the new screen row's
context. Previously, characters already measured on the old screen row were
carried over, so a hard tab carried onto a new screen row could be measured
at the wrong tab stop. The fast path depends on this, since both paths must
produce the same layout.

Commit 3: Apply BufferRowReader more broadly

The BufferRowReader class we introduced in commit 2 was worth applying in several other places. In particular it’s worth swapping in wherever the existing approach is to call getCharacterAtPosition directly on the buffer in a loop.

This is a small performance boost compared to other commits, but it’s dead-simple, and helped improve time to rewrap 50 screen rows (on our 1M-character test buffer) from 6ms to 1.2ms.

Commit 4: Re-highlight buffers in increments of screen rows, not buffer rows

In Pulsar, the editor’s language mode is in charge of telling the buffer when certain regions of the buffer need to re-apply syntax highlighting. Internally, text-buffer remembered that invalidated range, but then expanded the range’s end to the nearest buffer-row boundary when driving re-highlighting: it took the HighlightIterator given to it by the editor and called #seek(startingPoint, endingBufferRow). That was the signature of HighlightIterator#seek, and it obligated both text-buffer and Pulsar to do more work than was strictly necessary.

This commit fixes that. First, a reported highlighting change clears only the cached screen rows that touch the invalidated region, rather than all screen rows of buffer rows affected by the edit.

Next, the second argument to HighlightIterator#seek is now a point rather than a row, and it represents the exact position at which highlighting can stop. This allows the editor (which is in charge of telling text-buffer about all scope boundaries and positions) to minimize the amount of work it does to prepare that information.

This is a breaking change, so it means bumping the major version to 16; it also means that we should not bump text-buffer in Pulsar until we have updated all HighlightIterator implementations to understand the new signature. (We can do duck-typing on the second parameter and fall back to the original behavior if it’s an integer; this is responsible and keeps support for any, uh, “unofficial” consumers of highlight iterators. Retaining the first argument as a Point, rather than consolidating the two Point parameters into a single Range, is also necessary because the second argument is optional and is ignored for TextMate-style grammars.)

The performance improvement we get for this one is harder to quantify since half of the work has to be done on the Pulsar side. But in our 1M-character test buffer, an edit that affected only one screen row invalidated only one screen row instead of 2,041 — thereby reducing re-render time from 4.8ms to 0.3ms (~16x improvement!). That undercounts the performance benefit because it skips the cost of highlighting — which, for a Tree-sitter grammar, is a function of how much got invalidated at once (and therefore how much of the tree we must run a highlights.scm query against). The screencasts below do a better job of illustrating the scale of the improvement.

Screencasts

Sounds great in the abstract — but what do all these performance boosts gain us in practical terms?

I pointed my Pulsar installation to this branch’s text-buffer and wrote some quick code to adapt to the new HighlightIterator API. Here are before/after screencasts for a nightmare scenario: a JSON dump that has zero newlines and is over 100K.

First, the control from 1.133.1:

before-optimization.mp4

You can tell when the editor is freezing because the cursor stops blinking; that's also when I start twirling my mouse pointer. It happens on both attempts to delete text.

Now here’s the “after” screencast, though stay tuned for a caveat:

after-optimization.mp4

I was vexed by the freeze that happened in the middle; it happened when I pressed Opt+Delete, which is the macOS keybinding for editor:delete-to-previous-word-boundary. I opened the profiler and tried to narrow it down.

It turns out that’s because I’d had atom-ide-hyperclick configured to show clickable symbols when I hold down Opt — so just pressing it on the way to pressing Opt+Delete triggered a buffer-wide tags.scm query. That’s another pitfall of an obscenely large buffer that we can tackle later; one strategy there might be to allow symbol-provider-tree-sitter to subtract from its own “suitability” score (part of how symbols-view picks a provider) when the buffer is large enough to produce this sort of freeze.

Anyway, that’s my way of saying that the freeze in the middle is unrelated. Once I disabled atom-ide-hyperclick temporarily, it went away. You can see that the editing experience is tolerable and that syntax highlighting updates rather quickly. This is especially impressive because JSON is a near-worst-case scenario for syntax highlighting; any edit that turns the JSON invalid (in Tree-sitter’s view) ends up triggering a highlighting update for the entire buffer, since it likely changes the structure of most or all of the syntax tree.

In fact, here's a third screencast in greater detail:

after-optimization-2.mp4

This is me deleting and restoring text at the end of a buffer row that is 1.3M+ characters long. Edits are near-instantaneous and just as fast as the edits in screencast 2 that happened much earlier in the buffer row. I widen the window in the middle of the screencast just to make the buffer coordinates visible in the status bar.

The improvement is honestly much, much better than I was expecting. I figured we'd have to disable syntax highlighting for huge files altogether, but these changes alone might be enough to make editing of long lines tolerable.

Tests

The implementations of these optimizations were achieved using one talented robot named Claude and one extremely skeptical human who asks lots of questions and does not trust the robot. I pored over each commit’s diff to make sure I understood the code; and, as with #21, I wrote this entire PR description by hand to ensure my understanding was deep enough to explain to others.

The specs got updated in the following ways:

• The first commit makes the pre-existing randomized spec seeded so that we can reproduce a specific randomization if it helps us diagnose something. The new verifyScreenLinesForRandomRange helper is related to this.
• Commit 2 adds a verifySpatialIndexMatchesNewDisplayLayer helper to supplement the checks that were already present. Without it, we’d have no way of detecting an incorrect incremental update to the index. It also introduces a new randomized spec that tests row re-wrap behavior, plus one that deterministically tests re-wrap behavior in a scenario where an edit on screen line 3 affects screen line 1.
• Commit 3 didn’t add any specs, since it didn’t change any behavior. The suite stayed green, proving it was a more efficient way of doing the same work.
• Commit 4 changes specs so that they test the new seek contract. It adds a spec to verify that invalidating just one screen row does not also invalidate other screen rows on that buffer row. It makes one of the randomized specs occasionally perform a highlighting-only change. The existing "emits update events from the display layer when the language mode's highlighting changes" spec is unchanged and helps prove the soundness of the approach and the lack of regressions.

What’s next?

This implements most of what’s mentioned in #21. As for the rest:

• We can introduce more opportunities to soft-wrap (e.g., on punctuation), but that change needs to happen in Pulsar.
• The “Make smaller buffer text lookups” idea is somewhat implemented (BufferRowReader did it, mostly), but there’s a related aspect that isn’t yet optimized: skipping needing to walk the entire start of the line when soft-wrap is disabled. Pulsar, by enforcing eventual soft-wrap even when editor.softWrap is false, helps us avoid the worst-case scenario here, but it might still be worth fixing this in text-buffer. It would require keeping track of hard tabs on a row-by-row basis — not just the number of hard tabs in each line but their exact positions.

But there are other items we could explore that aren’t in #21:

• We can improve upon commit 2’s behavior by narrowing the cases in which it skips the optimization (i.e., only when a half-width character is present).
• We could add folded text as another random effect in the randomized long-buffer-row spec.
• Commit 2 has another optimization skip: it bails out when the buffer row has text that is hidden by a fold. This, too, is safe but overly broad; we could narrow the exclusion further.
• There are a few other places we could use BufferRowReader, though the performance gains would be much more modest.