[zahariash]

This work was produced with the help of language models.

Fixes #1301. Part of #1290.

The first query in a process deserialized the lexer's ATN, which builds large Unicode character sets one interval at a time, and IntervalSet::add walked the whole interval list for each one: about half of the first query's 9.39 ms compile time for RETURN 1 (query-summary time, median of 8 processes). The 0.18 ms in #1290 is a later query.

Fix. An interval that goes into an empty set, or starts after the last interval without touching it, is appended directly; everything else takes the existing loop, so the set comes out the same. Upstream 4.13.2 and the dev branch have the same loop, so the change is a patch in third_party/ports/antlr4_runtime, next to three older local changes to the vendored 4.13.1 runtime that were recorded nowhere.

Performance. First query compile 9.39 → 5.16 ms, 77.7M → 42.0M instructions; later queries unchanged. Mallocs are unchanged too: about 68.4K for the first query (within run-to-run noise) and 601 per later RETURN 1 at one thread. The rest of the first compile is ANTLR filling its prediction cache on first use.

Tests. AntlrIntervalSetTest.AddKeepsCanonicalForm: random and sorted inserts plus touching intervals, against brute-force membership and the canonical sorted, disjoint, non-adjacent form. EofMergesOnlyWithTouchingRange: EOF (-1) merges with a range starting at 0 and stays separate from one starting at 1. EmptyIntervalAndRangeLimits: an empty interval is ignored, and appends and merges near SSIZE_MAX give the same set. All three fail if the fast path appends an interval that touches the last one. AppendAfterRangeMinimum: a set starting at SSIZE_MIN takes the fast path without signed overflow (fails under UBSan if the check uses adjacent()). Full ctest on this branch: 3104/3104.

Independent of #1297 and #1300; with all three the first query compiles in 5.1 ms and executes in 0.33 ms (query-summary median of 8 processes).

[adsharma]

How about something like:

  if (_intervals.empty() ||
      (_intervals.back().b < addition.a && addition.a != _intervals.back().b + 1)) {

since adjacent() doesn't check for overflow/underflow?

Also, we should track this under third_party/ports if we're going to maintain patches against upstream.

[zahariash]

Done in 9bbd8b6 and 4eecc2b.

The fast path uses your condition. The new AppendAfterRangeMinimum test starts a set at SSIZE_MIN; under -fsanitize=signed-integer-overflow it fails on adjacent()'s a - 1 with the old condition and passes with this one. The upstream loop below still calls adjacent(); left as ANTLR has it.

The change is now third_party/ports/antlr4_runtime/patches/interval-set-append.patch, next to three older changes to the vendored runtime that were recorded nowhere (the <chrono> include for Windows, the non-virtual ParseTree::operator== for GCC 13, the deprecation pragmas in Vocabulary.cpp). ./pkg antlr4_runtime update reproduces the vendored tree byte for byte and keeps our CMakeLists.txt.