Repository navigation
perf(resolve): keep the members an import surfaces for the query asking again - #1358
Merged
Merged
Conversation
…ng again Co-Authored-By: jason.han <hanhuijun@gmail.com>
Contributor
Author
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
A kept enumeration is replayed into its reader's frame (what it read is captured as the collisions memo does) so an edit of a document it read drops the reader too, is not read while a filter condition's own names resolve (which admit every member), and is counted by MemoSize. Co-Authored-By: jason.han <hanhuijun@gmail.com>
A read inside a nested capturing was appended to every capture open above it and a hit replayed its reads into them again, once per nesting level; with kept enumerations that made an eight-package import cycle five times slower to analyse. A capture records its own reads and takes the captures settled or replayed inside it as children, folded into its lists once on its first replay. Co-Authored-By: jason.han <hanhuijun@gmail.com>
…shot A capture replayed from parallel batch workers must not be written after it settles: flatten now builds a separate snapshot of the capture's reads and its children's, under the capture's Once, reading only the children's own lists; capturing settles every capture, adopted or top-level, as before. Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
This was referenced Oct 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
The 0.11.0 performance record against v0.10.1 (#1337) leaves one regression after #1348: the language server's
ReferencesCold(44 ms → ~1.0 s, 9.8 MiB → ~950 MiB per query) andRenameWarm(20 ms → ~1.7 s, 1.4 MiB → ~1.5 GiB), bisected by exact allocation counts to #1062 (hidden colliding imports). #1348 removed the per-member garbage; what remains is the member lists themselves, rebuilt over and over: since #1062 made enumerating what a wildcard import surfaces expensive, every document a references query indexes and every occurrence a rename rewrites enumerated the same imports again. The resolver's per-edge memo (enumMembers, issue #633) was cleared when the outermost enumeration returned, so it only ever served one enumeration.The enumeration is now kept for the next one asking the same:
enumMembers, the per-edge memo (the members an import edge surfaces into its own namespace), stays as it was: provisional, filled within one enumeration and cleared when the outermost returns — inside a re-import cycle every edge is cut, and that memo is what keeps a cycle linear rather than once per path. Beside it,enumKeptkeeps across enumerations the edges whose result was not cut, andenumResultskeeps whatimportedMembersInto/ImportedMembersUnfilteredanswer for a (target namespace, referring namespace, import, member kinds) — the top-level enumeration, which the per-edge memo never covered.journalNew) with the import's document, so an edit that invalidates that document drops them, as the resolver's other tables. Each kept entry also carries what its computation read (keptMembers: the members and areadCapturetaken withcapturing), replayed into the reader's frame on every hit the waySharedCollisionsdoes, so a later document reading a kept enumeration depends on the documents it read and is dropped with them.MemoSizecounts both tables.hidesBelow) applies:enumCutcounts the cuts — an import edge skipped because it is on the enumeration stack (a re-import cycle), an import target still resolving (resolvingImports), a filter condition in progress (inCondition) — and a result computed while the count moved, or while either is set, is returned but not kept. Neither table is read while a filter condition's own names resolve (inCondition), which admit every member (KerML 8.2.4): a kept filtered enumeration would otherwise answer unfiltered.The members returned are the same; callers iterate the slices or copy from them (
withoutHiddenMembersfilters into a fresh slice), none appends to one.How it was verified
TestImportedMembersAreEnumeratedOncePerResolver(internal/semantic/resolve/enumeration_memo_test.go): enumerating the same import again allocates 26 objects on develop (the whole enumeration, the three members re-exported through two wildcard imports), 0 on this branch.TestImportedMembersKeptAreDroppedOnEdit: with the tracked index offrames_test.go, the imported package gains a member throughput(the workspace's invalidation path) and the import enumerates again and surfaces it.TestImportedMembersKeptMakeTheirReaderDepend: two tracked frames enumerate the same import; the second, which reads the kept enumeration, is among the imported document's dependents and is dropped by an edit of it (on the first head of this branch the dependents were the first reader only).TestImportedMembersInAConditionAreNotTheFilteredOnes:Lib::*[@Safety]enumerated once brings the one admitted member, insideInConditionboth, outside again one (on the first head the condition got the kept filtered one).TestImportedMemberEnumerationDoesNotWalkRoutesPerMember(perf(resolve): enumerate imported members without walking their routes #1348's test) now measures the enumeration itself (enumerateImported) rather than its kept result, so it keeps its per-member meaning: 1.09 allocations per member.go test ./internal/semantic/... ./internal/frontend/lsp/... ./tests/resolve/... ./internal/check/...,gofmt -l .empty,go vet ./...,go build ./...,python3 scripts/changelog.py check.A/B over six interleaved rounds, develop a662c2b | this branch at its final head (
benchstat, n=6, allocations exact; this machine's timings carry ±5–30%):ReferencesColdRenameWarmReferencesWarmWorkspaceUpdateFormatEditsWorkspaceEdit/vehicle/reindex+diagnosticsGRPCParseFileUncachedWorkspaceEdit/synthetic(+ reindex),Analyze/synthetic,ImportValues/*(8 sizes)For comparison, 0.10.1 measured 44 ms / 9.8 MiB / 44.6k on
ReferencesColdand 20 ms / 1.4 MiB / 13.7k onRenameWarm(the record, #1337); the rest of the distance is #1062's enumeration itself, carried in the record.The two gates a resolver change must hold, both sides on this machine:
TestPackagesImportingEachOtherInACycleAnalysePromptly/n=8(tests/model,-race) 0.50 s on develop, 0.54 s here — a first form of the reads replay, which appended each read to every capture open above it, had taken it to 2.44 s against its 5 s deadline, which is why a capture now records only its own reads and adopts the captures settled or replayed inside it as children, folded into a separate snapshot on its first replay (flatten; the snapshot keeps a settled capture immutable, asSharedCollisionsreplays from parallel batch workers —TestCapturesReplayInParallelWithoutRacing); andTestInterfaceRecordDifferential/examples/self-model(-race) 17.0 s on develop, 15.4 s here.TestCapturesNoteEachDocumentOnceHoweverOftenItRepeatsnow checks what the outer capture replays into a frame rather than its own list, and that a top-level capture is settled.Checklist
make testandmake lintpass locally (the packages above; the full suite runs in CI)changes/unreleased/memoize-imported-members.fixed.md, not as an edit toCHANGELOG.mdmake docs-countsrun if a gate count moved (none moved)F4,K5) in the body, docs, or changelogLink to Devin session: https://nasa-jpl-demo.devinenterprise.com/sessions/cf22d7a746814d20b2f0eaf11682fef9
Open in Devin Desktop: https://nasa-jpl-demo.devinenterprise.com/desktop/session/cf22d7a746814d20b2f0eaf11682fef9?variant=devin
Requested by: @HuiJun