Repository navigation
perf(repl): count a diagnostic's UTF-16 column without encoding the line - #1352
Merged
Merged
Conversation
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Contributor
Author
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
This was referenced Oct 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
Since #1205 (SARIF/CI formats count columns in UTF-16) every
sysmldiagnostic carries aUTF16Column, computed byutf16Columnaslen(utf16.Encode([]rune(string(content[lineStart:offset])))): an encoded copy of the line prefix per diagnostic (and two more per quick fix), thrown away after its length is read. The snippet path also converted the whole snippet source to[]bytetwice per diagnostic (utf16Column([]byte(sn.src), …),locatedFixes(…, []byte(sn.src), …)). In the 0.11.0 release-gate measurementinternal/frontend/replBenchmarkDiagnosticsreads 755 → 1 044 B/diagnostic (attributes=50), 638 → 929 (200), 574 → 829 (800) against 0.10.1; the allocation profile puts theutf16.Encodecopies at about half of the new allocations per diagnostic (the rest, and nearly all of the new bytes, is the semantics work of #1276/#1319 —AllSupertypesand the wider type sets — measured separately).The column is now counted in place:
utf16Unitswalks the prefix withutf8.DecodeRune, two units for an astral rune, one otherwise — the same countutf16.Encodegives, including one unit for an invalid byte (RuneError). The snippet source is converted to bytes once per snippet, beside the line index it already caches per snippet. Columns are unchanged.How it was verified
Failing-before test
TestUTF16ColumnCountsWithoutEncoding(internal/frontend/repl/utf16_column_test.go): for ASCII, two-byte, astral and invalid-byte lines the column equalslen(utf16.Encode(...)) + 1, and a repeated call allocates 0 times — on develop d8b60ed it allocates once per call for every line.A/B over six interleaved rounds on one machine (allocations and bytes are exact, so
benchstatreports p=0.002 on every row; sec/op was not usable on the loaded machine and is omitted). Columns: pre-feat(cli): SARIF and JUnit reports, a baseline and -warnings-as-errors for sysml -validate #1205731645777| developd8b60ed62+ fix(semantics): derive a declaration's implied generals once per symbol #1350 | this change + fix(semantics): derive a declaration's implied generals once per symbol #1350.Diagnostics/attributes=50Diagnostics/attributes=200Diagnostics/attributes=800tests/perfREPLLoadFiletests/perfREPLSubmitSnippetThe change removes one allocation per diagnostic (two with a quick fix) and the per-diagnostic snippet copies — half of the row's allocation growth since feat(cli): SARIF and JUnit reports, a baseline and -warnings-as-errors for sysml -validate #1205;
REPLLoadFile/REPLSubmitSnippetdo not render diagnostics and are unchanged, as expected.go build ./...,go vet ./internal/frontend/repl,gofmt -l .empty,go test ./internal/frontend/repl(with the training corpus present),python3 scripts/changelog.py check.Checklist
make testandmake lintpass locally (repl package run locally; the full suite runs in CI)changes/unreleased/<slug>.<section>.md, not as an edit toCHANGELOG.mdmake docs-countsrun if a gate count moved (no gate count moved)F4,K5) in the body, docs, or changelogLink to Devin session: https://nasa-jpl-demo.devinenterprise.com/sessions/cf22d7a746814d20b2f0eaf11682fef9
Open in Devin Desktop: https://nasa-jpl-demo.devinenterprise.com/desktop/session/cf22d7a746814d20b2f0eaf11682fef9?variant=devin
Requested by: @HuiJun