Skip to content

perf(semantics): memoize the base feature a declaration's kind implies - #1354

Merged
HuiJun merged 3 commits into
developfrom
perf/ci-runtime-budget
Oct 21, 2026
Merged

HuiJun merged 3 commits into
developfrom
perf/ci-runtime-budget

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

What and why

The CircleCI Go race tests (build 12113) and Go coverage shards (build 12104) jobs on develop died on go test's 30-minute per-package budget at d8b60ed, the merge of #1319 (usages subset the base library usage of their kind), and had passed on b60a18c just before. This PR measures where the time went and removes the remaining cost of that change on the hot paths.

Root cause of (a). After #1319 every ImplicitGenerals / DirectSupertypes derivation calls implicitUsageBaseFeature, implicitKerMLFeatureBase, implicitBaseUsage and declaredChainReachesKindBase, each of which re-walks the declared generalization chain (declaredGeneralizationReaches) and re-resolves the library name (LookupQualified), five to eight times per symbol. Counting over tests/stressmodel on develop: 178 300 computeImplicitGenerals calls (2.66 M memo hits), none provisional, so the memo itself works — each miss is simply expensive. The fix memoizes the kind base once per symbol, journaled and guarded exactly like implicitBases and ImplicitGenerals; semantics and outputs are unchanged.

Overlap with open PRs. None memoizes this. #1349 (conformsByName, filter.go), #1232 (API JSON, touches implicit.go for derived properties only), #1327 (crossing.go), #1314 (gather.go/model.go recording) and the runtime PRs #1340/#1308/#1310/#1311/#1312 do not touch the kind-base lookup.

Step 1 — where the time went (last 26 develop merges)

Three causes, separated:

CircleCI Go race tests (resource_class large, parallelism 5): per-node wall minutes and per-package go test seconds

develop commit PR CircleCI build job runtime: node min runtime: exec/runtime explore runtime: exec/runtime rest runtime: tests/wasm runtime: tools conformance model: node min model: tests/corpus model: exec/smt model: workspace/model export: node min export: tests/export export: tests/model export: check/passes rest: node min rest: tests/stressmodel rest: tests/codegen rest: tests/diagnostics migrate: node min migrate: tests/migrate
25909c6 #1300 11874 success 45 1315 – 714 178 17 763 899 – 30 1015 1768 837 32 157 820 237 36 2108
94ea870 #1301 11897 success 43 1217 – 685 214 14 633 731 – 30 1026 1756 845 32 162 854 233 35 2059
12156af #1303 11898 success 43 1240 – 691 191 16 751 883 – 28 920 1627 748 34 178 971 283 34 2000
e3ec3c2 #1304 11921 success 44 1254 – 735 193 17 788 960 – 29 981 1686 813 34 168 911 261 34 1981
43a831e #1306 11925 failed 43 1240 – 703 190 16 738 790 – 30 949 1739 787 35 190 992 289 35 2056
4317c4a #1316 11943 failed 46 1339 – 725 207 19 895 984 – 29 907 1676 759 35 166 907 262 37 2204
1195153 #1305 11957 failed 46 1378 – 747 213 16 714 884 – 32 1051 1872 887 36 184 970 275 38 2209
9c04259 #1302 11960 success 45 1297 – 753 202 19 911 1010 – 32 1048 1877 826 36 170 918 271 35 2078
47a75de #1315 11981 success 45 1247 – 744 201 21 1017 1037 – 29 983 1713 788 34 179 944 291 34 2015
2bb7df3 #1317 11985 success 47 1335 – 795 197 16 797 725 – 29 966 1699 778 33 163 877 240 36 2142
c5a59e1 #1318 11997 success 48 1415 – 812 198 18 879 919 – 26 912 1496 730 33 175 890 247 36 2103
f9f6d95 #1323 12007 success 45 1297 – 689 190 20 977 1031 – 32 1043 1854 823 31 181 897 282 36 2094
2de4131 #1325 12021 success 44 1250 – 710 183 18 900 916 – 29 1019 1669 823 32 171 882 244 38 2197
a270e66 #1313 12033 success 43 1168 – 754 189 19 923 1019 – 29 1048 1711 860 34 173 909 240 37 2149
4667955 #1326 12053 success 43 1178 – 748 189 22 1125 1138 – 30 1067 1735 884 32 176 922 270 33 1932
50885da #1329 12065 success 47 1351 – 763 204 22 1093 1164 – 28 1008 1648 822 35 179 939 266 41 2383
fce39e1 #1334 12066 success 47 1368 – 746 203 18 861 891 – 29 995 1685 806 34 183 912 274 35 2055
f53120a #1335 12078 success 43 1266 – 694 180 19 936 951 – 30 991 1752 778 36 186 1008 307 38 2250
b60a18c #1330 12092 success 44 1322 – 666 163 19 897 953 – 25 899 1478 720 32 166 877 265 37 2192
d8b60ed #1319 12113 failed 37 1803† timeout – – – 19 855 835 – 37 998 2189 903 35 513 941 263 39 2292
5b0c985 #1350 12116 success 46 1326 – 769 198 17 798 866 – 31 984 1786 852 36 194 941 281 35 2062
2ce843b #1347 12128 success 47 1426 – 738 178 19 940 990 – 32 1025 1852 881 36 213 1007 282 36 2125
1e51a00 #1348 12138 success 44 1294 – 700 183 17 838 880 – 27 945 1582 814 33 200 916 249 34 2015
c2a6582 #1351 12153 success 45 1357 – 724 182 20 978 1025 – 30 1032 1764 890 35 211 927 280 37 2211
e5bdb8f #1352 12164 success 44 1341 – 679 179 20 995 965 – 30 1058 1778 908 34 200 917 254 35 2095
a662c2b #1353 12174 success 45 1359 – 694 177 18 880 910 – 29 1003 1702 874 34 181 875 243 36 2108

CircleCI Go coverage shards (large, parallelism 5, -coverpkg=./...): per-node wall minutes and per-package seconds

develop commit PR CircleCI build job runtime: node min runtime: exec/runtime runtime: tests/wasm model: node min model: tests/corpus model: exec/smt model: workspace/model export: node min export: tests/export export: tests/model export: check/passes rest: node min rest: cmd/sysml rest: tests/codegen rest: tests/stressmodel rest: frontend/grpc migrate: node min migrate: tests/migrate
25909c6 #1300 11875 success 23 1102 702 14 666 814 461 15 461 870 440 28 1553 704 95 435 21 1199
94ea870 #1301 11886 success 22 1066 668 13 684 728 479 15 468 860 448 24 1353 632 92 376 20 1148
12156af #1303 11899 success 24 1155 752 14 722 823 522 15 482 871 456 27 1489 710 105 391 21 1217
e3ec3c2 #1304 11910 success 24 1173 736 14 724 816 520 16 493 900 478 28 1548 733 106 410 20 1171
43a831e #1306 11922 failed 24 1152 757 13 634 727 445 16 504 911 486 25 1428 693 98 375 21 1200
4317c4a #1316 11935 failed 24 1160 738 17 759 954 565 15 469 870 450 28 1564 733 97 407 21 1201
1195153 #1305 11947 failed 23 1135 715 13 681 766 516 16 505 921 491 27 1527 727 100 395 21 1203
9c04259 #1302 11959 success 26 1299 836 15 887 835 661 16 515 917 479 27 1517 713 97 429 22 1268
47a75de #1315 11970 success 23 1140 719 18 897 1031 667 16 517 910 491 29 1600 733 99 441 21 1217
2bb7df3 #1317 11983 success 24 1172 743 16 912 870 696 15 515 896 484 26 1474 701 102 394 22 1281
c5a59e1 #1318 11995 success 23 1147 734 15 860 901 641 16 524 909 495 26 1489 715 100 411 22 1262
f9f6d95 #1323 12008 success 26 1259 809 17 955 995 710 15 513 895 480 27 1500 713 102 414 20 1197
2de4131 #1325 12020 success 23 1139 761 15 879 895 640 16 522 911 494 26 1465 690 95 398 19 1133
a270e66 #1313 12030 success 24 1178 784 16 924 906 710 16 528 918 495 26 1495 719 100 412 21 1234
4667955 #1326 12043 success 25 1209 783 13 771 640 534 15 493 853 461 28 1569 733 96 417 21 1185
50885da #1329 12056 success 24 1185 786 15 864 840 623 15 510 891 477 32 1773 857 120 457 21 1224
fce39e1 #1334 12067 success 20 1018 619 18 1011 1019 819 16 517 898 488 25 1398 674 94 372 20 1195
f53120a #1335 12081 success 23 1139 723 15 888 894 655 16 510 902 474 28 1579 731 103 434 22 1270
b60a18c #1330 12093 success 24 1191 779 16 925 889 692 17 564 959 531 27 1524 708 100 416 21 1220
d8b60ed #1319 12104 failed 32 1671 687 15 866 753 738 24 566 1373 604 31 1800† timeout 946 498 447 20 1190
5b0c985 #1350 12114 success 25 1216 729 15 866 883 621 16 500 920 507 27 1523 713 116 406 21 1240
2ce843b #1347 12129 success 23 1154 670 15 868 796 640 16 510 937 518 27 1535 714 116 418 21 1206
1e51a00 #1348 12141 success 24 1219 722 17 912 964 665 15 464 860 468 27 1511 709 117 416 20 1158
c2a6582 #1351 12151 success 24 1188 727 16 862 912 632 15 445 860 452 27 1535 715 112 416 19 1116
e5bdb8f #1352 12165 success 23 1110 669 16 849 902 620 16 517 948 525 27 1550 739 118 415 21 1231
a662c2b #1353 12175 success 23 1144 693 15 878 876 614 17 537 958 544 28 1571 740 121 428 21 1224

GitHub pr.yml full-tier shards on each merged PR's head: Run Go tests step minutes

develop commit PR PR head pr.yml run runtime (min) runtime-corpus (min) model (min) export (min) rest (min) migrate (min)
25909c6 #1300 815012a success 31 33 18 30 39 15
94ea870 #1301 8205685 success 32 23 18 31 26 22
12156af #1303 3c0adf6 success 21 33 16 26 40 30
e3ec3c2 #1304 a7438e3 success 23 22 18 18 34 15
43a831e #1306 c1a8757 success 31 38 14 30 37 21
4317c4a #1316 4a72836 success – – – – – –
1195153 #1305 b021628 success 31 36 12 19 36 16
9c04259 #1302 3265d5d success 25 21 13 31 41 27
47a75de #1315 c4c6d9d success 33 37 20 29 29 27
2bb7df3 #1317 1676bc1 success – – – – – –
c5a59e1 #1318 7bdb1f1 success – – – – – –
f9f6d95 #1323 04d73e5 success 33 34 20 27 28 30
2de4131 #1325 e5f3a4c success 21 36 15 22 31 30
a270e66 #1313 5e16a37 success 32 34 20 17 41 22
4667955 #1326 5d47474 success 31 37 16 31 42 30
50885da #1329 f7841c9 success 32 35 21 29 42 30
fce39e1 #1334 cac9ca3 success 21 22 16 22 27 30
f53120a #1335 fd355a9 success – – – – – –
b60a18c #1330 dd25007 success – – – – – –
d8b60ed #1319 3156524 success 40 36 21 39 34 30
5b0c985 #1350 1ceae96 success 32 33 16 19 25 30
2ce843b #1347 2cff947 success 41 35 22 39 28 27
1e51a00 #1348 22a1a46 success 24 36 17 22 33 30
c2a6582 #1351 c291109 success 36 35 21 38 45 31
e5bdb8f #1352 106814f success 31 34 18 39 44 28
a662c2b #1353 606dac8 success 24 37 21 38 35 29

Step 2 — before/after, same machine (8 vCPU, go1.25.11), go test -count=1

target b60a18c (before #1319) a662c2b (develop) this PR
cmd/sysml -run TestExploreTablesTheSpacecraftRaceWithinItsBudget (package ok seconds) 135.5 s 145.7 s (+7.5 %) 124.2 s (−8.3 %)
internal/exec/runtime -run TestExploreWithIsExploreOverTheConformanceCorpus (wall) 40.4 s 41.8 s (+3.5 %) 36.6 s (−9.4 %)
tests/stressmodel (package ok seconds) 15.3 s 19.3 s (+26 %) 16.3 s (+6.5 %)

go tool pprof -top -cum -diff_base develop vs b60a18c on tests/stressmodel: ImplicitGenerals +4.21 s, computeImplicitGenerals +4.04 s, implicitUsageBaseFeature +2.24 s, DirectSupertypes +1.86 s, candidateBaseFeatures +1.64 s, declaredGeneralizationReaches +1.50 s, implicitBaseFeatures +1.34 s, declaredChainReachesKindBase +0.62 s; implicitBases −1.30 s (already memoized by #1350). On the spacecraft exploration and the conformance-corpus exploration the develop-vs-before diff is GC noise plus ≤0.5 s in TimeEventConforms; the perf PRs already merged recovered those.

Step 3 — CI configuration

With (a) removed, one package was still living on the budget's edge before #1319 for reasons (b)/(c): cmd/sysml under -coverpkg=./... -covermode=atomic took 1353–1773 s of the 1800 s budget on every one of the 19 green coverage runs before #1319 (1773 s at #1329, 27 s of headroom) and still 1511–1571 s on the six after it with the perf PRs in, while sharing the rest node's 4 vCPUs with ~2200 s of other packages (tests/codegen 709 s, internal/frontend/grpc 416 s, internal/frontend/repl 285 s, tests/diagnostics 174 s, internal/workspace/libs 141 s, cmd/sysml-grpc 127 s, …). The rest shard is also the longest GitHub pr.yml shard (25–45 min). So cmd/sysml becomes a shard of its own, cli:

  • scripts/race-shard.sh: cli → cmd/sysml, which rest no longer contains.
  • .circleci/config.yml: both jobs parallelism: 6, shard list runtime model export rest migrate cli; the node-count guard stays.
  • Makefile: COVERAGE_SHARDS gains cli; test-shard, test-quick and coverage-shard accept it.
  • .github/workflows/pr.yml: full tier gains the cli shard, the quick tier runs it with rest migrate, and the merge step expects 7 profiles.

No -timeout changed, nothing is skipped or marked short, and the node-count guards keep the three shard lists in step. CircleCI's build-test workflow is filtered to main/develop, so the cli node first runs on the develop pipeline after this merges; on this PR the same shard layout is exercised by the GitHub full tier (Go race tests (cli), Go race coverage merging 7 profiles).

How it was verified

  • TestKindBaseIsLookedUpOncePerSymbol (internal/semantic/semantics): the memoized lookups answer as the first query did and a repeated implicitUsageBaseFeature, implicitBaseUsage, declaredChainReachesKindBase and ImplicitGenerals query allocates nothing.
  • go build ./..., go vet ./..., gofmt -l . empty; go test -count=1 ./internal/semantic/semantics ./tests/stressmodel ./internal/exec/runtime ./cmd/sysml; full go test -count=1 ./... (race-free) locally.
  • scripts/race-shard.sh cli prints cmd/sysml, scripts/race-shard.sh rest no longer does; both workflow files parse; make -n coverage-shard SHARD=cli.
  • The before/after timings above, same machine, same session, go test -count=1.
  • GitHub full tier on 6218508, all green: Go race tests runtime 34 min, runtime-corpus 34, model 23, export 31, rest 30 (without cmd/sysml), migrate 17, cli 9; Go race coverage merged the 7 profiles. Devin Review: no findings.

Checklist

  • make test and make lint pass locally (race-free go test ./...; make lint unchanged surface)
  • Tests added or updated for the change
  • Documentation extended where it already covers the surface (see CONTRIBUTING.md) — the shard lists live in the Makefile help and scripts/race-shard.sh usage, both updated
  • Changelog entry added as changes/unreleased/<slug>.<section>.md, not as an edit to CHANGELOG.md
  • baselines regenerated and make docs-counts run if a gate count moved (compliance rows need nothing: the census is counted at docs build)
  • No internal work-item labels (waves, slices, F4, K5) in the body, docs, or changelog

Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

devin-ai-integration Bot and others added 2 commits October 21, 2026 14:33
Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration
devin-ai-integration Bot marked this pull request as ready for review October 21, 2026 15:31

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@HuiJun
HuiJun merged commit c5eb6cc into develop Oct 21, 2026
51 checks passed
@HuiJun
HuiJun deleted the perf/ci-runtime-budget branch October 21, 2026 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant