[SarletteD]

Refs #1810. Based on main (the prerequisite #2396 is merged). This is the first of four Structured Text
PRs (1a language, 1b dialect normalizer, 1c export layout, 1d real-world dialect).

What does this PR do?

Adds IEC 61131-3 Structured Text as CBM_LANG_ST on .st, with the vendored grammar
HeytalePazguato/tree-sitter-iec61131-3-st
@ 00e24f50f8de (MIT, (c) 2026 Jorge Centeno), a content gate for the shared .st extension, and the
ST-specific extraction the graph needs:

• language.c / discover.c: .st is also used by Smalltalk (Pharo/Tonel, GNU Smalltalk) and
  StringTemplate, so like .m / .cls / .frm / .res it is routed by content. cbm_disambiguate_st
  reads the first 16 KB, skips a UTF-8 BOM, IEC comments ((* *), //, C-style block comments) and
  {...} pragmas, and accepts the file only when its first token is an IEC declaration keyword
  (PROGRAM, FUNCTION_BLOCK, FUNCTION, INTERFACE, CLASS, TYPE, NAMESPACE, CONFIGURATION,
  VAR_GLOBAL, USING, or an ABSTRACT/FINAL/INTERNAL modifier) followed by a name. Anything else,
  and a read failure, stays unindexed.
• lang_specs.c: node types for functions/methods (method_signature for INTERFACE members), the
  class-like POUs (FUNCTION_BLOCK, PROGRAM, INTERFACE, type_definition), properties, calls
  (call_expression, invocation_statement), USING imports, branches, variables, assignments.
• extract_defs.c: every base of EXTENDS/IMPLEMENTS is collected by st_base_classes (the grammar
  repeats the field per base, the generic probe kept one); a small class_base_names dispatches ST there
  and leaves extract_base_classes unchanged; find_class_body answers ST before the generic body probe,
  which would otherwise take a trailing body statement for the member container.
• extract_usages.c: declared VAR names bind (plural names field, named st_binding_nodes row), a
  declaration's initial_value stays a read; that check and the existing type-field check now share
  one helper, is_non_binding_decl_field.
• extract_unified.c: the body of a FUNCTION_BLOCK or PROGRAM is the code that runs when the block is
  called, so its calls and reads source to the block (its Class node) instead of the file's Module node;
  METHODs keep their own scope (st_enter_block_body, called after push_boundary_scopes).

Credit: @ysfsmet for the ST groundwork and the discussion in #1810; the extraction commit carries his
Co-authored-by trailer.

In scope / out of scope

• In: the grammar, language registration and content gate, count surfaces, definition/base/usage
  extraction and the body-call source node, tests.
• Out (following PRs): vendor dialect normalization (1b), the export layout with members after
  END_FUNCTION_BLOCK (1c), the remaining real-world dialect (1d). Without them most real exports parse
  only partially (see the table: 86 % of the proof project's files at this stage).

Vendoring checklist

• internal/cbm/vendored/grammars/st/LICENSE (MIT, upstream text)
• THIRD_PARTY.md entry
• SBOM: generated dynamically from the vendored tree, nothing to edit
• internal/cbm/vendored/grammars/MANIFEST.md: paragraph + table row + recounted summary/ABI lines;
  the paragraph says "Vendored unmodified." (the normalizer sentence arrives with 1b)
• scripts/vendored-checksums.txt regenerated; make -f Makefile.cbm security passes
• Count surfaces 162 -> 163: README, docs/index.html, docs/llms.txt, pkg/npm/README.md, chocolatey
  nuspec, scripts/package-release.sh, server.json. Four places that still read 158 on main
  (README.md, docs/index.html twice, docs/llms.txt) are corrected too; the language-count contract
  regex does not cover them.
• Same wiring as the Chialisp precedent: LANG_NAME_TABLE in userconfig.c, a
  CBM_GRAMMAR_CASES fixture, a LABEL_GOLDENS row, lang_ext_st, call-node manifest and language
  registry repro rows (the manifest assert message now states the real count).

Commits

1. feat(lang): add IEC 61131-3 Structured Text (.st) (e084b1c)
2. feat(st): extract definitions, bases and usages from .st (91b215a)

Size

Production +284 / -8 (language.c +108, discover.c/.h +15, lang_specs.c +39, other registration
+6, extract_defs.c +63 / -1, extract_usages.c +29 / -7, extract_unified.c +24), tests +435 / -16,
docs and metadata +64 / -30 (including MANIFEST.md and vendored-checksums.txt), vendored grammar
+97,336 (generated parser.c and friends, excluded from review size). Per commit: language +168 prod /
+189 -11 test; extraction +116 / -8 prod, +246 / -5 test.

Checks

• Focused suites (extraction grammar_regression grammar_labels language lang_contract discover): all
  pass except the known environment failure test_extraction.c: sp is NULL (WSL /tmp is a 7.8 GB tmpfs;
  main fails identically): 730 PASS lines on the first commit alone, 737 on the second
• scripts/lint.sh --ci (clang-format 20, cppcheck 2.20): rc 0
• lint-tidy-diff (clang-tidy 21.1.6, diff against main): one finding, in upstream code the change has
  to touch: is_binding_occurrence is over the cognitive-complexity threshold on main (33). This PR
  lowers it to 32 (the type-field check and the ST initial_value check now share
  is_non_binding_decl_field), but the diff-scoped run still reports the function because a changed
  line sits inside it. extract_base_classes, push_boundary_scopes and detect_file_language keep
  their main complexity (the new logic lives in small helpers next to them).
• make -f Makefile.cbm security: rc 0 (vendored checksums match)
• scripts/test.sh stops at its static step 0x on main as well: tests/test_version_metadata_contract.sh
  reports pkg/scoop/codebase-memory-mcp.json is pinned at 0.11.0, which is now the newest release
  (pre-existing, unrelated to this PR); the language-count contract (163) passes.

Proof on a real project

Stage	partial .st files	nodes	edges	index time, median (3 runs)
upstream main @ 80eb92a	.st not indexed	297	300	9.87 s (10.39 / 9.81 / 9.87)
1a language (91b215a)	822 / 956 (86.0 %)	3430	8562	9.56 s (10.60 / 9.56 / 8.15)
1b normalizer (2215fc1)	263 / 956 (27.5 %)	3431	8415	7.84 s (8.01 / 7.84 / 7.53)
1c export layout (d6fd533)	156 / 956 (16.3 %)	5045	17536	7.92 s (7.76 / 7.92 / 8.41)
1d real-world (037be4d)	6 / 956 (0.6 %)	5108	17561	7.92 s (7.92 / 7.92 / 7.43)

Machine: Linux 6.18.33.2-microsoft-standard-WSL2 x86_64 (WSL2 on Windows 11), 13th Gen Intel Core i9-13950HX, 32 logical CPUs, 15 GiB RAM visible to WSL. Release CLI (make -f Makefile.cbm cbm, -O2), cli index_repository on ThorstenBrach/SRCI @ 67fd773 (956 .st files, all of which pass the content gate at every stage), fresh CBM_CACHE_DIR and CBM_RUNTIME_DIR per run, median of 3 wall-clock runs, nothing else running. Wall time includes process start and the non-ST files; it stays flat because the ST files are small (it is higher than in the first revision of this text for main too, so the machine, not the change, moved).

Edges by type, 1a -> 1b: USAGE 4923 -> 4774. 1b -> 1c: CALLS 104 -> 4007, DEFINES_METHOD 2 -> 1606, USAGE 4774 -> 5803, WRITES 0 -> 460, OVERRIDE 0 -> 401. 1c -> 1d: DEFINES 3740 -> 3803 (types that parse now), USAGE 5803 -> 5765 (struct field names no longer read as usages).

Changes against the first revision of this table (review round 1): the .st content gate admits all 956 files, so files, nodes and partial counts are unchanged at every stage. Edges move only because FUNCTION_BLOCK/PROGRAM body calls and reads now source to the block instead of the Module: at 1a +242 USAGE edges (4681 -> 4923; 242 targets in 119 files are now used both from the block and from the Module, the latter by METHODs exported after END_FUNCTION_BLOCK that 1a cannot attach yet; no new targets), at 1b +262. From 1c on the counts per type are unchanged except SEMANTICALLY_RELATED 123 -> 109 (derived from the moved sources; not investigated further); the sources move: at 1c CALLS Module 24 -> Class 24, USAGE Module 1483 -> Class 806 + Module 677, WRITES Module 8 -> Class 8. Wall times of this revision were measured on a busier machine than the first (main itself 7.67 s then, 9.87 s now) and are noisy; compare within the column.

Known limitations (this stage)

• The content gate reads the first 16 KB: an ST file whose header comment is longer than that stays
  unindexed. A non-ST .st file that happens to open with, say, Program Foo would still be taken as ST.
• Real vendor exports mostly parse partially until 1b-1d land (86 % partial on the proof project here).

Written with Claude Code (Anthropic) on behalf of @SarletteD, who reviewed and signs off every commit.

[github-actions]

Thanks for opening this — it has been seen, and it is queued.

This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence.

Current review status: working through a backlog. 0.9.1-rc.1 is out, so the release freeze that held reviews is over — but it left a large queue of open pull requests behind it, and we are reading through them oldest-first. The background is in discussion #1144.

What that means for this PR, concretely:

• It will not be closed for inactivity. No stale bot touches pull requests here.
• It may still sit a while before a human reads it. That is on us, not on you.
• Older PRs are read first, so a recent one is not being skipped — it is behind a queue.

Things that will genuinely speed it up whenever review does happen:

• Keep it rebased on main — the tree is moving quickly right now, and a conflicting branch cannot be reviewed as the diff you intended.
• Get CI green, or say which failures you believe are pre-existing.
• Keep the change to one claim. Bundled features and refactors get split before they get merged, which costs you a round trip.
• Every commit needs a sign-off (git commit -s) — CI enforces DCO.

If this fixes a bug, a reproduction we can run is worth more than a description of the symptom.

Thanks for contributing, and sorry in advance for the wait.

[DeusData]

Thank you, @SarletteD. This is a very carefully built first slice. A few things make a language foundation reviewable, and you did all of them:

• vendoring the grammar unmodified (every file is byte-identical to upstream 00e24f50f8de, which made provenance trivial to verify);
• keeping extract_base_classes byte-identical by routing ST through class_base_names;
• fixing the stale count strings the contract test can't see;
• backing it all with a public proof on ThorstenBrach/SRCI at a pinned commit.

The 1a->1d table also makes it clear what this slice does and doesn't claim.

A few things before it lands:

1. Gate .st by content. .st is shared with Pharo/Tonel (*.class.st), GNU Smalltalk and StringTemplate; even GitHub's own classifier labels SRCI as Smalltalk. Graph quality comes first here, so we'd like a small content check following cbm_disambiguate_m / cls / frm in src/discover/language.c. Route a .st file to ST only when its head carries an IEC POU keyword (PROGRAM, FUNCTION_BLOCK, FUNCTION, TYPE, INTERFACE, CLASS ... as you see fit), and otherwise leave it unindexed. That way a Pharo repository doesn't pick up a pile of partial parses. A test with a Smalltalk-shaped .st file asserting it is not indexed as ST would pin it.

2. Credit for @ysfsmet in the history. Thank you for crediting #1810 in the description. The extraction commit follows #1810's design closely in three places:

   • collecting every repeated extends/implements field;
   • answering find_class_body with the declaration node;
   • the variable_declaration binding row, with initial_value kept as a read.

   PR descriptions don't survive in git log, so could you add Co-authored-by: Yusuf S. Demirci <ysf-samet@outlook.com> to 4c7bc08? That keeps the credit with the code, the way we credited earlier groundwork in #2083.

3. MANIFEST.md:39 says the dialect "is normalized in internal/cbm/st_dialect.c", but that file arrives with 1b. The manifest is the ledger we audit vendoring against, so it should describe this tree. Could you move that sentence to 1b and regenerate vendored-checksums.txt?

4. Merge order: we'll take #2396 first, and this sits cleanly on top.

Optional:

• st_type_alias_keeps_its_name exercises a STRUCT, so a rename (or an extra TYPE T : INT; END_TYPE case) would match its name.
• A named st_binding_nodes array would read like the neighbouring rows.
• A question for the series: calls made directly in a FUNCTION_BLOCK/PROGRAM body seem to fall back to the file's Module node. If 1c changes that, great; otherwise a test pinning the intended source node would help.
• IEC identifiers are also case-insensitive while resolution is case-sensitive, which is worth keeping in mind for the resolution PRs.

Thank you again for picking this up so generously, and for splitting it so cleanly.

[SarletteD]

Thank you for the careful review, @DeusData. All four points are addressed; the branch is rebased on
current main (no longer carries #2396) and now has two commits:
e084b1cb (language) and 91b215a1 (extraction).

1. .st gated by content. cbm_disambiguate_st() in src/discover/language.c, called from
detect_file_language like the .m / .cls / .frm / .res ones. It reads the first 16 KB (the
.cfc bound; library sources often open with a long licence header), skips a UTF-8 BOM, IEC comments
((* *), //, and C-style block comments) and {...} pragmas, and routes the file to ST only when the
first token is an IEC declaration keyword (PROGRAM, FUNCTION_BLOCK, FUNCTION, INTERFACE, CLASS,
TYPE, NAMESPACE, CONFIGURATION, VAR_GLOBAL, USING, or an ABSTRACT/FINAL/INTERNAL
modifier; the grammar's top-level items plus CLASS) followed by a name. That last condition is what
keeps Tonel's Class { and StringTemplate's program(x) ::= out. Anything else, and a read failure,
leaves the file unindexed. Tests: Pharo/Tonel, GNU Smalltalk (bracket and message syntax) and
StringTemplate (template, group, raw text) heads are not ST; a 40-line // header plus a block comment
and a pragma before FUNCTION_BLOCK is ST; every declaration head, a BOM, lower case; prefix words
(PROGRAMS), an unterminated header comment and TYPE: are not; plus an end-to-end discovery test
(Foo.class.st unindexed, an ST file indexed). On SRCI all 956 .st files still pass the gate: files,
nodes and partial count are unchanged (956 files, 3430 nodes, 822 partial, as before).

2. Credit. 91b215a1 now carries
Co-authored-by: Yusuf S. Demirci <ysf-samet@outlook.com> next to the existing trailers.

3. MANIFEST.md. In this PR the st paragraph ends with "Vendored unmodified."; the
st_dialect.c sentence moved into the normalizer commit of the follow-up, and
scripts/vendored-checksums.txt is regenerated in each commit that touches the manifest.

4. Merge order. Thanks for taking #2396; this branch is rebased onto main after it.

Optional points

• st_type_alias_keeps_its_name is now st_struct_type_keeps_its_name, and a new
  st_type_alias_keeps_its_name covers TYPE T_Speed : INT; END_TYPE.
• The ST row uses a named st_binding_nodes array.
• Source node of a call in a FUNCTION_BLOCK/PROGRAM body: you were right, it was the file's Module
  node on this PR, and 1c does not change it (1c only moves members exported after
  END_FUNCTION_BLOCK back into their block). The body of a FUNCTION_BLOCK or PROGRAM is the code that
  runs when the block is called, so the intended source is the block itself. That was a small change in
  the extraction commit: while the walk is inside one of those two blocks, enclosing_func_qn is the
  block's QN (set by a small st_enter_block_body right after push_boundary_scopes pushed the class
  scope, and restored when that scope pops); a METHOD inside still pushes its own scope.
  st_body_call_sources_to_its_block pins it: a body call of an FB and of a PROGRAM source to the
  block, a call inside a METHOD to the method, and a VAR
  type read of the FB to the FB. It failed before the change ("t.Motor" != "t.Motor.FB_Motor"). Effect
  on SRCI: at this stage +242 USAGE edges (4681 -> 4923): 242 targets in 119 files are now used both from
  the block and from the Module, the latter by METHODs exported after END_FUNCTION_BLOCK that only 1c
  attaches; no new targets. From 1c on the edge counts per type stay the same and only the sources move
  (at 1c: CALLS Module 24 -> Class 24, USAGE Module 1483 -> Class 806 + Module 677, WRITES Module 8 ->
  Class 8), apart from SEMANTICALLY_RELATED 123 -> 109, which moves with them (I have not traced that
  pass in detail).
• Case-insensitive identifiers: agreed; this PR adds no ST-specific resolution. Our fork already
  resolves ST method names case-insensitively for typed calls (a call through a variable of a declared
  FB type), and the resolution PRs will carry that.

Checks (WSL2, same machine as the PR text): focused suites (extraction grammar_regression grammar_labels language lang_contract discover) pass on each commit except the known WSL sp is NULL
spill-store failure; scripts/lint.sh --ci rc 0; make security rc 0; lint-tidy-diff against main
reports only the disclosed is_binding_occurrence (33 on main, 32 here); the new logic sits in small
helpers so push_boundary_scopes and detect_file_language are untouched in complexity.
scripts/test.sh stops at step 0x on main too (pkg/scoop/codebase-memory-mcp.json is pinned at 0.11.0, which is now the newest release), unrelated to this PR.

The PR description is refreshed accordingly (proof table, MANIFEST line, the .st collision limitation
is resolved here).

Written with Claude Code (Anthropic) on behalf of @SarletteD.

[DeusData]

Thanks for addressing the Structured Text review points, preserving the groundwork credit, and updating the real-project table. I have read the follow-up; the revised first slice remains pending maintainer re-review. No additional scope is requested here.

[DeusData]

Thank you again for addressing the Structured Text review and preserving the groundwork credit. The revised first slice is still waiting on our re-review, without additional scope requested. Sorry for the delay—I’ll follow up soon.

[DeusData]

Thank you so much, @SarletteD, for the careful rebase and for moving the .st check into the shared language classifier yourself - that was exactly the port we had in mind, and the new probe and classify tests make it easy to trust. We re-checked the vendored grammar against upstream (all six files identical to the upstream main tip, MIT, LICENSE vendored) and the recounted MANIFEST and checksums, and everything lines up.

Main has moved since your last push, including changes in several files this PR touches, so we merged current main into your branch (one signed-off merge commit; it merged cleanly and your commits are untouched). The required checks will now run against today's tree. If you push to this branch again, please pull first so the merge commit is kept. We will come back here once CI is green.