Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
[![CI](https://img.shields.io/github/actions/workflow/status/DeusData/codebase-memory-mcp/release.yml?branch=main&label=main%20CI)](https://github.com/DeusData/codebase-memory-mcp/actions/workflows/release.yml)
[![Native and frontend tests](https://img.shields.io/badge/tests-native_%2B_frontend-blue)](#tests)
[![MCP tools](https://img.shields.io/badge/MCP_tools-17-blue)](#mcp-tools)
[![Languages](https://img.shields.io/badge/languages-162-orange)](https://github.com/DeusData/codebase-memory-mcp)
[![Languages](https://img.shields.io/badge/languages-163-orange)](https://github.com/DeusData/codebase-memory-mcp)
[![Hybrid LSP](https://img.shields.io/badge/Hybrid_LSP-type_resolution-blue)](#hybrid-lsp)
[![Agents](https://img.shields.io/badge/agent_surfaces-45-purple)](https://github.com/DeusData/codebase-memory-mcp)
[![Pure C](https://img.shields.io/badge/pure_C-no_language_runtime-blue)](https://github.com/DeusData/codebase-memory-mcp)
Expand Down Expand Up @@ -394,7 +394,7 @@ In this repository, test narrowing is wired to the Linux x86_64 test leg when `T
- `SEMANTICALLY_RELATED` (vocabulary-mismatch, same-language, score ≥ 0.80)

### Indexing pipeline
- **162 vendored tree-sitter grammars** compiled into the binary
- **163 vendored tree-sitter grammars** compiled into the binary
- **Generic package / module resolution**: bare specifiers like `@myorg/pkg`, `github.com/foo/bar`, `use my_crate::foo` resolved via manifest scanning (`package.json`, `go.mod`, `Cargo.toml`, `pyproject.toml`, `composer.json`, `pubspec.yaml`, `pom.xml`, `build.gradle`, `mix.exs`, `*.gemspec`)
- **Infrastructure-as-code indexing**: Dockerfiles (`Dockerfile.*`, `*.Dockerfile`, `Containerfile`: bases, what the container runs, `ENV`), container images (`Image`: the normalised repository, tag and digest on the edges; a repository built from several Dockerfiles, one per component tag, gets an `Image` per component), docker compose services (`Resource`), Kubernetes manifests (`Resource`, with `group`, `version`, `kind`, `namespace`, `labels` as `k=v,k2=v2`), Kustomize overlays (`IMPORTS` to their entries and patch files, `OVERRIDE` from their patches), Helm charts (`Chart`, `DEPENDS_ON`), Terraform / OpenTofu (`.tf`, `.tofu`, `.tfvars`) as graph nodes. A Service walks to the code it runs: Service → workload → `Image` → Dockerfile → `main`
- **[Hybrid LSP semantic type resolution](#hybrid-lsp)** for Python, TypeScript / JavaScript / JSX / TSX, PHP, C#, Go, C, C++, Java, Kotlin, Rust, and Perl; a lightweight C implementation of language type-resolution algorithms, structurally inspired by and compatible with major language servers including tsserver / typescript-go, pyright, gopls, Roslyn, Eclipse JDT, and rust-analyzer (parameter binding, return-type inference, generic substitution, JSX component dispatch, JSDoc inference for plain JS files, namespace + trait + late-static-binding resolution for PHP, file-scoped namespaces + records + LINQ method syntax for C#, class-hierarchy + overload + lambda resolution for Java, extension-function + scope-function resolution for Kotlin, trait-method + UFCS resolution for Rust)
Expand Down Expand Up @@ -1066,14 +1066,14 @@ codebase-memory-mcp ships a **lightweight C implementation of language type-reso

**Two-layer architecture:**

1. **Tree-sitter pass**: fast, syntactic, runs for every one of the 162 languages. Extracts definitions, calls, imports.
1. **Tree-sitter pass**: fast, syntactic, runs for every one of the 163 languages. Extracts definitions, calls, imports.
2. **Hybrid LSP pass**: type-aware, runs above the tree-sitter pass per-language. Refines call edges using the import graph plus a per-file or pre-built cross-file definition registry. Languages without a Hybrid LSP pass yet fall back to textual resolution, with coverage varying by language and construct.

The result is a knowledge graph accurate enough to drive `trace_path` across packages, inheritance hierarchies, and stdlib calls; without paying for a language server process per project.

## Language Support

162 languages, counted by the vendored grammar directories in [the parser manifest](internal/cbm/vendored/grammars/MANIFEST.md). Benchmarked against 64 real open-source repositories (78 to 49K nodes):
163 languages, counted by the vendored grammar directories in [the parser manifest](internal/cbm/vendored/grammars/MANIFEST.md). Benchmarked against 64 real open-source repositories (78 to 49K nodes):

| Tier | Score | Languages |
|------|-------|-----------|
Expand All @@ -1100,7 +1100,7 @@ src/
ui/ Local HTTP server + embedded CBM Lens frontend
foundation/ Platform abstractions (threads, filesystem, logging, memory)
graph-ui/ CodeAtlasWeb: React, Monaco, and Three.js browser frontend
internal/cbm/ Vendored tree-sitter grammars (162 languages) + AST extraction engine
internal/cbm/ Vendored tree-sitter grammars (163 languages) + AST extraction engine
```

## Security
Expand Down
13 changes: 12 additions & 1 deletion THIRD_PARTY.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ The core runtime headers in `internal/cbm/vendored/common/tree_sitter/`

## Tree-sitter Grammars

160 pre-generated parsers are vendored in `internal/cbm/vendored/grammars/<lang>/`
163 pre-generated parsers are vendored in `internal/cbm/vendored/grammars/<lang>/`
(generated `parser.c` plus `scanner.c` where applicable, compiled statically).
Each grammar is the work of its upstream authors and each grammar directory
contains the upstream `LICENSE` file.
Expand Down Expand Up @@ -96,6 +96,17 @@ License summary:
`internal/cbm/vendored/grammars/MANIFEST.md`. PL/SQL support was originally
contributed in PR #1033 by Oğuz (@ouzsrcm).

### tree-sitter-iec61131-3-st

- **Project:** [HeytalePazguato/tree-sitter-iec61131-3-st](https://github.com/HeytalePazguato/tree-sitter-iec61131-3-st)
- **License:** MIT
- **Copyright:** (c) 2026 Jorge Centeno
- **Vendored at:** `internal/cbm/vendored/grammars/st/`
- **Pinned commit:** `00e24f50f8de`
- **Notes:** Community-maintained grammar for IEC 61131-3 Structured Text; not in
the nvim-treesitter or Helix registries (`community-niche` in the manifest).
Ships `parser.c` plus an external `scanner.c`. No local patches.

### tree-sitter-objectscript (UDL + routine)

- **Project:** [intersystems/tree-sitter-objectscript](https://github.com/intersystems/tree-sitter-objectscript)
Expand Down
1 change: 1 addition & 0 deletions internal/cbm/cbm.h
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,7 @@ typedef enum {
// operations are Routes (openapi.c)
CBM_LANG_DIAGRAM, // diagram files (PlantUML, DOT, D2, draw.io, Excalidraw), read
// without a grammar (diagram.c)
CBM_LANG_ST, // IEC 61131-3 Structured Text (.st)
CBM_LANG_COUNT
} CBMLanguage;

Expand Down
64 changes: 63 additions & 1 deletion internal/cbm/extract_defs.c
Original file line number Diff line number Diff line change
Expand Up @@ -4602,6 +4602,46 @@ static const char **extract_julia_base_classes(CBMArena *a, TSNode node, const c
return result;
}

/* IEC 61131-3 ST: `FUNCTION_BLOCK A EXTENDS B IMPLEMENTS I1, I2` and
* `INTERFACE I EXTENDS J, K` tag EVERY base name with a repeated `extends` /
* `implements` field (the separating ',' tokens carry the field too), so the
* first-match field probe of extract_base_classes would keep only one base.
* Collect every named child in either field — identifier or
* qualified_identifier. Whether an edge becomes IMPLEMENTS or INHERITS is
* decided later from the target's label. */
static const char **st_base_classes(CBMArena *a, TSNode node, const char *source) {
const char *pbases[MAX_BASES];
int pc = 0;
uint32_t nc = ts_node_child_count(node);
for (uint32_t i = 0; i < nc && pc < MAX_BASES_MINUS_1; i++) {
const char *fn = ts_node_field_name_for_child(node, i);
if (!fn || (strcmp(fn, "extends") != 0 && strcmp(fn, "implements") != 0)) {
continue;
}
TSNode bn_node = ts_node_child(node, i);
if (!ts_node_is_named(bn_node)) {
continue;
}
char *bn = cbm_node_text(a, bn_node, source);
if (bn && bn[0]) {
pbases[pc++] = bn;
}
}
if (pc == 0) {
return NULL;
}
const char **result =
(const char **)cbm_arena_alloc(a, (pc + NULL_TERM) * sizeof(const char *));
if (!result) {
return NULL;
}
for (int i = 0; i < pc; i++) {
result[i] = pbases[i];
}
result[pc] = NULL;
return result;
}

static const char **extract_base_classes(CBMArena *a, TSNode node, const char *source,
CBMLanguage lang) {
// ObjectScript: `Class X Extends (A, B)` — bases are class_name children of
Expand Down Expand Up @@ -4887,6 +4927,16 @@ static const char **extract_base_classes(CBMArena *a, TSNode node, const char *s
return find_base_from_children(a, node, source, base_types);
}

/* Base names of a class-like definition: ST collects its repeated fields in
* st_base_classes, every other language goes through extract_base_classes. */
static const char **class_base_names(CBMArena *a, TSNode node, const char *source,
CBMLanguage lang) {
if (lang == CBM_LANG_ST) {
return st_base_classes(a, node, source);
}
return extract_base_classes(a, node, source, lang);
}

// Classify class label from AST node kind
static const char *class_label_for_kind(const char *kind) {
if (strcmp(kind, "interface_declaration") == 0 || strcmp(kind, "interface_type") == 0 ||
Expand Down Expand Up @@ -8298,7 +8348,7 @@ static void emit_class_def(CBMExtractCtx *ctx, TSNode node, const CBMLangSpec *s
if (ctx->language == CBM_LANG_CSHARP && !ctx->enclosing_class_qn) {
def.decl_namespace = cs_type_decl_namespace(a, node, ctx->source);
}
def.base_classes = extract_base_classes(a, node, ctx->source, ctx->language);
def.base_classes = class_base_names(a, node, ctx->source, ctx->language);
def.decorators = extract_decorators(a, node, ctx->source, ctx->language, spec);
def.docstring = extract_docstring(ctx, node, name);

Expand Down Expand Up @@ -8509,6 +8559,18 @@ static void drop_c_typedefs_shadowed_by_tags(CBMExtractCtx *ctx, int first) {

// Find the body/members node inside a class node
static TSNode find_class_body(TSNode class_node, CBMLanguage lang) {
/* Structured Text must be answered BEFORE the generic field-name probe
* below. A FUNCTION_BLOCK / PROGRAM / INTERFACE keeps its METHOD, PROPERTY
* and VAR members as DIRECT children, but the ST grammar also tags a
* trailing body statement with the field name "body". The probe would grab
* that single statement and return it as if it were the member container,
* so every member of a block that ends in a statement would be lost.
* Iterate the declaration itself, as ObjC/Squirrel/Smali do below - those
* simply never expose a "body" field, which is why answering them after
* the probe is harmless. */
if (lang == CBM_LANG_ST) {
return class_node;
}
// Try field names first
static const char *body_fields[] = {"body", "members", "class_body", "declaration_list", NULL};
for (const char **f = body_fields; *f; f++) {
Expand Down
24 changes: 24 additions & 0 deletions internal/cbm/extract_unified.c
Original file line number Diff line number Diff line change
Expand Up @@ -2639,6 +2639,29 @@ static void push_boundary_scopes(CBMExtractCtx *ctx, TSNode node, const CBMLangS
}
}

/* ST: the body of a FUNCTION_BLOCK or PROGRAM is the code that runs when the
* block is called, so its calls and reads source to the block, not to the
* Module. Runs after push_boundary_scopes: the block's class frame at this
* depth becomes the enclosing routine until that frame pops (and restores
* the Module). Its METHODs push their own function scope. */
static void st_enter_block_body(const CBMExtractCtx *ctx, TSNode node, WalkState *state,
uint32_t depth) {
if (ctx->language != CBM_LANG_ST) {
return;
}
const char *kind = ts_node_type(node);
if (strcmp(kind, "function_block_declaration") != 0 &&
strcmp(kind, "program_declaration") != 0) {
return;
}
for (int i = state->scope_top - SKIP_ONE; i >= 0 && state->scopes[i].depth == depth; i--) {
if (state->scopes[i].kind == SCOPE_CLASS && state->scopes[i].qn) {
state->enclosing_func_qn = state->scopes[i].qn;
return;
}
}
}

void cbm_extract_unified(CBMExtractCtx *ctx) {
const CBMLangSpec *spec = cbm_lang_spec(ctx->language);
if (!spec) {
Expand Down Expand Up @@ -2740,6 +2763,7 @@ void cbm_extract_unified(CBMExtractCtx *ctx) {
scan_infra_bindings(ctx, node);

push_boundary_scopes(ctx, node, spec, &state, depth, &invocation);
st_enter_block_body(ctx, node, &state, depth);
}

/* Lexer-terminal trivia has no semantic work and no descendants. Avoid
Expand Down
36 changes: 29 additions & 7 deletions internal/cbm/extract_usages.c
Original file line number Diff line number Diff line change
Expand Up @@ -458,6 +458,7 @@ static const char *const meson_write_nodes[] = {"operatorunit", NULL};
static const char *const gn_write_nodes[] = {"assignment_statement", NULL};
static const char *const objectscript_binding_nodes[] = {"argument", "tag_parameter", NULL};
static const char *const objectscript_write_nodes[] = {"set_argument", NULL};
static const char *const st_binding_nodes[] = {"variable_declaration", NULL};

/* Parallel to lang_specs: occurrence semantics evolve without adding fields to
* the positional CBMLangSpec initializer used by every language. */
Expand Down Expand Up @@ -506,6 +507,13 @@ static const CBMOccurrenceSpec occurrence_specs[CBM_LANG_COUNT] = {
CBM_OCCURRENCE_STANDARD, true},
[CBM_LANG_OBJECTSCRIPT_ROUTINE] = {objectscript_binding_nodes, objectscript_write_nodes,
CBM_OCCURRENCE_STANDARD, true},
/* IEC 61131-3 ST: every VAR/VAR_INPUT/VAR_OUTPUT/VAR_IN_OUT/VAR_TEMP entry is a
* variable_declaration whose names sit in the plural `names` field, which
* binding_fields does not list — so without this row a declared name was
* emitted as a usage. All VAR sections precede the body and are visible for
* the whole POU, so parameter-like whole-scope binding is the ST semantics.
* The `type` field and (see is_binding_occurrence) `initial_value` stay reads. */
[CBM_LANG_ST] = {st_binding_nodes, NULL, CBM_OCCURRENCE_STANDARD, false},
};

static bool text_equals(CBMExtractCtx *ctx, TSNode node, const char *expected) {
Expand Down Expand Up @@ -1137,6 +1145,26 @@ static bool elixir_binary_operator_binds(CBMExtractCtx *ctx, TSNode node) {
text_equals(ctx, operator_node, "->") || text_equals(ctx, operator_node, "\\\\");
}

/* A declaration field whose symbols are reads, not the declared name.
* A declaration container binds its name/pattern, not the symbols used by its
* type annotation. Both TypeScript (`cfg: Config`) and Go (`cfg Config`)
* expose that annotation through the exact `type` field; stop before the
* whole-parameter binding rule can swallow `Config`. The emitted occurrence
* remains an ordinary USAGE, never a callable reference merely because the
* target happens to be a type. ST names a declaration's initializer
* `initial_value` ("f : INT := g;"): a read like any value field, but
* is_value_field stays generic, since Pine exposes a field of the same name
* with its own occurrence policy. */
static bool is_non_binding_decl_field(const CBMExtractCtx *ctx, const char *field) {
if (!field) {
return false;
}
if (strcmp(field, "type") == 0) {
return true;
}
return ctx->language == CBM_LANG_ST && strcmp(field, "initial_value") == 0;
}

static bool is_binding_occurrence(CBMExtractCtx *ctx, TSNode node, const CBMLangSpec *spec,
WalkState *state) {
const CBMOccurrenceSpec *occurrence = &occurrence_specs[ctx->language];
Expand All @@ -1158,13 +1186,7 @@ static bool is_binding_occurrence(CBMExtractCtx *ctx, TSNode node, const CBMLang
if (is_value_field(field)) {
return false;
}
/* A declaration container binds its name/pattern, not the symbols used
* by its type annotation. Both TypeScript (`cfg: Config`) and Go
* (`cfg Config`) expose that annotation through the exact `type` field;
* stop before the whole-parameter binding rule can swallow `Config`.
* The emitted occurrence remains an ordinary USAGE, never a callable
* reference merely because the target happens to be a type. */
if (field && strcmp(field, "type") == 0) {
if (is_non_binding_decl_field(ctx, field)) {
return false;
}

Expand Down
4 changes: 4 additions & 0 deletions internal/cbm/grammar_st.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
// Vendored tree-sitter grammar: st
// Each grammar compiled as separate unit (conflicting static symbols).
#include "vendored/grammars/st/parser.c"
#include "vendored/grammars/st/scanner.c"
39 changes: 39 additions & 0 deletions internal/cbm/lang_specs.c
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,39 @@ extern const TSLanguage *tree_sitter_plsql(void);
// -- Empty sentinel --
static const char *empty_types[] = {NULL};

// ==================== STRUCTURED TEXT (IEC 61131-3) ====================
// Every node name below was taken from the grammar's node-types.json at the
// pinned commit, not guessed.
extern const TSLanguage *tree_sitter_iec61131_3_st(void);
// An INTERFACE member is a *_signature, not a *_declaration - verified by
// parsing: an INTERFACE contains method_signature/property_signature and no
// method_declaration/property_declaration at all.
// FUNCTION_BLOCK and PROGRAM are class-like: they OWN methods and properties.
// They must not be in function_node_types: the def-walker would match the
// block, emit one node spanning all of it and stop descending (it descends
// into a matched function only for an allowlist of languages that does not
// include ST), so the block's own methods and properties would be lost, while
// an INTERFACE's members still arrive via the class-body path.
// Do NOT list them in both arrays: the function branch runs first and wins.
static const char *st_func_types[] = {"method_declaration", "method_signature",
"function_declaration", NULL};
// "TYPE T_X : INT; END_TYPE" parses as a NAMELESS type_declaration wrapping a
// type_definition that carries name='T_X'. Selecting the wrapper loses the name.
static const char *st_class_types[] = {"function_block_declaration", "program_declaration",
"interface_declaration", "type_definition", NULL};
static const char *st_field_types[] = {"property_declaration", "property_signature", NULL};
static const char *st_module_types[] = {"source_file", NULL};
static const char *st_call_types[] = {"call_expression", "invocation_statement", NULL};
// The dependency is "USING N;" (using_directive). A namespace_declaration is the
// file DECLARING a namespace, which is the opposite of an import - registering it
// produced an import edge pointing at the file's own namespace.
static const char *st_import_types[] = {"using_directive", NULL};
static const char *st_branch_types[] = {"if_statement", "case_statement", "for_statement",
"while_statement", "repeat_statement", NULL};
static const char *st_var_types[] = {"variable_declaration", NULL};
static const char *st_assign_types[] = {"assignment_statement", "reference_assignment_statement",
NULL};

// ==================== GO ====================
static const char *go_func_types[] = {"function_declaration", "method_declaration", "method_elem",
"func_literal", NULL};
Expand Down Expand Up @@ -2189,6 +2222,12 @@ static const CBMLangSpec lang_specs[CBM_LANG_COUNT] = {
empty_types, chialisp_var_types, empty_types, empty_types, NULL,
empty_types, NULL, NULL, tree_sitter_chialisp, NULL},

// CBM_LANG_ST
[CBM_LANG_ST] = {CBM_LANG_ST, st_func_types, st_class_types, st_field_types, st_module_types,
st_call_types, st_import_types, empty_types, st_branch_types, st_var_types,
st_assign_types, empty_types, NULL, empty_types, NULL, NULL,
tree_sitter_iec61131_3_st, NULL},

// CBM_LANG_FENNEL
[CBM_LANG_FENNEL] = {CBM_LANG_FENNEL, fennel_func_types, empty_types, empty_types,
fennel_module_types, fennel_call_types, empty_types, empty_types,
Expand Down
Loading
Loading