Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Full release notes with details on each version: [GitHub Releases](https://githu

## 0.9.61 (unreleased)

- Fix: Elixir `alias`/`import`/`require`/`use` edges now land on the `defmodule` node they name instead of being pruned as dangling. Module definitions are keyed `_make_id(file_stem, name)` while references were keyed `_make_id(name)`, so the two could never match and build dropped every Elixir alias/import edge, leaving the graph almost entirely intra-file. The extractor now stamps the written module name on the edge and a corpus-wide resolver repoints it onto the single module with that exact name; ambiguous names and genuine externals (`Ecto.Query`, `Logger`) are left alone rather than guessed at, exactly as in the Kotlin fix (#2526) this mirrors. Resolution runs against the unchanged-corpus context too, so an alias into a file you did not touch still resolves on an incremental/`watch` rebuild, and a self-import (`__using__`/`quote`) is left alone so it cannot overwrite the file's `contains` edge. `_AST_CACHE_SCHEMA` is bumped to 3: the extractor's output changed, so a cache written by an earlier build would otherwise keep serving pre-fix results and make the fix silently inert (#3562).
- Fix: `graphify.serve` now imports cleanly on Python 3.12 and 3.13. The `chinese` extra pins `jieba-py` from 3.12 onward (0.9.60 mistakenly kept the old `jieba` until 3.14, and its invalid regex escapes are a hard error on 3.12+), and the jieba import now suppresses the tokenizer's `SyntaxWarning` regardless of message or line so it never escalates under `-W error`.
- Fix: the git hook's rebuild-root guard now rejects a symlink-loop or dangling `.graphify_root` on Python 3.13, whose `Path.resolve()` no longer raises on a loop — the saved root must resolve to a real directory inside the repo before it is adopted.

Expand Down
2 changes: 1 addition & 1 deletion graphify/cache.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
_EXTRACTOR_VERSION = "unknown"

# Bump when AST cache-key semantics change independently of the package version.
_AST_CACHE_SCHEMA = 2
_AST_CACHE_SCHEMA = 3

# Version dirs already swept this process — cleanup runs once per (base, version).
_cleaned_ast_dirs: set[str] = set()
Expand Down
113 changes: 105 additions & 8 deletions graphify/extract.py
Original file line number Diff line number Diff line change
Expand Up @@ -4530,6 +4530,87 @@ def _resolve_kotlin_import_targets(
e["target"] = candidates[0]


_ELIXIR_MODULE_LABEL_RE = re.compile(r"^[A-Z][A-Za-z0-9_]*(?:\.[A-Z][A-Za-z0-9_]*)*$")


def _resolve_elixir_import_targets(
per_file: list[dict],
all_nodes: list[dict],
all_edges: list[dict],
) -> None:
"""Rewrite Elixir ``imports`` edge targets from the bare module name to the
``defmodule`` node that name actually defines (#3562).

``extract_elixir`` mints module DEFINITIONS as ``_make_id(file_stem, name)``
(the corpus-wide invariant documented on ``_file_stem`` — same-named symbols
in different files must not collide) but emits
``file --imports--> _make_id(module_name)`` for every
``alias``/``import``/``require``/``use``, with the written name stamped as
``metadata.target_fqn``. The two ids can never be equal, so build's
``src not in node_set or tgt not in node_set`` prune dropped EVERY Elixir
alias/import edge — the same failure Kotlin had in #2526, and the reason
``extract_elixir`` deliberately exempts ``imports`` from its own dangling-edge
filter: it assumes a downstream pass will resolve them.

An Elixir alias names a BEAM module, not a path, so the per-file extractor
cannot resolve it — and guessing a file from ``Macro.underscore`` conventions
would break on umbrella apps, ``:elixirc_paths`` config, acronym modules and
multi-module files. Hence a corpus-wide pass here instead, matching the
written name against module labels EXACTLY (the label is the alias text as
written, so no normalisation is involved and confidence stays EXTRACTED). A
name defined exactly ONCE in the corpus is rewritten to that node id; zero or
several candidates leave the edge untouched, dangling like other languages'
external imports (``Ecto.Query``, ``Logger``, any dependency), so no stub node
is ever fabricated.

Must run BEFORE the shared call pass builds its import-evidence index, for the
same reason the Kotlin pass does — see the call site in ``extract()``.
"""
# label -> [module node ids]. Restricted to nodes that came from an Elixir
# source file and whose label has the shape of a module alias: extract_elixir
# emits exactly three node kinds, and neither file nodes (basename, e.g.
# `user.ex`) nor function nodes (`create()`) can match this pattern.
modules_by_name: dict[str, list[tuple[str, str]]] = {}
for n in all_nodes:
# `_elixir_module` is stamped by extract_elixir on TOP-LEVEL defmodule
# nodes only. Gating on it (rather than on a label regex) excludes
# nested modules — whose label is the written inner name, not their real
# BEAM name, so they would capture aliases meant for a different module —
# and also excludes any same-labelled node a semantic/LLM tier might add,
# which would otherwise make the name ambiguous and suppress resolution.
if not n.get("_elixir_module"):
continue
label = str(n.get("label") or "")
if not _ELIXIR_MODULE_LABEL_RE.match(label):
continue
modules_by_name.setdefault(label, []).append(
(n["id"], str(n.get("source_file") or ""))
)
if not modules_by_name:
return
for e in all_edges:
if e.get("relation") != "imports":
continue
if not str(e.get("source_file") or "").endswith((".ex", ".exs")):
continue
fqn = str((e.get("metadata") or {}).get("target_fqn") or "")
if not fqn:
continue
candidates = modules_by_name.get(fqn, [])
if len(candidates) != 1: # single-candidate guard: never fabricate
continue
target_id, target_src = candidates[0]
# Self-import: `defmodule MyApp.Context` whose own `__using__`/`quote`
# body does `import MyApp.Context`. Retargeting would land a second
# edge on the `file --contains--> module` pair; the graph is a plain
# (non-multi) Graph, so one relation would silently overwrite the
# other and destroy the `contains` edge. A self-import carries no
# cross-file information, so leave it dangling as before.
if target_src and target_src == str(e.get("source_file") or ""):
continue
e["target"] = target_id


def _resolve_csharp_qualified_calls(
per_file: list[dict],
all_nodes: list[dict],
Expand Down Expand Up @@ -4723,6 +4804,14 @@ def _resolve_kotlin_qualified_calls(
"kotlin_import_targets", frozenset({".kt", ".kts"}), _resolve_kotlin_import_targets
)

# Elixir import-target resolution (#3562) — same shape, same reason, same early
# slot as the Kotlin pass above: alias/import targets are name-derived and must be
# repointed onto the real `defmodule` node before import-evidence promotion reads
# the edges.
_ELIXIR_IMPORT_TARGET_RESOLVER = LanguageResolver(
"elixir_import_targets", frozenset({".ex", ".exs"}), _resolve_elixir_import_targets
)


# Register the cross-file, language-specific member-call resolvers into the shared
# registry (framework lives in graphify.resolver_registry). A new language plugs in
Expand Down Expand Up @@ -7276,15 +7365,23 @@ def _looks_like_bash(result: object) -> bool:
# them from the indirect_call guard below to avoid false edges (#2137).
class_nids = {n["id"] for n in resolution_nodes if n.get("_callable_class")}

# Kotlin import targets (#2526): rewrite each `imports` edge from the bare
# last-segment id to the node its written FQN names, via the per-file
# package declarations. Runs HERE — after the id-remap/disambiguation passes
# (ids are final) but before the import-evidence index just below reads the
# edges — so genuine imported calls get promoted INFERRED -> EXTRACTED. The
# tail registry run (run_language_resolvers below) would be too late.
# Kotlin (#2526) and Elixir (#3562) import targets: rewrite each `imports`
# edge from its name-derived id to the node its written FQN names. Runs HERE —
# after the id-remap/disambiguation passes (ids are final) but before the
# import-evidence index just below reads the edges — so genuine imported calls
# get promoted INFERRED -> EXTRACTED. The tail registry run
# (run_language_resolvers below) would be too late.
# `resolution_nodes`, NOT `all_nodes` (#2406): on an incremental rebuild
# `all_nodes` holds only the re-extracted batch, so every alias pointing into
# an UNCHANGED file would fail to resolve and be pruned — the graph would
# decay back toward the bug on exactly the `graphify watch` / hook path. It
# would also bypass the single-candidate guard, since a name that is
# ambiguous corpus-wide can look unique within the changed subset and get an
# arbitrary, fabricated target. Inert for Kotlin, which indexes off
# `per_file`.
run_language_resolvers(
paths, per_file, all_nodes, all_edges,
resolvers=[_KOTLIN_IMPORT_TARGET_RESOLVER],
paths, per_file, resolution_nodes, all_edges,
resolvers=[_KOTLIN_IMPORT_TARGET_RESOLVER, _ELIXIR_IMPORT_TARGET_RESOLVER],
)

# Build evidence index from import edges so cross-file calls backed by an
Expand Down
32 changes: 30 additions & 2 deletions graphify/extractors/elixir.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
from typing import Any

from graphify.extractors.base import _LANGUAGE_BUILTIN_GLOBALS, _file_stem, _make_id
from graphify.security import sanitize_metadata


def extract_elixir(path: Path) -> dict:
Expand All @@ -30,6 +31,7 @@ def extract_elixir(path: Path) -> dict:
edges: list[dict] = []
seen_ids: set[str] = set()
function_bodies: list[tuple[str, Any]] = []
module_def_nids: set[str] = set()

def add_node(nid: str, label: str, line: int) -> None:
if nid not in seen_ids:
Expand All @@ -39,12 +41,15 @@ def add_node(nid: str, label: str, line: int) -> None:

def add_edge(src: str, tgt: str, relation: str, line: int,
confidence: str = "EXTRACTED", weight: float = 1.0,
context: str | None = None) -> None:
context: str | None = None,
metadata: dict | None = None) -> None:
edge = {"source": src, "target": tgt, "relation": relation,
"confidence": confidence, "source_file": str_path,
"source_location": f"L{line}", "weight": weight}
if context:
edge["context"] = context
if metadata:
edge["metadata"] = sanitize_metadata(metadata)
edges.append(edge)

file_nid = _make_id(str(path))
Expand Down Expand Up @@ -118,6 +123,16 @@ def walk(node, parent_module_nid: str | None = None) -> None:
return
module_nid = _make_id(stem, module_name)
add_node(module_nid, module_name, line)
# Only TOP-LEVEL modules are indexable by the corpus-wide resolver.
# A nested `defmodule Supervisor` inside `MyApp.Application` is
# labelled with the written inner name, not its real BEAM name
# (`MyApp.Application.Supervisor`), so indexing it would let it
# capture `use Supervisor` from unrelated files and assert a false
# EXTRACTED edge. Marker rides on the node dict so it survives the
# id-remap/disambiguation passes (same rationale as `_callable`,
# #1566); export strips `_`-prefixed attrs.
if parent_module_nid is None:
module_def_nids.add(module_nid)
add_edge(file_nid, module_nid, "contains", line)
if do_block_node:
for child in do_block_node.children:
Expand Down Expand Up @@ -164,15 +179,28 @@ def walk(node, parent_module_nid: str | None = None) -> None:

if keyword in _IMPORT_KEYWORDS and arguments_node:
for module_name in _get_alias_modules(arguments_node):
# An Elixir `alias`/`import`/`require`/`use` names a BEAM module,
# not a path, so a per-file extractor cannot know which file
# defines it. Target stays the name-derived id for now;
# `_resolve_elixir_import_targets` rewrites it to the real
# `defmodule` node id via the `target_fqn` stamped here, once the
# corpus-wide module index exists (#3562). Targets that resolve
# to nothing (Ecto, Logger, any dep) stay dangling like other
# languages' external imports and are pruned by build.
tgt_nid = _make_id(module_name)
add_edge(file_nid, tgt_nid, "imports", line, context="import")
add_edge(file_nid, tgt_nid, "imports", line, context="import",
metadata={"target_fqn": module_name})
return

for child in node.children:
walk(child, parent_module_nid)

walk(root)

for n in nodes:
if n["id"] in module_def_nids:
n["_elixir_module"] = True

label_to_nid: dict[str, str] = {}
for n in nodes:
normalised = n["label"].strip("()").lstrip(".")
Expand Down
Loading