hrw4u: record spans, comments and operand forms in the AST - #13665
hrw4u: record spans, comments and operand forms in the AST#13665masaori335 wants to merge 2 commits into
Conversation
ast_visitor.py is test-only today, and it cannot reproduce the file it
parsed. Four things are dropped, each of which changes either the
emitted .conf or the location a diagnostic can name.
A span names a file, line and column rather than a line alone.
Procedure inlining splices nodes in from a second file, so a line
number can no longer say where a diagnostic belongs. Node.line is
replaced outright rather than kept as a property over span: nothing
outside the tests reads it, and keeping it would let new code go on
producing locations that cannot name their own file.
Comments, because five .conf goldens carry # comments at exact
positions.
Parentheses, because a parenthesized factor emits cond %{GROUP}.
Set and iprange operands, which until now both became a bare tuple:
`in [1.2.3.4]` and `in {1.2.3.4}` produced byte-identical ASTs. They
are not interchangeable -- visitor.py emits `lhs (...)` for a set and
`lhs {...}` for an iprange, and only the set path is sandbox-checked.
Each keeps the source text the emitter already uses, so SetValue.raw
matches visitor.py's set_().getText()[1:-1] and IpRangeValue.raw its
iprange().getText().
Target.from_dotted guessed a namespace/field split that no caller
wanted; every table lookup matches the whole dotted name.
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Pull request overview
This PR updates the HRW4U AST and visitor to preserve source fidelity (file/line/column spans, comments, grouping parentheses) and to disambiguate operand forms that previously collapsed into identical structures.
Changes:
- Replace
Node.linewithSpan(file, line, column)and thread spans through all AST nodes viaASTVisitor. - Preserve comments as first-class AST nodes across program/section/block/vars bodies, keeping original ordering and indentation/column.
- Introduce
Group,SetValue, andIpRangeValueto retain parentheses and distinguishin [...]vsin {...}semantics; simplify assignments to storenamerather than a splitTarget.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| tools/hrw4u/tests/test_ast_visitor.py | Updates tests to assert spans, comment preservation, grouping behavior, and new operand/assignment representations; adds strict parsing in tests. |
| tools/hrw4u/tests/test_ast_nodes.py | Replaces Target tests with Span immutability/hashability/equality tests. |
| tools/hrw4u/src/ast_visitor.py | Emits spans on all nodes, preserves comments, wraps parenthesized conditions in Group, and distinguishes set vs iprange RHS forms. |
| tools/hrw4u/src/ast_nodes.py | Adds Span, Comment, Group, SetValue, IpRangeValue; removes Target and replaces line with span; updates node/value unions accordingly. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
The eight "Unhandled ... at line N" raises predate the span and cannot name the file, which is the case procedure inlining creates. They go through one _unhandled() helper now. test_ast_nodes.py asserted AttributeError for assignment to an unknown Span field, which proved nothing: FrozenInstanceError subclasses AttributeError, so frozen intercepted the write and slots was never exercised. Check the layout instead. ValueExpr admits IpRangeValue but not SetValue because the grammar's value rule admits iprange and not set_; say so where the alias is defined.
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
|
[approve ci autest 2of4] |
ast_visitor.py is test-only today, and it cannot reproduce the file it parsed. Four things are dropped, each of which changes either the emitted .conf or the location a diagnostic can name.
A span names a file, line and column rather than a line alone. Procedure inlining splices nodes in from a second file, so a line number can no longer say where a diagnostic belongs. Node.line is replaced outright rather than kept as a property over span: nothing outside the tests reads it, and keeping it would let new code go on producing locations that cannot name their own file.
Comments, because five .conf goldens carry # comments at exact positions.
Parentheses, because a parenthesized factor emits cond %{GROUP}.
Set and iprange operands, which until now both became a bare tuple:
in [1.2.3.4]andin {1.2.3.4}produced byte-identical ASTs. They are not interchangeable -- visitor.py emitslhs (...)for a set andlhs {...}for an iprange, and only the set path is sandbox-checked. Each keeps the source text the emitter already uses, so SetValue.raw matches visitor.py's set_().getText()[1:-1] and IpRangeValue.raw its iprange().getText().Target.from_dotted guessed a namespace/field split that no caller wanted; every table lookup matches the whole dotted name.