Skip to content

emit-portable: token alignment window for Doc/edit (S2, #57)#63

Merged
johnsoncodehk merged 1 commit into
masterfrom
portable-align-s2
Jul 11, 2026
Merged

emit-portable: token alignment window for Doc/edit (S2, #57)#63
johnsoncodehk merged 1 commit into
masterfrom
portable-align-s2

Conversation

@johnsoncodehk

Copy link
Copy Markdown
Owner

Summary

Slice 2 of #57 (portable incremental edit). Builds on the S1 Doc/edit API (#61):

  • All three portable targets (TS/Go/Rust) now retain token metadata (kind/off/end/nl) across edits. After each edit(), the doc relexes in full and computes a token alignment window {oldN, newN, prefix, suffix} — the longest common token prefix (matching kind/offset/end/newline-flag + source text) and suffix (same, shifted by the byte-length delta), constrained so prefix + suffix ≤ min(oldN, newN).
  • The edit-session runner protocol emits the alignment quadruple as one JSON line on stderr (before parse outcome), so the gate can assert it even on rejected final texts. Plain-mode stdout/stderr are unchanged.
  • Gate: 10 edit scenarios (incl. 2 large-doc ~1.2k/2.4k char cases) × 3 targets assert the full quadruple against oracle-lexer-derived expectations; existing edit-session ≡ fresh-parse checks retained.
  • Attached fix: edit() now clamps start/end to [0, len] with end ≥ start in all three targets, matching String.prototype.slice semantics (Go/Rust previously panicked on out-of-range edits that JS silently clamped).

This slice computes the damage window but does not yet use it — parse paths are untouched (still full reparse). Windowed relexing and tree reuse come in later slices.

Validation

  • node test/check.ts emit-parity — 5/5 gates
  • node test/portable-targets.ts — calc/javascript: 5/5 edit-sessions + 5/5 token-alignments × 3 targets
  • node test/newline-mode.ts — 29/29
  • npx tsc --noEmit — clean
  • Independent adversarial probe: same-length token replace, newline-flag flip, token merge (a bab), first-token change — alignment parity across all 3 targets vs oracle-lexer expectations

@johnsoncodehk johnsoncodehk merged commit 860c6c4 into master Jul 11, 2026
3 checks passed
@johnsoncodehk johnsoncodehk deleted the portable-align-s2 branch July 11, 2026 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant