Fix/qualified tagged template tags - #306
Merged
aspeddro merged 2 commits intoSep 25, 2026
Merged
Conversation
`in_backticks` never tracked what its name says. The opening backtick was fused into the template token, so it never set the flag; TEMPLATE_CHARS clears the flag when it reaches the closing backtick, and the BACKTICK branch then toggles it at that same backtick. The flag therefore ended up false inside template literals and true outside them — inverted — and it feeds `in_string`, which gates whitespace skipping and block-comment scanning. Every template literal silently flipped the lexer into string mode for the code that followed it. Nothing needs the flag. The scanner learns it is lexing template text from `valid_symbols[TEMPLATE_CHARS]`, which the parser supplies and which is authoritative; the TEMPLATE_CHARS branch then consumes the whole run itself, so `/*` inside a template is never offered to the comment scanner. Quotes are different and keep `in_quotes`: string content is lexed by the grammar as `unescaped_double_string_fragment`, so the scanner does run between fragments and does need to be told to stand down. Removing `in_quotes` as well fails "Strings containing comment-like content". Both backticks become ordinary grammar tokens. Also drops `_unescaped_template_string_fragment`, dead since template content moved to the scanner, and adds a test for a block comment following a template substitution — the case the inverted flag corrupted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The tag was a lexical prefix fused into the template token, matching only
`[a-z_][a-zA-Z0-9_']*`, so a module-qualified tag failed to parse at all:
let _ = await sql->(SQL.query`SELECT 1`)
ReScript treats the tag as a function. `bsc -dparsetree` gives `Pexp_apply`
of the `Pexp_ident` `SQL.query` over two arrays, so the tag is a value
reference — plain or qualified by a module — rather than part of the
literal. Model it that way, with the `function` and `arguments` fields
`call_expression` already uses. `SQL` and `query` become separate nodes,
which lets highlights scope them as a module and a call.
The compiler settles the edges: whitespace before the literal is allowed,
`obj.field` and `f(1)` are rejected as tags, escaped identifiers are
accepted, and in JSX children no tagged template is formed — `<div>
sql`x` </div>` is two children, which the added conflict reproduces.
Parenthesized tags are left out; accepting an arbitrary parenthesized
expression before every template is not worth the ambiguity.
Injections had to follow: `%raw`/`%graphql`/`%relay` matched content with
`(_ (_))`, which under the extra node captured the tag and the backticks
as injected source. They now name the shapes they accept.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Member
|
@codex review |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi there,
The current implementation errors on a tagged template literals like:
which are a perfectly valid ReScript.
This PR fixes this by making literal prefixes what they really are: a function application. I also deleted an unused and actually misleading state flag from the C lexer that I've introduced many years ago. Thanks to Claude. His technical explanation goes below
Why
SQL.query`…`doesn't parse: the tag was a lexical prefix matching only[a-z_][a-zA-Z0-9_']*.bsc -dparsetreeshows a tagged template isPexp_applyof aPexp_ident, sothe tag is a value reference rather than part of the literal — hence the node,
and the
function/argumentsfields borrowed fromcall_expression.The scanner commit
in_backtickswas inverted. The opening backtick was fused into the templatetoken so it never set the flag, while
TEMPLATE_CHARScleared it andBACKTICKtoggled it at the same closing backtick — leaving it false inside templates and
true outside. It feeds
in_string, which gates whitespace skipping andblock-comment scanning. Nothing needs it:
valid_symbols[TEMPLATE_CHARS]alreadytells the scanner when it is in template text.
in_quotesis not symmetric and stays — string content is grammar-lexed, so thescanner does run between fragments. Removing it fails Strings containing
comment-like content.