Skip to content

Traqula migration

Jump to bottom
Martynas Jusevičius edited this page Oct 1, 2026 · 1 revision

Traqula migration plan

Migrating the browser's SPARQL parsing and generation off the deprecated SPARQL.js 2.x onto Traqula (@traqula/parser-sparql-1-1, @traqula/generator-sparql-1-1, and @traqula/rules-sparql-1-1 for its AstFactory).

sparqljs is marked deprecated on npm (3.7.4, February 2026). LinkedDataHub bundles sparqljs 2.x as js/SPARQL.js, two generations back, and since #401 retired SPARQLBuilder it is the only SPARQL parser and generator in the browser, so this is a single-library swap. Traqula is the Comunica project's successor, and its own migration guide says it "does not form a drop in replacement on AST level".

Scope

What stays: the pipeline. Parse to JSON in JS, json-to-xml, rewrite with XSLT modes, xml-to-json, generate in JS. The 19 modes in client/query-transforms.xsl keep their names, parameters and callers.

What changes: the vocabulary of the tree every mode matches on and emits, the term representation everywhere, the server-side twin ParseQuery.java, the update builder, the script that ships, and the glue params in client.xsl.

Decisions

  • Straight to Traqula, skipping SPARQL.js 3.x. 3.x would cost the same term-representation rewrite and is itself deprecated.
  • XSLT written against the Traqula tree directly. No JS adapter: an adapter is a hand-maintained library again, which is what #401 just removed.
  • No round-tripping, for now. The parser is built with an AstFactory({ tracksSourceLocation: false }), so every parsed node carries loc: { sourceLocationType: 'autoGenerate' }, and every node the XSLT builds carries the same marker. The generator then regenerates the whole query from the tree, which is exactly what SPARQL.js did. Round-tripping (verbatim reproduction of untouched regions via origSource) is a later, optional phase; it is what would make the sibling-GRAPH merge class of bug (see the comment in client/tree.xsl) impossible.
  • Prefixes resolved in XSLT. Traqula does not expand foaf:name; a parsed prefixed IRI is { value: 'name', prefix: 'foaf' } and the query's context array holds the definitions. One function ldh:iri($term, $context) returns the full IRI, used wherever a predicate or value is compared against a URI.

Rename table: SPARQL.js 2.x tree to Traqula tree

Terms. The whole tree uses these, including expressions and VALUES rows. Every node also carries loc, omitted below.

SPARQL.js 2.x (string) Traqula (node)
?name { type: 'term', subType: 'variable', value: 'name' }
http://x/p { type: 'term', subType: 'namedNode', value: 'http://x/p' }; a parsed ex:p is { ..., value: 'p', prefix: 'ex' }
"lex" { type: 'term', subType: 'literal', value: 'lex', langOrIri: undefined }
"lex"@en { ..., langOrIri: 'en' }
"lex"^^http://dt { ..., langOrIri: { type: 'term', subType: 'namedNode', value: 'http://dt' } }
_:b0 { type: 'term', subType: 'blankNode', label: 'b0' }
* in a projection or COUNT(*) { type: 'wildcard' }

Structure.

SPARQL.js 2.x Traqula
{ type: 'query', queryType: 'SELECT' } { type: 'query', subType: 'select', context: [], datasets, solutionModifiers: {}, where, variables } (context, datasets and solutionModifiers are required keys, possibly empty)
variables: ['?x', { expression, variable }] variables: [ TermVariable, { type: 'pattern', subType: 'bind', expression, variable } ] or [ Wildcard ]
where: [ pattern, ... ] (array) where: { type: 'pattern', subType: 'group', patterns: [...] } (one group node)
{ type: 'bgp', triples: [{ subject, predicate, object }] } { type: 'pattern', subType: 'bgp', triples: [{ type: 'triple', subject, predicate, object }] }
{ type: 'group' | 'optional' | 'minus', patterns } { type: 'pattern', subType: ..., patterns }
{ type: 'union', patterns: [ bgp, graph ] } { type: 'pattern', subType: 'union', patterns: [ PatternGroup, PatternGroup ] }. Each arm must be a group. ldh:GraphUnionPattern puts a bgp and a graph directly under the union today; both get wrapped
{ type: 'graph', name: '?g', patterns } { type: 'pattern', subType: 'graph', name: TermVariable, patterns }
{ type: 'filter', expression } { type: 'pattern', subType: 'filter', expression }
{ type: 'bind', variable: '?v', expression } { type: 'pattern', subType: 'bind', variable: TermVariable, expression }
{ type: 'values', values: [{ '?v': term }] } { type: 'pattern', subType: 'values', variables: [ TermVariable ], values: [{ v: term }] } (bare key, plus the variables list)
{ type: 'operation', operator: 'in' | '!' | 'isliteral' | 'regex', args } { type: 'expression', subType: 'operation', operator, args: Expression[] }. The IN list is no longer a nested array of args; the exact shape is confirmed in the Phase 0 spike
{ type: 'operation', operator: 'exists', args: [ group ] } { type: 'expression', subType: 'patternOperation', operator: 'exists', args: PatternGroup }
{ expression: '?x', type: 'aggregate', aggregation: 'count', distinct } { type: 'expression', subType: 'aggregate', aggregation: 'count', distinct, expression: [ TermVariable ] | [ Wildcard ] } (one-element array)
limit: 10, offset: 20 (top-level) solutionModifiers.limitOffset: { type: 'solutionModifier', subType: 'limitOffset', limit, offset }
order: [{ expression, descending }] solutionModifiers.order: { type: 'solutionModifier', subType: 'order', orderDefs: [{ expression, descending: boolean }] }
group: [{ expression }] solutionModifiers.group: { type: 'solutionModifier', subType: 'group', groupings: [ Expression ] }
from: { default: [iri], named: [iri] } datasets: { type: 'datasetClauses', clauses: [{ clauseType: 'default' | 'named', value: TermIri }] }
prefixes: { ex: 'http://...' } context: [{ type: 'contextDef', subType: 'prefix', key: 'ex', value: TermIriFull }]
template: [ triple ] (CONSTRUCT) template: { type: 'pattern', subType: 'bgp', triples }
{ type: 'path', pathType: '|', items } { type: 'path', subType: '|', items }
{ type: 'update', updates: [{ updateType: 'insertdelete', delete, insert, where }] } { type: 'update', updates: [{ context: [], operation: { type: 'updateOperation', subType: 'modify', graph: undefined, insert: Quads[], delete: Quads[], from, where } }] }
generator.stringify(tree) generator.generate(tree)
new Parser() new Parser({ defaultContext: { astFactory: new AstFactory({ tracksSourceLocation: false }) } }), constructed once: Chevrotain pays its optimisation at construction

Inventory of what the tree touches today

Area What
client/query-transforms.xsl 19 modes, ~1100 lines: replace-variables, replace-limit, replace-offset, wrap-describe, strip-from, strip-order-by, link-predicates (with ldh:GraphUnionPattern), add-parallax-step, replace-order-by, toggle-desc, bgp-value-counts, filter-in, set-filter-in-values, result-count, container-select, add-constructor-triple, add-regex-filter, add-filter-in, append-values
Term layer, shared SSR/CSR imports/default.xsl ldh:triples-to-descriptions (regex parsing of 2.x strings, to go); client/functions.xsl ldh:parse-query, ldh:parse-rdf-post, ldh:descriptions-to-triples, ldh:insertdelete-update; ldh:construct-instance, ldh:instance-term
Server twin writer/function/ParseQuery.java (139 lines): emits the 2.x subset (template, where) behind ldh:parse-query under Saxon so constructors render identically on both products
Readers of keys outside the transforms client/modal.xsl (64 references), client/block/view.xsl (55, including ldh:alt-path-uris on pathType), client/functions.xsl (49), client/form.xsl (23), imports/default.xsl (22), client/constructor.xsl (15), client/map.xsl (6), client/block.xsl (5), client/block/chart.xsl (1). Keys: order, limit, offset, variables, descending, where, template, from, distinct
Glue client.xsl $sparql-parser and $sparql-generator; layout.xsl $load-sparql-js and its script tag; js/SPARQL.js
Siblings REST-VKG / Web-Algebra: no consumer of the SPARQL.js JSON found; the ldh-* operations go through the platform's HTTP API

Phase 0: bundle and spike (must come first)

  1. Bundle. Traqula is ESM/CJS only, with no standalone browser build (parser 331 KB, rules 1.6 MB, core 0.8 MB unpacked; the minified bundle size is unknown and is measured here). Add platform/traqula/ with a package.json, an index.js exporting { Parser, Generator, AstFactory } onto window.traqula, and an esbuild invocation; commit the output as js/traqula.js the way SPARQL.js was committed, so the Maven/Docker build (COPY src) needs no Node step. A make traqula target rebuilds it.
  2. Spike in Node, with the harness the project already uses for XSLT functions (a scratch stylesheet run by npx xslt3-he, SaxonJS 3.0.0-beta2, the browser's engine): parse a view SELECT with the non-tracking factory, JSON.stringify it, confirm every loc is autoGenerate; build a SELECT entirely by hand with autoGenerate locs and generate it; confirm the generator accepts a root with no origSource. Record the exact tree for FILTER(?x IN (...)), EXISTS, COUNT(DISTINCT *), VALUES, FROM, and a prefixed IRI with its context.
  3. Decide the IN shape and the union-arm wrapping from the spike output; both change what filter-in, set-filter-in-values, add-filter-in and ldh:GraphUnionPattern emit.

Exit criteria: the bundle loads from a <script> tag, parse and generate round-trip the spike queries, and the measured bundle size is recorded on this page.

Phase 1: test harness for the transforms

Commit tests/xsl/query-transforms/: a Node runner that, per fixture, parses input.rq with the bundle, applies one mode with its parameters through xslt3-he (the stylesheet included from an entity-expanded copy, ac:uuid() stubbed), generates the result and diffs it against expected.rq. One fixture per mode, cut from the queries the tests/ui specs exercise (view facets, parallax, sort, pager, count, class list, search, geo, constructor), including prefixed queries so the prefix-resolution change is caught.

Written first, against SPARQL.js, so the expected strings are today's behaviour; each mode is then ported until its fixture is green again under Traqula. make xsl-tests runs it; CI runs it beside tests/http.

Exit criteria: every mode has a green fixture on SPARQL.js.

Phase 2: term layer, both products

  • ldh:parse-query (CSR) returns the Traqula tree; ParseQuery.java (SSR) emits the same subset with Traqula term nodes (template.triples[] with term-node subjects, predicates and objects; the where presence flag unchanged). The Javadoc's lockstep note moves with it.
  • ldh:triples-to-descriptions reads subType, value, langOrIri and label instead of regex-splitting strings; the "quoting sucks in 2.x" TO-DO and the XMLLiteral regex go.
  • ldh:descriptions-to-triples and ldh:parse-rdf-post emit term nodes; ldh:instance-term prefixes label rather than the _: string.
  • New ldh:iri($term, $context) in imports/default.xsl (shared) resolving prefix against context; ldh:alt-path-uris and every URI comparison use it.

Exit criteria: constructor forms render identically SSR and CSR (overlays/modal/constructor-editor.spec.mjs, forms/row-form.spec.mjs); the RDF/POST round trip is unchanged.

Phase 3: query-transforms.xsl, mode by mode

Port each mode against its Phase 1 fixture. Patterns that recur:

  • matching /json:map/json:number[@key = 'limit'] becomes /json:map/json:map[@key = 'solutionModifiers']/json:map[@key = 'limitOffset']/json:number[@key = 'limit'], and absence means creating the modifier node, not just the key;
  • json:array[@key = 'where'] becomes json:map[@key = 'where']/json:array[@key = 'patterns'];
  • every emitted node gets <json:map key="loc"><json:string key="sourceLocationType">autoGenerate</json:string></json:map>, through one named template ldh:Loc so it is written once;
  • ldh:GraphUnionPattern wraps each arm in a group;
  • VALUES rows drop the ? and gain the variables list;
  • aggregates wrap expression in a one-element array.

Exit criteria: all fixtures green under Traqula.

Phase 4: call sites and glue

  • client.xsl: $sparql-parser and $sparql-generator constructed from window.traqula with the non-tracking factory; stringify becomes generate at the 11 serialise sites and in ldh:insertdelete-update, whose tree becomes a modify operation.
  • Readers in modal.xsl, block/view.xsl, form.xsl, constructor.xsl, map.xsl, block.xsl, block/chart.xsl and navigation.xsl: re-point the key paths (order, limit, offset, variables, descending, from, template) per the rename table.
  • layout.xsl: the load flag loads js/traqula.js and is renamed $load-sparql-parser.
  • Remove js/SPARQL.js; README lists Traqula; CHANGELOG entry.

Exit criteria: make sef compiles; tests/ui green, in particular document/blocks/view/*, document/blocks/map, forms/combobox, overlays/modal/constructor-editor, shell/drawer/* and document/action-bar/create.

Phase 5 (optional, later): round-tripping

Switch the parser to the tracking factory and pass origSource to generate; mark only the nodes the XSLT changes as nodeReplace or autoGenerate. Untouched regions of an authored query then come back byte-identical, which removes the sibling-GRAPH merge and prefix-abbreviation concerns for good. Not needed for parity; listed so it is not forgotten.

Risks and unknowns

  • Bundle size after minification is unknown until Phase 0: three packages plus @traqula/chevrotain. If it is far above SPARQL.js's 73 KB, load it deferred as today and measure first paint.
  • autoGenerate at the root without origSource: the type comment says it "must have an ancestor of type nodeReplace". The non-tracking factory produces exactly this tree for every parse, so it has to work, but Phase 0 proves it.
  • Prefix non-expansion is a semantic change relative to both 2.x and 3.x and bites every authored query with a PREFIX. ldh:iri must be used at every comparison; the Phase 1 fixtures include prefixed queries to catch a missed one.
  • Chevrotain parse speed on large authored queries: construct the parser once (already the case through the client.xsl param) and measure in the spike.
  • The generator is single-threaded ("cannot generate multiple queries in parallel"): fine under SaxonJS's synchronous calls.

Estimate

Phase Effort
0 bundle and spike 1 day
1 harness and fixtures 1.5 days
2 term layer (XSL and Java) 1.5 days
3 transforms 2.5 days
4 call sites, glue, cleanup, UI run 1.5 days
Total ~8 days

Clone this wiki locally