Skip to content

clarify where execution variables resolve and how a run gets them - #271

Open
seebi wants to merge 3 commits into
mainfrom
feature/execution-variables-documentation
Open

seebi wants to merge 3 commits into
mainfrom
feature/execution-variables-documentation

Conversation

@seebi

@seebi seebi commented Sep 14, 2026

Copy link
Copy Markdown
Member

Follow-up to the execution variables call on 2026-09-14 (Sebastian, Robert, Andreas, Saipraneeth), which ended with "add a task: improve the documentation".

Everything is in docs/build/variables/index.md, in the Execution Variables section.

One question for @robertisele — please answer before merging

Do the SPARQL Select and Update query templates resolve execution. references?

You said in the call that they do, next to the Evaluate template operator. I could not corroborate it from anything readable, and one source points the other way:

  • plugin_describe(sparqlUpdateOperator) reports sparqlUpdateTemplate as parameterType: "code-sparql", not the template parameter type that marks a per-execution parameter. plugin_describe(Template) does report parameterType: "template".
  • The generated reference page for the SPARQL Update task enumerates its template variables as input.entity, input.config, output.config, project, global — no execution.
  • DataIntegration's own conceptual documentation (the variables topic served by the embedded MCP server) names only the Template custom task as the reliable per-execution parameter and does not mention the SPARQL tasks.

This PR therefore names only the Evaluate template operator and transformer. If the SPARQL tasks do support it, say so and I will add them here and raise a generator ticket for the missing execution entry in both SPARQL reference pages.

What changed

  • Where execution variables resolve. Stated as a rule about resolution time: project and global variables are resolved when a task is saved, so they work in any field with the {#} toggle; an execution variable only has a value during a run, so it takes effect in parameters evaluated once per execution.
  • The stale-template trap. "Not supported in ordinary parameter fields" turned out to be inaccurate — any string parameter accepts a template, but one in an ordinary field is resolved at task-load time and cached until the task is reloaded. Referencing an execution variable there does not fail, it silently returns the same value on every run. The page now says this and points at the config port for per-run plugin parameters.
  • How a run's variable set is built, in three steps, with the two consequences the call surfaced: a variable used by an inner task or sub-workflow must be defined on the executed workflow or supplied at run start, and names form one flat set, so the same name in two tasks always gets the same value.
  • Both REST forms. The page documented only the executionVariables JSON body, which is unusable on endpoints whose body carries a replaceable input dataset. ?variable-<name>=value now comes first as the form that always works.
  • The transitive lookup, two sentences — see below.
  • Titles. The H1 and the navigation entry named two of the three scopes the page documents; they are now Build Variables and Variables.

Worth checking before anyone builds it

GET /api/core/variableTemplate/variables with the transitive flag looks like it already shipped in 26.2 — the release notes describe exactly the endpoint the call spent about fifteen minutes specifying, recursion through sub-workflows and enclosing-workflow precedence included. @msaipraneeth is blocked waiting for it.

Follow-ups not in this PR

  • Generator: the SPARQL Update reference lists its template variables without execution. (depends on the question above).
  • Generator: the Evaluate template transformer describes only the global scope, although it is one of the two confirmed places where execution variables resolve.
  • cmemc section still covers the project variable command group only; execution variable support is the work starting now.

task check passes.

🤖 Generated with Claude Code

https://claude.ai/code/session_017QSztDcCXEweTw91E3c9Vm

The page did not say where an execution.* reference actually takes
effect, which is the question users brought to the review meeting. It
also documented only one of the two ways to pass variables into a run,
and that one does not work on endpoints accepting a replaceable input
dataset.

- state the rule: project and global variables resolve when a task is
  saved, execution variables only in parameters evaluated per run
- warn that a template in an ordinary parameter field is resolved once
  at task load and cached, so it never changes between runs
- describe how the variable set of a run is built, that inner tasks and
  sub-workflows contribute no defaults, and that names are flat
- document the variable- query parameter form first, the JSON body form
  second, and the transitive variable lookup
- rename the page and its navigation entry to cover all three scopes
@seebi
seebi requested a review from robertisele September 14, 2026 11:01
@github-actions

github-actions Bot commented Sep 14, 2026

Copy link
Copy Markdown

Test Results

0 tests   0 ✅  0s ⏱️
0 suites  0 💤
1 files    0 ❌

Results for commit c4cf3d8.

♻️ This comment has been updated with latest results.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants