Rework the GraphQL task: encrypted token, output modes, schema derived from the query by seebi · Pull Request #12 · eccenca/cmem-plugin-graphql

seebi · 2026-09-15T21:42:18Z

Breaking changes throughout, so this is a 7.0.0 release. Verified against a running
deployment as well as in the test suite: a workflow that queries GitLab and writes its
result to a JSON dataset runs through at every step below.

Added

Access token, typed as a password so the value is encrypted instead of sitting as
readable text in the task configuration. Sent as Authorization: Bearer <token>; a GitLab
personal, project or group access token works. It sits directly under Endpoint and is
not an advanced field, because a token is ordinary configuration for most endpoints.

Output mode, choosing the shape the responses leave in:

entities — the default, and what the task did so far.
file — every response of the run written to one graphql-result.json and handed on as
a file, for a chain that stores or uploads the result rather than mapping it.

Breaking

The plugin identifier moved from the generated
cmem_plugin_graphql-workflow-graphql-GraphQLPlugin to cmem_plugin_graphql-Query. A
task configured before this release stops resolving: its workflow fails with Invalid
plugin cmem_plugin_graphql-workflow-graphql-GraphQLPlugin. Rebuilding the task works, but
rewriting the stored task's type is cheaper and keeps its id, its place in every
workflow, its parameter templates and its stored token. Both routes are in the changelog.
Target JSON Dataset is gone. Setting it used to remove the output port, so the result
could only be reached by reading the dataset back. Connect a dataset to the output port.
OAuth access token is gone. Move the value to Access token and rotate the token.

Changed

The output schema is derived from the query wherever the query describes its own response,
so the next task is offered the paths while the workflow is drawn rather than after a first
run. Only the top level is described — a field selecting sub fields becomes a relation, and
the entities behind it follow the response, as cmem-plugin-llm does for a nested
structured output. Aliases are honoured; a field selected twice is one path, as GraphQL
merges it into one key.

Cardinality is not in a query, so every path is declared as possibly multi valued and the
entities are built to match. A field answered with a single object therefore arrives as a
one element list, and a connected JSON dataset holds an array in that place. The
alternative — guessing single — loses data whenever the endpoint answers with a list.

The task no longer talks to Corporate Memory at all. The dataset write was its only use, so
the integration tests became ordinary ones and the suite needs no deployment.

Fixed

The one that took a real workflow down: a field the endpoint answers with null for some
records and an object for others made a JSON dataset write fail outright with Current
context not Array but Object. build_entities_from_data describes such a field as a
relation and then writes [""] where the answer was null, and an empty string is not a
sub entity URI. Found by bisecting a GitLab query against the sink —
latestVersion.createdByUser is null for versions created by a job. Reported upstream as
eccenca/cmem-plugin-base#48; the workaround here goes once that ships.

Also fixed, most of it found by /code-review on this branch:

Jinja values were HTML escaped into the query and the variables, so O'Brien & Co
reached the endpoint as O'Brien & Co — a query asked about, and a mutation
stored, a string nobody typed. Autoescaping is off now, S701 suppressed with its reason
as .claude/rules/copier-template.md prescribes.
A field answered with text in one record and an object in another crashed the task with a
bare AttributeError after every query of the run had been sent. It now names the field.
Objects behind one path are built across the whole run, so a path is described once
instead of once per response with contradictory schemas.
Values that are not text keep their JSON form: true, null, {"major": 1}.
Root entities carry an identifier of their own per run, so two runs and two such tasks in
one workflow no longer collide.
The per entity loop honours cancellation; a failed entity logs the message, not just the
error class; an empty response object counts as a result; an anonymous { ... } query is
reported as a read.

Tests

The endpoint tests, three of them mutations against a public endpoint nobody here owns, are
guarded on TESTING_GRAPHQL_ENDPOINT — they used to run on every push of every branch, and
the suite failed offline. 35 tests run offline in seconds; 47 with the endpoint named.
task check is green: ruff, mypy, deptry, trivy.

Known limitations, deliberately left

A Jinja query is still not syntax checked. Rendering its placeholders as null and
parsing that would catch a typo, but a template supplying a field name renders to null
where GraphQL needs a name, so the check would reject good configurations.
Jinja values are substituted literally now that escaping is gone, so a value carrying a
quote still breaks the JSON it lands in. tojson is the way to place one safely.
The mirror of the upstream defect — object first, null later — still flattens the object
into its Python string form. build_entities_from_data: stop a null object from corrupting the entities cmem-plugin-base#48 fixes it at the source.
The temporary directory the file shape writes into is never deleted; the consumer reads
the file after the task returns. Two other plugins in the fleet do the same.
Input ports are still not declared, so the editor shows DataIntegration's default.

🤖 Generated with Claude Code

The endpoint token was only configurable through "OAuth access token", a plain string parameter, so it was stored unencrypted in the task configuration and in every project export. The new "Access token" parameter uses PasswordParameterType and is sent as the same Authorization: Bearer header. The old parameter keeps working while the new one is empty, and logs a warning when it is used, so existing tasks are not broken by the change. It is documented as removed in version 7.0.0. Tests cover both parameters, their precedence and the warning without needing a deployment, plus a GitLab query guarded by TESTING_GITLAB_TOKEN that proves the header is honoured by asking for currentUser. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

write_to_dataset() from cmem_plugin_base carries no deprecation marker itself, but calls setup_cmempy_user_access() and cmempy's post_resource() internally, so every execution against a target dataset raised a DeprecationWarning saying cmempy is going away. It was the last cmempy path left in the plugin. The upload now goes through get_client(context).datasets.post_file_resource(). The combined "<project>:<dataset>" id existed only because write_to_dataset wanted one and is gone. A failed write therefore raises httpx.HTTPStatusError rather than requests.HTTPError. The test asserting a 404 on an unknown dataset is updated, as is a suppress() in the project fixture that had been naming the requests exception while already guarding a cmem-client call. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The Target JSON Dataset parameter took the result out of the workflow: setting it removed the output port, so the only way to reach the response was to read the dataset back with another task. The response now always leaves on the output port, one entity per query execution, and a workflow that wants it in a dataset connects a task that writes one. The deprecated OAuth access token parameter goes in the same release, as its deprecation note announced. Its value belongs in Access token, which keeps it encrypted rather than in plain text in the task configuration and in project exports, so move it over and rotate the token. Writing the dataset was the only thing this plugin used Corporate Memory for. Nothing here talks to a deployment now, which is why the integration tests became ordinary ones: they assert on the returned entities instead of reading an uploaded file back, and the whole suite runs without credentials. Both parameters are breaking removals - a task configured with either has to be reconfigured. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

A GraphQL query names the fields it asks for, so the paths of the entities the response becomes are known before the endpoint is called. The output port now carries them as a fixed schema, which lets DataIntegration offer the paths while the workflow is drawn instead of only after the task has run once. Only the top level of the query is described: a field that selects sub fields becomes a relation path, and the schema behind it is still discovered from the response, the way cmem-plugin-llm handles a nested structured output. A field is named by its alias where it has one, since that is the key the response uses. What a query does not state is how many values a field carries - that is in the endpoint's schema - so every path is declared as possibly multi valued. Declaring a single value and receiving a list is the direction that loses data, so the returned root schema may say is_single_value where the declared one does not; the path names and relation flags always agree. A query that does not parse as GraphQL on its own, which a Jinja template usually does not, one holding several operations, and one whose top level is a fragment describe no schema in advance and keep the unknown schema port. Also returns an empty Entities carrying that schema when nothing was queried at all. build_entities_from_data answers None for an empty payload, which execute() passed straight on while declaring it returns Entities. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Two defects, both reached through a JSON dataset connected to the output port, both found by running a real workflow against a deployment. A field the endpoint answers with null for some items and with an object for others took the whole write down with "Current context not Array but Object". build_entities_from_data() calls such a field a relation, because one item does carry an object, and then writes [""] for every item where the answer was null. An empty string is not a sub entity URI, so the sink follows it, lands nowhere and aborts before writing a single entity. The placeholder is now removed and the path left empty. A GitLab query asking for a createdByUser that nobody set hits this, and nothing shallower does: flat entities, deep nesting and nested lists all write fine. Separately, the declared schema no longer disagrees with the entities. Cardinality is in the endpoint's schema rather than in the query, so every path is declared as possibly multi valued - the only declaration that carries both a single object and a list - and the root entities are now built to match the declaration instead of being read off the response, which described the same query differently per response. A field answering with one object therefore arrives as a one element list, and a JSON dataset holds an array where the endpoint answered with an object. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

github-actions · 2026-09-15T21:43:23Z

Coverage Report

File	Stmts	Miss	Cover	Missing
init.py	0	0	100%
workflow/init.py	0	0	100%
workflow/graphql.py	145	27	81%	240-241 270-272 274-275 277 286-291 306 313-315 318 336 345-350 406
workflow/utils.py	93	0	100%
TOTAL	238	27	89%

Tests	Skipped	Failures	Errors	Time
47	13 💤	0 ❌	0 🔥	3.196 ⏱

A token is ordinary configuration for most endpoints, so hiding it behind the advanced section made it easy to miss. DataIntegration renders parameters in the order of the constructor signature - retrieve_parameters() walks inspect.signature(__init__) and matches the decorator entries by name - so the position is settled by the constructor, not by the order of the @plugin parameter list. Both are reordered to keep them readable together. Access token therefore precedes a parameter that has no default and so cannot carry one either. That does not make it mandatory: required-ness comes from PluginParameter.default_value, which stays "". Verified against a deployment, where the plugin reports Endpoint, Access token, Query, Query variables with advanced false and required limited to graphql_url and graphql_query. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The documentation had followed the task through two reshapes and no longer matched it. Rewritten against the code: It describes what leaves on the output port and how the schema follows from the query, and it names the consequence a user actually meets - a field the endpoint answers with a single object arrives as a list holding one entity, so a JSON dataset connected to the port holds an array in that place. It regained the beat saying where the task sits in a chain, which was lost when the target dataset parameter went. The Jinja caveats stay, since both of them still bite. It no longer claims an input port accepts entities. The task never declares input_ports, so what the editor offers is DataIntegration's default rather than something this code states, and the documentation describes the behaviour that is verifiable instead: Jinja turns the task into a per-entity loop, plain text is sent once and anything connected is ignored. Each parameter description now leads with what that parameter controls rather than with a definition of GraphQL, and the query example is a fenced code block instead of a loose indented blob. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Some chains do not want the response taken apart. A task that stores or uploads what it is handed wants the bytes, and mapping the response into entities only to serialise it again on the other side is work with a shape change in the middle of it. Output mode chooses between the two. It defaults to entities, which is what the task did so far, so an existing task keeps its behaviour without being touched - DataIntegration fills the parameter in from its default. The file shape collects the responses of the whole run into one JSON file and hands that on, described by FileEntitySchema, the way cmem-plugin-jira and cmem-plugin-yaml hand files on. The file is written into a directory of its own so it can be called graphql-result.json rather than carrying a temporary name into the next task, and ensure_ascii stays off so non-ASCII text reaches it as text. It sits under Access token, which means the constructor again, since that is the order DataIntegration renders. Two of the three leading parameters now carry no Python default, so the tests build the plugin through a helper rather than repeating both at every call, and a new test pins the parameter surface - order, advanced flags and defaults - as the descriptor reports it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The block had grown to the longer end of what this fleet ships - two output shapes, the cardinality rule, three cases with no derivable schema and two Jinja traps - and read as one run of paragraphs, so finding one of those meant reading all of them. Headings now mark the beats the text already followed, and the caveats became a list with the claim in bold at the front of each, following cmem-plugin-graphsigning, which documents the same beats that way. No claim changed; this is the same text, cut into sections. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Both described what happens when a Jinja-templated query or variables text has nothing connected as input. Removed on Sebastian's call. The behaviour is unchanged and both descriptions were accurate: the templated query is still sent unrendered and fails the task, and templated variables with no input still send nothing and complete as a run with no work in it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The null-object repair only covered the branch that derives a schema from the query. A query carrying Jinja syntax derives none and leaves through build_entities_from_data instead, which handed the placeholder on untouched - so the repair missed the per entity mode, which is the mode the fault was found in. Both branches now pass through the same sanitising step. Jinja rendering had autoescaping on, which rewrites the values on their way into the query and the variables: O'Brien & Co reached the endpoint as O'Brien & Co, so a query asked about, and a mutation stored, a string nobody typed. Autoescaping is off now, suppressing S701 with the reason in place, which is what .claude/rules/copier-template.md prescribes for exactly this case. Values are still substituted literally, so tojson remains the way to place an untrusted one. Twelve tests reached a public GraphQL endpoint unconditionally after the integration tests were converted, three of them mutations writing to an endpoint nobody here owns on every push of every branch, and the suite failed offline. They are guarded on TESTING_GRAPHQL_ENDPOINT now: 26 tests run offline in seconds, 38 with the endpoint named. Also from the review: the per entity loop honours cancellation, a failed entity is logged with the error message rather than only its class, CLAUDE.md no longer documents the parameters and code paths this branch deleted, the breaking removals sit under the heading this project uses for them, and cmem-client is gone from the dev dependencies with the last cmem_client import. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

A field answered with text in one record and with an object in another crashed the task with a bare AttributeError from inside cmem-plugin-base, after every query of the run had been sent and any mutation among them carried out. Such a response still cannot become entities, but the error now names the field and points at selecting the fields individually or taking the result as a file. The objects behind one relation path are built across the whole run rather than once per response. Built per response, a path holding an object in one and null in another produced two collections contradicting each other about the same path, and a consumer resolving relation URIs was handed both. A field selected twice at the top level is one key in the response, because GraphQL merges the selections, so it is described once rather than twice and its object is built once rather than twice. Values that are not text keep their JSON form: true, null and {"major": 1} rather than Python's True, None and {'major': 1}. A field selected without sub fields can still answer with an object where the endpoint types it as a custom JSON scalar. Root entities get an identifier of their own per run instead of a counter, which two runs and two of these tasks in one workflow were handing out identically. Smaller: an empty response object counts as a result rather than a failed entity, the report calls an anonymous query a read by reading the parsed operation rather than the first word, and the temporary directory the file shape writes into carries a prefix naming its owner. Not fixed, deliberately: a Jinja query is still not syntax checked. Rendering the placeholders as null and parsing that would catch a typo, but a template supplying a field name renders to null where GraphQL needs a name, so the check would reject configurations that are perfectly good. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The identifier was generated from the module path and the class name, so moving this module or renaming the class would have changed the identity of every task deployed from it and orphaned the workflows referencing them - silently, with no error anywhere. It is written out now, verbatim as generated: cmem_plugin_graphql-workflow-graphql-GraphQLPlugin. Not shortened to the <package>-<Name> form a new plugin would take, because that shorter name is a different identifier and would cause exactly the breakage pinning it prevents. A test holds it to this value. Nothing changes for a configured task. Verified against a deployment: a task created in February still resolves and its workflow runs through. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Setting the identifier explicitly stops the module path and the class name from deciding it. The previous commit pinned the generated value to avoid breaking deployed tasks; this one takes the shorter <package>-<Name> form instead, on Sebastian's call that a release already removing two parameters is the release to move it in, rather than carrying the crooked name through another major version. This breaks harder than the parameter removals do. A task configured before 7.0.0 does not merely lose a field, it stops resolving: the workflow fails with "Invalid plugin cmem_plugin_graphql-workflow-graphql-GraphQLPlugin. Expected a single Python plugin, but got: List()". Verified against a deployment, and so is the way out - rewriting the stored task's type to the new identifier restores it while keeping its id, its place in the workflow, its parameter templates and its stored token, which rebuilding the task by hand would all lose. The changelog carries both routes under Breaking Change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Validated every entry against the code. Four things were wrong about the entries themselves rather than about what they describe. Losing the Corporate Memory dependency sat under Fixed, where it is not a fix, and now sits under Changed with what it means for running the task. The plugin identifier appeared twice, once as a plain change and once as the breaking change it actually is; only the latter remains. A blank line split the Changed list in two and one entry carried a ragged wrap from an earlier edit. The note on the removed OAuth parameter claimed the token was readable "in every project export". Its being readable in the task configuration was verified; an export was never opened, so the entry now says that much and no more. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

seebi and others added 5 commits September 15, 2026 22:11

seebi and others added 10 commits September 15, 2026 23:51

seebi changed the title ~~Encrypted access token, output port only, schema derived from the query~~ Rework the GraphQL task: encrypted token, output modes, schema derived from the query Sep 16, 2026

seebi merged commit 9b8910c into main Sep 16, 2026
2 checks passed

seebi deleted the feature/accessTokenParameter branch September 16, 2026 08:12

seebi mentioned this pull request Sep 16, 2026

Declare the input port, naming the paths the Jinja templates ask for #13

Merged

Navigation Menu

Rework the GraphQL task: encrypted token, output modes, schema derived from the query - #12

Rework the GraphQL task: encrypted token, output modes, schema derived from the query#12
seebi merged 15 commits into
mainfrom
feature/accessTokenParameter

seebi commented Sep 15, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 15, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

seebi commented Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Added

Breaking

Changed

Fixed

Tests

Known limitations, deliberately left

Uh oh!

github-actions Bot commented Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

seebi commented Sep 15, 2026 •

edited

Loading

github-actions Bot commented Sep 15, 2026 •

edited

Loading