Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
"dataiku": {
"type": "stdio",
"command": "uv",
"args": ["run", "--quiet", "--locked", "--script", "${CLAUDE_PLUGIN_ROOT}/bin/run_mcp.py"]
"args": ["run", "--quiet", "--locked", "--script", "${CLAUDE_PLUGIN_ROOT}/runtime/run_mcp.py"]
}
}
}
42 changes: 35 additions & 7 deletions .github/workflows/bump.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,20 +74,24 @@ jobs:
- name: Verify PEP 723 script lockfile is current
run: |
set -euo pipefail
uv lock --script bin/run_mcp.py --check
uv lock --script bin/run_mcp.py --upgrade
if ! git diff --quiet -- bin/run_mcp.py.lock; then
echo "::error::bin/run_mcp.py.lock is stale. Regenerate, test, and commit it before releasing."
git --no-pager diff -- bin/run_mcp.py.lock
uv lock --script runtime/run_mcp.py --check
uv lock --script runtime/run_mcp.py --upgrade
if ! git diff --quiet -- runtime/run_mcp.py.lock; then
echo "::error::runtime/run_mcp.py.lock is stale. Regenerate, test, and commit it before releasing."
git --no-pager diff -- runtime/run_mcp.py.lock
exit 1
fi
- id: before
name: Record current tag
run: echo "tag=$(git describe --tags --abbrev=0 2>/dev/null || true)" >> "$GITHUB_OUTPUT"
# Keep the bump local until manifests are verified. commitizen-action's
# push defaults to true, which would publish a release tag before we can
# fail on a silent version_files miss.
- name: Bump version, changelog and tag
uses: commitizen-tools/commitizen-action@338bbd841b75aaee6bf5340e1fa12f6ab58ff9ff # 0.27.1
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
push: false
# 3 = no commits found, 21 = nothing to bump: treat both as a no-op
# rather than a failed run.
no_raise: "3,21"
Expand All @@ -109,16 +113,17 @@ jobs:
# but a version_files entry whose version string no longer matches is
# skipped *silently* — the run stays green while the plugin manifests keep
# advertising the old version, which is exactly the field harnesses use to
# decide whether users get an update. Fail loudly instead.
# decide whether users get an update. Fail loudly instead, before any push.
- name: Verify plugin manifests carry the bumped version
id: version
run: |
set -euo pipefail
project=$(python3 -c 'import tomllib; print(tomllib.load(open("pyproject.toml", "rb"))["project"]["version"])')
portable=$(jq -r '.version' plugin.json)
claude=$(jq -r '.version' .claude-plugin/plugin.json)
codex=$(jq -r '.version' .codex-plugin/plugin.json)

for pair in "claude:${claude}" "codex:${codex}"; do
for pair in "portable:${portable}" "claude:${claude}" "codex:${codex}"; do
name=${pair%%:*}
found=${pair#*:}
if [ "${found}" != "${project}" ]; then
Expand All @@ -139,8 +144,31 @@ jobs:
exit 1
fi

# Guard the Agent Plugins schema identifiers: a botched version rewrite
# must not rewrite the 1.0.0 schema path segment.
for schema_file in plugin.json mcp.json; do
schema=$(jq -r '."$schema"' "${schema_file}")
case "${schema}" in
https://agent-plugins.org/schemas/1.0.0/*) ;;
*)
echo "::error::${schema_file} \$schema drifted to ${schema}."
exit 1
;;
esac
done

echo "version=${project}" >> "${GITHUB_OUTPUT}"

# Publish the local bump commit + vX.Y.Z tag only after verification.
- name: Push bump commit and version tag
if: steps.after.outputs.tag != ''
env:
TAG: ${{ steps.after.outputs.tag }}
run: |
set -euo pipefail
git push origin "HEAD:${{ github.ref_name }}"
git push origin "refs/tags/${TAG}"

# Plugin release tag, in the form `claude plugin tag` produces. Separate
# from commitizen's vX.Y.Z: it marks the commit a harness resolves a plugin
# install to. Derived from the tag commitizen actually created, so the two
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:
- name: Install locked dependencies
run: uv sync --locked
- name: Verify PEP 723 script lockfile
run: uv lock --script bin/run_mcp.py --check
run: uv lock --script runtime/run_mcp.py --check
# Runs every hook in .pre-commit-config.yaml (ruff lint, file hygiene,
# uv-lock) against all files. commit-msg hooks are skipped by --all-files.
- name: Run pre-commit
Expand Down
2 changes: 1 addition & 1 deletion .mcp.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
"--quiet",
"--locked",
"--script",
"./bin/run_mcp.py"
"./runtime/run_mcp.py"
],
"cwd": ".",
"env_vars": [
Expand Down
11 changes: 6 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,16 +16,16 @@ Use this file when changing this repository. It is not an operating guide for us
- `dataiku_mcp/tools/cobuild.py` owns retained Cobuild conversations. Cobuild is the default path for constructing or modifying project-level flow and analytic assets.
- Cobuild owns construction and modification of project-level flow and analytic assets. Direct writes are allowed only for the fixed exception categories defined by the **Cobuild Write-Routing Convention** in `CODING_STANDARDS_AND_STRUCTURE.md`; do not infer permission for a new direct write from existing implementation.
- `skills/dataiku-headless/SKILL.md` is the operator-facing router. Its `references/` directory owns object-specific inspection, mutation, and verification workflows.
- `bin/run_mcp.py` is the entry point used by manifests through `uv run --quiet`. It owns the PEP 723 runtime metadata. Stdout is reserved for MCP JSON-RPC; diagnostics belong on stderr. `bin/launcher.sh` is inactive legacy fallback code retained for possible future use.
- `runtime/run_mcp.py` is the entry point used by manifests through `uv run --quiet`. It owns the PEP 723 runtime metadata. Stdout is reserved for MCP JSON-RPC; diagnostics belong on stderr. `runtime/launcher.sh` is inactive legacy fallback code retained for possible future use.

## Sources of truth

- Registered MCP surface: `tests/test_tool_surface.py`.
- User-facing capability boundary and tool inventory: `docs/capabilities.md`, enforced by `tests/test_capabilities_doc.py`.
- Tool registration imports: `dataiku_mcp/__init__.py`.
- Project dependencies, Python support, version, and CLI entry points: `pyproject.toml`.
- Standalone server dependency pins and Python floor: the PEP 723 block in `bin/run_mcp.py`.
- Plugin launch configuration: `.mcp.json`, `.claude-plugin/plugin.json`, and `.codex-plugin/plugin.json`.
- Standalone server dependency pins and Python floor: the PEP 723 block in `runtime/run_mcp.py`.
- Portable Agent Plugins package: root `plugin.json` + `mcp.json` (skills under `skills/`). Harness-specific launch config: `.mcp.json`, `.claude-plugin/plugin.json`, and `.codex-plugin/plugin.json`.
- User-facing installation and architecture overview: `README.md`.
- Release behavior: `RELEASE.md` and `.github/workflows/bump.yml`.
- CI behavior: `.github/workflows/ci.yml` and `.github/workflows/pr-title.yml`.
Expand All @@ -49,8 +49,8 @@ Do not copy volatile inventories, versions, or command details into this file wh

- Preserve Cobuild conversation ownership by instance and project, one active turn per conversation, exact turn IDs for answers, and recoverable polling after timeouts or cancellation.
- Keep general SDK work and long-running Cobuild calls on their separate executors.
- Keep `bin/launcher.sh` POSIX `/bin/sh` compatible and silent on stdout if modifying its retained legacy fallback behavior.
- When runtime dependencies or the Python floor change, update both `pyproject.toml` and the PEP 723 block in `bin/run_mcp.py`; `tests/test_pep723_launcher.py` enforces their relationship.
- Keep `runtime/launcher.sh` POSIX `/bin/sh` compatible and silent on stdout if modifying its retained legacy fallback behavior.
- When runtime dependencies or the Python floor change, update both `pyproject.toml` and the PEP 723 block in `runtime/run_mcp.py`; `tests/test_pep723_launcher.py` enforces their relationship.
- When manifest behavior changes, inspect every manifest rather than assuming their schemas or path interpolation rules are identical.
- When changing package contents or entry points, build the distributions and smoke-test the wheel as CI does.

Expand Down Expand Up @@ -78,6 +78,7 @@ Useful focused checks include:
```bash
uv run pytest tests/test_tool_surface.py
uv run pytest tests/test_pep723_launcher.py
uv run pytest tests/test_agent_plugins.py
uv run pytest tests/test_cobuild.py
```

Expand Down
9 changes: 5 additions & 4 deletions CODING_STANDARDS_AND_STRUCTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ export DKU_API_KEY="your-api-key"
| Workflow prompts | `dataiku_mcp/prompts/workflows.py` |
| Project/dataset/folder/recipe/ML skills | `skills/**/SKILL.md` |
| Cobuild conversation tools | `dataiku_mcp/tools/cobuild.py` |
| Portable Agent Plugins package | root `plugin.json` + `mcp.json` (keep harness manifests in sync for MCP launch) |

## Error Handling
- Prefer simple, readable tool handlers: keep top-level control flow short, avoid repeated Dataiku lookups, and use local helpers only when they improve clarity.
Expand Down Expand Up @@ -97,14 +98,14 @@ PYTHONPYCACHEPREFIX=/tmp/pycache uv run python -m py_compile $(find dataiku_mcp
Run the MCP server locally to verify end-to-end:

```bash
uv run --quiet --locked --script ./bin/run_mcp.py # exactly what every manifest runs
uv run --quiet --locked --script ./runtime/run_mcp.py # exactly what every manifest runs
```

`uv` 0.12.0 or later is a runtime prerequisite for the plugin. **`bin/run_mcp.py`** is the server entry point: its [PEP 723](https://peps.python.org/pep-0723/) inline metadata declares pinned dependencies and `requires-python`, so uv creates an isolated cached environment without a project install. `dataiku_mcp` is imported from the working tree, so source edits take effect immediately, while local edits to dependencies do not.
`uv` 0.12.0 or later is a runtime prerequisite for the plugin. **`runtime/run_mcp.py`** is the server entry point: its [PEP 723](https://peps.python.org/pep-0723/) inline metadata declares pinned dependencies and `requires-python`, so uv creates an isolated cached environment without a project install. `dataiku_mcp` is imported from the working tree, so source edits take effect immediately, while local edits to dependencies do not.

**`bin/launcher.sh`** is inactive legacy code retained for possible future fallback use. No manifest invokes it; do not re-enable it without explicitly reviewing the platform behavior and updating all manifests.
**`runtime/launcher.sh`** is inactive legacy code retained for possible future fallback use. No manifest invokes it; do not re-enable it without explicitly reviewing the platform behavior and updating all manifests.

The inline metadata and its adjacent `bin/run_mcp.py.lock` resolve independently of the project `uv.lock`. The `==` pins are the direct dependency constraints for plugin launches; the script lock records the complete direct and transitive resolution. Bump direct pins deliberately, then regenerate and commit the script lock with `uv lock --script bin/run_mcp.py`. Every launcher uses `--locked`, so a stale or absent script lock fails before server startup rather than resolving on a user's machine.
The inline metadata and its adjacent `runtime/run_mcp.py.lock` resolve independently of the project `uv.lock`. The `==` pins are the direct dependency constraints for plugin launches; the script lock records the complete direct and transitive resolution. Bump direct pins deliberately, then regenerate and commit the script lock with `uv lock --script runtime/run_mcp.py`. Every launcher uses `--locked`, so a stale or absent script lock fails before server startup rather than resolving on a user's machine.

The inline dependency list duplicates `[project].dependencies`; `tests/test_pep723_launcher.py` fails if the two drift apart.

Expand Down
30 changes: 20 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@

Dataiku Headless is an MCP server with tools for working in Dataiku, plus skills that teach AI assistants how to use them. Connect it to a Dataiku instance, and your AI assistant can build data pipelines, models, dashboards, agents, and more.

Install it from the [Claude Code](#claude-code-cli) or [Codex](#codex-cli) plugin marketplace, or install it as an agent plugin from this GitHub repository for Cursor, Snowflake CoCo, AWS Kiro, OpenCode, and more.
Install it from the [Claude Code](#claude-code-cli) or [Codex](#codex-cli) plugin marketplace. The repository also ships an [Agent Plugins](https://agent-plugins.org/) v1.0.0 package for Cursor and other conforming clients.

## Requirements

Expand All @@ -47,7 +47,7 @@ Here, we use the Claude Code CLI to build a visual pipeline to clean up hospital

## Install with another agent

`dataiku-headless` also works with Snowflake CoCo (Cortex Code), Cursor, OpenCode, and custom MCP-compatible agents. Each plugin starts the same local MCP server; after installation, use the same setup flow above.
`dataiku-headless` also works with Cursor, OpenCode, and other Agent Plugins-compatible clients. The portable package uses the root `plugin.json`, root `mcp.json`, and the shared `skills/` directory; dedicated manifests remain for Claude Code and Codex.

> **First launch:** If Dataiku Headless tools are unavailable, first check that `uv` is installed and on your `PATH`:
>
Expand All @@ -64,6 +64,12 @@ codex plugin marketplace add https://github.com/dataiku/dataiku-headless.git
codex plugin add dataiku-headless@dataiku
```

### Agent Plugins (portable)

This repository is an [Agent Plugins](https://agent-plugins.org/) v1.0.0 package: root `plugin.json`, root `mcp.json`, and Agent Skills under `skills/`. Any client that implements the standard can load the portable core directly from this directory.

Harness-specific manifests (`.claude-plugin/`, `.codex-plugin/`, …) remain for install paths those clients already support. They are additive compatibility layers; the portable files are the cross-client floor.

### Claude Code CLI

```bash
Expand All @@ -77,12 +83,12 @@ claude plugin install dataiku-headless@dataiku
grok plugin install dataiku/dataiku-headless --trust
```

### Cursor Agent CLI
### Cursor

```bash
cursor agent plugin marketplace add github.com/dataiku/dataiku-headless
# Tip: use /plugins in interactive mode to install `dataiku-headless` plugin from this marketplace.
```
Open **Customize** in the Cursor sidebar, add this GitHub repository as a
plugin source, then install `dataiku-headless` at your preferred user or project
scope. Cursor detects the root Agent Plugins manifest and loads the bundled
skills and MCP server.

### Snowflake CoCo

Expand All @@ -101,13 +107,15 @@ Add the following to your `.mcp.json` from a checkout of this repository:
"mcp": {
"dataiku": {
"type": "local",
"command": ["uv", "run", "--quiet", "--locked", "--script", "./bin/run_mcp.py"],
"command": ["uv", "run", "--quiet", "--locked", "--script", "./runtime/run_mcp.py"],
"enabled": true
}
}
}
```

Portable Agent Plugins clients read root `mcp.json` instead. It launches the same locked `uv` script entry point as the existing manifests.

#### Skills

The `skills/*/SKILL.md` files follow the universal skill format:
Expand Down Expand Up @@ -193,7 +201,7 @@ Auth resolution order:
Every install path above has your harness launch the server itself. Run it standalone only if you're testing it directly — from a clone of this repo:

```bash
uv run --quiet --locked --script ./bin/run_mcp.py # same command the plugin manifests use
uv run --quiet --locked --script ./runtime/run_mcp.py # same command the plugin manifests use
```

## Project Structure
Expand Down Expand Up @@ -252,10 +260,12 @@ uv run --quiet --locked --script ./bin/run_mcp.py # same command the plugin ma
│ ├── agents.md # Agent and agent-tool inspection
│ ├── ... # Additional references for dashboards, insights, scenarios, wikis, migrations, and more
│ └── recipes/ # Nested recipe-family and shared recipe references
├── bin/
├── runtime/
│ ├── launcher.sh # Inactive legacy fallback retained for possible future use
│ ├── run_mcp.py # Server entry point: PEP 723 script pinning the runtime deps inline
│ └── run_mcp.py.lock # Committed, full dependency resolution for the entry point
├── plugin.json # Agent Plugins v1.0.0 portable manifest
├── mcp.json # Agent Plugins portable stdio MCP config
├── .claude-plugin/
│ ├── plugin.json # Claude Code plugin manifest (skills + unconfigured stdio MCP)
│ └── marketplace.json # Marketplace catalog (single-plugin, source: "./")
Expand Down
17 changes: 10 additions & 7 deletions RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,14 @@ artifacts:
| GitHub release | The published, browsable release notes |

The version number still matters even without an index: Commitizen keeps it in
lockstep across `pyproject.toml` and the three plugin manifests, and the manifest
version is how a harness notices there's a newer plugin to install. `bump.yml`
verifies that lockstep held before it tags anything — a `version_files` entry
whose version string stops matching is skipped *silently* by Commitizen, which
would otherwise ship a release whose manifests still advertise the old version.
lockstep across `pyproject.toml` and the plugin manifests (portable Agent Plugins
`plugin.json`, plus the Claude Code and Codex compatibility manifests), and the
manifest version is how a harness notices there's a newer plugin to install.
`bump.yml` bumps locally first (`push: false`), verifies that lockstep held
across all manifests (and that Agent Plugins `$schema` URLs were not rewritten),
and only then pushes the bump commit and tags — a `version_files` entry whose
version string stops matching is skipped *silently* by Commitizen, which would
otherwise ship a release whose manifests still advertise the old version.

---

Expand All @@ -48,9 +51,9 @@ PR and is the gate before anything merges to `main`.
Before Commitizen can create a version, `bump.yml` also verifies the PEP 723
script lockfile is valid and refreshes it in the disposable runner. If the
latest allowed transitive dependency resolution differs from
`bin/run_mcp.py.lock`, the workflow fails before changing the version or
`runtime/run_mcp.py.lock`, the workflow fails before changing the version or
creating tags. Regenerate the lock, run the normal checks, and commit it in a
PR; direct dependencies remain deliberately pinned in `bin/run_mcp.py`.
PR; direct dependencies remain deliberately pinned in `runtime/run_mcp.py`.

### Workflow files

Expand Down
17 changes: 17 additions & 0 deletions mcp.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"$schema": "https://agent-plugins.org/schemas/1.0.0/mcp.schema.json",
"mcpServers": {
"dataiku": {
"type": "stdio",
"command": "uv",
"args": [
"run",
"--quiet",
"--locked",
"--script",
"${PLUGIN_ROOT}/runtime/run_mcp.py"
],
"cwd": "${PLUGIN_ROOT}"
}
}
}
14 changes: 14 additions & 0 deletions plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json",
"name": "dataiku-headless",
"version": "0.6.0",
"description": "Connect your agent to Dataiku: inspect projects, datasets, recipes, ML, and agents with typed MCP tools, and drive Dataiku Cobuild to build project-level assets.",
"author": {
"name": "Dataiku",
"url": "https://www.dataiku.com/"
},
"homepage": "https://github.com/dataiku/dataiku-headless",
"repository": "https://github.com/dataiku/dataiku-headless",
"license": "Apache-2.0",
"keywords": ["dataiku", "cobuild", "mcp", "agent"]
}
Loading