Skip to content

Write a skill in the conversation, not by retyping it into a form - #316

Draft
jerelvelarde wants to merge 1 commit into
CopilotKit:mainfrom
jerelvelarde:jerel/skill-creator-in-chat-upstream
Draft

Write a skill in the conversation, not by retyping it into a form#316
jerelvelarde wants to merge 1 commit into
CopilotKit:mainfrom
jerelvelarde:jerel/skill-creator-in-chat-upstream

Conversation

@jerelvelarde

Copy link
Copy Markdown
Contributor

Recording to follow. Every other claim below is verified. The card is a browser surface and
deserves a recording; the machine this was written on cannot boot the runtime (no licence token, no
model key), and the browser session available to me clips frame capture after the first input event.
Draft until that is attached rather than opening with a still.

The problem

A skill is the unit of reuse here: four fields, a / command, and an instruction a Bot follows. The
hard field is the instruction, because it is the one that decides whether the skill works — and the
only place to write it is a textarea on /skills.

That is backwards. The person best placed to draft that instruction is the Bot in the same tab: it
knows which tools it holds, it has just done the work three times, and it can be asked what should
happen when a step turns up nothing. What happens instead is that somebody has the conversation, gets
a good draft in the transcript, then copies it out and retypes it into a form. Mostly they do not
bother, which is why a deployment can run for months with no skills in it.

examples/fintech/skills.yaml already makes this argument one step earlier. It says the tool
declarations ship with the package rather than being left to a screen, because left to a screen the
narrowing "is switched off on every clone until somebody sits down and maps tools to skills by hand".
A screen is where skill authoring goes to not happen. The same is true of the skill itself.

The approach

The example package ships a deployment skill, skill-creator, whose instruction is how to interview
somebody about the skill they want: capture what it is for, interview until you could do the task
yourself, look before you name, write it, declare the tools, rehearse it against one realistic
request, save it, and say what is left. agents.yaml grants it to general-assistant, and a Bot
holding it is offered four tools the app registers — list_skills, read_skill, list_skill_tools,
and save_skill.

The write is a card, not a handler. save_skill suspends the run and draws the command, the
title, the declared refs and the whole instruction, and writes nothing until a button is pressed. Two
reasons, neither about permission: this is a named thing that will appear in everybody's / menu with
somebody's name on it, and saving is how an edit is spelled here, so an unattended save can replace a
skill somebody is already using.

They run in the browser, as the signed-in person. Whether somebody may take a slug is a question
about them, which POST /api/plugins/skills already answers — their own, an administrator's for the
deployment, a refusal naming the slug otherwise. Riding that endpoint inherits the ownership rules and
the configuration.changed audit row instead of growing a second copy of them. Server-side, the tool
would have to carry an actor into runs that do not have one — a routine, a Slack thread, a schedule —
and the first way that goes wrong is a skill written under the wrong name.

The grant is the gate. The four tools appear only while the Bot holds skill-creator. Four extra
tools on every run costs the per-run narrowing more than it buys, and a Bot for looking up
transactions has no business drafting skills.

What it costs, said plainly. It works only where there is a browser, so no Slack thread and no
schedule — acceptable rather than merely tolerated, because authoring is an interview and there is
nobody to interview in an unattended run. And the app now knows one slug by name,
SKILL_CREATOR_SLUG, a coupling that breaks silently in both directions: rename it in the YAML and
the tools are never offered, grant it to nobody and there is no Bot to write with. Both look like a
healthy deployment, which is why skill-creator-slug.test.ts exists.

What is not covered

  • No grant from the card. A saved skill is on no Bot, so nobody can invoke it yet. Granting is a
    decision about a Bot with its own ownership rules, so it stays on /skills; the card links there
    and the instruction tells the Bot to say so.
  • Personal skills only. The tool never sends global, so every skill written this way is the
    author's. An administrator writing for the whole deployment still uses the form.
  • No new audit event. A skill written in a conversation lands the same configuration.changed /
    skill_installed row as one typed into the form. The trail does not distinguish a skill a Bot
    drafted from one somebody wrote by hand; skills.origin is where that would go if it is wanted.
  • The card cannot repair a draft. A proposal the fields refuse is answered back to the model with
    the problems named by field, and the person is never asked.
  • No recording yet. See the note at the top.

Verification

Baseline on upstream/main at b85f361, same worktree and same database: 2147 pass, 0 fail. This
branch: 2180 pass, 0 fail under bun run test:ci — exactly the 33 cases added below.
bun run typecheck clean across all four workspaces, bun run lint clean over 517 files,
bun run --filter app build clean, and drizzle-kit check reports no snapshot problems (this branch
adds no migration).

The loop has also been driven on a live internal deployment, which is the only way the model half can
be exercised: /skill-creator, one request, and the Bot called list_skills before naming anything,
drew the card, saved on the press, showed the "put it on a Bot" link, and the audit trail carried
configuration.changed on the new slug.

New tests:

  • app/tests/skill-proposal.test.ts — a model's proposal held to the same schema the form uses:
    problems named by field, a lone tool string read as one tool, and the sentence the card answers with
    pinned against the reader that decides whether to offer the follow-up link.
  • app/tests/proposed-skill-card.test.tsx — the card's states, that a server refusal leaves the run
    suspended rather than ending the turn, and that an invalid draft answers exactly once across
    re-renders. Follows button-native.test.tsx: the window is installed and removed around the file
    and queries come off render()'s return, so the suite can be walked in any order.
  • app/tests/skill-creator-slug.test.ts — the app's gate slug against what the package ships and
    grants.
  • server/tests/tenant-package.test.ts — extended for the shipped skill and the pairing that makes it
    reachable. It previously pinned general-assistant at skills: [].

Merge notes

app/src/lib/copilot/provider.tsx gains one mounted component, so any branch adding a tool component
there conflicts on the same three lines. server/tests/tenant-package.test.ts now expects
skills: ["skill-creator"] for general-assistant, so a branch also editing
examples/fintech/agents.yaml must take both halves together.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant