Write a skill in the conversation, not by retyping it into a form - #316
Draft
jerelvelarde wants to merge 1 commit into
Draft
Write a skill in the conversation, not by retyping it into a form#316jerelvelarde wants to merge 1 commit into
jerelvelarde wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
A skill is the unit of reuse here: four fields, a
/command, and an instruction a Bot follows. Thehard field is the instruction, because it is the one that decides whether the skill works — and the
only place to write it is a textarea on
/skills.That is backwards. The person best placed to draft that instruction is the Bot in the same tab: it
knows which tools it holds, it has just done the work three times, and it can be asked what should
happen when a step turns up nothing. What happens instead is that somebody has the conversation, gets
a good draft in the transcript, then copies it out and retypes it into a form. Mostly they do not
bother, which is why a deployment can run for months with no skills in it.
examples/fintech/skills.yamlalready makes this argument one step earlier. It says the tooldeclarations ship with the package rather than being left to a screen, because left to a screen the
narrowing "is switched off on every clone until somebody sits down and maps tools to skills by hand".
A screen is where skill authoring goes to not happen. The same is true of the skill itself.
The approach
The example package ships a deployment skill,
skill-creator, whose instruction is how to interviewsomebody about the skill they want: capture what it is for, interview until you could do the task
yourself, look before you name, write it, declare the tools, rehearse it against one realistic
request, save it, and say what is left.
agents.yamlgrants it togeneral-assistant, and a Botholding it is offered four tools the app registers —
list_skills,read_skill,list_skill_tools,and
save_skill.The write is a card, not a handler.
save_skillsuspends the run and draws the command, thetitle, the declared refs and the whole instruction, and writes nothing until a button is pressed. Two
reasons, neither about permission: this is a named thing that will appear in everybody's
/menu withsomebody's name on it, and saving is how an edit is spelled here, so an unattended save can replace a
skill somebody is already using.
They run in the browser, as the signed-in person. Whether somebody may take a slug is a question
about them, which
POST /api/plugins/skillsalready answers — their own, an administrator's for thedeployment, a refusal naming the slug otherwise. Riding that endpoint inherits the ownership rules and
the
configuration.changedaudit row instead of growing a second copy of them. Server-side, the toolwould have to carry an actor into runs that do not have one — a routine, a Slack thread, a schedule —
and the first way that goes wrong is a skill written under the wrong name.
The grant is the gate. The four tools appear only while the Bot holds
skill-creator. Four extratools on every run costs the per-run narrowing more than it buys, and a Bot for looking up
transactions has no business drafting skills.
What it costs, said plainly. It works only where there is a browser, so no Slack thread and no
schedule — acceptable rather than merely tolerated, because authoring is an interview and there is
nobody to interview in an unattended run. And the app now knows one slug by name,
SKILL_CREATOR_SLUG, a coupling that breaks silently in both directions: rename it in the YAML andthe tools are never offered, grant it to nobody and there is no Bot to write with. Both look like a
healthy deployment, which is why
skill-creator-slug.test.tsexists.What is not covered
decision about a Bot with its own ownership rules, so it stays on
/skills; the card links thereand the instruction tells the Bot to say so.
global, so every skill written this way is theauthor's. An administrator writing for the whole deployment still uses the form.
configuration.changed/skill_installedrow as one typed into the form. The trail does not distinguish a skill a Botdrafted from one somebody wrote by hand;
skills.originis where that would go if it is wanted.the problems named by field, and the person is never asked.
Verification
Baseline on
upstream/mainatb85f361, same worktree and same database: 2147 pass, 0 fail. Thisbranch: 2180 pass, 0 fail under
bun run test:ci— exactly the 33 cases added below.bun run typecheckclean across all four workspaces,bun run lintclean over 517 files,bun run --filter app buildclean, anddrizzle-kit checkreports no snapshot problems (this branchadds no migration).
The loop has also been driven on a live internal deployment, which is the only way the model half can
be exercised:
/skill-creator, one request, and the Bot calledlist_skillsbefore naming anything,drew the card, saved on the press, showed the "put it on a Bot" link, and the audit trail carried
configuration.changedon the new slug.New tests:
app/tests/skill-proposal.test.ts— a model's proposal held to the same schema the form uses:problems named by field, a lone tool string read as one tool, and the sentence the card answers with
pinned against the reader that decides whether to offer the follow-up link.
app/tests/proposed-skill-card.test.tsx— the card's states, that a server refusal leaves the runsuspended rather than ending the turn, and that an invalid draft answers exactly once across
re-renders. Follows
button-native.test.tsx: the window is installed and removed around the fileand queries come off
render()'s return, so the suite can be walked in any order.app/tests/skill-creator-slug.test.ts— the app's gate slug against what the package ships andgrants.
server/tests/tenant-package.test.ts— extended for the shipped skill and the pairing that makes itreachable. It previously pinned
general-assistantatskills: [].Merge notes
app/src/lib/copilot/provider.tsxgains one mounted component, so any branch adding a tool componentthere conflicts on the same three lines.
server/tests/tenant-package.test.tsnow expectsskills: ["skill-creator"]forgeneral-assistant, so a branch also editingexamples/fintech/agents.yamlmust take both halves together.