Skip to content

Make a coworker by talking to one, and grant it nothing - #331

Open
jerelvelarde wants to merge 1 commit into
CopilotKit:mainfrom
jerelvelarde:jerel/bot-creator-in-chat
Open

Make a coworker by talking to one, and grant it nothing#331
jerelvelarde wants to merge 1 commit into
CopilotKit:mainfrom
jerelvelarde:jerel/bot-creator-in-chat

Conversation

@jerelvelarde

@jerelvelarde jerelvelarde commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Making a coworker by talking to one

A vague request, an interview that asks rather than assumes, a card carrying the whole instruction the
coworker will run on, and then a conversation with the thing that was just made — which correctly says
it can reach nothing. Full size, as video:
bot-creator.mp4

Rebuilt on current main as one commit, 14 files. It no longer stacks on
#299 and depends on nothing unmerged.

The problem

#317 replaced the create form with a questionnaire, and that is a better form: it asks one thing at a
time and explains itself. What it cannot do is the part this addresses. A questionnaire asks a fixed
set of questions, so it cannot ask the follow-up that the answer to the last one calls for; it cannot
read the roster and say "there is already a Ticket Triage that does this"; and it cannot be told "one
like Ticket Triage, but for renewals" and go and read what that coworker actually runs on.

Underneath both surfaces is the same hard field. role_description is not a label — with no endpoint
it becomes the standing instruction handed to a model on every turn in every channel that coworker is
in. The three things that make one useful are the three nobody volunteers: what it must never
conclude, what it says when the evidence is thin, and whether the job is already covered. A form of
any shape has to ask for those up front and accept whatever comes back. A conversation can push.

The approach

A bot-creator skill, and the grant is the gate. The package ships it and grants it to
general-assistant. A Bot holding it is offered list_bots, read_bot, list_bot_skills and
save_bot; every other Bot is offered none of them, because four more tools on every run costs the
per-run narrowing more than it buys and a Bot for looking up transactions has no business making
coworkers.

Browser-side, so nothing is re-implemented. POST /api/agents already answers who may create a
coworker and writes bot.created with the actor on it; POST /api/plugins/grants puts a skill on
one. A tool riding the signed-in session inherits both unchanged. A server-side tool would have to
carry an actor into runs that do not always have one — a routine, a Slack thread, a schedule — and the
first way that goes wrong is a coworker created under a name nobody chose. Nothing is lost by it
living in the browser: making a coworker is an interview, so there is nobody to interview where there
is no browser.

The card is the whole tool. save_bot has no handler: the run suspends, and the card shows the
name, the job, the skills and the entire role description, scrolled rather than clamped. That text
runs on somebody's behalf, so a clamp would hide half of what they are agreeing to from the one person
being asked to agree to it.

It grants skills and nothing else. No address, no connector, no tool, no boundary, no visibility.
The coworker arrives private, holding what was agreed, and everything it might reach is granted
afterwards on its profile by somebody who may. A card that could grant capability would make "ask the
Bot for it" the fastest route around the screens that exist to decide it.

The create path grows a system prompt, without which the interview would have to end by asking
somebody for a URL. store.create writes a built_in coworker running on its role description where
there is neither an endpoint nor a Bot in the box — which on the one-container image the README
recommends is every coworker — and the route passes that text when no endpoint was given. Give an
address and it is a remote Bot exactly as before.

store.update had to move with it, and this is the half worth reading. A built_in coworker is
given its configuration.systemPrompt and no standing role message, so that column is the whole of
what it is ever told: agentProfiles.roleDescription never reaches it. Update wrote only the profile,
so editing such a coworker changed what every screen shows and nothing the Bot follows — permanently,
with nothing anywhere to say so, which is the same shape as the endpoint bug the code above it already
warns about. It now carries the text into the prompt for built_in rows and only those. A remote Bot
must not acquire a prompt it never had, and the tenant package's own Bots — whose system_prompt is
deliberately not their role_description — cannot reach update at all, because requireManageable
throws ProtectedAgentError for anything the package owns.

Creating and granting are two calls and the pair is not atomic, so a grant that fails after the
coworker exists is named in the answer rather than rounded up. Rolling the coworker back would delete
something the person just watched being made over a skill they can add in two clicks.

What is not covered

  • It does not replace Make and manage coworkers in dialogs #317's dialog, and should not. Somebody who knows what they want should fill
    in four fields, not conduct an interview. This is the other end of that range.
  • No editing. The card creates; it never replaces. Changing a coworker stays on the profile: a
    coworker is a thing people have been talking to, and an unattended overwrite is not undoable.
  • No test drives a real conversation. The suite covers the card, the answers and the package
    coupling. The leg from interview to created coworker needs a model and a platform, so it is verified
    by hand against a running deployment.
  • No provenance. The trail cannot tell a coworker a Bot drafted from one somebody typed into the
    dialog. bot.created records the actor either way, which is the fact that matters for permission
    and not the one that matters for review.
  • The interview is only as good as the model. The skill instructs one question at a time; a
    complete brief goes straight to the card, which is right, and a vague one may take several turns.
    Nothing enforces a number.

Verification

format:check, lint and typecheck clean. bun run test:ci reports 2167 pass, 23 skip, 0 fail
across 179 files
, with 17 cases in the three new files and three more in the profile store's.

Two existing tests were updated rather than left. server/tests/agent-routes.test.ts asserts the exact
object handed to store.create, which now carries the prompt on the no-endpoint path and still does
not on update. server/tests/tenant-package.test.ts pins the shipped roster, and it is what caught
the grant landing on Knowledge instead of the general assistant.

Driven end to end against a running deployment, which is how both defects below were found: a vague
request, an interview, a card, Create it, and then a conversation with the coworker that now exists.
Afterwards the database holds bot.created for the agent and configuration.changed for each grant,
and the coworker answers a question about a document by naming the terms it would have searched and
saying it has no source — a coworker granted nothing, saying so.

Two defects the tests would not have found, because both were about state rather than logic. The
first conversation this feature ever had answered "No skills exist here yet" to a deployment holding
nine: the model called list_bot_skills in the first second of the run, the handler read a useQuery
result that had not come back, and an empty list and an unloaded list were sharing a code path. The
Bot then told the person that as a fact about their deployment. The answers now fetch when they are
asked, and live in a module a test can call against a query client nothing has warmed. Second, the
link on a completed card lived in component state — so it would have drawn once for the person who
pressed the button and silently stopped for anybody who reloaded, since a card is re-rendered from the
transcript with the arguments and the answer and nothing else. The coworker's id travels in the answer
now, read back by a function paired with the sentence and pinned by a test.

File What it holds down
app/tests/proposed-bot-card.test.tsx The whole role description unclipped; that rendering writes nothing; a partial grant named rather than rounded up; a refusal leaving the run open; the completed card's link rebuilt from the answer alone
app/tests/bot-tool-answers.test.ts Every answer fetched against a cold query client, so an unloaded list can never read as an empty one — and a genuinely empty deployment still says it is empty
app/tests/bot-creator-slug.test.ts That the package ships the slug the app gates on, that a Bot holds it, and that it declares no tools
server/tests/agent-profile-store.integration.test.ts That a coworker with nowhere to send it is stored built_in on its role description, that editing one moves the instruction it actually runs on, and that a coworker at its own address never acquires a prompt

The card test follows this repository's DOM harness — register in beforeAll, unregister in
afterAll, queries off render() rather than the global screen. It did not at first, and the
symptom is worth knowing: the file passed alone and every case in it failed in a full run, because a
neighbour's afterAll unregistered the document underneath it.

Merge notes

server/src/agents/profile-store.ts and the create handler in server/src/agents/routes.ts carry the
same in-process binding as #299 — that PR needs it for importing a template, this one needs it for the
interview, and whichever lands second should drop those hunks. Nothing else here overlaps.

One asymmetry worth carrying across: #299 has the create half without the update half, so a
coworker imported from a template there has the same silent divergence until this PR's update change
lands with it. Whichever merges first, that hunk should go in.

examples/fintech/agents.yaml gains a skills: block on general-assistant, which is the gate for
the whole feature: bot-creator-slug.test.ts and the tenant-package assertion both fail loudly if it
is dropped, because without them removing that line breaks making a coworker in a conversation with
nothing on any screen to say why.

Making a coworker means answering a form, and the field that decides
everything is the one a form is worst at. `role_description` is not a
label: with no endpoint it becomes the standing instruction handed to a
model on every turn in every channel that coworker is in. People write a
sentence, get a coworker that answers vaguely, and never go back.

The three questions that make one useful are the ones nobody answers
unprompted — what it must never conclude, what it says when the evidence
is thin, and whether something here already does the job. A form cannot
ask a follow-up. A conversation can.

So the deployment ships a `bot-creator` skill whose instruction is how to
run that interview, and a Bot holding it is offered four tools:
`list_bots`, `read_bot`, `list_bot_skills` and `save_bot`. The grant is
the gate rather than a flag — four more tools on every run costs the
per-run narrowing more than it buys, and a Bot for looking up
transactions has no business making coworkers.

The tools ride the signed-in session from the browser, so `POST
/api/agents` answers who may and writes `bot.created` with the actor on
it, unchanged. A server-side tool would have to carry an actor into runs
that have none — a routine, a Slack thread, a schedule — and the first
way that goes wrong is a coworker created under a name nobody chose.

The card is the whole tool; there is no handler behind it. It shows the
entire role description, unclipped, because that text will run on
somebody's behalf and a clamp would hide half of what they are agreeing
to. It grants skills and nothing else: no address, no connector, no tool,
no boundary, no visibility. A card that could grant capability would make
asking a Bot the fastest route around the screens that decide it.

The create path grows a system prompt, because without it the interview
would have to end by asking somebody for a URL. `store.create` writes a
`built_in` coworker on its role description where there is neither an
endpoint nor a Bot in the box — which on the one-container image is every
coworker — and the route passes that text when no endpoint was given.

`store.update` had to move with it, and this is the half worth reading.
A `built_in` coworker is given its `configuration.systemPrompt` and no
standing role message, so that column is the whole of what it is ever
told and `agentProfiles.roleDescription` never reaches it. Update wrote
only the profile, so editing such a coworker changed what every screen
showed and nothing the Bot followed — permanently, silently. It now
carries the text into the prompt for `built_in` rows and only those: a
remote Bot must not acquire a prompt it never had, and the package's own
Bots, whose `system_prompt` is deliberately not their `role_description`,
cannot reach `update` at all because `requireManageable` refuses anything
the package owns.

Two defects found by using it, both about state rather than logic. The
first conversation answered "No skills exist here yet" to a deployment
holding nine, because the handler read a query that had not loaded; an
empty list and an unloaded list were sharing a code path, and the Bot
repeated the tidier one as fact. The answers now fetch when asked, from a
module a test calls against a cold client. And the link on a completed
card lived in component state, so it would have drawn once and stopped
for anybody who reloaded — a card is re-rendered from the transcript with
the arguments and the answer and nothing else. The coworker's id travels
in the answer now, read back by a function paired with the sentence.

Verification: 2167 pass, 23 skip, 0 fail across 179 files; format, lint
and typecheck clean. The card test follows this repository's DOM harness
— register in `beforeAll`, unregister in `afterAll`, queries off
`render()` — because a file that registers at module scope and queries
the global `screen` passes alone and fails every case in a full run.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant