Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion .github/workflows/agent-workflow-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ jobs:
env:
GH_TOKEN: ${{ github.token }}
run: >-
gh aw compile backlog-assessment discussion-draft --strict --no-check-update --no-logo
gh aw compile backlog-assessment discussion-draft issue-implementation --strict --no-check-update --no-logo
--action-tag 32ecafbc061c9af57e9361942099e381102ae304
- name: Check generated files are current
run: |
Expand All @@ -69,3 +69,18 @@ jobs:
run: |
npm ci --ignore-scripts
npm test
implementation-sandbox:
name: Verify the credential-free implementation test container
runs-on: ubuntu-latest
timeout-minutes: 90
steps:
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
with:
persist-credentials: false
- uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
with:
node-version-file: .nvmrc
- name: Run real isolated TypeScript and Python checks on a synthetic patch
env:
TYPECHAT_SANDBOX_SMOKE: "true"
run: node --test tools/tests/implementation-sandbox.integration.test.mjs
2,367 changes: 2,367 additions & 0 deletions .github/workflows/issue-implementation.lock.yml

Large diffs are not rendered by default.

596 changes: 596 additions & 0 deletions .github/workflows/issue-implementation.md

Large diffs are not rendered by default.

145 changes: 139 additions & 6 deletions AUTOMATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -457,14 +457,15 @@ its model setting does not configure GitHub Copilot PR reviews.
### Validation and rollout

Changes to the workflow are checked by compiling its Markdown source with
gh-aw **v0.93.5**. Maintainers should regenerate and commit both the
gh-aw **v0.93.5**. Maintainers should regenerate and commit the
[compiled assessment](./.github/workflows/backlog-assessment.lock.yml) and
[compiled discussion drafter](./.github/workflows/discussion-draft.lock.yml), with
[compiled discussion drafter](./.github/workflows/discussion-draft.lock.yml), plus
the [compiled implementation pilot](./.github/workflows/issue-implementation.lock.yml), with
[compiler action pins](./.github/aw/actions-lock.json) together, rather than
edit the generated workflow directly:

```shell
gh aw compile backlog-assessment discussion-draft --strict --no-check-update --no-logo --action-tag 32ecafbc061c9af57e9361942099e381102ae304
gh aw compile backlog-assessment discussion-draft issue-implementation --strict --no-check-update --no-logo --action-tag 32ecafbc061c9af57e9361942099e381102ae304
```

The action reference above pins the v0.93.5 runtime. Keep the compiler pin file
Expand Down Expand Up @@ -774,9 +775,11 @@ assessment's GPT-6.1-sol/max configuration, restricted tools, 1,000-call limit,
60-minute execution limit, and 100-minute enclosing job. The backlog assessment
keeps its **400-credit** main budget. Detection retains its **separate
400-credit default**, so 50 credits is not a combined workflow cap.
Publishing a response or acceptance invokes no model. A **1,000-credit implementation budget** is a
future owner-approved implementation-stage setting only; this pilot enables
neither that spending nor a build command.
Publishing a response or acceptance invokes no model. The separate
[implementation pilot](#third-pilot-owner-triggered-issue-implementation)
configures a **1,000-credit main budget** and **400-credit detection budget**,
but remains disabled until later owner activation. Discussion commands do not
activate either allowance.

Context, candidate, scan-binding, and sealed draft artifacts are retained for seven days.
Approval validity ends at the earlier of artifact expiry or seven days after
Expand Down Expand Up @@ -875,6 +878,136 @@ We aim to keep the entire process on GitHub:
Accepting a proposal is not the same as accepting its implementation. Both
decisions should be clear and visible.

## Third pilot: owner-triggered Issue implementation

**Prepared for review; disabled and not live-verified.** The
[accepted agreement for #504](https://github.com/microsoft/TypeChat/issues/504#issuecomment-6562465109)
authorizes development of this workflow, not an automated coding run. The new
[workflow](./.github/workflows/issue-implementation.md) requires
`TYPECHAT_IMPLEMENTATION_ENABLED` to equal `true` and a fresh owner Issue comment:

```text
/typechat build <acceptance-comment-id>:<acceptance-sha256>
```

The identifier names the durable public acceptance comment, not an expiring
draft artifact. A manual dispatch of **TypeChat accepted Issue implementation**
with `issue` and `acceptance` inputs verifies the record and prints a copyable
command without model execution or App-token creation. Only the exact canonical,
unchanged Actions-bot record, backed by an unchanged prior owner acceptance
command with the same hash, is supported. Ordinary manual acceptance remains
valid human governance but is not machine-readable authority for this pilot.
Discussion builds and inline notes remain unsupported; #502 is still deferred.

### Frozen authorization and bounded spending

Preflight verifies owner identity, default-branch workflow code, the current
scope policy and its ancestry, acceptance, and existing contributions. Open
linked or possibly competing PRs stop the run, including ambiguous references.
The agreement, discussion, exact source revision, control fingerprints, and
short-lived App-access proof are frozen. An immutable spending reservation
precedes inference. The model-start guard repeats authorization and rejects App
preflight older than 15 minutes. A build command expires after 24 hours.

Detection is eligible only after successful preparation and successful, sole-`noop`
agent completion. Its inference credential is released only after the trusted
authorization and exact-patch staging step succeeds and preceding job steps
have not failed or been cancelled. This explicit credential
gate is necessary because the pinned runtime can still launch detector setup
after a failed custom pre-step; that failed step must not grant inference access.

One execution permits at most **1,000 main-agent AI credits** and **400 separate
detection credits**, not a combined 1,000-credit allowance. The main agent uses
the established GPT-6.1-sol/max configuration, at most 1,000 calls, 60 minutes
of execution, and a 100-minute enclosing job. Detection has a 30-minute enclosing
job; deterministic validation and publication are also time-bounded. Native
usage outputs must be present, positive, and within their independent limits.
Incomplete, capped, failed, missing, or mixed native completion signals cannot
publish. Runtime budget overrides are not enabled.

Re-running a command cannot spend again. A confirmed PR is returned without
coding or writes; otherwise a reserved or partial run stops for inspection.
A genuinely new attempt after a failed coding run needs a fresh owner command.
An unresolved publication intent or branch still blocks it. Source commits,
policy, acceptance, or substantive discussion changes stop publication rather
than silently rebasing or expanding scope.

### Isolation, exact patch checks, and publication

The existing `typeagent-bot` App and its existing
`DEPENDABOT_APP_ID`/`DEPENDABOT_APP_PRIVATE_KEY` bindings are reused. No new App,
PAT, credential rotation, App setting change, or repository permission change
is needed. The [trusted App helper](./tools/implementation-app.mjs) verifies App
and installation identity, explicitly requests only TypeChat's `contents: write`
and `pull_requests: write`, checks returned permissions and repository selection,
masks credentials, and revokes each token. Unrelated vulnerability-alert rights
are rejected, not inherited. Missing/excessive permissions or revocation failures
are errors. The preflight token is revoked before coding; publication mints a
new one and verifies the installation again.

App credentials exist only in trusted preflight/publisher steps, never in the
coding agent, detector, generated tests, or artifacts. The coding agent has
read-only repository permissions inside the pinned runtime's firewall.
Generated tests run in a separate non-root container with no network, host
workspace, Docker socket, or credential environment. Its Node image is digest
pinned to the repository's Node version. Dependencies are installed from the
unchanged base before the patch is applied; install scripts are disabled for
npm. TypeScript validation explicitly builds the root before the workspaces,
then runs the existing tests. Python uses Hatch tests and the existing Pyright
check; site changes use the existing site build. Documentation-only changes do
not require unrelated language builds.

The first pilot supports at most 20 ordinary UTF-8/LF text-file changes under
`typescript/`, `python/`, or `site/`, plus the root `README.md`. Candidate JSON
is bounded at 64 KiB and the entire patch at 128 KiB. Dependencies, configuration,
automation, policy, permissions, credentials, agent instructions, symlinks,
binary or executable-mode changes, generated trees, and case collisions are
rejected. Unsupported work remains maintainer work; nothing is silently omitted.
Windows-incompatible trailing-dot components and case aliases of directories
are rejected as well. Failed dependency enumeration, source extraction, or
dependency installation cannot produce a partial candidate or passing check
evidence.

The [protocol](./tools/issue-implementation.mjs) validates the candidate against
the fixed source tree and stages the complete patch, candidate, and accepted
agreement for strict detection. Scan and check evidence bind that exact content
and run. The publisher independently verifies evidence, usage, freshness, App
access, and contribution history, then creates Git blobs/trees via the API.
It never checks out or executes the generated patch while holding a write token.
The complete resulting tree must preserve every unmodified file. No force-push,
existing-branch takeover, automatic approval, merge, issue closure, or release
is implemented.

A stable acceptance-owned branch and PR marker provide durable deduplication.
A publication intent precedes all writes; branch/PR uncertainty blocks blind
retries and preserves evidence. Context, reservations, candidate, scan, checks,
and results are retained for 30 days, longer than the 24-hour authorization
window. Do not delete intents to force a retry. GitHub does not provide an atomic
transaction across all these resources; late authorization changes can leave
an unpublished or partially published branch requiring human inspection.

### Activation and first live verification

Review and merge the workflow through the ordinary owner process first.
The existing default token permissions, PR-creation/approval setting, Dependabot
workflow, library behavior, and discussion commands stay unchanged. Compiler
safe-update approval of the existing App binding is a code-review step, not
deployment or budget activation.

After a separate owner decision, set the repository variable
`TYPECHAT_IMPLEMENTATION_ENABLED=true`. Choose genuinely agreed, supported
library/documentation work with no competing contribution, generate its command,
and explicitly authorize that one run. Do not use #504 to make the future agent
rewrite its own automation. Disabling the variable prevents new builds; also
cancel an active run or withdraw its command when stopping work in progress.

Record whether the resulting App-created PR actually starts normal CI and
receives Copilot review. Neither is claimed successful in advance; both remain
separate from human approval and merging. The CI sandbox smoke test uses a
synthetic TypeScript/Python patch and no inference or App credentials. It verifies
container behavior, not a live coding run. Discussion-path verification and
receipt rollout remain their separate milestones.

## Existing issues and contributions

The experiment will include the existing backlog, not only new submissions.
Expand Down
38 changes: 36 additions & 2 deletions OWNER-CHEATSHEET.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ Manual publication and the Discussion path still need live verification.
| Post the proposed response | Review the exact draft, then copy its publish command into the same thread. | One Actions-bot comment, not approval to build. |
| Accept an agent-prepared proposal | [Review and approve its exact terms](#accept-an-exact-proposal), or [record a human decision manually](#accept-work-manually). The Issue path is live-verified. | A visible scope agreement; no coding agent starts automatically. |
| Move an existing PR forward | [Review the latest revision](#review-and-merge-manually); support the existing contributor rather than duplicate their PR. | A human decision to request changes or merge. |
| Prepare one bounded coding run | After review/rollout, [prepare its exact build command](#prepare-an-implementation-run-disabled-by-default). | A copyable command, not spending or merge approval. |
| Resolve unclear or out-of-scope work | Pause dependent work. If changing direction, approve and merge a [policy PR](./GOALS.md#who-decides) first. | An explicit scope decision; popularity or a label is not approval. |

## Refresh the backlog (live)
Expand Down Expand Up @@ -170,6 +171,38 @@ assign a coding agent. Exact-proposal acceptance is also available on Issues;
- Merge only after that decision. Record the outcome in the original thread.
Releases remain a separate human-approved operation.

## Prepare an implementation run (disabled by default)

The [#504 implementation pilot](./AUTOMATION.md#third-pilot-owner-triggered-issue-implementation)
is prepared for review, not activated or live-verified. Do not post build commands
yet. After review and merge, and a separate decision to activate:

1. Choose supported, explicitly accepted Issue work with no competing PR.
The workflow requires the exact public `/typechat accept` record and hash,
not an ordinary free-form acceptance comment or expired draft artifact.
2. Run **TypeChat accepted Issue implementation** manually in Actions. Enter the
Issue number and the numeric acceptance-comment ID. This only prints a
copyable command; it uses no model or App token.
3. Review that exact agreement. The owner must separately enable the repository
variable `TYPECHAT_IMPLEMENTATION_ENABLED=true`, then post the generated
`/typechat build <acceptance-comment-id>:<sha256>` as a fresh Issue comment.
This authorizes one run with **1,000 main credits plus 400 detection credits**.
4. Follow Actions, then the resulting PR. Build commands do not add an eyes/start
comment. The PR links the agreement, exact patch origin, checks, usage, and
limitations. Verify normal PR CI and Copilot review actually ran before review
and any separate merge decision.

Do not use #504 itself as the first coding target: generated changes to the
agent's own automation are forbidden. Dependency/configuration changes, binary
files, Discussions, and inline notes are outside this first pilot.

Commands expire after 24 hours; delayed preflight, changed source/discussion,
budget exhaustion, failed checks, or an incomplete scan stop the run. Reruns do
not spend again. Inspect failures before posting new authorization, and never
delete retained intents or overwrite a partial branch to force recovery.
To stop an active run, cancel it or withdraw the command as well as disabling
the activation variable. No activation or live coding run was performed here.

## If something stalls

| Symptom | Safe next step |
Expand All @@ -190,8 +223,9 @@ assign a coding agent. Exact-proposal acceptance is also available on Issues;
**Spending reminder:** main assessment **400 AI credits/run**; main
drafting/reply generation **50/run**. Each has a **separate detection budget, default 400**,
so those main-agent limits are not whole-workflow caps. Publishing and acceptance use no
model. Copilot PR reviews are billed separately. The future **1,000-credit
implementation allowance** and per-run budget overrides are **not enabled**.
model. Copilot PR reviews are billed separately. The prepared implementation
configuration proposes **1,000 main credits plus 400 detection credits**; the
activation gate remains **off**. Per-run budget overrides are **not enabled**.

Keep this sheet's date and availability notes aligned with the
[verified rollout history](./AUTOMATION.md#status-and-implementation-history).
Expand Down
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,11 @@ not publish draft content or accept work. Generation keeps its 50-credit main bu
separately budgeted detection. Acceptance does not launch a coding agent or
enable the future implementation budget.

The [owner-triggered implementation pilot](./AUTOMATION.md#third-pilot-owner-triggered-issue-implementation)
implements the workflow accepted in [#504](https://github.com/microsoft/TypeChat/issues/504)
as a separate, disabled-by-default extension. Review, activation, and a fresh
owner build command remain distinct; no coding run or merge happens automatically.

| Milestone | Implemented | Current status |
| --- | --- | --- |
| Community-facing automation plan | 2026-10-26 | Documented in [AUTOMATION.md](./AUTOMATION.md). |
Expand Down
16 changes: 11 additions & 5 deletions tools/discussion-draft.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -377,12 +377,12 @@ export async function verifyCommand(github, request) {
return comment;
}

export async function readControls(github, ref) {
export async function readControls(github, ref, paths = CONTROL_PATHS) {
const result = await github.rest.git.getTree({ ...API_REPO, tree_sha: ref, recursive: "1" });
const tree = result.data;
requireValue(tree?.truncated === false && Array.isArray(tree.tree), "The control-file tree is incomplete.");
const fingerprints = {};
for (const file of CONTROL_PATHS) {
for (const file of paths) {
const entry = tree.tree.find(item => item.path === file);
requireValue(entry?.type === "blob" && entry.mode === "100644" && SHA1.test(entry.sha),
`Missing or invalid trusted control file: ${file}`);
Expand Down Expand Up @@ -736,13 +736,19 @@ export function readJson(directory, filename, maximum) {
export function stageCandidateForDetection(directory, detectionDirectory) {
const text = readArtifact(directory, "discussion-draft.json", MAX_DRAFT_BYTES);
validateDraft(parseJson(text));
stageDetectionContent(detectionDirectory, "discussion-draft", text);
return digest(text);
}

export function stageDetectionContent(detectionDirectory, type, content) {
identifier(type, "Missing detection content type.");
string(content, "Missing bounded detection content.", MAX_CONTEXT_BYTES);
const outputPath = path.join(detectionDirectory, "agent_output.json");
const output = fs.existsSync(outputPath) ? parseJson(fs.readFileSync(outputPath, "utf8")) : { items: [] };
requireValue(output && Array.isArray(output.items), "The detector's agent output is malformed.");
output.items.push({ type: "discussion-draft", content: text });
output.items.push({ type, content });
fs.mkdirSync(detectionDirectory, { recursive: true });
fs.writeFileSync(outputPath, JSON.stringify(output) + "\n");
return digest(text);
}

export function createCandidateScan(hash, outcome, runId, attempt) {
Expand Down Expand Up @@ -905,7 +911,7 @@ function publicationIntentName(publication) {
`${INTENT_PREFIX}${publication.request.command.action === "accept" ? "accept-" : ""}${publication.artifact.id}`;
}

async function priorIntent(github, name, excludeId) {
export async function priorIntent(github, name, excludeId) {
// At most one result can be this run's new intent; any other match blocks publication.
const data = (await github.rest.actions.listArtifactsForRepo({ ...API_REPO, name, per_page: 2, page: 1 })).data;
requireValue(data && Array.isArray(data.artifacts) && Number.isSafeInteger(data.total_count) &&
Expand Down
Loading
Loading