From 6bb04a4652258ff50d8cc8c087c15b35a2352670 Mon Sep 17 00:00:00 2001 From: Jordan Ritter Date: Wed, 28 Oct 2026 10:49:21 -0700 Subject: [PATCH] docs: drop the Contents grant for evidence reads; say not to add it --- docs/deployment.md | 9 ++++----- docs/support-agent.md | 2 +- 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/docs/deployment.md b/docs/deployment.md index 094ecc9..a58aa17 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -93,10 +93,9 @@ All default AI stages use `gpt-5.6-luna`: support investigation, independent con 1. Set `OPENAI_API_KEY` on `outpost-worker` and on `outpost-web`. These are the two services that make model calls. The other services only format and post replies. 2. Clear any `claude-*` value from `AI_RESPONSE_MODEL`, `AI_CONFIDENCE_MODEL`, `AI_CLASSIFIER_MODEL` and `AI_SENTIMENT_MODEL`. `validateConfig` rejects a Claude model under the OpenAI provider. `AI_LEGACY_RESPONSE_MODEL` only applies to the legacy generator and can stay. -3. Grant the GitHub App **Contents: Read-only**. An org owner or a GitHub App manager adds it under the App's **Permissions & events**, and the installation accepts the updated permissions. `read_source` and `read_release` run only on the OpenAI path, and they ask for a token narrowed to `contents: read`. An App without Contents fails every evidence read from the moment the OpenAI default deploys. -4. Set `GITHUB_APP_ID`, `GITHUB_PRIVATE_KEY` and `GITHUB_INSTALLATION_ID` on `outpost-web` too, if web QA evidence reads should be authenticated. The web QA route (`/api/qa`) runs the same support agent as the worker, and it reads these three variables from its own service. Without them, web QA evidence reads are anonymous and rate-limited, on the shared 60 requests/hour budget. Set all three or none: a partial set fails every web QA evidence read. +3. Set `GITHUB_APP_ID`, `GITHUB_PRIVATE_KEY` and `GITHUB_INSTALLATION_ID` on `outpost-web` too, if web QA evidence reads should be authenticated. The web QA route (`/api/qa`) runs the same support agent as the worker, and it reads these three variables from its own service. Without them, web QA evidence reads are anonymous and rate-limited, on the shared 60 requests/hour budget. Set all three or none: a partial set fails every web QA evidence read. -Missing step 1 or 2 does not fail at boot. `validateConfig` runs only when a pipeline is built, so every `AI_RESPONSE` job fails before its pending response row exists, then retries and dead-letters. The reporter gets no reply, and no human gets a handoff. Sentiment analysis meanwhile logs an error line, but nothing fails. Account scoring skips the sentiment update, so each account keeps its last sentiment. Missing step 3 does not stop the job either. Every source and release read fails instead, as [GitHub App Setup](#github-app-setup) describes. To stay on Anthropic instead, set `AI_RESPONSE_PROVIDER=anthropic` (the rollback above) before the release lands. +Missing step 1 or 2 does not fail at boot. `validateConfig` runs only when a pipeline is built, so every `AI_RESPONSE` job fails before its pending response row exists, then retries and dead-letters. The reporter gets no reply, and no human gets a handoff. Sentiment analysis meanwhile logs an error line, but nothing fails. Account scoring skips the sentiment update, so each account keeps its last sentiment. To stay on Anthropic instead, set `AI_RESPONSE_PROVIDER=anthropic` (the rollback above) before the release lands. ## GitHub App Setup @@ -105,10 +104,10 @@ Outpost's GitHub integration (`apps/github-app`) responds to issues and discussi 1. **Create the App.** GitHub → Settings → Developer settings → GitHub Apps → **New GitHub App**. - Webhook URL: `https:///api/webhooks/github` — needs a public URL (the deployed `outpost-github-app` Railway service, or a tunnel like ngrok for local dev on port 3200). - Webhook secret: generate a random string and save it — this becomes `GITHUB_WEBHOOK_SECRET`. - - Permissions: **Issues: Read & write**, **Discussions: Read & write**, **Contents: Read-only**. (**Metadata: Read-only** is granted to every App automatically.) + - Permissions: **Issues: Read & write**, **Discussions: Read & write**. (**Metadata: Read-only** is granted to every App automatically.) - Subscribe to events: **Issues**, **Issue comment**, **Discussions**. - Contents is for the support agent's evidence reads. `read_source` and `read_release` ask GitHub for an installation token narrowed to `contents: read` (`packages/outpost/ai/src/github-evidence-auth.ts`). If the App does not have Contents, GitHub refuses that token request, and every evidence read fails, even though the same public reads would succeed without a token. The model's tool result reports `auth_unavailable`. The worker log line, `[SupportAgent] GitHub evidence authentication unavailable:`, carries the category `token_exchange_failed`, the same one any failed token exchange gets. To add it to an App that already exists, an org owner or a GitHub App manager sets **Contents: Read-only** under the App's **Permissions & events**, and the installation then has to accept the updated permissions. GitHub issues no narrowed token until it does. + Do not grant **Contents**. The support agent's evidence reads (`read_source`, `read_release`) only target the public CopilotKit/CopilotKit and ag-ui-protocol/ag-ui repos, which need no Contents permission, so their installation token is narrowed to `metadata: read`. Contents on an org-wide install would apply to every repo in the org, private ones included. GitHub has no webhook event for comment reactions, so 👍/👎 feedback is picked up by a 24-hour poll job instead (`GITHUB_REACTION_POLL`) — no extra event subscription is needed for that. diff --git a/docs/support-agent.md b/docs/support-agent.md index 3ff374f..0e91154 100644 --- a/docs/support-agent.md +++ b/docs/support-agent.md @@ -23,7 +23,7 @@ Set `OPENAI_API_KEY` on the worker and web service. The default investigator, in - `AI_LEGACY_RESPONSE_MODEL` controls direct legacy-generator use when the pipeline provider is OpenAI. - `AI_DRAFT_LINT_MODE=report` records existing draft-rule violations. `enforce` routes blocking violations to review. Review false positives before enabling enforcement. Evidence/schema validation and groundedness checks are always enforced. - `OPENAI_AGENTS_DISABLE_TRACING=1` disables SDK tracing. Otherwise traces exclude sensitive generation/tool payloads. Responses requests set `store: false`. -- `GITHUB_APP_ID`, `GITHUB_PRIVATE_KEY`, and `GITHUB_INSTALLATION_ID` — the same App credentials the rest of the worker already uses — authenticate source and release evidence reads with an installation token narrowed to `contents: read`. Web QA runs the same agent, so `outpost-web` reads them too, from its own environment. Set them on each service whose evidence reads should be authenticated. The App must hold **Contents: Read-only** (see [GitHub App Setup](deployment.md#github-app-setup)); without it GitHub refuses the narrowed token, every evidence read reports `auth_unavailable` to the model, and the service's log line carries the category `token_exchange_failed`. Set all three or none: with none set, that service's evidence reads fall back to anonymous public reads on the shared 60 requests/hour budget; set partially or malformed, evidence reads fail rather than degrading to an anonymous read, and the service's log names the category so the host can be repaired. +- `GITHUB_APP_ID`, `GITHUB_PRIVATE_KEY`, and `GITHUB_INSTALLATION_ID` — the same App credentials the rest of the worker already uses — authenticate source and release evidence reads with an installation token narrowed to `metadata: read`. Web QA runs the same agent, so `outpost-web` reads them too, from its own environment. Set them on each service whose evidence reads should be authenticated. Evidence reads only target public repositories, so the App intentionally does not hold **Contents**, and it must not be added: on the org-wide install it would apply to every repo in the org (see [GitHub App Setup](deployment.md#github-app-setup)). Set all three or none: with none set, that service's evidence reads fall back to anonymous public reads on the shared 60 requests/hour budget; set partially or malformed, evidence reads fail rather than degrading to an anonymous read, and the service's log names the category so the host can be repaired. ## Investigation and publication