Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ All shots are the standalone Web (`dotnet run --project src/AgentOs.Web`).
| Boards → Tickets → Sessions | Settings |
|:---:|:---:|
| [![Board — connect a GitHub/ADO board and drive remote repo execution](docs/images/app-board.png)](docs/images/app-board.png) | [![Settings — agent profiles, providers, governance, platform](docs/images/app-settings.png)](docs/images/app-settings.png) |
| Connect a GitHub / Azure DevOps board, decompose an idea into tickets, drive repo-execution sessions. | One searchable rail: AI (Model keys, Routing, Agent profiles, Prompts) · Connections · Governance · People & data · Personal · System. Ctrl+K finds any setting. Changes apply on the next call. |
| Connect a GitHub / Azure DevOps board, decompose an idea into tickets, drive repo-execution sessions. | One searchable rail: AI (Model keys, Routing, Agent profiles, Prompts, Autopilot) · Connections · Governance · People & data · Personal · System. Ctrl+K finds any setting. Changes apply on the next call. |

| Platform governance | Terminal — runner session feed |
|:---:|:---:|
Expand Down Expand Up @@ -319,6 +319,10 @@ in the desktop and applies on the next call:
- **Settings → Platform** (`platform-admin` only) — remote auto-approve, fail-closed tool policy,
allowed self-hosted Git hosts, build sandbox (mode, image, quotas), offline fallback.
- **Settings → Prompts** — per-tenant system prompts for the Requirement, Coding, Testing and QA agents.
- **Settings → Autopilot** — how Autopilot runs a project (Board › Autopilot): who decides (Orchestrator agent or the
fixed rules), how each ticket runs (MAF workflow or step by step), tasks and acceptance tests at once, automatic fix
rounds (one count per task across engines and restarts), Orchestrator retries, and the checks wait. When your
computer's Claude / Codex plan hits its limit, Autopilot waits for the reset — no tries spent — and carries on.
- **Settings → Audit log** — who changed what (settings, keys — never their values —, members, sign-in), CSV export.

**Your repo's own rules.** When a Board ticket runs against a repo, every agent follows the repo's agent
Expand Down
25 changes: 25 additions & 0 deletions docs/CONFIG.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,31 @@ for models that reject sampling params; `MaxTokens` ≤ 128 000.
`Pipeline:MaxIterations` (QA-loop cap when a story sets no `NMax`) — default 3, range 1–10.
An unpriced model is charged at the fail-closed $15/$75 per 1M rate until added to `CostCalculator`.

### Settings → Autopilot

How Autopilot runs a project once a person turns it on in **Board › Autopilot**. Tenant scope (admins); read on every
pass / call, so a change applies within a minute, no restart. The Board's Autopilot tab shows the live values.

| Key | Default | Range | Notes |
|---|---|---|---|
| `Autopilot:DecidedBy` | `Orchestrator agent` | `Orchestrator agent` / `Rules` | Who picks the next moves. The agent works within the rules; `Rules` = the fixed rules alone (saves the member's plan). |
| `Autopilot:TicketEngine` | `MAF workflow` | `MAF workflow` / `Step by step` | How each started ticket runs: one Microsoft Agent Framework graph per ticket, or Autopilot driving each step every minute. |
| `Autopilot:MaxParallel` | `2` | 1–6 | Tasks running at once on a board (coding, fixing, reviewing, testing). |
| `Autopilot:MaxParallelTests` | `1` | 1–3 | Acceptance runs at once **per member's computer** (each brings the app up on its fixed ports). |
| `Autopilot:MaxFixRounds` | `3` | 0–10 | Times a change goes back to the Coding agent (red checks, conflicts, review / acceptance findings) before a person is asked. Counted **per task in total** from the persisted `FixRounds` — the same count in both engines and across restarts. |
| `Autopilot:MaxAgentRetries` | `2` | 0–5 | Extra guided retries the Orchestrator agent may give a stuck task after the fix rounds are used up. |
| `Autopilot:ChecksTimeoutMinutes` | `60` | 5–240 | How long a ticket's MAF workflow waits for a pull request's checks before handing it back to the step-by-step passes. |
| `Autopilot:ChecksPollSeconds` | `60` | 15–600 | How often it looks at them. |

Fixed rules (not settings): *Build and merge* merges a change only when its checks are green, the Reviewer agent
approved it and the Testing agent passed it — which needs the member's computer connected. When that computer's
Claude / Codex **plan hits its limit** (e.g. `You've hit your session limit · resets 11:50pm (Asia/Ho_Chi_Minh)`), the
RemoteAgent provider raises a typed `PlanLimitException` and records the member's wait (`IPlanLimits`); Autopilot
then starts nothing, retries nothing and spends no fix round until the reset (+1 minute; 30 minutes when the CLI gives
no time), logs one line in the Autopilot activity, shows it on Board › Autopilot and Board › Computers, and picks the
cut-short work up again afterwards. The wait is in memory: after a restart the next run meets the limit again (an
immediate answer) and the wait is back.

### Settings → Platform (platform admins)

**Run limits (per tenant)** — `Pipeline:Limits:MaxConcurrentRuns` (default `4`) and `Pipeline:Limits:RunsPerMinute`
Expand Down
2 changes: 1 addition & 1 deletion docs/design/design-system.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ in `app.css`; the gallery of every role × state is on `/_/components`.
on focus; CSD drag/resize/maximize; grab a maximized window's titlebar to restore it under the cursor.
- **Settings hub** (`SettingsHub.razor`, pinned "Settings") — every setting behind **one rail** (GNOME
Settings style, never a rail inside a rail): a search field, then pages under group headings — **AI**
(Model keys, Routing, Agent profiles, Prompts) · **Connections** (GitHub, MCP servers, Plugins) ·
(Model keys, Routing, Agent profiles, Prompts, Autopilot) · **Connections** (GitHub, MCP servers, Plugins) ·
**Governance** (Tool policy, Activity log — changes & sign-in | tool calls) · **People & data** (Users, API tokens, Data) ·
**Personal** (General, Appearance, Date & time, Notifications, Session) · **System** (About, Diagnostics,
Platform). Deep-link with `SettingsLinks.Open(WM, "<page>")` (e.g. `keys`; legacy `llm/keys` still
Expand Down
21 changes: 14 additions & 7 deletions docs/plans/autonomy-levels.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,25 +35,32 @@ others learned. The board shows each ticket's phase (Discovery → Coding → Te
approved* starts only tickets whose dependencies are merged, at most N at a time, and re-checks when one merges.
Today it starts everything at once — #12 (import) would race #11 (the module it needs).
2. ✅ (#242, Autopilot) **Self-repair on CI.** When a task's pull request goes red, run *Fix the failing checks* automatically, at most
3 rounds, then escalate (a failure card with "This needs a person").
*Automatic fix rounds* (Settings › Autopilot, default 3), then escalate (a failure card with "This needs a person").
The rounds are the task's persisted `FixRounds` — one count for both engines and across restarts.
3. ✅ (#244) **Acceptance agent.** What a human reviewer did by hand on the lab: read the ticket's criteria and the project's
`docs/ACCEPTANCE.md`, run the unit tests, bring the stack up and run the Playwright specs, run data checks
(e.g. the CBL corpus), and post the evidence on the PR. Pass → approve; fail → *Ask for changes* with the
findings.
3c. ✅ (#270) **One ticket = one MAF workflow.** The Autopilot Orchestrator agent decides which ticket starts; a
Microsoft Agent Framework Workflow (`TicketWorkflow`) then runs it — Coding → checks → Reviewer → Testing → merge —
with conditional edges back to Coding (red checks, conflicts, requested changes, failed acceptance; 3 rounds). A
flow that ends without a merge, or one a restart cut short, is picked up step by step. Settings › Autopilot › *Each
ticket runs as* (MAF workflow / Step by step).
with conditional edges back to Coding (red checks, conflicts, requested changes, failed acceptance; the same fix
rounds, seeded from the task's persisted count). A flow that ends without a merge, or one a restart cut short, is
picked up step by step. Settings › Autopilot › *Each ticket runs as* (MAF workflow / Step by step); its checks wait
and poll are settings too.
3b. ✅ (#264) **Reviewer agent.** Between green CI and acceptance testing: a read-only agent reviews the diff in
context and posts its review on the PR; must-fix findings go back to the Coding agent (same 3 bounded rounds).
4. ✅ (#242, *Build and merge*: on green CI; acceptance agent next) **Merge policy.** Per project: merge on green + acceptance pass (squash, delete branch), or wait for a person.
context and posts its review on the PR; must-fix findings go back to the Coding agent (same bounded rounds).
4. ✅ (#242, #264, #244) **Merge policy.** Per project: *Build and merge* merges only on green checks + review approved +
acceptance passed (which needs the member's computer connected) — squash, delete branch — or *Build* waits for a person.
4b. ✅ **Plan limits.** When the member's computer answers with its Claude / Codex plan limit, Autopilot waits until the
reset (+1 min) — no new work, no retries, no fix rounds spent — says so once in the Autopilot activity and on the
Board (Autopilot + Computers tabs), and picks the cut-short work up again afterwards.
5. **Environment care.** The known stack landmines (stale Keycloak/Postgres volumes, orphaned containers, ports)
handled by the acceptance agent's stack step, not by a person.

## What L5 adds

- **Autopilot** per board: a loop that runs 1–4 whenever there is a ready ticket, inside a monthly budget and a
parallelism cap, with Stop.
parallelism cap (Settings › Autopilot: tasks at once, acceptance tests at once, fix rounds, Orchestrator retries,
checks wait), with Stop.
- **Escalation** in words: a "Needs you" badge + notification with the one decision required.
- **Report**: a daily summary per project — merged, in progress, blocked and why, spend.
124 changes: 114 additions & 10 deletions src/AgentOs.Modules.Pipeline/Agents/AutopilotSettings.cs
Original file line number Diff line number Diff line change
@@ -1,14 +1,45 @@
// Who decides Autopilot's next moves on a board (ADR-0011: a UI setting, per tenant). The Orchestrator agent by
// default; "Rules" keeps the fixed rules only (the agent's guardrails), e.g. to save the member's subscription.
// How Autopilot runs a board (ADR-0011: UI settings, per tenant — Settings › Autopilot). Who decides the next moves
// (the Orchestrator agent by default; "Rules" keeps the fixed rules only, e.g. to save the member's subscription), how
// each ticket runs (a MAF workflow, or step by step), and the bounds every engine keeps: tasks at once, acceptance runs
// at once, automatic fix rounds, the Orchestrator agent's extra retries, and how long to wait for a pull request's
// checks. Read per pass / per call, so a change applies to the next pass without a restart.

using System;
using System.Collections.Generic;
using System.Globalization;
using System.Threading;
using System.Threading.Tasks;
using AgentOs.Domain.Configuration;

namespace AgentOs.Modules.Pipeline.Agents;

/// <summary>Declares <c>Autopilot:DecidedBy</c>.</summary>
/// <summary>The bounds Autopilot keeps on a board, as configured (<see cref="AutopilotSettings"/>).</summary>
/// <param name="MaxParallel">Tasks running at once on a board.</param>
/// <param name="MaxParallelTests">Acceptance runs at once on a member's computer (each brings the app up on fixed ports).</param>
/// <param name="MaxFixRounds">Automatic "send it back to the Coding agent" rounds before a person is asked — the total
/// across engines and restarts (a task's persisted fix rounds count against it).</param>
/// <param name="MaxAgentRetries">Extra retries the Orchestrator agent may spend on a stuck task, with guidance.</param>
/// <param name="ChecksTimeout">How long a ticket's MAF workflow waits for a pull request's checks.</param>
/// <param name="ChecksPoll">How often it looks at them.</param>
public sealed record AutopilotLimits(int MaxParallel, int MaxParallelTests, int MaxFixRounds, int MaxAgentRetries, TimeSpan ChecksTimeout, TimeSpan ChecksPoll)
{
/// <summary>The built-in values (the settings' defaults).</summary>
public static AutopilotLimits Default { get; } = new(
AutopilotSettings.DefaultMaxParallel, AutopilotSettings.DefaultMaxParallelTests, AutopilotSettings.DefaultMaxFixRounds,
AutopilotSettings.DefaultMaxAgentRetries, TimeSpan.FromMinutes(AutopilotSettings.DefaultChecksTimeoutMinutes),
TimeSpan.FromSeconds(AutopilotSettings.DefaultChecksPollSeconds));

/// <summary>These limits with no slot caps — every legal move is offered (the Orchestrator agent picks, then
/// the caps are applied to its picks).</summary>
public AutopilotLimits Uncapped => this with { MaxParallel = int.MaxValue, MaxParallelTests = int.MaxValue };
}

/// <summary>Declares the <c>Autopilot:*</c> settings.</summary>
public sealed class AutopilotSettings : ISettingsContributor
{
/// <summary>The settings category (and the Settings › Autopilot page).</summary>
public const string Category = "Autopilot";

/// <summary>The setting key.</summary>
public const string DecidedByKey = "Autopilot:DecidedBy";

Expand All @@ -27,26 +58,99 @@ public sealed class AutopilotSettings : ISettingsContributor
/// <summary>Autopilot drives each step itself, pass by pass.</summary>
public const string StepByStep = "Step by step";

/// <summary>Tasks running at once on a board.</summary>
public const string MaxParallelKey = "Autopilot:MaxParallel";

/// <summary>Acceptance runs at once on a member's computer.</summary>
public const string MaxParallelTestsKey = "Autopilot:MaxParallelTests";

/// <summary>Automatic fix rounds per task.</summary>
public const string MaxFixRoundsKey = "Autopilot:MaxFixRounds";

/// <summary>The Orchestrator agent's extra retries per stuck task.</summary>
public const string MaxAgentRetriesKey = "Autopilot:MaxAgentRetries";

/// <summary>How long to wait for a pull request's checks (minutes).</summary>
public const string ChecksTimeoutMinutesKey = "Autopilot:ChecksTimeoutMinutes";

/// <summary>How often to look at them (seconds).</summary>
public const string ChecksPollSecondsKey = "Autopilot:ChecksPollSeconds";

internal const int DefaultMaxParallel = 2;
internal const int DefaultMaxParallelTests = 1;
internal const int DefaultMaxFixRounds = 3;
internal const int DefaultMaxAgentRetries = 2;
internal const int DefaultChecksTimeoutMinutes = 60;
internal const int DefaultChecksPollSeconds = 60;

/// <summary>Each bound's key, range and default — the descriptors and the typed read share it.</summary>
private static readonly (string Key, int Min, int Max, int Default)[] Bounds =
[
(MaxParallelKey, 1, 6, DefaultMaxParallel),
(MaxParallelTestsKey, 1, 3, DefaultMaxParallelTests),
(MaxFixRoundsKey, 0, 10, DefaultMaxFixRounds),
(MaxAgentRetriesKey, 0, 5, DefaultMaxAgentRetries),
(ChecksTimeoutMinutesKey, 5, 240, DefaultChecksTimeoutMinutes),
(ChecksPollSecondsKey, 15, 600, DefaultChecksPollSeconds),
];

/// <summary>True when the stored value means a MAF workflow (anything but "Step by step").</summary>
public static bool UsesMafWorkflow(string? value) => !string.Equals(value?.Trim(), StepByStep, System.StringComparison.OrdinalIgnoreCase);
public static bool UsesMafWorkflow(string? value) => !string.Equals(value?.Trim(), StepByStep, StringComparison.OrdinalIgnoreCase);

/// <inheritdoc />
public IEnumerable<SettingDescriptor> Describe()
{
yield return new(TicketEngineKey, SettingKind.Choice, "Autopilot", "Each ticket runs as")
yield return new(DecidedByKey, SettingKind.Choice, Category, "Decided by")
{
DefaultValue = Agent,
Choices = [Agent, Rules],
Help = "Orchestrator agent: an AI lead reads the whole project and picks the next moves, retries stuck tasks with guidance and asks you precise questions — always within the rules. Rules: the fixed rules alone.",
};
yield return new(TicketEngineKey, SettingKind.Choice, Category, "Each ticket runs as")
{
DefaultValue = MafWorkflow,
Choices = [MafWorkflow, StepByStep],
Help = "MAF workflow: a Microsoft Agent Framework graph runs Coding → checks → Reviewer → Testing → merge for the ticket, routing every verdict. Step by step: Autopilot drives each step itself every minute. Both use the same agents, rules and evidence.",
};
yield return new(DecidedByKey, SettingKind.Choice, "Autopilot", "Decided by")
yield return Bound(MaxParallelKey, "Tasks at once",
"How many tickets the AI works on at the same time on a project (coding, fixing, reviewing or testing). Each one uses your computer.");
yield return Bound(MaxParallelTestsKey, "Acceptance tests at once",
"How many changes the Testing agent tries at the same time on one computer. Each brings the app up on its usual ports, so more than one only works when the app can run twice.");
yield return Bound(MaxFixRoundsKey, "Automatic fix rounds",
"How many times a change goes back to the Coding agent (red checks, a conflict, review or acceptance findings) before a person is asked. Counted per task, in total — a restart grants no new rounds.");
yield return Bound(MaxAgentRetriesKey, "Orchestrator retries",
"Extra tries the Orchestrator agent may give a stuck task, with its own guidance, after the automatic fix rounds are used up.");
yield return Bound(ChecksTimeoutMinutesKey, "Wait for checks (minutes)",
"How long a ticket's MAF workflow waits for the pull request's checks before handing it back to Autopilot's step-by-step passes.");
yield return Bound(ChecksPollSecondsKey, "Look at checks every (seconds)",
"How often the MAF workflow looks at the pull request's checks while it waits.");
}

private static SettingDescriptor Bound(string key, string label, string help)
{
var b = Array.Find(Bounds, x => x.Key == key);
return new(key, SettingKind.WholeNumber, Category, label)
{
DefaultValue = Agent,
Choices = [Agent, Rules],
Help = "Orchestrator agent: an AI lead reads the whole project and picks the next moves, retries stuck tasks with guidance and asks you precise questions — always within the rules. Rules: the fixed rules alone.",
DefaultValue = b.Default.ToString(CultureInfo.InvariantCulture),
Min = b.Min,
Max = b.Max,
Help = help,
};
}

/// <summary>True when the stored value means the agent decides (anything but "Rules").</summary>
public static bool AgentDecides(string? value) => !string.Equals(value?.Trim(), Rules, System.StringComparison.OrdinalIgnoreCase);
public static bool AgentDecides(string? value) => !string.Equals(value?.Trim(), Rules, StringComparison.OrdinalIgnoreCase);

/// <summary>The tenant's Autopilot bounds, read now (each clamped to its range — a seeded value can't escape it).</summary>
public static async ValueTask<AutopilotLimits> ReadLimitsAsync(ISettings settings, string tenantId, CancellationToken ct = default)
{
ArgumentNullException.ThrowIfNull(settings);
var v = new int[Bounds.Length];
for (var i = 0; i < Bounds.Length; i++)
{
var (key, min, max, fallback) = Bounds[i];
v[i] = Math.Clamp(await settings.GetIntAsync(key, tenantId, fallback, ct).ConfigureAwait(false), min, max);
}
return new AutopilotLimits(v[0], v[1], v[2], v[3], TimeSpan.FromMinutes(v[4]), TimeSpan.FromSeconds(v[5]));
}
}
Loading
Loading