feat(website): one spec declares a site, composes it and checks any server (FD-4683)

packages/agentenv-protocol/examples/website/notes/Dockerfile

@@ -14,7 +14,7 @@ WORKDIR /app
 RUN apt-get update && apt-get install -y --no-install-recommends curl && rm -rf /var/lib/apt/lists/*
 COPY packages/agentenv-protocol /tmp/agentenv-protocol
 RUN pip install --no-cache-dir /tmp/agentenv-protocol "uvicorn>=0.30" && rm -rf /tmp/agentenv-protocol
-COPY ${NOTES}/site.py /app/site.py
+COPY ${NOTES}/site.py ${NOTES}/website.toml /app/
 COPY ${NOTES}/data /app/data
 COPY --from=web /web/dist /app/web/dist
 ENV PORT=8000

packages/agentenv-protocol/examples/website/notes/README.md

@@ -6,19 +6,23 @@ them. The page shows the universe's time.
 
 | File | What it is |
 | --- | --- |
-| `site.py` | The server: the `Notes` WebsiteEnvironment, its `@api` routes, and `serve()` on `PORT` (8000). |
+| `website.toml` | The site's spec: its name, domains, health path, source, seed, collections, personas, roles, routes, state read and clock (see `docs/website-spec.md`). |
+| `site.py` | The server: the `Notes` WebsiteEnvironment, declared from `website.toml`, its `@api` routes, and `serve()` on `PORT` (8000). |
 | `data/seed.json` | The seed data: two users (`ann`, `ben`), their notes, and `current_user`, the acting user. |
 | `web/` | A minimal Vite app (`index.html`, `src/main.js`, `src/style.css`). Its assets (`base: "./"`) and API calls are relative to the page, so it works at `/` and under the gateway's `/website/notes/` alike. |
 | `Dockerfile` | The server image: Node builds `web/`, then Python installs this repo's `packages/agentenv-protocol` with uvicorn. It installs `curl`, which the gateway's compose health check runs. |
-| `env.toml` | The website env as a bundle entry. |
+| `env.toml` | The website env as a bundle entry, pointing at `website.toml`. |
 
 ## What it exercises
 
-- **Site declaration.** `@website(domains=["notes.test"], static="web/dist", health="/healthz", data="data/seed.json")`.
-  The card advertises website/v1 with the domains, the health path and every route by name.
+- **Site declaration.** `@website(spec="website.toml")`. The spec gives the name, domains, static folder, health path
+  and seed, and the class is refused when its `@api` routes aren't the spec's. The card's website/v1 params are the
+  spec's `to_card_extension()`: the domains, the health path, every route by name, the session, the state read and
+  the clock.
 - **Routes.** `list_users`, `sign_in`, `list_notes` (with an optional `?done=`), `create_note` (201), `get_note`,
   `edit_note`, `toggle_note` and `delete_note`. Errors are raised as `NotFound`, `BadRequest` and `Unauthorized`.
-- **Roles.** `all_notes` (`GET /api/admin/notes`) is declared with `roles=["admin"]`. It answers only a caller that
+- **Roles.** `all_notes` (`GET /api/admin/notes`) is declared with `roles=["admin"]`, and the spec's `[roles]` grants
+  it to `admin`. It answers only a caller that
   sends `AgentEnv-Role: admin` with `AgentEnv-Role-Key` equal to the container's `WEBSITE_ROLE_KEY`; any other caller
   gets 403. A deploy with `keyed_roles` gives the site its own role key as `WEBSITE_ROLE_KEY` and its gateway sends it
   on only beside a role it verified with the same key, so a page cannot claim `admin` through it. Any other deploy sets
@@ -72,9 +76,11 @@ python -m agentenv_protocol.website.conformance http://localhost:18080 --clock-h
 curl -s http://localhost:18080/api/notes
 ```
 
-Use `--clock-host` so the container can reach the checker's clock. `--strict` fails on a recommended check's warning
-too (the card, `X-Universe-Now`, the clock, forwarded headers, cookies); without it only the contract's required checks
-fail the run, and Notes passes every check either way. The clock check leaves the server following a
+Use `--clock-host` so the container can reach the checker's clock, and add
+`--spec packages/agentenv-protocol/examples/website/notes/website.toml` to check the server against its spec too.
+`--strict` fails on a recommended check's warning too (the card, `X-Universe-Now`, the clock, forwarded headers,
+cookies); without it only the contract's required checks, and the spec's when given, fail the run, and Notes passes
+every check either way. The clock check leaves the server following a
 clock that is gone once the check ends: later timestamps start from 2031 until the server is synced again, so restart
 the container before using it.
 
@@ -86,10 +92,10 @@ agent-env env website put --id notes-site \
   --server-docker-context . --health-path /healthz
 ```
 
-Run it from the repo root. `environment_name` comes from the card (`@environment_card(name="notes")`). The stored env
+Run it from the repo root. `environment_name` comes from the card, which the spec names `notes`. The stored env
 has a server image and no frontend image. Pass `--skip-validation` to register without deploying.
 
-`env.toml` describes the same env for a bundle. Its keys are `type`, `environment_name`, `health` and
-`backend_image`. Domains are not one of them: they come from the card. A bundle builds `backend_image` with this
+`env.toml` describes the same env for a bundle. Its keys are `type`, `spec` and `backend_image`; the env takes its
+`environment_name`, health path and domains from the spec. A bundle builds `backend_image` with this
 folder as the build context, but this Dockerfile needs the repo root. Until the protocol package is installed from a
 published release instead, register the site with `put` as above.

packages/agentenv-protocol/examples/website/notes/env.toml

@@ -1,4 +1,3 @@
 type = "website"
-environment_name = "notes"
-health = "/healthz"
+spec = "website.toml"
 backend_image = { dockerfile = "Dockerfile" }

packages/agentenv-protocol/examples/website/notes/site.py

@@ -11,7 +11,6 @@
 
 from pydantic import BaseModel
 
-from agentenv_protocol import environment_card
 from agentenv_protocol.website import (
     ApiContext,
     BadRequest,
@@ -43,8 +42,7 @@ class SignIn(BaseModel):
     user: str
 
 
-@environment_card(name="notes")
-@website(domains=["notes.test"], static="web/dist", health="/healthz", data="data/seed.json")
+@website(spec="website.toml")
 class Notes(WebsiteEnvironment):
     @ready_check
     def seeded(self) -> bool:

packages/agentenv-protocol/examples/website/notes/website.toml

@@ -0,0 +1,41 @@
+[site]
+name = "notes"
+domains = ["notes.test"]
+health = "/healthz"
+
+[source]
+sdk = "site:Notes"
+assets = "web/dist"
+
+[data]
+seed = ["data/seed.json"]
+
+[data.collections.users]
+
+[data.collections.notes]
+owner = "owner"
+
+[users]
+personas = [{ id = "ann", name = "Ann Lee" }, { id = "ben", name = "Ben Ortiz" }]
+acting_user = "ann"
+sign_in = "cookie"
+
+[roles]
+admin = ["all_notes"]
+
+[routes.ops]
+list_users = "GET /api/users"
+sign_in = "POST /api/sign-in"
+list_notes = "GET /api/notes"
+create_note = "POST /api/notes"
+get_note = "GET /api/notes/{note_id}"
+edit_note = "PATCH /api/notes/{note_id}"
+toggle_note = "POST /api/notes/{note_id}/toggle"
+delete_note = "DELETE /api/notes/{note_id}"
+all_notes = "GET /api/admin/notes"
+
+[state]
+read = "go"
+
+[clock]
+follow = "header"

packages/agentenv-protocol/examples/website/specs/components/sso.toml

@@ -0,0 +1,19 @@
+[site]
+name = "sso"
+
+[source]
+sdk = "fake_sso:FakeSso"
+
+[data.collections.accounts]
+
+[data.collections.grants]
+owner = "user"
+
+[users]
+sign_in = "cookie"
+
+[routes.ops]
+authorize = "GET /authorize"
+login = "POST /login"
+token = "POST /token"
+userinfo = "GET /userinfo"

packages/agentenv-protocol/examples/website/specs/intranet-ben.toml

@@ -0,0 +1,8 @@
+extends = "intranet.toml"
+
+[users]
+personas = ["ben", "cy"]
+acting_user = "ben"
+
+[roles]
+editor = false

packages/agentenv-protocol/examples/website/specs/intranet.toml

@@ -0,0 +1,30 @@
+[site]
+name = "intranet"
+domains = ["intranet.test", "*.intranet.test"]
+
+[source.image]
+dockerfile = "Dockerfile.intranet"
+
+[data.collections.users]
+
+[data.collections.pages]
+owner = "author"
+
+[users]
+personas = ["ann", "ben"]
+acting_user = "ann"
+
+[roles]
+editor = ["publish_page"]
+
+[routes.ops]
+list_pages = "GET /api/pages"
+publish_page = "POST /api/pages"
+
+[[uses]]
+spec = "components/sso.toml"
+mount = "/sso"
+data = { accounts = "users" }
+
+[state]
+read = "go"

packages/agentenv-protocol/examples/website/specs/slack-live.toml

@@ -0,0 +1,23 @@
+[site]
+name = "slack"
+domains = ["slack.com", "*.slack.com"]
+health = "/api/api.test"
+
+[source.live]
+allow_hosts = ["*.slack-edge.com", "*.slack-imgs.com"]
+credentials = { token = "secret:SLACK_USER_TOKEN" }
+
+[users]
+sign_in = "live"
+
+[routes.ops]
+list_conversations = "GET /api/conversations.list"
+history = "GET /api/conversations.history"
+post_message = "POST /api/chat.postMessage"
+add_reaction = "POST /api/reactions.add"
+
+[state]
+read = "diff"
+
+[clock]
+follow = "none"

packages/agentenv-protocol/examples/website/specs/slack.toml

@@ -0,0 +1,11 @@
+extends = "slack-live.toml"
+
+[source.replay]
+capture = "captures/slack-workspace.har"
+allow_hosts = ["*.slack-edge.com"]
+
+[routes.ops]
+post_message = { served_by = "slack_synthetic:SlackChat" }
+
+[clock]
+follow = "date_patch"

packages/agentenv-protocol/pyproject.toml

@@ -25,6 +25,7 @@ dependencies = [
     "pydantic>=2.11",
     "starlette>=0.37",
     "httpx>=0.27",
+    "tomli>=1.1; python_version < '3.11'",
 ]
 
 [project.optional-dependencies]

packages/agentenv-protocol/src/agentenv_protocol/website/__init__.py

@@ -1,6 +1,7 @@
 """The website SDK: a site's server, meeting ``urn:agentenv:website/v1``. Subclass WebsiteEnvironment, declare the
 site with ``@website(domains=[...], static="dist/")``, mark its API with ``@api.get/post/put/patch/delete(path)``, and
-``serve()`` it; it needs only the plain package, and uvicorn to serve. The class is optional: a server in any language
+``serve()`` it, or declare it from a website spec with ``@website(spec="website.toml")``; it needs only the plain
+package, and uvicorn to serve. The class is optional: a server in any language
 that meets the contract works, which ``python -m agentenv_protocol.website.conformance <url>`` checks."""
 
 from __future__ import annotations
@@ -19,12 +20,14 @@
 from .errors import ApiError, BadRequest, Conflict, Forbidden, NotFound, Unauthorized
 from .routes import ROLE_KEY_ENV, ROLE_KEY_HEADER, ApiContext, ApiRoute, api, reserved_path
 from .session import SESSION_COOKIE, SessionSigner
+from .spec import NameTable, SpecError, WebsiteSpec, from_card, from_environment, load as load_spec, name_table
 from .store import Collections
 
 __all__ = [
     "WebsiteEnvironment", "WebsiteDeclaration", "website", "api", "ApiContext", "ApiRoute", "ready_check",
     "ApiError", "BadRequest", "Unauthorized", "Forbidden", "NotFound", "Conflict",
     "Collections", "UniverseClock", "SessionSigner", "reserved_path",
+    "WebsiteSpec", "SpecError", "NameTable", "load_spec", "from_card", "from_environment", "name_table",
     "WEBSITE_EXTENSION_URI", "CLOCK_EXTENSION_URI", "UNIVERSE_NOW_HEADER", "DEFAULT_HEALTH_PATH", "SESSION_PATH",
     "STATE_PATH", "SESSION_COOKIE", "ROLE_KEY_ENV", "ROLE_KEY_HEADER",
 ]