feat(agent-device): devices, uploads, artifacts and diagnostics through a gateway #516

Part of #511.

Scope

After this PR, through a gateway:

• agent-device devices lists the token's devices across every worker that
  holds one;
• uploads, artifact downloads and diagnostics reach the worker holding the
  agent-device lease they name in the x-agent-device-lease-id header, and
  an upload ticket works though the worker never saw the agent's token;
• such a request with no header is refused with refused: this agent-device is too old to use through a gateway; update it, as feat(agent-device): open, commands, heartbeats and close through a gateway #515 already answers.

On a worker nothing changes: the header is not read, and devices behaves
as today.

The CLI sends the header from callstack/agent-device#3594, merged into
agent-device's simlock-integration branch at 253dca023. Tests here send
it by hand.

In short

New term:

• lease header: x-agent-device-lease-id: <agent-device lease id>,
  which agent-device sends on every upload, artifact download and
  diagnostics request once it holds a lease. Example:
  x-agent-device-lease-id: 3f2a… on POST /v1/agent-device/upload/preflight.

flowchart LR
  R[upload / artifact / diagnostics via gateway] --> H{lease header?}
  H -->|absent or empty| X[refused: update agent-device]
  H -->|not 32 hex| B[refused: header is not a lease id, new]
  H -->|lease id| L[relay host link: place + owner check, #515]
  L -->|not this token's| N[refused: not one of yours]
  L -->|this token's| F[relayed to that lease's worker, new]
  D[devices via gateway] --> FAN[one forward per held lease, each cut to its own device, joined, new]

On a worker, every row answers as it does today.

If this goes wrong, an upload lands on a worker that does not hold the
device, a token reads another token's artifacts, or devices misses a
device.

Technical spec

Modules touched

flowchart LR
  F["front: front.ts createFront, forwardCommand *"] --> HL["host-link port *"]
  F --> DF["front: devices-filter *"]
  RL["front: relay host link *"] -.->|implements| HL
  SH["shared: header name *"]
  F --> SH

(* = changed.)

• Header. The name lives once, in src/agent-device/shared/, as
  LEASE_HEADER = "x-agent-device-lease-id".
• Who reads it. The host-link port gains readsLeaseHeader (true for
  the relay host link, false for InProcessHostLink). Only when it is true
  does the front read the header or fan out devices, so a worker never
  does.
• Non-RPC routes through a gateway (createFront in front.ts, every
  route but /health):
  • An absent or empty header: forwarded with no lease id; the relay host
    link refuses it (feat(agent-device): open, commands, heartbeats and close through a gateway #515).
  • A value that is not 32 lower-case hex characters (the form leaseIdOf
    in shared/rpc.ts accepts): refused: the x-agent-device-lease-id header is not a lease id (status 400). The value is never echoed.
  • A lease id: the front reads status(leaseId); unavailable answers the
    notAvailable reply with its reason. Otherwise it forwards with that
    lease id, and the relay host link places it and checks its owner, the
    one place that rule lives on a gateway (feat(agent-device): open, commands, heartbeats and close through a gateway #515). A lease the token held
    that has since ended still routes through the link's memory, and
    agent-device answers for it.
• Upload tickets through a gateway. The relay drops the caller's
  authorization and x-agent-device-token (feat(fleet): a gateway relays an HTTP exchange to a worker module over the uplink #514), so the worker's proxy
  builds a ticket without them. For a relayed upload/preflight, the front
  adds the caller's own authorization and x-agent-device-token to the
  ticket's upload.headers in the answer, since the CLI sends the ticket's
  headers with its PUT.
• devices through a gateway (forwardCommand and devices-filter.ts):
  • With no lease held (tokenDevices), answers the empty list without
    forwarding. A LeaseCheckFailed answers today's could not check your leases; try again failure.
  • Otherwise it forwards the devices call once per held lease, all at
    once, each with that lease id, each bounded by feat(fleet): a gateway relays an HTTP exchange to a worker module over the uplink #514's relay deadline and
    carrying the CLI's signal, so a hang-up cancels them all.
  • Each answer is read as today's filter reads one, then cut to the device
    whose id is that lease's device address. The answers are joined in
    lease order. Two workers' emulators may share an id such as
    emulator-5554; cutting each answer to its own lease's device before
    joining means neither hides the other and no other token's device gets
    through.
  • The joined answer is the first lease's reply, with data.devices
    replaced.
  • When any forward fails, the rest are cancelled, and the command answers
    the failure of the first failing lease in lease order: an error reply
    from agent-device is passed on as today; a reply with no usable list
    answers failure("could not read the device list from the worker holding lease <id>") (status 502).
• RPC calls with no lease id. release_materialized_paths carries a
  materializationId, not a lease id; through a gateway it stays refused as
  a request naming no lease (feat(agent-device): open, commands, heartbeats and close through a gateway #515). No client code of agent-device outside
  its daemon builds it (checked at 253dca023). install_from_source names
  its lease like a command and routes by it.

Contract and event changes

None: the header is agent-device's wire, read by the edge.

Other code on the same state

The state is the front map and the token's simlock leases.

• Relay host link (feat(agent-device): open, commands, heartbeats and close through a gateway #515): the only owner check for a header's lease on
  a gateway; this task adds no second one.
• tokenDevices (token-devices.ts): read by devices only, as
  today.
• request-rules.ts: checks RPC calls only; non-RPC routes never pass
  through it.
• The host's ticket rewrite (host.ts #withTicketPrefix): still adds
  /v1 to the ticket URL on the worker; the gateway's front adds the
  credentials after it.
• The vendored proxy forwards only a fixed list of request headers
  upstream (FORWARDED_REQUEST_HEADERS), so the lease header never reaches
  the agent-device daemon; into an upload ticket it copies only
  authorization and x-agent-device-token. The direct-upload PUT
  carries the lease header because the CLI adds it when the ticket's origin
  is its base URL's, which through a gateway it is.

Rules in play

• safety.md: the header is a claim; its lease is placed and its owner
  checked before anything is relayed; a malformed value is never echoed; a
  worker never holds an agent's gateway credentials.
• architecture.md: the header name lives once; the owner rule stays in
  the relay host link.

Tests

Seam: real gateway and worker daemons in one process, HTTP into the
gateway's /v1/agent-device, FakeAgentDevice per worker, as #515's tests;
the worker-mode lines through src/agent-device/edge.test.ts.

• through the gateway, an upload preflight with the header naming the token's lease on W2 reaches W2's agent-device daemon and not W1's
• the preflight's ticket URL names the gateway's origin, its upload.headers carry the caller's own authorization, W2 received no authorization or x-agent-device-token, and a PUT to it with the lease header and those headers reaches W2
• a finalize, a POST /upload, a GET artifacts/<id> and a diagnostics GET, each with the header, reach the lease's worker
• through the gateway, the same requests without the header, or with an empty one, and a GET artifacts list, are refused with refused: this agent-device is too old to use through a gateway; update it
• through the gateway, a header naming another token's lease is refused refused: lease <id> is not one of yours on this gateway with status 403, and no agent-device daemon receives it
• through the gateway, a header of 40 characters, and one with upper-case hex, are refused refused: the x-agent-device-lease-id header is not a lease id with status 400, and the reply does not contain the value
• through the gateway, a header naming the token's lease whose worker stopped being capable answers the notAvailable reply naming that reason
• through the gateway, /health with a header naming another token's lease answers the proxy's health
• through the gateway, an upload naming the token's lease after simlock release --all reaches that lease's worker
• on a worker, a header naming another token's lease, a malformed header and no header are each forwarded, as today
• through the gateway, devices with no lease held answers an empty list and no agent-device daemon receives it
• through the gateway, devices with leases on W1 and W2 answers both devices, each once, and no device of another token
• through the gateway, devices with leases on W1 and W2 whose emulators are both emulator-5554 answers both, each from its own worker
• through the gateway, devices with two leases on one worker answers both devices, each once
• through the gateway, devices when W2's agent-device answers an error passes that error on; when W2 answers a body with no list, it answers could not read the device list from the worker holding lease <id> with status 502
• through the gateway, a CLI that hangs up during devices aborts every forward
• through the gateway, release_materialized_paths is refused refused: this agent-device is too old to use through a gateway; update it, and an install_from_source naming the token's lease reaches its worker
• on a worker, devices answers exactly as today (the existing devices tests in edge.test.ts pass unchanged)

Done when

• Every Tests line passes.
• docs/HTTP-API.md (/v1/agent-device/*) describes the lease header and
  the refusals through a gateway.

Assumptions

• Assumption: a relayed upload ticket's x-agent-device-token on W2 equals the proxy's own daemon token, never the caller's (decided in delivery, pending the maintainer).

Out of scope

• The agent-device change that sends the header (done as
  feat(remote): name the held lease on upload, artifact download and diagnostics requests callstack/agent-device#3594).
• A disconnected worker (feat(agent-device): a disconnected worker is not an ended lease, and a gateway restart keeps the device #517).
• Any change on a worker.

Depends on

• feat(agent-device): open, commands, heartbeats and close through a gateway #515

Approval

• Approved for delivery

Written by an agent.