Optional modules share one contract and load only when turned on #591

Request: none

Problem

Simlock has optional modules: parts a worker runs only when the operator
turns them on. agent-device is the only one on main. A second one, screen
streaming, is being built in #576.

Each optional module is wired into the daemon by hand. The daemon's start-up
file imports 14 names from agent-device and wires them in about 12 places,
for the worker and the gateway. #576 adds about 180 lines more for streaming,
in the same file.

Because nothing ties a module's parts together, they already disagree:

• A module's name is typed out in several places. In feat(stream): a lease's screen, streamed to whoever is looking at it (PoC) #576, the capability is
  called screen-stream and the relay target is called stream.
• A worker with agent-device turned off reports agent-device: off. A
  worker with streaming turned off reports nothing.
• The daemon loads agent-device's code even when it is turned off.
• simlock doctor has a finding kind made for agent-device only, so every
  new module would add its own kind to the wire contract.

Who it is for

• A maintainer or agent adding an optional module, who today has to edit
  the daemon's start-up file, the core config, the doctor and the contract,
  and keep the module's name in step by hand.
• An operator reading simlock doctor or a worker's status, who should see
  every optional module reported the same way.

Outcome

• An optional module is one directory and one line in a list of modules.
  Adding one changes nothing in the daemon's start-up file, the core config,
  the doctor or the contract.
• A module has one name. Its capability, its relay target, its doctor
  findings and its config section under capabilities all use it.
• A module that is turned off is never loaded and runs nothing. Every
  module that is turned off is reported as off.
• simlock doctor reports any optional module that is turned on but cannot
  run with one finding kind that names the module.
• agent-device is the first module built this way. It behaves as today,
  except the changes listed under "How it works for the user".

How it works for the user

flowchart TD
  S[simlock daemon starts] --> C[reads config.json]
  C --> E{module turned on?}
  E -->|no| OFF["status: name off<br/>module not loaded"]
  E -->|yes| L[module loaded and started]
  L --> R{can it run?}
  R -->|yes| AV["status: name available"]
  R -->|no| UN["status: name unavailable, reason<br/>doctor: module-unavailable finding"]

An operator's config.json moves agent-device under capabilities, the
word status and lease requests already use.

Before:

{ "agentDevice": { "enabled": true, "command": "agent-device" } }

After:

{ "capabilities": { "agent-device": { "enabled": true, "command": "agent-device" } } }

simlock config get prints the same new layout.

simlock doctor --json with an agent-device that is too old.

Before:

{ "findings": [{ "kind": "agent-device-unavailable",
  "message": "found 0.20.1, need 0.21.25 or later", "remedy": "…" }] }

After:

{ "findings": [{ "kind": "module-unavailable", "module": "agent-device",
  "message": "found 0.20.1, need 0.21.25 or later", "remedy": "…" }] }

Five things change for users. agent-device's integration has not been
released yet, so none of them needs a migration:

• The config layout above. An old agentDevice key is warned about as an
  unknown key and ignored.
• The doctor finding above. This is a breaking change to the doctor's
  JSON, and the protocol version moves.
• A gateway's warning about agent-device's command uses the generic
  text, shown under Examples.
• simlock daemon stop waits at most 30 s for each of agent-device's stop
  steps. Today it waits without a limit. agent-device's slowest stop takes
  about 22 s, so this shows only when agent-device hangs.
• If simlock crashes while agent-device is on, and the operator restarts it
  with agent-device turned off, the agent-device daemon the crash left
  keeps running. Today simlock stops it at that start. It is stopped once
  agent-device is turned on again, or when the machine restarts.

Examples

What could go wrong

• A module stops one step too late or too early on daemon stop, and
  simlock daemon stop hangs or leaves an agent-device process behind.
  Likely to show in tests, since today's stop order is kept and tested.
• A config.json written for the unreleased build keeps agentDevice, so
  agent-device stays off with only a warning. Likely for anyone testing
  that build; the warning names the key.
• A script that matches agent-device-unavailable in simlock doctor --json stops matching. Certain for such scripts. The changelog says so.
• An agent-device daemon left by a crash keeps running while agent-device
  is off. Unlikely: it needs a crash and a config change at the same time.

Words used

• optional module: a part of simlock a worker runs only when its config
  turns it on, e.g. agent-device.
• capability: what an optional module offers, under its name. The config
  turns it on (capabilities.agent-device.enabled), the worker's status
  reports its state (available, unavailable with a reason, or off),
  and a lease request can require it.
• relay target: the name a gateway uses to pass an HTTP request to a
  module on a worker, e.g. agent-device.
• module contract: the shape every optional module has, so the daemon can
  load, start and stop any module the same way.

Non-goals

• Moving screen streaming (feat(stream): a lease's screen, streamed to whoever is looking at it (PoC) #576) onto the contract. That happens when
  streaming becomes a feature. The ADR names it as the second module.
• Modules installed from outside this repository. Every module ships with
  simlock.
• Stopping a leftover agent-device daemon while agent-device is turned off.
• Any change to lease requires, CAPABILITY_MISSING, relay behaviour or
  the agent-device HTTP routes.

Completion conditions

• The daemon's start-up file names no optional module and imports nothing
  from agent-device.
• Adding a module means adding its directory and one line in the module
  list. A test that registers a fake module proves it gets its capability,
  relay target, routes, doctor findings and stop steps with no other edit.
• With capabilities.agent-device.enabled: false, or no section for it, a
  worker reports agent-device: off. agent-device's loaders are never
  called, and the dependency rules allow no static import of its code.
• simlock config get shows module config under capabilities, keyed by
  the module's name.
• With agent-device on but unable to run, simlock doctor --json reports
  { "kind": "module-unavailable", "module": "agent-device", … }.
• Every existing agent-device test still passes, with only the edits the
  tasks name: renamed config keys and finding kinds, the gateway warning's
  new text, the agentDevice* start options replaced by
  optionalModules: [agentDeviceModuleWith(...)], imports that follow
  moved code, and the config tests moved next to agent-device. The two tests for a supervisor turned off
  (stopping a leftover daemon, and a failed ps while off) are removed
  with that path.
• simlock daemon stop still stops agent-device's fronts before leasing
  drains, and stops its daemon before the drivers are released.

Open questions

Decisions

• ADR 0030 — Optional modules share one contract and load only when turned
  on (docs(adr): 0030 optional modules share one contract and load only when turned on #592): the module definition, its handle and hooks, config under
  capabilities, the gateway context and the doctor finding.

Shared values

Tasks

flowchart LR
  T596["#596 one doctor finding"] --> T595["#595 module host + capabilities config"]
  T595 --> T597["#597 agent-device on workers"]
  T597 --> T598["#598 agent-device on gateways"]

• Doctor reports every optional module with one module-unavailable finding #596 Doctor reports every optional module with one module-unavailable
  finding — a script reading simlock doctor's report sees one kind with
  the module's name. Risk: a client on the old protocol cannot read the
  report.
• Optional modules load through one host and are turned on under capabilities #595 Optional modules load through one host and are turned on under
  capabilities — simlock config get shows capabilities, and a module
  added later needs no change outside its directory. Risk: daemon stop
  hangs, or a worker's status stops parsing.
• agent-device runs as an optional module on workers #597 agent-device runs as an optional module on workers — agent-device
  is turned on under capabilities.agent-device, and its code never loads
  when off. Risk: agent-device does not start, or its process outlives
  daemon stop.
• agent-device's gateway side runs as an optional module #598 agent-device's gateway side runs as an optional module — nothing
  changes for users. Risk: agents on a gateway are told no worker can
  serve agent-device when one can.

Written by an agent.