diff --git a/.github/workflows/dependabot-automerge.yml b/.github/workflows/dependabot-automerge.yml new file mode 100644 index 0000000..35b2cd8 --- /dev/null +++ b/.github/workflows/dependabot-automerge.yml @@ -0,0 +1,24 @@ +name: Dependabot auto-merge + +on: pull_request + +permissions: + contents: write + pull-requests: write + +jobs: + dependabot: + runs-on: ubuntu-latest + if: ${{ github.event.pull_request.user.login == 'dependabot[bot]' }} + steps: + - name: Fetch Dependabot metadata + id: metadata + uses: dependabot/fetch-metadata@v2 + with: + compat-lookup: true + - name: Enable auto-merge for patch and minor updates + if: ${{ steps.metadata.outputs.update-type == 'version-update:semver-patch' || steps.metadata.outputs.update-type == 'version-update:semver-minor' }} + run: gh pr merge --auto --merge "$PR_URL" + env: + PR_URL: ${{ github.event.pull_request.html_url }} + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/pyrefly.yml b/.github/workflows/pyrefly.yml new file mode 100644 index 0000000..9586c0c --- /dev/null +++ b/.github/workflows/pyrefly.yml @@ -0,0 +1,28 @@ +name: pyrefly + +on: + push: + branches: [main, develop] + pull_request: + branches: [main, develop] + +jobs: + pyrefly: + runs-on: ubuntu-latest + timeout-minutes: 10 + + steps: + - name: Check out repository code + uses: actions/checkout@v7 + + - name: Set up Python + uses: astral-sh/setup-uv@v10.0.1 + with: + enable-cache: true + python-version: "3.13" + + - name: Install packages + run: uv sync --all-extras --dev + + - name: Run pyrefly + run: uv run pyrefly check diff --git a/.github/workflows/ruff.yml b/.github/workflows/ruff.yml new file mode 100644 index 0000000..cf01182 --- /dev/null +++ b/.github/workflows/ruff.yml @@ -0,0 +1,28 @@ +name: ruff + +on: + push: + branches: [main, develop] + pull_request: + branches: [main, develop] + +jobs: + ruff: + runs-on: ubuntu-latest + timeout-minutes: 10 + + steps: + - name: Check out repository code + uses: actions/checkout@v7 + + - name: Set up Python + uses: astral-sh/setup-uv@v10.0.1 + with: + enable-cache: true + python-version: "3.13" + + - name: Install packages + run: uv sync --all-extras --dev + + - name: Run ruff + run: uv run ruff check modules/ pyobs_web_admin/ diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml new file mode 100644 index 0000000..11bf7ba --- /dev/null +++ b/.github/workflows/tests.yml @@ -0,0 +1,29 @@ +name: Tests + +on: + push: + branches: [main, develop] + pull_request: + branches: [main, develop] + +jobs: + test: + name: Run Django test suite + runs-on: ubuntu-latest + steps: + - name: Checkout the repository + uses: actions/checkout@v7 + + - name: Install uv + uses: astral-sh/setup-uv@v10.0.1 + with: + enable-cache: true + + - name: Install dependencies + run: uv sync --locked + + - name: Run Django system checks + run: uv run python manage.py check + + - name: Run tests + run: uv run python manage.py test diff --git a/README.md b/README.md index be24e3f..ccaac57 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,8 @@ A web-based administration interface for [pyobs](https://github.com/pyobs/pyobs- the robotic telescope framework. It lets you start, stop, and restart modules, tail and filter their logs, and view and edit their configuration files — all from a browser. +![Dashboard showing modules grouped under Stopped and Deactivated headings, with summary tiles and per-row quick-action buttons](docs/source/_static/screenshots/dashboard.jpg) + ## Features - **Dashboard** — sortable list view of all modules with: @@ -18,21 +20,21 @@ filter their logs, and view and edit their configuration files — all from a br - Responsive: on small screens the table collapses to status dot + name + log counts + actions - **Module detail** — per-module view with four tabs: - *Overview* — current status, PID, uptime, CPU and memory usage, running `pyobs-*` package versions (flagged when they lag the installed set), per-level log message counts (last 24 h), XMPP connection state (if enabled), start/restart/stop/activate/deactivate control - - *Logs* — live log tail with text filter, time-range filter (set a start date to load all logs since that instant, or click a line to set it), colour-coded by severity, auto-refresh; scrolling to the top auto-loads older entries (journald-backed modules, or file-backed modules once a start date is set, see [journald-logs.md](specs/design/journald-logs.md)) + - *Logs* — live log tail with text filter, time-range filter (set a start date to load all logs since that instant, or click a line to set it), colour-coded by severity, auto-refresh; scrolling to the top auto-loads older entries (journald-backed modules, or file-backed modules once a start date is set) - *Config* — YAML editor with syntax highlighting and colour-coded `{include}` lines; included shared configs are shown as clickable links - *ACL* — point-and-click editor for the module's `acl:` block: click to allow/deny known modules, add other callers, toggle enforce/log mode - **New module** — a "+" next to the sidebar's Modules section creates a brand-new `.yaml` config (a minimal starter with just a `class:` key) and takes you straight to its Config tab to fill in the rest - **Shared configs** — `*.shared.yaml` config fragments listed in a separate sidebar section with a YAML-highlighted config editor (no start/stop controls) -- **Packages** (`/packages/`) — every installed `pyobs-*` package (plus anything else listed in `PYOBS_MANAGED_PACKAGES`) with its installed and latest-PyPI version, and a one-click Update button; git/URL-installed packages get a Reinstall action instead (see [Package management](#package-management)) +- **Packages** (`/packages/`) — every installed `pyobs-*` package (plus anything else listed in `PYOBS_MANAGED_PACKAGES`) with its installed and latest-PyPI version, and a one-click Update button, plus an *Update all* bulk action that queues every outdated package sequentially; git/URL-installed packages get a Reinstall action instead (see [Package management](#package-management)) - **Overview** (`/overview/`) — fleet-wide summary, one row per configured host: reachable or not, running/stopped/total counts, aggregate CPU/RAM, linking into that host's own Dashboard, plus a package-version matrix (one row per `pyobs-*` package, one column per host) so version drift across the fleet is visible at a glance. Deliberately no bulk or per-module actions — those stay on the per-host Dashboard, since a fleet-wide "Stop All" from one button is a real footgun - **All Logs** (`/logs/`) — fleet-wide live log tail across every module on every configured host, same filtering and scroll-to-load-older behaviour as a module's own Logs tab -- **ACL Matrix** (`/acl/`) — fleet-wide read-only matrix of which module can call which, merged across every configured host (see [acl-matrix.md](specs/design/acl-matrix.md)) +- **ACL Matrix** (`/acl/`) — fleet-wide read-only matrix of which module can call which, merged across every configured host - **Hub mode** — control multiple remote pyobs hosts from a single browser tab; remote hosts are listed in the sidebar and all actions are proxied transparently - **ejabberd / XMPP status** (optional) — dashboard summary tile and per-module connected/not-connected indicator, plus a session/last-seen/registered-account block on each module's own page, for modules with a `comm.user` in their config — closes the gap between "the process is running" and "the module is actually reachable over XMPP" (see [ejabberd integration](#ejabberd-integration)) - **ejabberd / XMPP user management** (optional, builds on the above) — register, reset password, ban/unban, unregister, and kick XMPP accounts, either from a module's own Overview tab or from a fleet-wide **Users** page (`/xmpp-users/`) listing every registered account across every host, cross-referenced against which module(s) use it and which one is actually running. Safe by design for an identity shared across more than one module's `comm.user` — a password reset writes back to every module sharing it, and destructive actions name which other modules are affected before you confirm (see [ejabberd user management](#ejabberd-user-management)) - **Responsive** — works on mobile with a slide-in sidebar - **No pyobs-core dependency** — communicates with `pyobs` directly via subprocess; no Python imports from pyobs-core -- **Keycloak login** (optional) — SSO on top of the default shared admin/password login, with per-person activate/deactivate via Django's built-in admin site (see [Keycloak login](#keycloak-login)) +- **Keycloak login** (optional) — SSO on top of the default shared admin/password login, with per-person access granted/revoked via Keycloak group membership (see [Keycloak login](#keycloak-login)) ## Technology @@ -42,7 +44,7 @@ filter their logs, and view and edit their configuration files — all from a br | WSGI server | Gunicorn | | Frontend | Bootstrap 5 (CDN), CodeMirror 5 (CDN), vanilla JS | | Package manager | uv | -| Auth | Shared admin/password login (cookie sessions, no database) plus optional Keycloak SSO (SQLite, one table, for the Keycloak-linked `User` only) | +| Auth | Shared admin/password login plus optional Keycloak SSO; both share one SQLite db (sessions table for both, `User` table for Keycloak-linked accounts only) | | Hub auth | Pre-shared token in `X-Hub-Token` header; CSRF bypassed for hub requests | --- @@ -184,6 +186,13 @@ ADMIN_PASSWORD_HASH = "pbkdf2_sha256$..." # see generation command above # "IDP_HINT": "gwdg", # "IDP_LABEL": "GWDG", # "USER_RESOLVER": "pyobs_web_admin.authentication.keycloak.resolve_user", +# # Only members of this Keycloak group are authorized to use web-admin - create it (and add +# # people to it) in the Keycloak admin console before anyone logs in. +# "REQUIRED_GROUPS": ["/pyobs-web-admin"], +# # Keycloak-independent kill switch, layered on top of REQUIRED_GROUPS above: an admin can +# # deactivate a specific local User (Django admin) regardless of their Keycloak group +# # membership. +# "ENFORCE_LOCAL_ACTIVE": True, # } # pyobs paths @@ -193,13 +202,12 @@ PYOBS_LOG_DIR = "/opt/pyobs/log" # directory containing *.log files PYOBS_RUN_DIR = "/opt/pyobs/run" # directory for PID files PYOBS_LOG_LEVEL = "info" # log level passed to pyobs on start PYOBS_LOG_BACKEND = None # None (default): auto-detect from pyobsd's own - # config; "file" or "journald" to override -- - # see specs/design/journald-logs.md. If "journald": - # the account running pyobs-web-admin needs journal - # read access — `usermod -aG systemd-journal - # ` (preferred over `adm`, see the doc's - # "Which group to grant") — otherwise logs come - # back silently empty, no error. + # config; "file" or "journald" to override. If + # "journald": the account running pyobs-web-admin + # needs journal read access — `usermod -aG + # systemd-journal ` (preferred over `adm`, + # which grants broader log access than needed) — + # otherwise logs come back silently empty, no error. # Packages page (optional — see Package management section) PYOBS_MANAGED_PACKAGES = [] # e.g. ["pyobs-core[full]", "my-custom-driver", @@ -227,9 +235,9 @@ EJABBERDCTL = "ejabberdctl" # required for user management w The shared admin/password login above is the default and always available — Keycloak is an additive option on top of it, not a replacement, so it stays working as a break-glass fallback -if Keycloak itself is unreachable. Unlike the shared account, each Keycloak-linked account can -be individually activated or deactivated, so you can grant or revoke one person's access without -touching anyone else's or handing out the shared password. +if Keycloak itself is unreachable. Unlike the shared account, a Keycloak-linked person's access +is granted or revoked individually via Keycloak group membership, without touching anyone else's +access or handing out the shared password. ### Enabling it @@ -237,13 +245,23 @@ touching anyone else's or handing out the shared password. `https://your.domain.com/accounts/keycloak/callback/`, post-logout redirect URI `https://your.domain.com/`). 2. Add the `PYOBS_AUTH` block shown in [Configuration](#configuration) to `local_settings.py`, - filling in `SERVER_URL`, `CLIENT_ID`, and `CLIENT_SECRET`. -3. Run `manage.py migrate` (creates `db.sqlite3` — only used for the Keycloak-linked `User` - table; the shared admin/password login and sessions stay fully DB-free either way). + filling in `SERVER_URL`, `CLIENT_ID`, and `CLIENT_SECRET`. `REQUIRED_GROUPS` defaults to + `["/pyobs-web-admin"]` — create that group in the realm and add whoever should have access to + it (Keycloak admin console) before anyone tries to log in, or every login is refused as "not + authorized". +3. Run `manage.py migrate` (creates `db.sqlite3`, used for the Keycloak-linked `User` table and, + as of this version, sessions too — `SESSION_ENGINE` is now database-backed rather than signed + cookies, since a Keycloak session carries a refresh token that shouldn't be serialized into + the browser. This applies to every session app-wide, including the shared admin/password + login's — it's no longer fully DB-free, but it's the one Django setting shared by both login + paths, so it can't be split per-path). Expired sessions aren't purged automatically — run + `manage.py clearsessions` periodically (e.g. a daily cron/systemd timer) or `django_session` + grows unbounded in a long-lived deployment. The login page then shows a "Log in with Keycloak" button. A first-time Keycloak login mints a -local `User` (linked to an existing one by email, falling back to username, if either matches) -with `is_active=False` — it can't do anything until activated. +local `User` (linked to an existing one by email, falling back to username, if either matches), +active immediately — whether they're actually let in is decided by `REQUIRED_GROUPS` above, not +by anything in this app's own database. Setting `IDP_HINT` (plus `IDP_LABEL` for the button text) switches the login page to two buttons: "Log in with ``" goes straight to that identity provider (skipping Keycloak's own @@ -251,13 +269,16 @@ login/IdP-selection page), and "Log in with local Keycloak account" keeps the lo reachable for anyone without that IdP's identity. Leave `IDP_HINT` unset for the single-button behavior above. -### Activating / deactivating a user +### Granting or revoking a person's access -Django's built-in admin site is mounted at `/admin/` for exactly this — it isn't gated behind -this app's own `/login/` page (Django's own is-staff-gated login handles it instead), and both -the shared admin/password account and any other Django superuser can get in. Under **Users**, -toggle **Active** on a Keycloak-linked account to grant or revoke their access; nothing else -about that page needs touching. +This is now done in the Keycloak admin console, not this app's Django admin: add or remove the +person from the `/pyobs-web-admin` group (or whatever `REQUIRED_GROUPS` is set to). The change +takes effect at their next login. Django's `/admin/` site is still mounted (the shared +admin/password account and any other Django superuser can still get in there for other reasons), +with the `ENFORCE_LOCAL_ACTIVE` setting above (recommended, and the example config's default), +toggling a Keycloak-linked account's **Active** flag there still works too, as an additional, +Keycloak-independent kill switch on top of the group gate. Leave `ENFORCE_LOCAL_ACTIVE` unset if +you'd rather Keycloak group membership be the only thing that matters. The shared admin/password account works at `/admin/` directly (no prior visit to `/login/` needed first) because `manage.py migrate` syncs a matching superuser `User` automatically — see @@ -389,7 +410,7 @@ it as-is; `mod_http_api` can also expose account-management commands trusted, no password or token is involved. This blocks the network (a request from outside the host is rejected), but **not** other processes on the same machine, which get the same access pyobs-web-admin does. That's an accepted tradeoff for a dedicated, single-purpose -observatory control host — see `specs/design/ejabberd-integration.md` if your threat model is different. +observatory control host, not a shared one — reassess if that's not your deployment. --- @@ -463,8 +484,11 @@ host with one admin identity, not a shared or multi-tenant one. The **Packages** page (`/packages/`) lists every installed `pyobs-*` package (plus anything extra listed in `PYOBS_MANAGED_PACKAGES`) alongside its latest release on PyPI, and lets you -update any of them with one click. It always reflects `pip`'s own view of the environment -`PYOBS_EXEC` runs in (via the sibling `pip` next to it) — nothing here is invented or cached. +update any of them with one click, or all outdated ones at once via **Update all**. Updates are +serialized host-wide (only one `pip install` runs at a time), so Update all queues them +sequentially rather than firing them in parallel — one package failing doesn't stop the rest from +being attempted. It always reflects `pip`'s own view of the environment `PYOBS_EXEC` runs in (via +the sibling `pip` next to it) — nothing here is invented or cached. The fleet-wide **Overview** page (`/overview/`) additionally shows a package-version matrix across every configured host, so a package that's drifted out of sync on one host is easy to spot. @@ -539,12 +563,12 @@ end up as a literal `pip install` command-line argument, visible to any other lo - **Discovery** — all `*.yaml` files in `PYOBS_CONFIG_DIR` (excluding `*.shared.yaml`) are treated as modules. `*.shared.yaml` files are listed separately as shared configs. - **Creating a module** — the "New module" button writes a fresh `.yaml` with a minimal starter (`class:` key only); `PYOBS_CONFIG_DIR` is created automatically if it doesn't exist yet. - **Activate / Deactivate** — deactivating a module renames its config from `name.yaml` to `_name.yaml` (stopping it first if running); activating renames it back. Deactivated modules are excluded from *Start All* and *Restart All*. -- **Start** — runs `pyobs --pid-file /.pid --log-file /.log --log-level `. pyobs daemonises itself via `python-daemon`. If the effective log backend is `"journald"` (see below), `--syslog` is passed instead of `--log-file` — pyobs then logs directly to the systemd journal, tagged `SYSLOG_IDENTIFIER=pyobs` and `PYOBS_MODULE=` (see [journald-logs.md](specs/design/journald-logs.md)). +- **Start** — runs `pyobs --pid-file /.pid --log-file /.log --log-level `. pyobs daemonises itself via `python-daemon`. If the effective log backend is `"journald"` (see below), `--syslog` is passed instead of `--log-file` — pyobs then logs directly to the systemd journal, tagged `SYSLOG_IDENTIFIER=pyobs` and `PYOBS_MODULE=`. - **Stop** — sends `SIGTERM` to the PID in the PID file; falls back to `SIGKILL` after 5 s. - **Restart** — stop followed by start. - **Status** — checks whether the process with the stored PID is alive (`os.kill(pid, 0)`). - **Resource usage** — uptime, CPU %, and RSS memory read via `psutil` on every status poll. -- **Logs** — read from `PYOBS_LOG_DIR`'s flat files by default, or from the systemd journal via `journalctl` if the effective log backend is `"journald"`; the log viewer and per-level counts work identically either way. The effective backend is `PYOBS_LOG_BACKEND` if set explicitly, otherwise auto-detected from `pyobsd`'s own config file (`~/.config/pyobs.yaml`, `/etc/pyobs.yaml`, or `/opt/pyobs/storage/pyobs.yaml`, first found wins) — the same file `pyobsd` (`pyobs-core`'s daemon manager) reads to decide whether *it* starts modules with `--syslog`, so this can't silently drift out of sync with it. Scrolling a log window to the top auto-loads older entries via journalctl's `--until`, for journald-backed modules only — the file backend's plain `tail -n` has no seek/offset to page further back with, so it reports nothing older available instead (see [journald-logs.md](specs/design/journald-logs.md)). +- **Logs** — read from `PYOBS_LOG_DIR`'s flat files by default, or from the systemd journal via `journalctl` if the effective log backend is `"journald"`; the log viewer and per-level counts work identically either way. The effective backend is `PYOBS_LOG_BACKEND` if set explicitly, otherwise auto-detected from `pyobsd`'s own config file (`~/.config/pyobs.yaml`, `/etc/pyobs.yaml`, or `/opt/pyobs/storage/pyobs.yaml`, first found wins) — the same file `pyobsd` (`pyobs-core`'s daemon manager) reads to decide whether *it* starts modules with `--syslog`, so this can't silently drift out of sync with it. Scrolling a log window to the top auto-loads older entries via journalctl's `--until`, for journald-backed modules only — the file backend's plain `tail -n` has no seek/offset to page further back with, so it reports nothing older available instead. - **Log counts** — per-level message counts (DEBUG / INFO / WARNING / ERROR / CRITICAL) for the last 24 h, using binary search on the log file to avoid reading the whole file. --- diff --git a/docs/requirements.txt b/docs/requirements.txt index 631b7d7..e4eed6f 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -1,6 +1,10 @@ sphinx-rtd-theme django>=6.0.7 +gunicorn>=26.0.0 +packaging>=24.0 psutil>=6.0 +pyobs-auth>=2.0.0.dev9 pyyaml>=6.0 requests>=2.32 ruamel.yaml>=0.18 +whitenoise>=6.8 diff --git a/docs/source/_static/screenshots/config-editor.jpg b/docs/source/_static/screenshots/config-editor.jpg new file mode 100644 index 0000000..411da7d Binary files /dev/null and b/docs/source/_static/screenshots/config-editor.jpg differ diff --git a/docs/source/_static/screenshots/dashboard.jpg b/docs/source/_static/screenshots/dashboard.jpg new file mode 100644 index 0000000..6e7616b Binary files /dev/null and b/docs/source/_static/screenshots/dashboard.jpg differ diff --git a/docs/source/api_endpoints.rst b/docs/source/api_endpoints.rst index ec0dc5e..26c760f 100644 --- a/docs/source/api_endpoints.rst +++ b/docs/source/api_endpoints.rst @@ -105,11 +105,14 @@ Logs - ``/api/modules//logs/`` - Query params: ``lines`` (default 300, capped at 2000), ``filter`` (substring/regex applied server-side), ``since`` (ISO-8601 instant -- the time-range start date, so - only entries at or after it are returned), and ``before`` (ISO-8601 instant -- returns - the last ``lines`` entries at or before it, for the log pane's scroll-to-top "load - older logs" feature). ``since`` and ``before`` combine into a ``[since, before]`` - window; ``before`` on its own is journald-only (the file backend returns ``[]`` - unless ``since`` is also set). ``{"lines": [...]}``. + only entries at or after it are returned), ``until`` (ISO-8601 instant -- the + time-range end date, so only entries at or before it are returned), and ``before`` + (ISO-8601 instant -- returns the last ``lines`` entries at or before it, for the log + pane's scroll-to-top "load older logs" feature). ``since``/``until`` form a + ``[since, until]`` window; ``before`` (the page-back cursor) combines with ``until`` + as a second upper bound, the earlier of the two winning. ``before`` on its own is + journald-only (the file backend returns ``[]`` unless ``since`` is also set). + ``{"lines": [...]}``. * - GET - ``/api/modules//log-stats/`` - 24h level counts. ``{"stats": {"DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"}}``. @@ -118,9 +121,9 @@ Logs - Fleet-wide merged tail, independent of the active host. Query params: ``lines``, ``filter``, ``modules`` -- a comma-separated list of ``:`` tokens (``host`` is ``"localhost"`` or a ``HUB_HOSTS`` name); omit ``modules`` entirely for - every module on every configured host -- plus ``since`` and ``before`` (same - time-range start date and "load older logs" semantics as the per-module endpoint - above, forwarded unchanged to each configured host). ``{"lines": [...], + every module on every configured host -- plus ``since``, ``until`` and ``before`` + (same time-range start/end date and "load older logs" semantics as the per-module + endpoint above, forwarded unchanged to each configured host). ``{"lines": [...], "unreachable_hosts": [{"name", "error"}, ...]}``; each line is tagged ``[host]`` when more than one host is selected. * - GET diff --git a/docs/source/features/dashboard.rst b/docs/source/features/dashboard.rst index 5cc0060..62a9d63 100644 --- a/docs/source/features/dashboard.rst +++ b/docs/source/features/dashboard.rst @@ -1,6 +1,16 @@ Dashboard and module detail ############################ +.. image:: ../_static/screenshots/dashboard.jpg + :alt: Dashboard showing six modules grouped under Stopped and Deactivated headings, with + Total/Running/Stopped/RAM/CPU summary tiles and per-row quick-action buttons. + :width: 100% + +.. image:: ../_static/screenshots/config-editor.jpg + :alt: Module detail Config tab showing a syntax-highlighted YAML editor for a module's + config file. + :width: 100% + Dashboard ********* diff --git a/docs/source/features/logging.rst b/docs/source/features/logging.rst index 1d1af74..5a0dcab 100644 --- a/docs/source/features/logging.rst +++ b/docs/source/features/logging.rst @@ -58,15 +58,20 @@ the line you were looking at doesn't jump. Setting a start date in the time-range filter turns it into a server-side ``since`` bound rather than just a client-side filter, so the pane loads logs *since* that date and you can -page back through them. The initial fetch and every scroll-to-top page-back send the start -date as ``since``; the server bounds the query accordingly (``journalctl --since`` on the -journald backend, a timestamp window on the file backend). +page back through them. Setting an end date likewise becomes a server-side ``until`` bound, +so a past window is fetched exactly as ``[since, until]`` instead of the newest ``lines`` +entries being filtered down client-side (which previously showed "no matching log lines" +for any range that ended before the newest activity). The initial fetch and every +scroll-to-top page-back send both dates as ``since``/``until``; the server bounds the query +accordingly (``journalctl --since/--until`` on the journald backend, a timestamp window on +the file backend). Scrolling to the top pages back with ``before`` (the oldest currently-loaded line's own timestamp). For the journald backend that's a ``journalctl --until`` cutoff alongside the existing ``-n ``, giving the next page of entries immediately before what's already -on screen. The file backend supports the same page-back once a start date is set (the window -is bounded by ``since``), but a plain ``tail -n`` has no seek/offset to page further back -*without* a start date -- so on a file-backed module with no start date the pane reports -"Beginning of available logs" the first time you scroll to the top, rather than silently -re-serving the same tail on every scroll. +on screen; the end date combines with the cursor as a second upper bound, the earlier of the +two winning. The file backend supports the same page-back once a start date is set (the +window is bounded by ``since``), but a plain ``tail -n`` has no seek/offset to page further +back *without* a start date -- so on a file-backed module with no start date the pane +reports "Beginning of available logs" the first time you scroll to the top, rather than +silently re-serving the same tail on every scroll. diff --git a/modules/admin.py b/modules/admin.py index 8c38f3f..b97a94f 100644 --- a/modules/admin.py +++ b/modules/admin.py @@ -1,3 +1,2 @@ -from django.contrib import admin # Register your models here. diff --git a/modules/ejabberd.py b/modules/ejabberd.py index 8101b2e..e7b00b6 100644 --- a/modules/ejabberd.py +++ b/modules/ejabberd.py @@ -48,6 +48,7 @@ def _ctl_returncode(command: str, *args: str) -> int: def _parse_session_line(line: str, has_jid: bool) -> dict: parts = line.split("\t") + session: dict[str, str | int] if has_jid: jid, *rest = parts session = {"jid": jid, **dict(zip(_SESSION_FIELDS, rest))} diff --git a/modules/models.py b/modules/models.py index 71a8362..35e0d64 100644 --- a/modules/models.py +++ b/modules/models.py @@ -1,3 +1,2 @@ -from django.db import models # Create your models here. diff --git a/modules/proxy.py b/modules/proxy.py index 0a0a622..3a40ca9 100644 --- a/modules/proxy.py +++ b/modules/proxy.py @@ -1,5 +1,4 @@ import requests as _http - from django.conf import settings diff --git a/modules/pyobs_config.py b/modules/pyobs_config.py index ea8d7df..a130184 100644 --- a/modules/pyobs_config.py +++ b/modules/pyobs_config.py @@ -105,4 +105,4 @@ def replace_aliases(matches: list[tuple[str, str]], anchor_filename: str, alias_ return alias_string -__all__ = ["pre_process_yaml"] \ No newline at end of file +__all__ = ["pre_process_yaml"] diff --git a/modules/services.py b/modules/services.py index adb9e6d..750bcd8 100644 --- a/modules/services.py +++ b/modules/services.py @@ -9,12 +9,10 @@ import time from collections import deque from concurrent.futures import ThreadPoolExecutor -from datetime import datetime, timedelta, timezone +from datetime import UTC, datetime, timedelta from pathlib import Path from typing import Any, NamedTuple -_LOG_LEVEL_RE = re.compile(r'\[(DEBUG|INFO|WARNING|ERROR|CRITICAL)\]') - import psutil import requests import yaml @@ -24,6 +22,8 @@ from modules.pyobs_config import pre_process_yaml +_LOG_LEVEL_RE = re.compile(r'\[(DEBUG|INFO|WARNING|ERROR|CRITICAL)\]') + # Used only to serialize a *fresh* acl: block for _replace_local_acl_block -- ruamel's # round-trip dumper reads more like hand-written YAML (indented block sequences, minimal # quoting) than plain pyyaml's default output. Not used for reading/round-tripping a whole @@ -1127,6 +1127,7 @@ def _get_module_versions_file(name: str) -> dict[str, str] | None: grep = subprocess.Popen( ["grep", "-m1", "-F", _LOADED_PACKAGES_PREFIX], stdin=tac.stdout, stdout=subprocess.PIPE, text=True ) + assert tac.stdout is not None tac.stdout.close() stdout, _ = grep.communicate() tac.wait() @@ -1141,7 +1142,7 @@ def _get_module_versions_journald(name: str, since_create_time: float) -> dict[s entry's MESSAGE. Bound to `--since @` (the module's own psutil create_time(), already used by get_module_stats) -- otherwise --grep scans the entire retained journal, and for a long-running module the version line is old.""" - since = datetime.fromtimestamp(since_create_time, tz=timezone.utc) + since = datetime.fromtimestamp(since_create_time, tz=UTC) entries = _journalctl_json([ "SYSLOG_IDENTIFIER=pyobs", f"PYOBS_MODULE={_journald_module_tag(name)}", "--since", f"{since:%Y-%m-%d %H:%M:%S} UTC", "--grep", _LOADED_PACKAGES_PREFIX, @@ -1264,7 +1265,7 @@ def _journal_entry_to_line(entry: dict) -> str: # file backend's shape), so it must actually *be* UTC regardless of the host OS's local # timezone -- the frontend (templates/modules/detail.html, all_logs.html: parseLogTime) # assumes exactly that when it parses these lines back out. - ts = datetime.fromtimestamp(int(entry["__REALTIME_TIMESTAMP"]) / 1_000_000, tz=timezone.utc) + ts = datetime.fromtimestamp(int(entry["__REALTIME_TIMESTAMP"]) / 1_000_000, tz=UTC) level = _JOURNALD_PRIORITY_TO_LEVEL.get(int(entry.get("PRIORITY", 6)), "INFO") module = entry.get("PYOBS_MODULE", "") # CODE_FILE is logging_journald's record.pathname (a full path), but pyobs's own journal @@ -1280,12 +1281,13 @@ def _journal_entry_to_line(entry: dict) -> str: return f"{ts:%Y-%m-%d %H:%M:%S} [{level}] ({module}) {code_file}:{code_line} {message}" -def _get_logs_journald(name: str, lines: int, before: datetime | None = None, since: datetime | None = None) -> list[str]: +def _get_logs_journald(name: str, lines: int, before: datetime | None = None, since: datetime | None = None, until: datetime | None = None) -> list[str]: args = ["SYSLOG_IDENTIFIER=pyobs", f"PYOBS_MODULE={_journald_module_tag(name)}"] if since is not None: args += ["--since", f"{since:%Y-%m-%d %H:%M:%S} UTC"] - if before is not None: - args += ["--until", f"{before:%Y-%m-%d %H:%M:%S} UTC"] + upper = _until_bound(before, until) + if upper is not None: + args += ["--until", f"{upper:%Y-%m-%d %H:%M:%S} UTC"] args += ["-n", str(lines)] entries = _journalctl_json(args) return [_journal_entry_to_line(e) for e in entries] @@ -1297,7 +1299,7 @@ def _get_log_stats_journald(name: str, since: datetime | None = None) -> dict: # window when it's more recent than the standard 24h rollup, but never widens it beyond # 24h -- an ack from days ago shouldn't suddenly pull that whole history back in. if since is not None: - cutoff = max(since, datetime.now(timezone.utc) - timedelta(hours=24)) + cutoff = max(since, datetime.now(UTC) - timedelta(hours=24)) since_arg = f"{cutoff:%Y-%m-%d %H:%M:%S} UTC" else: since_arg = "-24h" @@ -1331,19 +1333,19 @@ def _log_identities(name: str) -> list[str]: return [primary, comm_user] -def get_logs(name: str, lines: int = 300, filter_str: str = "", before: datetime | None = None, since: datetime | None = None) -> list[str]: +def get_logs(name: str, lines: int = 300, filter_str: str = "", before: datetime | None = None, since: datetime | None = None, until: datetime | None = None) -> list[str]: validate_name(name) identities = _log_identities(name) if _log_backend() == "journald": - line_lists = [_get_logs_journald(i, lines, before, since) for i in identities] + line_lists = [_get_logs_journald(i, lines, before, since, until) for i in identities] else: - line_lists = [_get_logs_file(i, lines, since, before) for i in identities] + line_lists = [_get_logs_file(i, lines, since, before, until) for i in identities] # A module whose comm user differs from its config name logs under both identities (see # _log_identities) -- merge the two tails into one timestamp-ordered stream, trimmed to # the overall last `lines`, the same per-module approximation the fleet-wide merge uses. log_lines = line_lists[0] if len(line_lists) == 1 else merge_log_lines(line_lists, lines) if filter_str: - log_lines = [l for l in log_lines if filter_str.lower() in l.lower()] + log_lines = [line for line in log_lines if filter_str.lower() in line.lower()] return log_lines @@ -1366,7 +1368,20 @@ def _file_line_ts(line: str) -> datetime | None: def _naive_utc(dt: datetime) -> datetime: """File-backend timestamps are naive and assumed UTC (see _journal_entry_to_line); convert an aware datetime to that same naive-UTC form before comparing against them.""" - return dt.astimezone(timezone.utc).replace(tzinfo=None) + return dt.astimezone(UTC).replace(tzinfo=None) + + +def _until_bound(before: datetime | None, until: datetime | None) -> datetime | None: + """The window's effective upper bound: the tighter of the page-back cursor (`before`, + the oldest currently-loaded line) and the time-range end date (`until`). journald takes + a single --until and the file backend a single `before` cutoff, so the two upper-bound + sources collapse into one -- `before` never exceeds `until` (the cursor only moves + backward from the newest loaded line, which is itself within the window).""" + if before is None: + return until + if until is None: + return before + return min(before, until) def _file_offset_of_last_line_before(f, file_size: int, cutoff: datetime) -> int: @@ -1388,35 +1403,43 @@ def _file_offset_of_last_line_before(f, file_size: int, cutoff: datetime) -> int return lo -def _get_logs_file(name: str, lines: int, since: datetime | None, before: datetime | None) -> list[str]: - """File-backend read of `name`'s flat log, bounded by [since, before] (both inclusive) - and trimmed to the last `lines` lines -- the same "tail within a window" semantics the - journald backend gets from `journalctl -n --since ... --until ...`. +def _get_logs_file(name: str, lines: int, since: datetime | None, before: datetime | None, until: datetime | None = None) -> list[str]: + """File-backend read of `name`'s flat log, bounded by [since, until] (both inclusive, + `until` capped further by the page-back cursor `before`) and trimmed to the last `lines` + lines -- the same "tail within a window" semantics the journald backend gets from + `journalctl -n --since ... --until ...`. The common tail/auto-refresh case (`before is None`) stays a plain `tail -n` so the every-few-seconds refresh never re-reads a whole day of history: timestamps are monotonic, - so `tail -n lines` either lies entirely inside [since, now] (when the window holds >= lines - lines) or already includes the whole window plus some older lines, which the since-filter - then collapses to exactly the window. The `before` case (a scroll-to-top page-back) reads - forward from the first line >= since and stops at the first line > before, keeping only the - last `lines`. Without a `since` the file backend still can't page back -- `tail -n` has no - seek/offset concept to page further back with, and reading from offset 0 on every scroll - would re-read the whole file -- so that returns [] as before (see journald-logs.md).""" + so `tail -n lines` either lies entirely inside [since, until] (when the window holds >= + lines lines) or already includes the whole window plus some older lines, which the + since/until filters then collapse to exactly the window. The `before` case (a scroll-to-top + page-back) reads forward from the first line >= since and stops at the first line > + before, keeping only the last `lines`. Without a `since` the file backend still can't page + back -- `tail -n` has no seek/offset concept to page further back with, and reading from + offset 0 on every scroll would re-read the whole file -- so that returns [] as before (see + journald-logs.md).""" log_file = _log_dir() / f"{_active_name(name)}.log" if not log_file.exists(): return [] if before is not None and since is None: return [] since_naive = _naive_utc(since) if since is not None else None - before_naive = _naive_utc(before) if before is not None else None + until_naive = _naive_utc(until) if until is not None else None - if before_naive is None: + if before is None: result = subprocess.run(["tail", "-n", str(lines), str(log_file)], capture_output=True, text=True) tail_lines = result.stdout.splitlines() - if since_naive is None: + if since_naive is None and until_naive is None: return tail_lines - return [l for l in tail_lines if (t := _file_line_ts(l)) is None or t >= since_naive] - + return [ + line for line in tail_lines + if (t := _file_line_ts(line)) is None + or ((since_naive is None or t >= since_naive) and (until_naive is None or t <= until_naive)) + ] + + # A page-back cursor and an end date are both upper bounds; the earlier one caps the window. + before_naive = min(_naive_utc(before), until_naive) if until_naive is not None else _naive_utc(before) with open(log_file, "rb") as f: f.seek(0, 2) file_size = f.tell() @@ -1440,7 +1463,7 @@ def _get_logs_file(name: str, lines: int, since: datetime | None, before: dateti return list(buf) -def _get_all_logs_journald(names: list[str] | None, lines: int, before: datetime | None = None, since: datetime | None = None) -> list[str]: +def _get_all_logs_journald(names: list[str] | None, lines: int, before: datetime | None = None, since: datetime | None = None, until: datetime | None = None) -> list[str]: # names is None means "no PYOBS_MODULE restriction at all" -- broader than "every # currently configured module," since it also surfaces entries from a module whose # config has since been removed/renamed. names == [] means the caller explicitly @@ -1454,8 +1477,9 @@ def _get_all_logs_journald(names: list[str] | None, lines: int, before: datetime args += [f"PYOBS_MODULE={_journald_module_tag(n)}" for n in names] if since is not None: args += ["--since", f"{since:%Y-%m-%d %H:%M:%S} UTC"] - if before is not None: - args += ["--until", f"{before:%Y-%m-%d %H:%M:%S} UTC"] + upper = _until_bound(before, until) + if upper is not None: + args += ["--until", f"{upper:%Y-%m-%d %H:%M:%S} UTC"] args += ["-n", str(lines)] entries = _journalctl_json(args) return [_journal_entry_to_line(e) for e in entries] @@ -1480,7 +1504,7 @@ def merge_log_lines(line_lists: list[list[str]], lines: int) -> list[str]: return [line for _, _, _, line in entries[-lines:]] -def _get_all_logs_file(names: list[str], lines: int, since: datetime | None, before: datetime | None) -> list[str]: +def _get_all_logs_file(names: list[str], lines: int, since: datetime | None, before: datetime | None, until: datetime | None = None) -> list[str]: # Each module's own file has no cross-module time index, so the merge reads the windowed # tail from every file independently, then sorts the union by each line's own leading # timestamp and trims to the overall last `lines` -- an approximation (a module with @@ -1490,12 +1514,12 @@ def _get_all_logs_file(names: list[str], lines: int, since: datetime | None, bef # search comment). line_lists = [] for name in names: - line_lists.append(_get_logs_file(name, lines, since, before)) + line_lists.append(_get_logs_file(name, lines, since, before, until)) return merge_log_lines(line_lists, lines) def get_all_logs( - names: list[str] | None = None, lines: int = 300, filter_str: str = "", before: datetime | None = None, since: datetime | None = None + names: list[str] | None = None, lines: int = 300, filter_str: str = "", before: datetime | None = None, since: datetime | None = None, until: datetime | None = None ) -> list[str]: if names is not None: for name in names: @@ -1513,11 +1537,11 @@ def get_all_logs( expanded.append(ident) names = expanded if _log_backend() == "journald": - log_lines = _get_all_logs_journald(names, lines, before, since) + log_lines = _get_all_logs_journald(names, lines, before, since, until) else: - log_lines = _get_all_logs_file(names if names is not None else list_modules(), lines, since, before) + log_lines = _get_all_logs_file(names if names is not None else list_modules(), lines, since, before, until) if filter_str: - log_lines = [l for l in log_lines if filter_str.lower() in l.lower()] + log_lines = [line for line in log_lines if filter_str.lower() in line.lower()] return log_lines @@ -1536,7 +1560,7 @@ def _get_log_stats_file(name: str, since: datetime | None = None) -> dict: # File-backend timestamps are naive and assumed UTC (see _journal_entry_to_line); # convert the aware `since` the same way before comparing, and only narrow the # window, never widen it beyond the standard 24h rollup. - since_naive = since.astimezone(timezone.utc).replace(tzinfo=None) + since_naive = since.astimezone(UTC).replace(tzinfo=None) cutoff = max(cutoff, since_naive) def _line_ts(line: str) -> datetime | None: @@ -2009,6 +2033,44 @@ def get_resolved_acl(name: str) -> tuple[dict | None, str | None]: return acl, _block_source_file(config_file.read_text(), "acl") +def get_module_class(name: str) -> str | None: + """Returns a module's configured top-level `class:` (fully-qualified class name), + resolved the same way get_resolved_acl resolves acl: -- via pre_process_yaml + + yaml.safe_load, since class: can equally arrive through a YAML anchor/merge key, not + just be written directly. None if the config doesn't exist, doesn't parse, or has no + top-level "class:" key. + """ + validate_name(name) + config_file = _config_dir() / f"{name}.yaml" + if not config_file.exists(): + return None + try: + resolved = yaml.safe_load(pre_process_yaml(str(config_file))) or {} + except (OSError, yaml.YAMLError): + return None + cls = resolved.get("class") + return cls if isinstance(cls, str) and cls else None + + +def build_module_classes() -> dict[str, str]: + """Maps every local module name to its configured class: (e.g. + "pyobs.modules.camera.BaseCamera") -- feeds api/modules/classes/ (issue #65), which lets + an external caller (e.g. pyobs-portal) filter modules by interface on its own + side, using its own pyobs-core install, without this app importing pyobs.interfaces or + the module's actual class itself. + + A module whose config can't be resolved, or that has no top-level "class:" key, is + simply omitted -- not included with a None/error value -- since the caller only cares + about modules it can actually resolve to a class. + """ + result: dict[str, str] = {} + for name in list_modules(): + cls = get_module_class(name) + if cls: + result[name] = cls + return result + + def get_resolved_comm(name: str) -> tuple[str | None, str | None, str | None]: """Returns (comm_user, comm_password, source) for a module's *effective* comm: block -- the same resolution get_resolved_acl uses for acl:, via pre_process_yaml + @@ -2140,7 +2202,7 @@ def _replace_comm_password(raw: str, new_password: str) -> str: break i += 1 - if block_start is None: + if block_start is None or block_end is None: raise ValueError("no top-level comm: block found") password_re = re.compile(r"^(\s*)password\s*:\s*.*$") @@ -2403,6 +2465,24 @@ def build_acl_matrix() -> dict: return {"targets": rows, "callers": caller_names} +def merge_module_classes(per_host: list[tuple[str, dict[str, str]]]) -> list[dict[str, str]]: + """Combines each host's build_module_classes()-shaped result (a flat {name: class} dict) + into one fleet-wide list, tagged by host -- see module-classes-fleet-aggregation.md. + per_host is a list of (host_name, classes) pairs, e.g. [("localhost", + build_module_classes()), ("MONETS", ), + ...]. + + No collision arbitration: a same-named module on two hosts becomes two distinct rows, + disambiguated by host, rather than one overwriting the other -- the same choice + merge_acl_matrices makes for ACL rows. + """ + return [ + {"name": name, "class": cls, "host": host_name} + for host_name, classes in per_host + for name, cls in classes.items() + ] + + def merge_acl_matrices(per_host: list[tuple[str, dict]]) -> dict: """Combines each host's build_acl_matrix()-shaped result into one fleet-wide matrix -- see acl-matrix.md, "Hub mode interaction". per_host is a list of (host_name, matrix) @@ -2422,4 +2502,4 @@ def merge_acl_matrices(per_host: list[tuple[str, dict]]) -> dict: for host_name, matrix in per_host for row in matrix["targets"] ] - return {"targets": rows, "callers": caller_names} \ No newline at end of file + return {"targets": rows, "callers": caller_names} diff --git a/modules/tests.py b/modules/tests.py index 974323d..e5b5c61 100644 --- a/modules/tests.py +++ b/modules/tests.py @@ -4,7 +4,7 @@ import tempfile import time import unittest -from datetime import datetime, timedelta, timezone +from datetime import UTC, datetime, timedelta from pathlib import Path from unittest.mock import MagicMock, patch @@ -19,11 +19,10 @@ from modules import ejabberd, services, views from modules.middleware import HubTokenMiddleware -from modules.views import _tag_host from modules.pyobs_config import include_parts, pre_process_yaml, reload_anchors +from modules.views import _tag_host from pyobs_web_admin.authentication.admin_sync import sync_admin_user - # ── include_parts ───────────────────────────────────────────────────────────── class IncludePartsTests(unittest.TestCase): @@ -260,6 +259,106 @@ def test_malformed_allow_reports_error_not_raise(self): self.assertIsNotNone(error) +# ── services.get_module_class / build_module_classes (issue #65) ──────────────── + +class GetModuleClassTests(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.TemporaryDirectory() + self.tmp_path = Path(self.tmp.name) + self._settings = override_settings(PYOBS_CONFIG_DIR=str(self.tmp_path), PYOBS_CONFIG_GIT_ENABLED=False) + self._settings.enable() + + def tearDown(self): + self._settings.disable() + self.tmp.cleanup() + + def _write(self, name: str, content: str) -> None: + (self.tmp_path / f"{name}.yaml").write_text(content) + + def test_missing_module_returns_none(self): + self.assertIsNone(services.get_module_class("nope")) + + def test_no_class_key_returns_none(self): + self._write("cam1", "comm:\n user: camera\n") + self.assertIsNone(services.get_module_class("cam1")) + + def test_class_defined_locally(self): + self._write("cam1", "class: pyobs.modules.camera.BaseCamera\n") + self.assertEqual(services.get_module_class("cam1"), "pyobs.modules.camera.BaseCamera") + + def test_class_via_include(self): + self._write("base.shared", "class: pyobs.modules.camera.BaseCamera\n") + self._write("cam1", "{include base.shared.yaml}\n") + self.assertEqual(services.get_module_class("cam1"), "pyobs.modules.camera.BaseCamera") + + def test_broken_config_returns_none_not_raise(self): + self._write("cam1", "class: [unterminated\n") + self.assertIsNone(services.get_module_class("cam1")) + + +class BuildModuleClassesTests(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.TemporaryDirectory() + self.tmp_path = Path(self.tmp.name) + self._settings = override_settings(PYOBS_CONFIG_DIR=str(self.tmp_path), PYOBS_CONFIG_GIT_ENABLED=False) + self._settings.enable() + + def tearDown(self): + self._settings.disable() + self.tmp.cleanup() + + def _write(self, name: str, content: str) -> None: + (self.tmp_path / f"{name}.yaml").write_text(content) + + def test_maps_every_module_with_a_class(self): + self._write("cam1", "class: pyobs.modules.camera.BaseCamera\n") + self._write("tel1", "class: pyobs.modules.telescope.BaseTelescope\n") + self.assertEqual( + services.build_module_classes(), + {"cam1": "pyobs.modules.camera.BaseCamera", "tel1": "pyobs.modules.telescope.BaseTelescope"}, + ) + + def test_omits_modules_with_no_resolvable_class(self): + self._write("cam1", "class: pyobs.modules.camera.BaseCamera\n") + self._write("broken", "class: [unterminated\n") + self.assertEqual(services.build_module_classes(), {"cam1": "pyobs.modules.camera.BaseCamera"}) + + def test_no_modules_returns_empty_dict(self): + self.assertEqual(services.build_module_classes(), {}) + + +# ── services.merge_module_classes ─────────────────────────────────────────────── + +class MergeModuleClassesTests(unittest.TestCase): + def test_tags_rows_with_their_host(self): + merged = services.merge_module_classes([ + ("localhost", {"cam1": "pyobs.modules.camera.BaseCamera"}), + ("MONETS", {"telescope": "pyobs.modules.telescope.BaseTelescope"}), + ]) + self.assertIn({"name": "cam1", "class": "pyobs.modules.camera.BaseCamera", "host": "localhost"}, merged) + self.assertIn( + {"name": "telescope", "class": "pyobs.modules.telescope.BaseTelescope", "host": "MONETS"}, merged + ) + + def test_same_named_module_on_two_hosts_becomes_two_rows(self): + """No collision arbitration -- disambiguated by host, same choice merge_acl_matrices + makes for ACL rows, rather than one host's entry silently overwriting the other's.""" + merged = services.merge_module_classes([ + ("localhost", {"cam1": "pyobs.modules.camera.BaseCamera"}), + ("MONETS", {"cam1": "pyobs.modules.camera.Sbig"}), + ]) + self.assertEqual(len(merged), 2) + self.assertIn({"name": "cam1", "class": "pyobs.modules.camera.BaseCamera", "host": "localhost"}, merged) + self.assertIn({"name": "cam1", "class": "pyobs.modules.camera.Sbig", "host": "MONETS"}, merged) + + def test_empty_input_returns_empty_list(self): + self.assertEqual(services.merge_module_classes([]), []) + + def test_host_with_no_modules_contributes_nothing(self): + merged = services.merge_module_classes([("localhost", {}), ("MONETS", {"cam1": "pyobs.modules.camera.Sbig"})]) + self.assertEqual(merged, [{"name": "cam1", "class": "pyobs.modules.camera.Sbig", "host": "MONETS"}]) + + # ── services.get_comm_user ──────────────────────────────────────────────────── class GetCommUserTests(unittest.TestCase): @@ -1338,7 +1437,7 @@ def test_get_log_stats_since_narrows_window_when_more_recent_than_24h(self, mock # A dashboard-supplied "last acknowledged" instant more recent than the standard 24h # rollup should become the actual --since cutoff, not just widen/ignore it. mock_run.return_value = self._mock_result("") - since = datetime.now(timezone.utc) - timedelta(hours=1) + since = datetime.now(UTC) - timedelta(hours=1) services.get_log_stats("camera_verify_test", since=since) mock_run.assert_called_once_with( ["journalctl", "SYSLOG_IDENTIFIER=pyobs", "PYOBS_MODULE=camera_verify_test", @@ -1352,13 +1451,13 @@ def test_get_log_stats_since_older_than_24h_falls_back_to_24h(self, mock_run): # An ack from days ago shouldn't pull that whole history back into the "unacknowledged" # count -- the window is still capped at the standard 24h rollup. mock_run.return_value = self._mock_result("") - since = datetime.now(timezone.utc) - timedelta(days=3) + since = datetime.now(UTC) - timedelta(days=3) services.get_log_stats("camera_verify_test", since=since) args = mock_run.call_args[0][0] since_arg = args[args.index("--since") + 1] self.assertTrue(since_arg.endswith(" UTC")) - cutoff = datetime.strptime(since_arg[:-len(" UTC")], "%Y-%m-%d %H:%M:%S").replace(tzinfo=timezone.utc) - expected = datetime.now(timezone.utc) - timedelta(hours=24) + cutoff = datetime.strptime(since_arg[:-len(" UTC")], "%Y-%m-%d %H:%M:%S").replace(tzinfo=UTC) + expected = datetime.now(UTC) - timedelta(hours=24) self.assertLess(abs((cutoff - expected).total_seconds()), 5) @override_settings(PYOBS_LOG_BACKEND="journald") @@ -1403,7 +1502,7 @@ def test_get_logs_before_adds_until_arg_ahead_of_lines_flag(self, mock_run): --until, mirroring get_log_stats' existing --since usage, so "-n" then returns the last N entries *at or before* that instant instead of the last N overall.""" mock_run.return_value = self._mock_result("") - before = datetime(2026, 7, 15, 10, 0, 0, tzinfo=timezone.utc) + before = datetime(2026, 7, 15, 10, 0, 0, tzinfo=UTC) services.get_logs("camera_verify_test", lines=300, before=before) mock_run.assert_called_once_with( ["journalctl", "SYSLOG_IDENTIFIER=pyobs", "PYOBS_MODULE=camera_verify_test", @@ -1417,7 +1516,7 @@ def test_get_logs_since_adds_since_arg_ahead_of_lines_flag(self, mock_run): """The time-range start date becomes journalctl's --since, so `-n` returns the last N entries *at or after* that instant instead of the last N overall.""" mock_run.return_value = self._mock_result("") - since = datetime(2026, 7, 15, 10, 0, 0, tzinfo=timezone.utc) + since = datetime(2026, 7, 15, 10, 0, 0, tzinfo=UTC) services.get_logs("camera_verify_test", lines=300, since=since) mock_run.assert_called_once_with( ["journalctl", "SYSLOG_IDENTIFIER=pyobs", "PYOBS_MODULE=camera_verify_test", @@ -1429,8 +1528,8 @@ def test_get_logs_since_adds_since_arg_ahead_of_lines_flag(self, mock_run): @patch("modules.services.subprocess.run") def test_get_logs_since_and_before_combine_both_bounds(self, mock_run): mock_run.return_value = self._mock_result("") - since = datetime(2026, 7, 15, 9, 0, 0, tzinfo=timezone.utc) - before = datetime(2026, 7, 15, 10, 0, 0, tzinfo=timezone.utc) + since = datetime(2026, 7, 15, 9, 0, 0, tzinfo=UTC) + before = datetime(2026, 7, 15, 10, 0, 0, tzinfo=UTC) services.get_logs("camera_verify_test", lines=300, since=since, before=before) mock_run.assert_called_once_with( ["journalctl", "SYSLOG_IDENTIFIER=pyobs", "PYOBS_MODULE=camera_verify_test", @@ -1439,6 +1538,56 @@ def test_get_logs_since_and_before_combine_both_bounds(self, mock_run): capture_output=True, text=True, ) + @override_settings(PYOBS_LOG_BACKEND="journald") + @patch("modules.services.subprocess.run") + def test_get_logs_until_adds_until_arg(self, mock_run): + """The time-range end date (`until`) becomes journalctl's --until, so `-n` returns the + last N entries *at or before* the end date instead of the last N overall -- without + this the server kept returning the newest lines and the client-side end filter wiped + out a past window entirely.""" + mock_run.return_value = self._mock_result("") + since = datetime(2026, 7, 15, 9, 0, 0, tzinfo=UTC) + until = datetime(2026, 7, 15, 10, 0, 0, tzinfo=UTC) + services.get_logs("camera_verify_test", lines=300, since=since, until=until) + mock_run.assert_called_once_with( + ["journalctl", "SYSLOG_IDENTIFIER=pyobs", "PYOBS_MODULE=camera_verify_test", + "--since", "2026-07-15 09:00:00 UTC", "--until", "2026-07-15 10:00:00 UTC", + "-n", "300", "-o", "json", "--no-pager"], + capture_output=True, text=True, + ) + + @override_settings(PYOBS_LOG_BACKEND="journald") + @patch("modules.services.subprocess.run") + def test_get_logs_before_and_until_tighter_bound_wins(self, mock_run): + """A page-back cursor (`before`) and an end date (`until`) are both upper bounds -- + journald takes a single --until, so the earlier of the two is used.""" + mock_run.return_value = self._mock_result("") + since = datetime(2026, 7, 15, 9, 0, 0, tzinfo=UTC) + before = datetime(2026, 7, 15, 9, 30, 0, tzinfo=UTC) + until = datetime(2026, 7, 15, 10, 0, 0, tzinfo=UTC) + services.get_logs("camera_verify_test", lines=300, since=since, before=before, until=until) + mock_run.assert_called_once_with( + ["journalctl", "SYSLOG_IDENTIFIER=pyobs", "PYOBS_MODULE=camera_verify_test", + "--since", "2026-07-15 09:00:00 UTC", "--until", "2026-07-15 09:30:00 UTC", + "-n", "300", "-o", "json", "--no-pager"], + capture_output=True, text=True, + ) + + @override_settings(PYOBS_LOG_BACKEND="journald") + @patch("modules.services.subprocess.run") + def test_get_logs_until_newer_than_cursor_cursor_wins(self, mock_run): + mock_run.return_value = self._mock_result("") + since = datetime(2026, 7, 15, 9, 0, 0, tzinfo=UTC) + before = datetime(2026, 7, 15, 11, 0, 0, tzinfo=UTC) + until = datetime(2026, 7, 15, 10, 0, 0, tzinfo=UTC) + services.get_logs("camera_verify_test", lines=300, since=since, before=before, until=until) + mock_run.assert_called_once_with( + ["journalctl", "SYSLOG_IDENTIFIER=pyobs", "PYOBS_MODULE=camera_verify_test", + "--since", "2026-07-15 09:00:00 UTC", "--until", "2026-07-15 10:00:00 UTC", + "-n", "300", "-o", "json", "--no-pager"], + capture_output=True, text=True, + ) + @override_settings(PYOBS_LOG_BACKEND="journald") @patch("modules.services.subprocess.run") def test_get_logs_merges_comm_user_identity(self, mock_run): @@ -1577,7 +1726,7 @@ def test_file_backend_before_returns_empty_list_not_a_tail(self, mock_run): log_file = Path(tmp) / "camera.log" log_file.write_text("2026-07-04 08:00:00 [INFO] (camera) x.py:1 hello\n") with override_settings(PYOBS_LOG_DIR=tmp, PYOBS_LOG_BACKEND="file"): - lines = services.get_logs("camera", before=datetime(2026, 7, 4, tzinfo=timezone.utc)) + lines = services.get_logs("camera", before=datetime(2026, 7, 4, tzinfo=UTC)) self.assertEqual(lines, []) mock_run.assert_not_called() @@ -1590,7 +1739,7 @@ def test_file_backend_since_filters_tail_to_window(self, mock_run): "2026-07-04 09:00:00 [INFO] (camera) x.py:2 new\n" ) mock_run.return_value = MagicMock(stdout=log_file.read_text()) - since = datetime(2026, 7, 4, 8, 30, 0, tzinfo=timezone.utc) + since = datetime(2026, 7, 4, 8, 30, 0, tzinfo=UTC) with override_settings(PYOBS_LOG_DIR=tmp, PYOBS_LOG_BACKEND="file"): lines = services.get_logs("camera", lines=300, since=since) self.assertEqual(lines, ["2026-07-04 09:00:00 [INFO] (camera) x.py:2 new"]) @@ -1605,12 +1754,92 @@ def test_file_backend_since_and_before_pages_within_window(self): "2026-07-04 09:00:00 [INFO] (camera) x.py:2 mid\n" "2026-07-04 10:00:00 [INFO] (camera) x.py:3 new\n" ) - since = datetime(2026, 7, 4, 8, 30, 0, tzinfo=timezone.utc) - before = datetime(2026, 7, 4, 9, 30, 0, tzinfo=timezone.utc) + since = datetime(2026, 7, 4, 8, 30, 0, tzinfo=UTC) + before = datetime(2026, 7, 4, 9, 30, 0, tzinfo=UTC) with override_settings(PYOBS_LOG_DIR=tmp, PYOBS_LOG_BACKEND="file"): lines = services.get_logs("camera", lines=300, since=since, before=before) self.assertEqual(lines, ["2026-07-04 09:00:00 [INFO] (camera) x.py:2 mid"]) + @patch("modules.services.subprocess.run") + def test_file_backend_until_caps_the_tail(self, mock_run): + """The time-range end date (`until`) bounds the plain tail from above, so the bug + scenario -- user picks a [since, until] window that ends before the newest activity -- + returns exactly the window's lines instead of the newest N lines (which the old + client-only end filter then wiped out entirely).""" + with tempfile.TemporaryDirectory() as tmp: + log_file = Path(tmp) / "camera.log" + log_file.write_text( + "2026-07-04 08:00:00 [INFO] (camera) x.py:1 before window\n" + "2026-07-04 09:00:00 [INFO] (camera) x.py:2 in window\n" + "2026-07-04 09:30:00 [INFO] (camera) x.py:3 in window\n" + "2026-07-04 14:00:00 [INFO] (camera) x.py:4 after window\n" + ) + mock_run.return_value = MagicMock(stdout=log_file.read_text()) + since = datetime(2026, 7, 4, 8, 30, 0, tzinfo=UTC) + until = datetime(2026, 7, 4, 10, 0, 0, tzinfo=UTC) + with override_settings(PYOBS_LOG_DIR=tmp, PYOBS_LOG_BACKEND="file"): + lines = services.get_logs("camera", lines=300, since=since, until=until) + self.assertEqual(lines, [ + "2026-07-04 09:00:00 [INFO] (camera) x.py:2 in window", + "2026-07-04 09:30:00 [INFO] (camera) x.py:3 in window", + ]) + mock_run.assert_called_once_with( + ["tail", "-n", "300", str(log_file)], capture_output=True, text=True + ) + + def test_file_backend_until_without_since_filters_tail_to_end_date(self): + """An end date alone is a plain upper bound -- the last N lines at or before it, no + page-back semantics involved (unlike a bare `before`, which needs a `since`).""" + with tempfile.TemporaryDirectory() as tmp: + (Path(tmp) / "camera.log").write_text( + "2026-07-04 08:00:00 [INFO] (camera) x.py:1 in window\n" + "2026-07-04 09:00:00 [INFO] (camera) x.py:2 in window\n" + "2026-07-04 14:00:00 [INFO] (camera) x.py:3 after window\n" + ) + until = datetime(2026, 7, 4, 10, 0, 0, tzinfo=UTC) + with override_settings(PYOBS_LOG_DIR=tmp, PYOBS_LOG_BACKEND="file"): + lines = services.get_logs("camera", lines=300, until=until) + self.assertEqual(lines, [ + "2026-07-04 08:00:00 [INFO] (camera) x.py:1 in window", + "2026-07-04 09:00:00 [INFO] (camera) x.py:2 in window", + ]) + + def test_file_backend_before_and_until_tighter_bound_wins(self): + """A scroll-to-top page-back (`before`) combined with an end date (`until`): the + earlier of the two caps the window, so paging back never resurfaces lines newer than + the end date.""" + with tempfile.TemporaryDirectory() as tmp: + (Path(tmp) / "camera.log").write_text( + "2026-07-04 08:00:00 [INFO] (camera) x.py:1 old\n" + "2026-07-04 09:00:00 [INFO] (camera) x.py:2 mid\n" + "2026-07-04 09:30:00 [INFO] (camera) x.py:3 late mid\n" + "2026-07-04 14:00:00 [INFO] (camera) x.py:4 after window\n" + ) + since = datetime(2026, 7, 4, 8, 30, 0, tzinfo=UTC) + before = datetime(2026, 7, 4, 9, 20, 0, tzinfo=UTC) # cursor older than until + until = datetime(2026, 7, 4, 10, 0, 0, tzinfo=UTC) # end date + with override_settings(PYOBS_LOG_DIR=tmp, PYOBS_LOG_BACKEND="file"): + lines = services.get_logs("camera", lines=300, since=since, before=before, until=until) + self.assertEqual(lines, ["2026-07-04 09:00:00 [INFO] (camera) x.py:2 mid"]) + + def test_file_backend_until_caps_page_back_when_newer_than_cursor(self): + """The other ordering: until (end date) is newer than the page-back cursor -- the + cursor still wins as the tighter upper bound.""" + with tempfile.TemporaryDirectory() as tmp: + (Path(tmp) / "camera.log").write_text( + "2026-07-04 08:00:00 [INFO] (camera) x.py:1 old\n" + "2026-07-04 09:00:00 [INFO] (camera) x.py:2 mid\n" + "2026-07-04 14:00:00 [INFO] (camera) x.py:3 after window\n" + ) + since = datetime(2026, 7, 4, 8, 30, 0, tzinfo=UTC) + before = datetime(2026, 7, 4, 15, 0, 0, tzinfo=UTC) # cursor newer than until + until = datetime(2026, 7, 4, 10, 0, 0, tzinfo=UTC) # end date + with override_settings(PYOBS_LOG_DIR=tmp, PYOBS_LOG_BACKEND="file"): + lines = services.get_logs("camera", lines=300, since=since, before=before, until=until) + self.assertEqual(lines, [ + "2026-07-04 09:00:00 [INFO] (camera) x.py:2 mid", + ]) + # ── journald PYOBS_MODULE version gating ───────────────────────────────────── @@ -1811,7 +2040,7 @@ def test_parses_newest_matching_entry(self, mock_run, mock_process_cls, _mock_al services.get_module_versions("camera"), {"pyobs-core": "2.0.0.dev76", "pyobs-fli": "2.0.0.dev7"}, ) - since = datetime.fromtimestamp(1700000000.0, tz=timezone.utc) + since = datetime.fromtimestamp(1700000000.0, tz=UTC) mock_run.assert_called_once_with( ["journalctl", "SYSLOG_IDENTIFIER=pyobs", "PYOBS_MODULE=camera", "--since", f"{since:%Y-%m-%d %H:%M:%S} UTC", "--grep", "Loaded pyobs packages: ", @@ -2048,7 +2277,7 @@ def test_file_backend_merges_and_sorts_across_modules_by_timestamp(self): def test_file_backend_get_log_stats_since_narrows_the_24h_window(self): with tempfile.TemporaryDirectory() as tmp: - now = datetime.now(timezone.utc) + now = datetime.now(UTC) old_ts = (now - timedelta(hours=2)).strftime("%Y-%m-%d %H:%M:%S") recent_ts = (now - timedelta(minutes=30)).strftime("%Y-%m-%d %H:%M:%S") (Path(tmp) / "camera.log").write_text( @@ -2109,7 +2338,7 @@ def test_filter_str_applies_after_merge(self): @patch("modules.services.subprocess.run") def test_journald_before_adds_until_arg_ahead_of_lines_flag(self, mock_run): mock_run.return_value = self._mock_result("") - before = datetime(2026, 7, 15, 10, 0, 0, tzinfo=timezone.utc) + before = datetime(2026, 7, 15, 10, 0, 0, tzinfo=UTC) services.get_all_logs(names=["camera"], lines=300, before=before) mock_run.assert_called_once_with( ["journalctl", "SYSLOG_IDENTIFIER=pyobs", "PYOBS_MODULE=camera", @@ -2124,7 +2353,7 @@ def test_file_backend_before_returns_empty_list(self, mock_run): with tempfile.TemporaryDirectory() as tmp: (Path(tmp) / "camera.log").write_text("2026-07-04 08:00:00 [INFO] (camera) x.py:1 hello\n") with override_settings(PYOBS_LOG_DIR=tmp, PYOBS_LOG_BACKEND="file"): - lines = services.get_all_logs(names=["camera"], before=datetime(2026, 7, 4, tzinfo=timezone.utc)) + lines = services.get_all_logs(names=["camera"], before=datetime(2026, 7, 4, tzinfo=UTC)) self.assertEqual(lines, []) mock_run.assert_not_called() @@ -2132,7 +2361,7 @@ def test_file_backend_before_returns_empty_list(self, mock_run): @patch("modules.services.subprocess.run") def test_journald_since_adds_since_arg_ahead_of_lines_flag(self, mock_run): mock_run.return_value = self._mock_result("") - since = datetime(2026, 7, 15, 10, 0, 0, tzinfo=timezone.utc) + since = datetime(2026, 7, 15, 10, 0, 0, tzinfo=UTC) services.get_all_logs(names=["camera"], lines=300, since=since) mock_run.assert_called_once_with( ["journalctl", "SYSLOG_IDENTIFIER=pyobs", "PYOBS_MODULE=camera", @@ -2149,7 +2378,7 @@ def test_file_backend_since_filters_across_modules(self): (Path(tmp) / "telescope.log").write_text( "2026-07-04 09:30:00 [INFO] (telescope) y.py:1 new telescope\n" ) - since = datetime(2026, 7, 4, 8, 30, 0, tzinfo=timezone.utc) + since = datetime(2026, 7, 4, 8, 30, 0, tzinfo=UTC) with override_settings(PYOBS_LOG_DIR=tmp, PYOBS_LOG_BACKEND="file"): lines = services.get_all_logs(names=["camera", "telescope"], lines=300, since=since) self.assertEqual(lines, [ @@ -2647,6 +2876,125 @@ def test_named_clients_and_legacy_token_coexist(self): self.assertEqual(request._hub_client, "default") +# ── api_module_classes ─────────────────────────────────────────────────────── + +class ApiModuleClassesTests(unittest.TestCase): + """Issue #68: api_module_classes went from "always local" (flat {name: class} dict) to + fleet-aggregating (loops ["localhost"] + HUB_HOSTS, merges, reports unreachable hosts), + matching api_all_logs' pattern. See module-classes-fleet-aggregation.md.""" + + def setUp(self): + self.factory = RequestFactory() + self.hosts = [{"name": "MONETS", "url": "http://monets", "token": "tok"}] + + def _request(self): + return self.factory.get("/api/modules/classes/") + + @patch("modules.services.build_module_classes") + def test_single_host_no_hub_hosts_returns_local_modules_in_new_shape(self, mock_build): + mock_build.return_value = {"cam1": "pyobs.modules.camera.BaseCamera"} + response = views.api_module_classes(self._request()) + self.assertEqual(response.status_code, 200) + data = json.loads(response.content) + self.assertEqual( + data, {"modules": [{"name": "cam1", "class": "pyobs.modules.camera.BaseCamera", "host": "localhost"}], + "unreachable_hosts": []}, + ) + + @override_settings(HUB_HOSTS=[{"name": "MONETS", "url": "http://monets", "token": "tok"}]) + @patch("modules.proxy.call") + @patch("modules.services.build_module_classes") + def test_multi_host_merges_local_and_remote(self, mock_build, mock_call): + mock_build.return_value = {"cam1": "pyobs.modules.camera.BaseCamera"} + mock_call.return_value = { + "modules": [{"name": "telescope", "class": "pyobs.modules.telescope.BaseTelescope", "host": "localhost"}], + "unreachable_hosts": [], + } + response = views.api_module_classes(self._request()) + data = json.loads(response.content) + self.assertCountEqual( + data["modules"], + [ + {"name": "cam1", "class": "pyobs.modules.camera.BaseCamera", "host": "localhost"}, + {"name": "telescope", "class": "pyobs.modules.telescope.BaseTelescope", "host": "MONETS"}, + ], + ) + self.assertEqual(data["unreachable_hosts"], []) + mock_call.assert_called_once_with(self.hosts[0], "GET", "/api/modules/classes/") + + @override_settings(HUB_HOSTS=[{"name": "MONETS", "url": "http://monets", "token": "tok"}]) + @patch("modules.proxy.call") + @patch("modules.services.build_module_classes") + def test_unreachable_host_is_reported_not_fatal(self, mock_build, mock_call): + mock_build.return_value = {"cam1": "pyobs.modules.camera.BaseCamera"} + mock_call.side_effect = Exception("connection refused") + response = views.api_module_classes(self._request()) + self.assertEqual(response.status_code, 200) + data = json.loads(response.content) + self.assertEqual( + data["modules"], [{"name": "cam1", "class": "pyobs.modules.camera.BaseCamera", "host": "localhost"}] + ) + self.assertEqual(len(data["unreachable_hosts"]), 1) + self.assertEqual(data["unreachable_hosts"][0]["name"], "MONETS") + + @override_settings(HUB_HOSTS=[{"name": "MONETS", "url": "http://monets", "token": "tok"}]) + @patch("modules.proxy.call") + @patch("modules.services.build_module_classes") + def test_nested_hub_preserves_sub_host_tags_instead_of_collapsing_them(self, mock_build, mock_call): + """MONETS is itself a hub with its own sub-host "south" -- its response is already + host-tagged (it went through this same view), including a "cam1" on both its own + localhost and "south". Re-flattening that into one {name: class} dict per remote + would silently drop one of the two "cam1" entries; each inner host must survive, + with only MONETS's own "localhost" rows re-tagged to "MONETS".""" + mock_build.return_value = {} + mock_call.return_value = { + "modules": [ + {"name": "cam1", "class": "pyobs.modules.camera.BaseCamera", "host": "localhost"}, + {"name": "cam1", "class": "pyobs.modules.camera.Sbig", "host": "south"}, + ], + "unreachable_hosts": [], + } + response = views.api_module_classes(self._request()) + data = json.loads(response.content) + self.assertCountEqual( + data["modules"], + [ + {"name": "cam1", "class": "pyobs.modules.camera.BaseCamera", "host": "MONETS"}, + {"name": "cam1", "class": "pyobs.modules.camera.Sbig", "host": "south"}, + ], + ) + + @override_settings(HUB_HOSTS=[{"name": "MONETS", "url": "http://monets", "token": "tok"}]) + @patch("modules.proxy.call") + @patch("modules.services.build_module_classes") + def test_nested_hubs_unreachable_sub_host_is_propagated(self, mock_build, mock_call): + mock_build.return_value = {} + mock_call.return_value = { + "modules": [], + "unreachable_hosts": [{"name": "south", "error": "connection refused"}], + } + response = views.api_module_classes(self._request()) + data = json.loads(response.content) + self.assertEqual(data["unreachable_hosts"], [{"name": "south", "error": "connection refused"}]) + + @override_settings(HUB_HOSTS=[{"name": "MONETS", "url": "http://monets", "token": "tok"}]) + @patch("modules.proxy.call") + @patch("modules.services.build_module_classes") + def test_remote_on_old_flat_dict_shape_is_reported_not_silently_dropped(self, mock_build, mock_call): + """A HUB_HOSTS remote not yet upgraded past #68 still answers with the pre-existing + flat {name: class} shape (no "modules" key) -- during a rolling deployment this must + surface as unreachable, not silently contribute zero modules with no explanation.""" + mock_build.return_value = {"cam1": "pyobs.modules.camera.BaseCamera"} + mock_call.return_value = {"telescope": "pyobs.modules.telescope.BaseTelescope"} + response = views.api_module_classes(self._request()) + data = json.loads(response.content) + self.assertEqual( + data["modules"], [{"name": "cam1", "class": "pyobs.modules.camera.BaseCamera", "host": "localhost"}] + ) + self.assertEqual(len(data["unreachable_hosts"]), 1) + self.assertEqual(data["unreachable_hosts"][0]["name"], "MONETS") + + class ApiAllLogStatsAcksTests(unittest.TestCase): """The dashboard sends its own localStorage "log-ack-" timestamps as an `acks` query param so its WARNING/ERROR/CRITICAL badges reflect unacknowledged issues rather than @@ -2668,7 +3016,7 @@ def test_acks_entry_is_parsed_into_a_since_datetime(self, mock_list_modules, moc mock_get_log_stats.return_value = {"DEBUG": 0, "INFO": 0, "WARNING": 0, "ERROR": 0, "CRITICAL": 0} response = views.api_all_log_stats(self._request({"camera": "2026-07-15T10:00:00.000Z"})) self.assertEqual(response.status_code, 200) - mock_get_log_stats.assert_called_once_with("camera", since=datetime(2026, 7, 15, 10, 0, 0, tzinfo=timezone.utc)) + mock_get_log_stats.assert_called_once_with("camera", since=datetime(2026, 7, 15, 10, 0, 0, tzinfo=UTC)) @patch("modules.services.get_log_stats") @patch("modules.services.list_modules") @@ -2720,7 +3068,7 @@ def setUp(self): def test_parse_before_accepts_iso_with_z_suffix(self): parsed = views._parse_ts("2026-07-15T10:00:00.000Z") - self.assertEqual(parsed, datetime(2026, 7, 15, 10, 0, 0, tzinfo=timezone.utc)) + self.assertEqual(parsed, datetime(2026, 7, 15, 10, 0, 0, tzinfo=UTC)) def test_parse_before_missing_is_none(self): self.assertIsNone(views._parse_ts(None)) @@ -2739,7 +3087,7 @@ def test_api_logs_forwards_before_to_get_logs(self, mock_list_modules, mock_get_ response = views.api_logs(request, "camera") self.assertEqual(response.status_code, 200) mock_get_logs.assert_called_once_with( - "camera", lines=300, filter_str="", before=datetime(2026, 7, 15, 10, 0, 0, tzinfo=timezone.utc), since=None + "camera", lines=300, filter_str="", before=datetime(2026, 7, 15, 10, 0, 0, tzinfo=UTC), since=None, until=None ) @patch("modules.services.get_logs") @@ -2750,7 +3098,7 @@ def test_api_logs_no_before_param_behaves_as_before(self, mock_list_modules, moc request = self.factory.get("/api/modules/camera/logs/", {"lines": 300}) request.session = {} views.api_logs(request, "camera") - mock_get_logs.assert_called_once_with("camera", lines=300, filter_str="", before=None, since=None) + mock_get_logs.assert_called_once_with("camera", lines=300, filter_str="", before=None, since=None, until=None) @patch("modules.services.get_all_logs") @patch("modules.services.list_modules") @@ -2764,7 +3112,7 @@ def test_api_all_logs_forwards_before_to_get_all_logs(self, mock_list_modules, m response = views.api_all_logs(request) self.assertEqual(response.status_code, 200) mock_get_all_logs.assert_called_once_with( - ["camera"], lines=300, filter_str="", before=datetime(2026, 7, 15, 10, 0, 0, tzinfo=timezone.utc), since=None + ["camera"], lines=300, filter_str="", before=datetime(2026, 7, 15, 10, 0, 0, tzinfo=UTC), since=None, until=None ) @patch("modules.services.get_logs") @@ -2777,7 +3125,7 @@ def test_api_logs_forwards_since_to_get_logs(self, mock_list_modules, mock_get_l response = views.api_logs(request, "camera") self.assertEqual(response.status_code, 200) mock_get_logs.assert_called_once_with( - "camera", lines=300, filter_str="", before=None, since=datetime(2026, 7, 15, 10, 0, 0, tzinfo=timezone.utc) + "camera", lines=300, filter_str="", before=None, since=datetime(2026, 7, 15, 10, 0, 0, tzinfo=UTC), until=None ) @patch("modules.services.get_all_logs") @@ -2792,7 +3140,39 @@ def test_api_all_logs_forwards_since_to_get_all_logs(self, mock_list_modules, mo response = views.api_all_logs(request) self.assertEqual(response.status_code, 200) mock_get_all_logs.assert_called_once_with( - ["camera"], lines=300, filter_str="", before=None, since=datetime(2026, 7, 15, 10, 0, 0, tzinfo=timezone.utc) + ["camera"], lines=300, filter_str="", before=None, since=datetime(2026, 7, 15, 10, 0, 0, tzinfo=UTC), until=None + ) + + @patch("modules.services.get_logs") + @patch("modules.services.list_modules") + def test_api_logs_forwards_until_to_get_logs(self, mock_list_modules, mock_get_logs): + """The time-range end date arrives as `until` (an ISO-8601 instant) and must be + forwarded to services.get_logs -- before this, the end date never reached the server, + so a range ending before the newest activity was wiped by the client-side filter even + though lines existed in the window.""" + mock_list_modules.return_value = ["camera"] + mock_get_logs.return_value = [] + request = self.factory.get("/api/modules/camera/logs/", {"lines": 300, "until": "2026-07-15T10:00:00.000Z"}) + request.session = {} + response = views.api_logs(request, "camera") + self.assertEqual(response.status_code, 200) + mock_get_logs.assert_called_once_with( + "camera", lines=300, filter_str="", before=None, since=None, until=datetime(2026, 7, 15, 10, 0, 0, tzinfo=UTC) + ) + + @patch("modules.services.get_all_logs") + @patch("modules.services.list_modules") + def test_api_all_logs_forwards_until_to_get_all_logs(self, mock_list_modules, mock_get_all_logs): + mock_list_modules.return_value = ["camera"] + mock_get_all_logs.return_value = [] + request = self.factory.get( + "/api/logs/", {"lines": 300, "modules": "localhost:camera", "until": "2026-07-15T10:00:00.000Z"} + ) + request.session = {} + response = views.api_all_logs(request) + self.assertEqual(response.status_code, 200) + mock_get_all_logs.assert_called_once_with( + ["camera"], lines=300, filter_str="", before=None, since=None, until=datetime(2026, 7, 15, 10, 0, 0, tzinfo=UTC) ) @@ -3562,7 +3942,7 @@ def test_wrong_password_does_not_create_or_touch_any_user(self): class AdminSyncTests(DjangoTestCase): """admin_sync.sync_admin_user is the primary way the settings-configured admin account gets created/kept in sync - wired to run after every `manage.py migrate` via the post_migrate - signal (AuthenticationConfig.ready()), same mechanism as pyobs-archive/pyobs-robotic-backend, + signal (AuthenticationConfig.ready()), same mechanism as pyobs-archive/pyobs-portal, so a fresh deployment doesn't need an interactive `createsuperuser` step.""" def test_sync_creates_a_staff_superuser_with_a_working_password(self): diff --git a/modules/urls.py b/modules/urls.py index 1a2eb45..36394d6 100644 --- a/modules/urls.py +++ b/modules/urls.py @@ -18,6 +18,7 @@ path("git-config/", views.git_config_page, name="git_config"), # API path("api/statuses/", views.api_all_statuses, name="api_all_statuses"), + path("api/modules/classes/", views.api_module_classes, name="api_module_classes"), path("api/packages/", views.api_packages, name="api_packages"), path("api/packages/update/status/", views.api_package_update_status, name="api_package_update_status"), path("api/packages//update/", views.api_package_update, name="api_package_update"), diff --git a/modules/views.py b/modules/views.py index 13acfa9..9cf96e0 100644 --- a/modules/views.py +++ b/modules/views.py @@ -15,7 +15,6 @@ from modules import ejabberd, proxy, services - # ── Auth ────────────────────────────────────────────────────────────────────── def login_view(request): @@ -32,7 +31,7 @@ def login_view(request): # /admin/ (is_staff-gated) too, without a second account to create/remember. The # matching superuser User is normally already synced by # pyobs_web_admin.authentication.admin_sync (post_migrate signal, same mechanism as - # archive/robotic-backend) - get_or_create here is just a safety net for a fresh + # archive/portal) - get_or_create here is just a safety net for a fresh # install that hasn't run `migrate` since ADMIN_PASSWORD_HASH was set. admin_user, _ = User.objects.get_or_create( username=username, @@ -221,7 +220,7 @@ def packages(request): def git_config_page(request): ctx = {"active_git_config": True} - + ctx["git_enabled"] = True ctx["config_dir"] = "" @@ -245,7 +244,7 @@ def git_config_page(request): ctx["push_disabled"] = not ctx["git_status"]["branch"] or (ctx["git_status"]["clean"] and ctx["git_status"].get("ahead", 0) == 0) ctx["reset_disabled"] = not ctx["git_status"]["dirty"] ctx["git_change_count"] = len(ctx["git_status"].get("new_files", [])) + len(ctx["git_status"].get("modified_files", [])) + len(ctx["git_status"].get("deleted_files", [])) - + return render(request, "modules/git_config.html", ctx) @@ -523,11 +522,12 @@ def api_deactivate(request, name: str): # ── Logs API ────────────────────────────────────────────────────────────────── def _parse_ts(raw: str | None) -> datetime | None: - """Parses a timestamp query param (an ISO-8601 instant) -- used for both `before` (the + """Parses a timestamp query param (an ISO-8601 instant) -- used for `before` (the oldest currently-loaded log line's own timestamp, sent by the log windows' scroll-to-top - "load older logs" fetch) and `since` (the time-range start date, sent once a start date is - set). Malformed/missing input is treated as "no cutoff" rather than a 400 -- same - tolerance api_all_log_stats' acks parsing already gives a per-module timestamp from the + "load older logs" fetch), `since` (the time-range start date, sent once a start date is + set) and `until` (the time-range end date, sent once an end date is set). Malformed/ + missing input is treated as "no cutoff" rather than a 400 -- same tolerance + api_all_log_stats' acks parsing already gives a per-module timestamp from the same frontend-supplied-Date.toISOString() source.""" if not raw: return None @@ -543,18 +543,22 @@ def api_logs(request, name: str): lines = int(request.GET.get("lines", 300)) before_raw = request.GET.get("before") since_raw = request.GET.get("since") + until_raw = request.GET.get("until") before = _parse_ts(before_raw) since = _parse_ts(since_raw) + until = _parse_ts(until_raw) if host: params = {"lines": lines} if before_raw: params["before"] = before_raw if since_raw: params["since"] = since_raw + if until_raw: + params["until"] = until_raw return _proxy(host, "GET", f"/api/modules/{name}/logs/", params=params) _get_module_or_404(name) filter_str = request.GET.get("filter", "") - log_lines = services.get_logs(name, lines=min(lines, 2000), filter_str=filter_str, before=before, since=since) + log_lines = services.get_logs(name, lines=min(lines, 2000), filter_str=filter_str, before=before, since=since, until=until) return JsonResponse({"lines": log_lines}) @@ -596,8 +600,10 @@ def api_all_logs(request): modules_param = request.GET.get("modules") before_raw = request.GET.get("before") since_raw = request.GET.get("since") + until_raw = request.GET.get("until") before = _parse_ts(before_raw) since = _parse_ts(since_raw) + until = _parse_ts(until_raw) all_host_names = ["localhost"] + [h["name"] for h in getattr(settings, "HUB_HOSTS", [])] if modules_param is None: @@ -620,7 +626,7 @@ def api_all_logs(request): if names is not None: for name in names: _get_module_or_404(name) - host_lines = services.get_all_logs(names, lines=min(lines, 2000), filter_str=filter_str, before=before, since=since) + host_lines = services.get_all_logs(names, lines=min(lines, 2000), filter_str=filter_str, before=before, since=since, until=until) else: host_cfg = proxy.get_host_config(host_name) if not host_cfg: @@ -638,13 +644,15 @@ def api_all_logs(request): params["before"] = before_raw if since_raw: params["since"] = since_raw + if until_raw: + params["until"] = until_raw data = proxy.call(host_cfg, "GET", "/api/logs/", params=params) host_lines = data.get("lines", []) except Exception as e: unreachable.append({"name": host_name, "error": str(e)}) continue multi_host = len(host_selections) > 1 - line_lists.append([_tag_host(l, host_name) for l in host_lines] if multi_host else host_lines) + line_lists.append([_tag_host(line, host_name) for line in host_lines] if multi_host else host_lines) log_lines = services.merge_log_lines(line_lists, lines) return JsonResponse({"lines": log_lines, "unreachable_hosts": unreachable}) @@ -749,6 +757,61 @@ def api_shared_config(request, name: str): return JsonResponse({"error": "Method not allowed"}, status=405) +@require_GET +def api_module_classes(request): + """Every configured module's class: across the whole fleet (issue #65, extended by #68) -- + fleet-aggregating like api_all_logs above, not "always local" anymore: loops + ["localhost"] + HUB_HOSTS, using proxy.get_host_config + proxy.call for the remote + branches (same pattern as api_all_logs), and merges with services.merge_module_classes. + This composes for nested hubs for free -- when a hub instance is asked, whatever + HUB_HOSTS *it* has configured gets folded in automatically: a remote's own response is + already host-tagged (it went through this same view), so each row's inner host is kept + as-is (only its "localhost" tag is rewritten to the outer hub's name) instead of being + re-flattened into one dict per remote -- otherwise a same-named module on two of a + remote's own sub-hosts would silently collide one level deeper than the exact failure + this endpoint exists to eliminate. Authenticated via the existing HUB_CLIENTS + shared-secret mechanism (modules/middleware.py's HubTokenMiddleware), not a new auth + scheme. Lets the caller (e.g. pyobs-portal) filter modules by interface (ICamera, + ITelescope, ...) on its own side, using its own pyobs-core install -- this app never + imports pyobs.interfaces or the module's actual class to answer this. + + Response shape (breaking change from the old flat {module_name: class} dict -- see + module-classes-fleet-aggregation.md): + {"modules": [{"name": ..., "class": ..., "host": ...}, ...], + "unreachable_hosts": [{"name": ..., "error": ...}, ...]} + """ + per_host: list[tuple[str, dict[str, str]]] = [] + unreachable: list[dict] = [] + for host_name in ["localhost"] + [h["name"] for h in getattr(settings, "HUB_HOSTS", [])]: + if host_name == "localhost": + per_host.append((host_name, services.build_module_classes())) + continue + host_cfg = proxy.get_host_config(host_name) + if not host_cfg: + continue + try: + data = proxy.call(host_cfg, "GET", "/api/modules/classes/") + except Exception as e: + unreachable.append({"name": host_name, "error": str(e)}) + continue + if "modules" not in data: + # A HUB_HOSTS remote still on the pre-#68 flat-dict shape (mid-rollout fleet) -- + # surfaced rather than silently contributing nothing, which .get("modules", []) + # would otherwise do. + unreachable.append({"name": host_name, "error": "unexpected response shape (host not upgraded?)"}) + continue + unreachable.extend(data.get("unreachable_hosts", [])) + by_host: dict[str, dict[str, str]] = {} + for m in data["modules"]: + inner_host = m.get("host") or "localhost" + host = host_name if inner_host == "localhost" else inner_host + by_host.setdefault(host, {})[m["name"]] = m["class"] + per_host.extend(by_host.items()) + + modules = services.merge_module_classes(per_host) + return JsonResponse({"modules": modules, "unreachable_hosts": unreachable}) + + @require_GET def api_acl_matrix(request): """Queried by another pyobs-web-admin instance acting as a hub, to fold this diff --git a/pyobs_web_admin/authentication/admin_sync.py b/pyobs_web_admin/authentication/admin_sync.py index 460dc3b..70fcfe3 100644 --- a/pyobs_web_admin/authentication/admin_sync.py +++ b/pyobs_web_admin/authentication/admin_sync.py @@ -1,7 +1,7 @@ """Settings-configured admin account: ADMIN_USERNAME/ADMIN_PASSWORD_HASH, synced to a real superuser after every `manage.py migrate` (post_migrate signal, wired up in AuthenticationConfig.ready()) rather than on every login (login_view used to do this inline - -moved here for consistency with pyobs-archive/pyobs-robotic-backend, which have no shared- +moved here for consistency with pyobs-archive/pyobs-portal, which have no shared- password login step to hook a sync into). ADMIN_PASSWORD_HASH is already a Django-format hash (make_password() output, per settings.py's own instructions) - assigned straight to User.password rather than re-hashed. diff --git a/pyobs_web_admin/authentication/keycloak.py b/pyobs_web_admin/authentication/keycloak.py index 0a9db2e..4943087 100644 --- a/pyobs_web_admin/authentication/keycloak.py +++ b/pyobs_web_admin/authentication/keycloak.py @@ -1,13 +1,13 @@ """pyobs-auth USER_RESOLVER for web-admin. -Mirrors pyobs-archive/pyobs-robotic-backend's resolver: Keycloak's `sub` claim is the join key +Mirrors pyobs-archive/pyobs-portal's resolver: Keycloak's `sub` claim is the join key (see pyobs-core's shared-auth design doc), stored on KeycloakIdentity. First Keycloak login for an existing local User (matched by email, falling back to username) links the two rather than -minting a second, disconnected User. Newly-minted accounts default to is_active=False - -pyobs-auth's CallbackView/KeycloakAuthentication refuse an inactive user, so a fresh Keycloak -login needs local activation (Django admin, or `manage.py shell`) before it can do anything - -this is also the revocation mechanism: flip is_active back to False to cut a single person's -access without touching anyone else or the shared admin/password login. +minting a second, disconnected User. Newly-minted accounts are active by default: authorization +is now the PYOBS_AUTH['REQUIRED_GROUPS'] claims gate (Keycloak group membership), not local +activation - see pyobs-core's specs/design/shared-authz-keycloak.md. Granting/revoking a +person's access to web-admin is done in the Keycloak admin console (group membership) rather +than this app's Django admin. """ from __future__ import annotations @@ -23,8 +23,8 @@ def resolve_user(claims: dict[str, Any]) -> User | None: sub = claims["sub"] try: - return KeycloakIdentity.objects.get(keycloak_sub=sub).user - except KeycloakIdentity.DoesNotExist: + return KeycloakIdentity.objects.get(keycloak_sub=sub).user # pyrefly: ignore [missing-attribute] + except KeycloakIdentity.DoesNotExist: # pyrefly: ignore [missing-attribute] pass email = claims.get("email") @@ -38,8 +38,10 @@ def resolve_user(claims: dict[str, Any]) -> User | None: user = User.objects.filter(username=username).first() if user is None: user = User.objects.create( - username=username, email=email or "", is_active=False + username=username, email=email or "", is_active=True ) - KeycloakIdentity.objects.update_or_create(user=user, defaults={"keycloak_sub": sub}) + KeycloakIdentity.objects.update_or_create( # pyrefly: ignore [missing-attribute] + user=user, defaults={"keycloak_sub": sub} + ) return user diff --git a/pyobs_web_admin/authentication/tests.py b/pyobs_web_admin/authentication/tests.py index b5f4855..7400a83 100644 --- a/pyobs_web_admin/authentication/tests.py +++ b/pyobs_web_admin/authentication/tests.py @@ -19,9 +19,11 @@ def test_creates_a_new_user_on_first_login(self): self.assertEqual(user.email, "new@example.org") self.assertEqual(KeycloakIdentity.objects.get(user=user).keycloak_sub, "sub-1") - def test_new_user_is_created_inactive(self): + def test_new_user_is_created_active(self): + # Authorization is now the PYOBS_AUTH['REQUIRED_GROUPS'] claims gate, not local + # activation - see pyobs-core's specs/design/shared-authz-keycloak.md. user = resolve_user({"sub": "sub-2", "email": "pending@example.org"}) - self.assertFalse(user.is_active) + self.assertTrue(user.is_active) def test_same_sub_resolves_to_the_same_user_on_a_later_login(self): first = resolve_user({"sub": "sub-3", "email": "person@example.org"}) diff --git a/pyobs_web_admin/local_settings.py.example b/pyobs_web_admin/local_settings.py.example index f975700..64e8a12 100644 --- a/pyobs_web_admin/local_settings.py.example +++ b/pyobs_web_admin/local_settings.py.example @@ -47,10 +47,11 @@ ADMIN_PASSWORD_HASH = "" # Keycloak login (optional addon on top of the shared admin/password login above, not a # replacement -- leave SERVER_URL unset to disable it entirely; the login page won't show the -# button either). Unlike the shared admin account, each Keycloak-linked User can be individually -# deactivated at /admin/ without affecting anyone else -- new accounts mint inactive by default -# and need local activation there before they can log in. Run `manage.py migrate` once before -# first use (creates db.sqlite3, only used for this User table). +# button either). Unlike the shared admin account, a Keycloak-linked person's access is granted +# or revoked via Keycloak group membership (REQUIRED_GROUPS below), not local activation -- new +# accounts mint active by default. Run `manage.py migrate` once before first use (creates +# db.sqlite3, holding the User table and, since sessions are now DB-backed, every session too -- +# shared admin/password login included). # PYOBS_AUTH = { # "SERVER_URL": "https://keycloak.example.org", # "REALM": "pyobs", @@ -65,6 +66,13 @@ ADMIN_PASSWORD_HASH = "" # "IDP_HINT": "gwdg", # "IDP_LABEL": "GWDG", # "USER_RESOLVER": "pyobs_web_admin.authentication.keycloak.resolve_user", +# # Only members of this Keycloak group are authorized to use web-admin -- create it (and +# # add people to it) in the Keycloak admin console before anyone logs in. +# "REQUIRED_GROUPS": ["/pyobs-web-admin"], +# # Keycloak-independent kill switch, layered on top of REQUIRED_GROUPS above: an admin can +# # deactivate a specific local User (Django admin) regardless of their Keycloak group +# # membership. +# "ENFORCE_LOCAL_ACTIVE": True, # } # Hub: allow this instance to be controlled by a hub (leave empty to disable). diff --git a/pyobs_web_admin/settings.py b/pyobs_web_admin/settings.py index 3c8e561..55e672c 100644 --- a/pyobs_web_admin/settings.py +++ b/pyobs_web_admin/settings.py @@ -25,6 +25,10 @@ "whitenoise.middleware.WhiteNoiseMiddleware", "django.contrib.sessions.middleware.SessionMiddleware", "django.contrib.auth.middleware.AuthenticationMiddleware", + # after AuthenticationMiddleware (needs request.user) - re-checks a Keycloak-backed session's + # authorization once its access token expires, instead of only at next login. See pyobs-auth's + # docs/source/configuration.rst, "Authorization: claims vs. local is_active". + "pyobs_auth.middleware.KeycloakSessionRefreshMiddleware", "django.contrib.messages.middleware.MessageMiddleware", "django.middleware.common.CommonMiddleware", "modules.middleware.HubTokenMiddleware", @@ -53,8 +57,8 @@ WSGI_APPLICATION = "pyobs_web_admin.wsgi.application" -# SQLite: only used for the Keycloak-linked User table (django.contrib.auth) - the shared -# admin/password login below stays fully DB-free. +# SQLite: the Keycloak-linked User table (django.contrib.auth) and, since SESSION_ENGINE is +# database-backed (below), every session - shared admin/password login included. DATABASES = { "default": { "ENGINE": "django.db.backends.sqlite3", @@ -62,8 +66,18 @@ } } -# Sessions themselves stay in signed cookies regardless (no session table needed) -SESSION_ENGINE = "django.contrib.sessions.backends.signed_cookies" +# Server-side sessions, backed by the same sqlite3 db as the Keycloak-linked User table above - +# NOT signed_cookies. pyobs-auth's CallbackView stores the Keycloak refresh token in the session +# so it can be silently renewed later; a cookie-backed session would serialize that (a bearer +# credential that can mint fresh access tokens indefinitely) into the browser - signed, but not +# encrypted, so readable by the client. `manage.py migrate` creates the `django_session` table +# (django.contrib.sessions is already in INSTALLED_APPS). +SESSION_ENGINE = "django.contrib.sessions.backends.db" + +# Distinct from other pyobs Django apps (e.g. portal) so browser cookies don't +# collide when both are run on localhost at once - cookies are scoped by host, not port. +SESSION_COOKIE_NAME = "web_admin_sessionid" +CSRF_COOKIE_NAME = "web_admin_csrftoken" LANGUAGE_CODE = "en-us" TIME_ZONE = "UTC" @@ -82,10 +96,12 @@ # Keycloak login (optional addon on top of the shared admin/password login above, not a # replacement - leave SERVER_URL unset to disable it entirely; the login page won't show the -# button either). Unlike the shared admin account, each Keycloak-linked User can be individually -# deactivated (Django admin, or `manage.py shell`) without affecting anyone else - see -# pyobs_web_admin.authentication.keycloak.resolve_user. New accounts mint inactive by default and -# need local activation before they can log in. +# button either). Authorization is the REQUIRED_GROUPS claims gate below (Keycloak group +# membership), not local activation - see pyobs-core's specs/design/shared-authz-keycloak.md. +# The Keycloak admin console is the people-management surface now: granting/revoking access to +# this service is done there (assign/remove someone from the web-admin group), not in this app's +# Django admin. (pyobs-auth's ENFORCE_LOCAL_ACTIVE setting is available if a deployment still +# wants a local, Keycloak-independent per-user kill switch on top of the group gate.) PYOBS_AUTH = { "SERVER_URL": "", "REALM": "pyobs", @@ -99,6 +115,15 @@ "IDP_HINT": "", "IDP_LABEL": "", "USER_RESOLVER": "pyobs_web_admin.authentication.keycloak.resolve_user", + # Claims-based authorization gate (pyobs-auth >=2.1): membership in this Keycloak group is + # what authorizes a user to use web-admin at all. Empty disables the gate entirely, so leave + # this set unless every Keycloak login should be authorized regardless of group membership. + "REQUIRED_GROUPS": ["/pyobs-web-admin"], + # Keycloak-independent kill switch, layered on top of REQUIRED_GROUPS above: an admin can + # deactivate a specific local User (Django admin) regardless of their Keycloak group + # membership. True preserves this service's pre-2.1 behavior, where is_active was always + # the gate. + "ENFORCE_LOCAL_ACTIVE": True, } # Hub: list of remote hosts this instance can control diff --git a/pyproject.toml b/pyproject.toml index 9b15f1c..e40a10a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "pyobs-web-admin" -version = "2.0.0.dev12" +version = "2.1.2" description = "Web-based administration interface for pyobs robotic telescope modules" authors = [ { name = "Tim-Oliver Husser", email = "thusser@uni-goettingen.de" }, @@ -11,9 +11,35 @@ dependencies = [ "gunicorn>=26.0.0", "packaging>=24.0", "psutil>=6.0", - "pyobs-auth>=2.0.0.dev9", + "pyobs-auth>=2.1.0", "pyyaml>=6.0", "requests>=2.32", "ruamel.yaml>=0.18", "whitenoise>=6.8", ] + +[dependency-groups] +dev = [ + "ruff>=0.11.0", + "pyrefly>=0.17.0", +] + +[tool.ruff] +target-version = "py312" +line-length = 120 + +[tool.ruff.lint] +select = ["E", "F", "W", "I", "UP", "G"] +ignore = ["E501"] + +[tool.pyrefly] +project-includes = ["modules", "pyobs_web_admin"] +project-excludes = ["**/docs/", "**/migrations/"] +python-version = "3.12.0" +preset = "legacy" +# local_settings.py is gitignored (optional per-deployment override, see settings.py's +# try/except ImportError) -- never present in CI or a fresh checkout. +ignore-missing-imports = ["pyobs_web_admin.local_settings"] + +[tool.pyrefly.errors] +redundant-cast = "warn" diff --git a/specs/index.md b/specs/index.md index 86ce4da..5809ad4 100644 --- a/specs/index.md +++ b/specs/index.md @@ -9,4 +9,6 @@ These are `pyobs-web-admin`-local docs. `pyobs-core` is the reference for the fu convention across the `pyobs` ecosystem (it additionally has `adrs/` and `steering/`, and its own `CLAUDE.md` explains when a doc belongs there instead of here — e.g. anything that also concerns another sibling repo). See `pyobs-core`'s `specs/` for cross-repo design docs, plans, and ADRs that -happen to touch this repo (tagged with a `Repos:` line there). +happen to touch this repo (tagged with a `Repos:` line there) — e.g. +`../../pyobs-core/specs/design/shared-authz-keycloak.md` (proposed: centralized authorization via +Keycloak groups; replaces this repo's Django-admin activation for Keycloak-linked users). diff --git a/specs/plans/2026-08-25-module-classes-fleet-aggregation.md b/specs/plans/2026-08-25-module-classes-fleet-aggregation.md new file mode 100644 index 0000000..012433f --- /dev/null +++ b/specs/plans/2026-08-25-module-classes-fleet-aggregation.md @@ -0,0 +1,149 @@ +# Module classes: fleet aggregation + +## Status + +Implemented on the pyobs-web-admin side (this repo). Not yet live-verified against a real +two-instance hub pair. Portal-side follow-up filed as pyobs-portal#119, not yet started. See +issue #68. + +## Motivation + +`api_module_classes` (`GET /api/modules/classes/`, `modules/views.py:753`) was built for +issue #65 so pyobs-portal could filter modules by interface (`ICamera`, `ITelescope`, ...) on +its own side for its script-builder dropdowns. It was deliberately built "always local" — +no `_active_host` proxying, no hub-fleet loop — because the caller was assumed to target one +specific host directly. + +Issue #68 found the gap that assumption left: portal only has one `WEBADMIN_URL` configured +and never loops hosts itself, so in practice it only ever sees the one host it's pointed at. +Modules on other hub clients are invisible to it, with no config knob on either side to fix +that. It's the only hub-facing endpoint in this app without a fleet-aggregating counterpart — +every other one (ACL matrix, All Logs, Users) already has one. + +## Current state + +Three hub-interaction patterns coexist in this app (see `specs/design/index.md`'s "Wide +conventions" section): + +1. **Single active-host, session-driven proxying** — `dashboard`, `module_detail`, etc. +2. **Fleet-wide aggregating endpoints** that loop `["localhost"] + HUB_HOSTS` and merge + server-side. Two flavors already exist: + - A dumb per-host raw endpoint (`api_acl_matrix`) plus a separate aggregating **page** + that does the looping/merging (`acl_matrix` view, `services.merge_acl_matrices`, + `modules/views.py:354`). The split exists because that page needs to render HTML; the + raw endpoint is what a hub instance calls per sub-host. + - A single **self-aggregating JSON endpoint** with no separate raw variant: + `api_all_logs` (`modules/views.py:587`) always loops `["localhost"] + HUB_HOSTS` itself + and returns merged JSON directly, using `proxy.get_host_config` + `proxy.call` + (`modules/proxy.py`) and reporting per-host failures via an `unreachable_hosts` list + rather than failing the whole request. +3. **Raw, always-local, hub-facing endpoints** meant to be queried by another instance doing + pattern 2's merge — `api_acl_matrix`, `api_module_classes` today, the comm-user-map + endpoint. + +`services.build_module_classes()` (`modules/services.py:2031`) returns a flat +`{module_name: class_fqcn}` dict for the local host only. `api_module_classes` just +`JsonResponse`s that dict unchanged. + +pyobs-portal's caller (`pyobs_portal/api/webadmin.py`, `get_module_classes()`) expects exactly +that flat `dict[str, str]` shape, validates it structurally, and caches it 30s. + +## Design + +**Follow the `api_all_logs` precedent, not the `acl_matrix` one.** `api_module_classes` has no +human-facing page consuming it — its only consumer is a machine (portal) that wants one +answer in one call. The raw+page split exists for acl_matrix because a page needs to query +hosts one at a time to render; there's no equivalent reason to keep a separate "raw" variant +here. So: **extend `api_module_classes` in place** to self-aggregate, no new endpoint, no +`?fleet=1` param. This also composes for nested hubs for free — when a hub instance is asked, +whatever `HUB_HOSTS` *it* has configured gets folded in automatically, exactly like +`api_all_logs` does. + +This is a breaking response-shape change for the one known caller, and that's deliberate, not +incidental — see below. + +**Response shape must change from a flat dict to a list tagged by host.** A flat +`{module_name: class}` merge silently overwrites on a same-named module across two hosts. +Portal will need the host anyway once it moves from "pick a module for this field" to +"call that module" at execution time, so this isn't just collision-avoidance — it's data the +caller actually needs. New shape: + +```json +{ + "modules": [ + {"name": "cam1", "class": "pyobs.modules.camera.BaseCamera", "host": "localhost"}, + {"name": "telescope", "class": "pyobs.modules.telescope.BaseTelescope", "host": "MONETS"} + ], + "unreachable_hosts": [{"name": "MONETS", "error": "..."}] +} +``` + +**Implementation:** + +- `services.build_module_classes()` stays unchanged — it's still the correct per-host raw + shape (flat dict), both for the local branch of the loop and for what a leaf host returns + when queried by a hub instance's `proxy.call`. +- Add `services.merge_module_classes(per_host: list[tuple[str, dict]]) -> list[dict]`, + mirroring `merge_acl_matrices` (`modules/services.py:2444`): flatten each host's dict into + `{name, class, host}` rows. No collision arbitration needed since rows aren't deduped by + name — a same-named module on two hosts just becomes two distinct rows, disambiguated by + `host`, same as `merge_acl_matrices` does for ACL rows. +- Rewrite `api_module_classes` (`modules/views.py:753`) to loop + `["localhost"] + HUB_HOSTS` like `api_all_logs` does: local branch calls + `services.build_module_classes()` directly; remote branches use + `proxy.get_host_config` + `proxy.call(host_cfg, "GET", "/api/modules/classes/")` wrapped in + try/except, appending to `unreachable_hosts` on failure (same pattern as + `api_all_logs:617-645`). Merge with `merge_module_classes` and return the shape above. + Update the view's docstring — it no longer describes "always local." +- Update `pyobs-portal`'s `get_module_classes()` (`pyobs_portal/api/webadmin.py`) to parse the + new `{"modules": [...], "unreachable_hosts": [...]}` shape and return + `list[dict]`/`None` instead of `dict[str, str]`/`None`, adjusting whatever consumes it + (`schema.module_ref_options()`) to filter/group by `host` where useful. **This is a + separate repo and a separate PR** — out of scope for the pyobs-web-admin change itself, but + must land before or alongside it since the response shape is breaking. Note this explicitly + in the PR description. + +## Open questions + +- Confirmed via issue #65/#68: portal is the only known caller of this endpoint. No other + caller needs the old flat-dict, local-only behavior preserved — grep pyobs-portal before + merging to be sure nothing else there hits this URL. +- Is `api_module_classes` the only pattern-3 endpoint missing a pattern-2 aggregator? Not + re-audited here beyond confirming the comm-user-map endpoint already has one (Users page). + Worth a quick pass over `modules/urls.py`'s remaining `api/*` GETs if this pattern needs + applying again elsewhere. + +## Work Plan + +- [x] `services.merge_module_classes` + unit tests in `modules/tests.py` (mirror + `MergeAclMatricesTests`, `modules/tests.py:771`) — host-tagging, no-collision-arbitration + behavior, empty-input case +- [x] Rewrite `api_module_classes` to loop + merge, matching `api_all_logs`'s + unreachable-host handling; update its docstring +- [x] Unit tests for the view: single host (no `HUB_HOSTS`), multi-host merge, one host + unreachable +- [ ] Verify live against a real two-instance hub pair if available (issue #68 mentions + `south/monet` / `south/frontend` already share matching `HUB_HOSTS`/`HUB_TOKEN`) +- [ ] Follow-up PR in pyobs-portal: update `get_module_classes()` for the new response shape + (tracked as pyobs-portal#119) + +### Progress log + +pyobs-web-admin side implemented: `services.merge_module_classes` added, `api_module_classes` +rewritten to loop `["localhost"] + HUB_HOSTS` and merge (mirroring `api_all_logs`). Not yet +verified against a real hub pair. Portal-side change filed as its own issue (pyobs-portal#119) +rather than implemented here, per this doc's note that it's a separate repo/PR. + +PR #72 review (2026-08-26) found the initial remote-branch handling re-flattened a remote's +already host-tagged rows into one `{name: class}` dict per remote, which silently collided +same-named modules on two of *that remote's own* sub-hosts when it was itself a hub -- one +level deeper than the exact bug this endpoint exists to fix. Fixed: each row's inner host is +now preserved (only an inner `"localhost"` tag is rewritten to the outer hub's name), so +nested-hub composition is actually "for free" as the docstring claims, not just for one hop. +Also fixed per that review: a remote's own `unreachable_hosts` are now propagated up instead +of swallowed, and a remote still answering with the pre-#68 flat-dict shape (mid-rollout +fleet) is now reported as unreachable instead of silently contributing zero modules. Added 3 +more tests for these cases (nested-hub tag preservation, nested unreachable propagation, +old-shape remote) -- 10 new tests total, 316/316 passing. Also fixed in the same PR: CI +pyrefly was red on `develop` (pre-existing from #70) over the gitignored +`local_settings.py` import; added it to `ignore-missing-imports`. diff --git a/specs/plans/2026-09-01-log-fullscreen-button.md b/specs/plans/2026-09-01-log-fullscreen-button.md new file mode 100644 index 0000000..3f9e4e4 --- /dev/null +++ b/specs/plans/2026-09-01-log-fullscreen-button.md @@ -0,0 +1,132 @@ +# Plan: fullscreen button for logs + +Status: implemented, closed (#74, PR #83) + +## Problem + +The two log views (`templates/modules/all_logs.html`, fixed `height: 600px`; +`templates/modules/detail.html` `#tab-logs`, fixed `height: 520px`) render the log console at a +fixed height that's too small to follow a lively stream during an observing run or a long update. +There's no way to enlarge it short of browser zoom / DevTools. No fullscreen code exists anywhere +in the repo yet. + +The log JS in the two templates is intentionally duplicated "in lockstep" (see the comment at the +top of the notifications block in `detail.html`) — no shared static JS/CSS file exists for logs — +so this must be added identically in both places. + +## Design + +Overlay-only approach (no native `Element.requestFullscreen()`): a CSS class toggle makes a +wrapper around the toolbar + status line + `
` fill the viewport via
+`position: fixed; inset: 0`. Chosen over the Fullscreen API because iOS Safari doesn't support
+`requestFullscreen()` on arbitrary elements, and the overlay is simpler to implement/test with one
+code path instead of two (native + fallback).
+
+**Implementation note:** the class must go on a wrapper (`#log-container`) around the toolbar and
+the `
`, not on the `
` alone — applying `position: fixed` to just the log box leaves the
+toolbar (a sibling `
` above it) behind the fixed layer, hiding filter/refresh/acknowledge/etc.
+entirely. Caught via a real browser check during implementation, not by DOM inspection alone (the
+DOM computed styles look fine either way — this is a visual/z-stacking issue).
+
+### 1. CSS (added to both templates — `extra_head` block)
+
+`all_logs.html` has no `extra_head` block today; add one. `detail.html` already has one
+(lines 5–13) — append there. Both templates wrap the toolbar + `#log-older-status` + `
` in a
+new `
`.
+
+```css
+.log-fullscreen {
+  position: fixed; inset: 0; z-index: 1046; /* above sidebar (1044/1045), mobile navbar (1043) */
+  background: var(--pyobs-surface-bg);
+  display: flex; flex-direction: column;
+  padding: 1rem; margin: 0; overflow: auto;
+}
+.log-fullscreen #log-output { flex: 1 1 auto; height: auto !important; min-height: 0; }
+```
+
+`min-height: 0` on the flex child is required so the `
` can shrink below its content height
+and scroll internally instead of the whole overlay growing past the viewport.
+
+Theme-aware via the existing `--pyobs-surface-bg` custom property (`templates/base.html:25-40`,
+defined for both light and dark).
+
+### 2. Toolbar button (both files)
+
+```html
+
+```
+
+`aria-pressed` is kept in sync by `toggleLogFullscreen()` so screen-reader users get the toggle
+state (the file's other icon-only buttons rely on `title` alone, but this one is a stateful
+toggle, which is exactly the case `aria-pressed` is for).
+
+- `all_logs.html`: in the toolbar div (~line 44-79), under the module checkboxes.
+- `detail.html`: in the `#tab-logs` toolbar (~line 138 area), same relative position.
+
+### 3. JS (added identically to both files, near the other log helpers)
+
+```js
+function toggleLogFullscreen() {
+  const container = document.getElementById('log-container');
+  const btn = document.getElementById('log-fullscreen-btn');
+  const icon = btn.querySelector('i');
+  const isFs = container.classList.toggle('log-fullscreen');
+  icon.className = isFs ? 'bi bi-fullscreen-exit' : 'bi bi-arrows-fullscreen';
+  btn.title = isFs ? 'Exit fullscreen' : 'Expand log to fullscreen';
+  btn.setAttribute('aria-pressed', String(isFs));
+  if (isFs) document.addEventListener('keydown', escExitLogFullscreen);
+  else document.removeEventListener('keydown', escExitLogFullscreen);
+}
+function escExitLogFullscreen(e) {
+  if (e.key === 'Escape') toggleLogFullscreen();
+}
+```
+
+No `fullscreenchange`/`fullscreenerror` handling needed (overlay-only, no native API) — `Esc` is
+handled via a manual keydown listener instead.
+
+### 4. Interactions that must keep working — verified against current code, no changes needed
+
+- Auto-refresh (`logTimer`, `setInterval(fetchLogs, 3000)` — `all_logs.html:455,532` /
+  `detail.html:864,1017`): untouched, keeps running regardless of the DOM class.
+- `renderLogs()`'s scroll-position preservation (`wasNearBottom` from `pre.scrollHeight` /
+  `scrollTop` / `clientHeight`): unaffected — computed from the live element either way.
+- Ack badge / `isNewIssue()` / click+Shift-click time-range on `.log-line` spans: all bound to
+  `#log-output`, which is never replaced, only reclassed.
+- `fetchOlderLogs()` scroll-to-top loader: listens on `#log-output` scroll; class toggle doesn't
+  touch that listener.
+
+### 5. Collapsed-state restoration
+
+Free by construction: toggling `.log-fullscreen` off just removes the `position: fixed` override;
+the inline `style="height: 600px/520px"` on the `
` (`all_logs.html:90`, `detail.html:141`) is
+never removed, so it re-applies automatically. Scroll position is preserved since it's the same
+DOM node throughout (never detached/re-created).
+
+## Acceptance criteria
+
+- [ ] Fullscreen toggle button on both the All Logs page and the per-module Logs tab.
+- [ ] Expanded log fills the whole viewport (no sidebar/navbar space eaten).
+- [ ] Toolbar and all existing controls remain usable while expanded.
+- [ ] Auto-refresh, NEW/ack badge, older-log loading on scroll-to-top, and click/Shift+click
+      time-range still work while expanded.
+- [ ] Exit via button and via `Esc`; icon state stays in sync.
+- [ ] Collapsed state looks/behaves exactly as before (fixed heights restored after exit).
+- [ ] Works in current Chrome, Firefox, Safari, and iOS Safari (overlay works everywhere since it
+      doesn't depend on the Fullscreen API).
+- [ ] Both templates updated in lockstep (same behavior, same helper code).
+
+## Out of scope
+
+- Native Fullscreen API (`requestFullscreen()`) — overlay-only per the design decision above.
+- Keyboard shortcut (e.g. `f`) to toggle fullscreen — optional nicety in the issue, not required.
+- Packages page transient operation-log panel (`#update-panel-log` in
+  `templates/modules/packages.html`) — different kind of log, explicitly out of scope in #74.
+
+## Related
+
+- #44 (browser notifications for log WARNING+) — shares the per-module Logs tab / All Logs page;
+  keep the notification flow working while expanded.
+- #59 (logs for both config name and comm name) — touches the same log plumbing.
diff --git a/specs/plans/2026-09-01-update-all-packages.md b/specs/plans/2026-09-01-update-all-packages.md
new file mode 100644
index 0000000..6ee3be7
--- /dev/null
+++ b/specs/plans/2026-09-01-update-all-packages.md
@@ -0,0 +1,145 @@
+# Plan: "Update all" button on the Packages page
+
+Status: implemented
+
+Landed: PR #81 ("Add \"Update all\" bulk action to the Packages page", commits `0129b78`/`fafba81`,
+merged `afea89b`) — the "Update all" button, `updateAllPackages()`'s sequential queue, and the
+`bulkRunning`/`awaitUpdateCompletion` guards on `templates/modules/packages.html`. No backend
+changes, per the proposal below. Closes #79.
+
+Related: builds on `specs/plans/2026-08-16-async-package-update.md` (the background-job/lock
+design below is consumed as-is, not changed).
+
+## Problem
+
+The Packages page (`templates/modules/packages.html`) updates one `pyobs-*` package at a time via
+its per-row Update/Reinstall button. #79 asks for a bulk "update all" action.
+
+This can't be a simple bulk-fire like Dashboard's `controlAll`/`controlOutdated`
+(`templates/modules/dashboard.html:325-374`), which fires staggered requests with a fixed 1s
+`sleep` and never waits for completion. Package updates are serialized host-wide by design:
+`update_package_start` (`modules/services.py:874`) takes an flock and refuses a second job while
+one is running (see the async-package-update plan's "one job at a time, host-wide" rationale —
+concurrent `pip install` against the same venv races on `dist-info`/`RECORD`). A staggered-fire
+loop would have every request after the first come back `{"ok": false, "message": "Already
+updating X"}`. "Update all" has to actually wait for each job to reach a terminal state before
+starting the next.
+
+## Proposal
+
+1. New "Update all (N)" button next to Refresh on the Packages page, mirroring the
+   `btn-restart-outdated` count-badge pattern on Dashboard. **Per-host**, following the page's
+   existing session-active-host model — no fleet-wide version, consistent with this app's
+   established stance that bulk actions are a footgun outside a single host's own page
+   (README.md, `specs/design/index.md:76-88`).
+2. Eligibility = whatever `pkgRow()` already renders as a non-disabled Update/Reinstall button
+   (`packages.html:93`) — outdated regular packages, plus **vcs "Reinstall" packages** (per
+   confirmed scope: include vcs, not just PyPI-version-bump packages). No new eligibility logic;
+   reuse the existing per-row `disabled` computation.
+3. Client-side sequential queue: POST `.../update/` for package *i*, await its terminal state via
+   the existing status endpoint, then move to *i+1*. No new backend endpoint and no server-side
+   queue/job-list — the existing single-job lock file is reused exactly as today, one name at a
+   time.
+4. **Continue past failures** ("install as much as possible"): a `failed` or `interrupted` result
+   for one package does not stop the queue — log it and proceed to the next. End with a per-package
+   outcome summary (updated / failed / interrupted / skipped-start), not just whatever the last
+   package's raw pip log happened to say.
+5. Guard against overlap with a manual single-package click (or a resumed in-flight job from
+   another tab/admin) racing the queue and hitting the server-side lock.
+
+## Implementation
+
+### 1. Extract an awaitable poll helper — `packages.html`
+
+`pollUpdateStatus()` today is fire-and-forget (`setTimeout` recursion, used for the single-click
+flow and `resumeUpdateIfActive`). Add `awaitUpdateCompletion()`: same 1.5s-interval poll against
+`api_package_update_status`, still calling `renderUpdateLog(status)` on every tick for live output,
+but as an `async` loop that `return`s the final status once `!status.active` instead of scheduling
+another `setTimeout`. Keep the existing `pollUpdateStatus` for the single-click path unchanged
+(same auto-hide-panel-on-success behavior); both it and the new queue function call the same
+underlying fetch/render step so the polling logic isn't duplicated.
+
+### 2. `updateAllPackages()` — `packages.html`
+
+```
+async function updateAllPackages() {
+  const btn = document.getElementById('btn-update-all');
+  const queue = [...document.querySelectorAll('#packages-tbody tr[data-package]')]
+    .filter(row => !row.querySelector('button').disabled)
+    .map(row => row.dataset.package);
+  if (!queue.length) return;
+
+  setBulkControlsDisabled(true);   // btn-update-all + every row's Update/Reinstall + btn-refresh
+  const outcomes = [];
+  for (let i = 0; i < queue.length; i++) {
+    const name = queue[i];
+    showQueueProgress(i + 1, queue.length, name);   // "Updating 3/8: pyobs-fli…"
+    const resp = await fetch(`/api/packages/${encodeURIComponent(name)}/update/`, {
+      method: 'POST', headers: {'X-CSRFToken': getCsrfToken()},
+    });
+    const data = await resp.json();
+    if (!data.ok) {
+      outcomes.push({name, state: 'not-started', detail: data.message || data.error});
+      continue;                      // per-#5: lock contention on this one -- move on regardless
+    }
+    const status = await awaitUpdateCompletion();
+    outcomes.push({name, state: status.state});
+  }
+  renderQueueSummary(outcomes);      // replaces the log panel with a per-package result list
+  setBulkControlsDisabled(false);
+  await loadPackages();
+}
+```
+
+Queue membership is captured once, up front, from the currently-rendered DOM — not re-derived
+mid-loop — so it can't be affected by `loadPackages()` re-renders triggered elsewhere (e.g. a
+concurrent Refresh click, which is why `btn-refresh` is disabled for the duration too).
+
+### 3. Guard against overlap
+
+- `setBulkControlsDisabled(true)` disables `btn-update-all`, `btn-refresh`, and every row's action
+  button before the loop starts — prevents a manual click during the queue from ever reaching the
+  server and getting a confusing "Already updating X" response.
+- `resumeUpdateIfActive()` (`packages.html:169-182`, runs on page load) already detects a job
+  in flight from *before* this page load (e.g. another admin/tab, or a page refresh mid-queue).
+  Extend it: if `status.active` on load, also disable `btn-update-all` until that job reaches a
+  terminal state (reuse `awaitUpdateCompletion()` then re-enable), so a second admin can't start a
+  queue on top of a job already running.
+- Queue itself never needs this guard mid-run — it always awaits terminal state before its own
+  next POST — so no client-side retry/backoff logic is needed for the "Already updating" case
+  during a queue's own iterations, only for something external racing it.
+
+### 4. UI additions — `packages.html`
+
+- `