Skip to content

docs: check MSP references are up to date in CI (backport to release/9.1) - #11946

Open
sensei-hacker wants to merge 6 commits into
iNavFlight:release/9.1from
sensei-hacker:rebase-pr11875-msp-docs-ci-release91
Open

sensei-hacker wants to merge 6 commits into
iNavFlight:release/9.1from
sensei-hacker:rebase-pr11875-msp-docs-ci-release91

Conversation

@sensei-hacker

Copy link
Copy Markdown
Member

Summary

Backport of #11875 onto release/9.1 — cherry-picks its three commits (CI check that MSP reference docs are regenerated and up to date, deterministic enum-generation ordering, and untracked-file detection via git status --porcelain), since this is a CI/tooling improvement rather than a breaking change and should land as early as possible so it flows forward into maintenance-10.x and master naturally.

Two additional commits were needed to make this actually work on release/9.1, since it diverges from maintenance-10.x:

  • docs: regenerate stale MSP reference docs on release/9.1docs/development/msp/README.md had drifted out of sync with msp_messages.json before this PR (missing MSP2_INAV_SET_AUX_RC/MSP2_INAV_WIND entries, and a stale hwVersion field description) since nothing enforced regeneration until now. Without this, the new CI check would fail immediately on landing.
  • Fix msp_docs CI job always failing due to gen_docs.sh's interactive promptrelease/9.1's docs/development/msp/gen_docs.sh still ends with an interactive read prompt (already removed in maintenance-10.x's rewritten version of this script). Under GitHub Actions' default non-interactive shell, that read fails on EOF and aborts the step before the actual staleness check runs, which would make the new job permanently red regardless of doc state. Gated the prompt behind a TTY check instead.

Changes

  • Add msp_docs CI job to .github/workflows/docs.yml, failing if docs/development/msp/gen_docs.sh's output differs from the checked-in README.md/inav_enums.json/inav_enums_ref.md
  • Make get_all_inav_enums_h.py's file iteration deterministic (sorted(...)) so regeneration doesn't flip entries depending on filesystem enumeration order
  • Regenerate the currently-stale MSP reference docs so the new check starts green
  • Fix gen_docs.sh's trailing interactive prompt so it doesn't break the new CI job under a non-interactive shell

Testing

  • Ran the exact msp_docs job logic locally end-to-end (bash gen_docs.sh + the git status --porcelain staleness check) under GitHub Actions' default bash -eo pipefail shell with closed stdin — confirmed it now passes cleanly and is idempotent (re-running produces zero diff)
  • Reproduced the original (pre-fix) failure mode directly: bash gen_docs.sh returns exit 1 on EOF stdin even when docs are fully up to date, and confirmed -e aborts the step before the staleness check ever runs
  • Verified git status --porcelain (vs. git diff --exit-code, which the original PR's own follow-up commit also moved away from) correctly detects a deleted-then-recreated-as-untracked generated file, by reproducing that exact scenario
  • Confirmed all files referenced by the new CI job's path filters (CMakeLists.txt, docs/development/msp/*) exist at the same paths on release/9.1

Code Review

Reviewed with the inav-code-review agent, which independently reproduced the interactive-read CI failure in a scratch worktree before the fix was applied, and confirmed the fix and the regeneration commit afterward. No other issues found.

Related

Supersedes/backports #11875 for release/9.1 (original PR targets maintenance-10.x; a maintainer should confirm this is later cherry-picked/isn't otherwise superseded once both branches converge).

bkleiner and others added 5 commits September 14, 2026 01:02
README.md hadn't been regenerated since MSP2_INAV_SET_AUX_RC and
MSP2_INAV_WIND were added to msp_messages.json, and since hwVersion's
field description was updated for uint8_t encoding - both source
changes landed without a matching `gen_docs.sh` run. The new CI check
being backported alongside this (PR iNavFlight#11875) diffs a fresh regen
against the checked-in copy, so it would fail immediately on release/9.1
without this.
…rompt

release/9.1's gen_docs.sh still ends with an interactive
`read -n 1 -s -r -p "Press any key to continue"`. Under GitHub
Actions' default non-interactive shell (bash -eo pipefail), reading
from a closed stdin fails and becomes the script's own exit status,
so `bash gen_docs.sh` returns nonzero even when the docs are
perfectly up to date - aborting the step (via -e) before the actual
staleness check ever runs. Gate the prompt behind a TTY check so it
still pauses for a human running it locally but is a no-op in CI.

maintenance-10.x's gen_docs.sh has already been rewritten without
this prompt, so this only affects the release/9.1 backport.
@sensei-hacker sensei-hacker added this to the 9.1 milestone Sep 14, 2026
@qodo-code-review

Copy link
Copy Markdown
Contributor

ⓘ Qodo reviews are paused because the subscription is no longer active. Ask your workspace admin to reactivate the subscription to resume reviews. Manage billing

@qodo-free-for-open-source-projects

Copy link
Copy Markdown

PR Summary by Qodo

Enforce up-to-date MSP reference docs in CI for release/9.1

✨ Enhancement 🐞 Bug fix 📝 Documentation ⚙️ Configuration changes 🧪 Tests 🕐 20-40 Minutes

Grey Divider

AI Description

• Add CI enforcement that regenerated MSP references match checked-in documentation.
• Make enum discovery deterministic and MSP generation safe in non-interactive shells.
• Regenerate stale MSP messages, enum references, revision, and checksum metadata.
Diagram

graph TD
  INPUTS["MSP Sources"] --> WORKFLOW["Docs Workflow"] --> GENERATOR["Docs Generator"] --> ENUMS["Enum Collector"] --> OUTPUTS["Reference Files"] --> STATUS{"Stale Files?"}
Loading
High-Level Assessment

The PR's regenerate-and-check approach is appropriate because it exercises the same generator developers use locally. Using only git diff was considered but would miss deleted files recreated as untracked; generating into temporary files would require duplicating or redesigning the existing generation pipeline. Deterministic traversal and TTY-gated prompting directly address the reliability requirements of CI.

Files changed (9) +75 / -15

Bug fix (2) +3 / -2
gen_docs.shAvoid interactive prompt failures in CI +2/-1

Avoid interactive prompt failures in CI

• Runs the final keypress prompt only when standard input is a TTY. Ensures the generator exits successfully in non-interactive CI after completing generation.

docs/development/msp/gen_docs.sh

get_all_inav_enums_h.pyMake enum source traversal deterministic +1/-1

Make enum source traversal deterministic

• Sorts recursively discovered source paths before extracting enums, preventing generated output from changing with filesystem enumeration order.

docs/development/msp/get_all_inav_enums_h.py

Documentation (6) +41 / -13
README.mdDocument expanded documentation freshness checks +2/-2

Document expanded documentation freshness checks

• Describes the workflow's broader source and MSP path triggers. Clarifies that CI regenerates settings and MSP references and rejects stale checked-in output.

.github/workflows/README.md

README.mdRegenerate stale MSP message reference documentation +31/-3

Regenerate stale MSP message reference documentation

• Advances the reference revision to 6, documents 'MSP2_INAV_SET_AUX_RC' and 'MSP2_INAV_WIND', and corrects the 'hwVersion' payload definition to its one-byte bit-field encoding.

docs/development/msp/README.md

inav_enums.jsonRegenerate deterministic enum source metadata +2/-2

Regenerate deterministic enum source metadata

• Refreshes selected source paths for duplicated enum definitions according to deterministic source traversal order.

docs/development/msp/inav_enums.json

inav_enums_ref.mdRegenerate ordered enum reference entries +4/-4

Regenerate ordered enum reference entries

• Reorders duplicate SD-card and system-state enum source entries to match deterministic generator output.

docs/development/msp/inav_enums_ref.md

msp_messages.checksumRefresh MSP message definition checksum +1/-1

Refresh MSP message definition checksum

• Updates the stored checksum to match the MSP message definitions used for regenerated revision 6 documentation.

docs/development/msp/msp_messages.checksum

revAdvance MSP documentation revision +1/-1

Advance MSP documentation revision

• Increments the generated MSP reference revision from 5 to 6.

docs/development/msp/rev

Other (1) +31 / -0
docs.ymlAdd MSP reference regeneration check to CI +31/-0

Add MSP reference regeneration check to CI

• Expands workflow path filters to include all inputs capable of affecting generated documentation. Adds an MSP job that regenerates references and uses 'git status --porcelain' to detect modified, deleted, or untracked outputs.

.github/workflows/docs.yml

@qodo-free-for-open-source-projects

qodo-free-for-open-source-projects Bot commented Sep 14, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Generator failures pass the docs check ✓ Resolved 🐞 Bug ☼ Reliability
Description
gen_docs.sh ends unconditionally with true and does not enable fail-fast handling, so failures
from any preceding Python generator are discarded. When a generator exits before changing its
checked-in output, the subsequent Git status check sees a clean tree and the new CI job reports
success despite not validating the references.
Code

docs/development/msp/gen_docs.sh[28]

+true
Evidence
The workflow relies on the exit status of bash gen_docs.sh before checking only whether three
files changed. The script invokes three Python programs without checking their statuses and then
forces its final status to zero, so an early generator failure that leaves those outputs untouched
is indistinguishable from successful regeneration.

docs/development/msp/gen_docs.sh[3-28]
.github/workflows/docs.yml[49-58]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`gen_docs.sh` unconditionally returns success after running several unchecked generator commands, so the MSP documentation CI job can pass when generation fails before modifying an output file.
## Fix Focus Areas
- docs/development/msp/gen_docs.sh[27-28]
- .github/workflows/docs.yml[49-53]
## Recommended Fix
Make `gen_docs.sh` fail immediately when any generator or cleanup command fails, such as by enabling `set -euo pipefail`, and replace the trailing short-circuit plus `true` with an explicit `if [ -t 0 ]; then read ...; fi` block that succeeds without masking earlier errors.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Tip of the day
💡 Did you know, you can ask Qodo to dismiss a finding you disagree with, with your reason on record

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread docs/development/msp/gen_docs.sh Outdated
@github-actions

github-actions Bot commented Sep 14, 2026

Copy link
Copy Markdown

RAM / Flash usage vs. base commit d5c29d6 — commit 991ac26

Target Flash Δ RAM Δ
MATEKF405 ±0 B (±0.00%) CCM: ±0 B (±0.00%)
RAM: ±0 B (±0.00%)
MATEKF722 ±0 B (±0.00%) ITCM_RAM: ±0 B (±0.00%)
RAM: ±0 B (±0.00%)
TCM: ±0 B (±0.00%)
MATEKF765 ±0 B (±0.00%) DTCM_RAM: ±0 B (±0.00%)
SRAM1: ±0 B (±0.00%)
MATEKH743 ±0 B (±0.00%) D2_RAM: ±0 B (±0.00%)
DTCM_RAM: ±0 B (±0.00%)
ITCM_RAM: ±0 B (±0.00%)
RAM: ±0 B (±0.00%)

See RAM/flash optimization guide for techniques to reduce usage.

@github-actions

github-actions Bot commented Sep 14, 2026

Copy link
Copy Markdown

Test firmware build ready — commit 991ac26

Download firmware for PR #11946

245 targets built. Find your board's .hex file by name on that page (e.g. MATEKF405SE.hex). Files are individually downloadable — no GitHub login required.

Development build for testing only. Use Full Chip Erase when flashing.

The TTY-gated prompt fix (previous commit) unconditionally ended the
script with `true`, so if any preceding Python generator step failed
partway through, the failure was discarded and the script still
exited 0 - the CI job's git-status check would then see whatever
partial/unchanged state was left and report success despite the
docs never having actually been (re)validated. Add `set -euo
pipefail` so any real failure aborts immediately, and replace the
bare `true` with a proper `if [ -t 0 ]; then read ...; fi` that
doesn't need a trailing catch-all to stay -e-safe.

Caught by Qodo's review on the PR carrying the previous fix.
@sensei-hacker

Copy link
Copy Markdown
Member Author

Qodo's finding on `gen_docs.sh` (generator failures could pass the docs check silently, since the script had no `set -e` and unconditionally ended with `true`) was correct - fixed in 991ac26: added `set -euo pipefail` and replaced the trailing prompt handling with a proper `if [ -t 0 ]; then read ...; fi` instead of the bare fallback. Verified by simulating a generator crash (`python` step exits nonzero) - the script now correctly aborts instead of reaching the end and reporting success.

@sensei-hacker

sensei-hacker commented Sep 14, 2026

Copy link
Copy Markdown
Member Author

@xznhj8129
Just a heads-up: this adds a new CI check (`msp_docs` job) that verifies `docs/development/msp/` reference docs stay in sync with `msp_messages.json`,

Orginally #11875 against `maintenance-10.x`, from bkleiner, but it needs to get to master, via 9).

Since you're the primary maintainer on the MSP reference docs, flagging in case you want to take a look before it merges — it also regenerates the docs here since they'd drifted out of sync on this branch.

@xznhj8129

Copy link
Copy Markdown
Contributor

@xznhj8129 Just a heads-up: this adds a new CI check (msp_docs job) that verifies docs/development/msp/ reference docs stay in sync with msp_messages.json,

Orginally #11875 against maintenance-10.x, from bkleiner, but it needs to get to master, via 9).

Since you're the primary maintainer on the MSP reference docs, flagging in case you want to take a look before it merges — it also regenerates the docs here since they'd drifted out of sync on this branch.

Alright, excellent. I guess i can remove the CI from mine and keep it to the code?

@sensei-hacker

Copy link
Copy Markdown
Member Author

@xznhj8129 Just a heads-up: this adds a new CI check (msp_docs job) that verifies docs/development/msp/ reference docs stay in sync with msp_messages.json,
Orginally #11875 against maintenance-10.x, from bkleiner, but it needs to get to master, via 9).
Since you're the primary maintainer on the MSP reference docs, flagging in case you want to take a look before it merges — it also regenerates the docs here since they'd drifted out of sync on this branch.

Alright, excellent. I guess i can remove the CI from mine and keep it to the code?

Oh I didn't realize you had a CI one. If you think the CI you had is better, we can use it. Or we can use this one, or a combination of the two?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants