Docs: creates markdown reference

[lukegalbraithrussell]

Summary

This PR replaces the html reference with a md-based on. Tighter integration with docusaurus, and allows easier agent access as the md files are accessible. Also will now be trackable in Google Analaytics

Testing

Category

• slack_bolt.App and/or its core components
• slack_bolt.async_app.AsyncApp and/or its core components
• Adapters in slack_bolt.adapter
• Document pages under /docs
• Others

Requirements

Please read the Contributing guidelines and Code of Conduct before creating this issue or pull request. By submitting, you are agreeing to those rules.

• I've read and understood the Contributing Guidelines and have done my best effort to follow them.
• I've read and agree to the Code of Conduct.
• I've run ./scripts/install_all_and_run_tests.sh after making the changes.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.54%. Comparing base (a70d247) to head (6353313).
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1565   +/-   ##
=======================================
  Coverage   91.54%   91.54%           
=======================================
  Files         228      228           
  Lines        7285     7285           
=======================================
  Hits         6669     6669           
  Misses        616      616           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

[WilliamBergamin]

Thanks for working on this 💯

Left some comments/questions before we can move forward with this 🙏

[WilliamBergamin]

This is a lot of custom code 😅 any chance we can rely on defaults or configs instead of having this?

[lukegalbraithrussell]

@WilliamBergamin Using a new library that's a bit more consistent. Still a lot of custom code, but that's just the nature of the beast given our unique docusaurus setup - it's mostly on that docusaurus side, not the python repo side

[WilliamBergamin]

I like like the direction here. Moving the reference to Markdown is a clear win over the pdoc3 HTML blob:

• the pages are agent-readable
• they can be tracked in GA
• they slot into Docusaurus natively.

griffe is also the right parser choice (it resolves our __all__ re-exports and parses the Google-style docstrings correctly).

My main concern is scripts/generate_api_docs.py as a long-term maintenance surface: ~640 lines of hand-rolled code is a lot to keep working as slack_bolt, griffe, and Docusaurus all evolve. I dug into how the other tools ship their reference and I think we can remove a big chunk of the hand rolled code.

node-slack-sdk is the closest working model. It ships a committed Markdown reference that docs.slack.dev consumes, and it does two things we're currently doing the hard way:

1. No hand-built sidebar.json. It uses a Docusaurus autogenerated sidebar (dirName + a generated-index link); deno-slack-sdk and slack-cli do the same. Worth noting: the docs site imports exactly one _sidebar.json per tool today and has no mechanism to import a second reference/sidebar.json, so the artifact we generate can't actually be wired in as-is. Switching to an autogenerated Reference category in our existing docs/english/_sidebar.json removes ~90 lines and the whole "how does the site pick this up" question.
2. Signatures in fenced code blocks. MDX leaves < and { literal inside code fences, so the escaping only really needs to cover prose docstring text, not signatures. That shrinks the escaping/hazard surface a lot.

What I'd suggest (keep griffe, cut the rest):

• Drop the sidebar.json generation; add an autogenerated Reference category to docs/english/_sidebar.json (node/deno/slack-cli pattern).
• Move the imperative signature/section rendering into a small set of Jinja2 templates (the mkdocstrings/quartodoc approach). Same output, roughly half the code, and future tweaks become template edits.
• Keep a minimal prose-level MDX escaper + the hazard gate, and wire the hazard gate into CI.
• Pin griffe in a requirements file (it's currently pip install -U griffe, so output can shift under us), and add a CI job that regenerates and fails on git diff drift, since the generated tree is committed and nothing guards it right now.

Happy to pair on the template refactor if useful.

[WilliamBergamin]

Thanks for working on this 🙏 but I'm starting to think we might want to break this out into 3 PRs

Splitting this PR into three stacked PRs (less custom code)

This PR migrates the API reference from pdoc3 HTML to a griffe-generated Markdown tree under docs/english/reference/, wired into Docusaurus via an autogenerated category in docs/english/_sidebar.json. The direction and griffe as the parser are good 🚀 the concern is that scripts/generate_api_docs.py (573 lines) is a hand-rolled maintenance surface, and the generated tree is committed but unguarded.

Much of the script is defensive logic that only exists to absorb docstring formatting problems at generation time:

• _reflow_indented_code: rewrites indented code blocks into fenced blocks. Dead code today (every docstring example is already fenced, so it never fires).
• _check_mdx_hazards: fails generation on MDX/JSX/ESM hazards (<, import, export at column 0), but only when someone manually regenerates — never on a PR.
• _escape_mdx: escapes </{ in prose so Docusaurus (which parses .md as MDX) doesn't read prose as JSX/JS. This is legitimate rendering and stays.

The generator can't shrink while it's the only thing guarding these invariants. The fix: move the guarantee into a test that runs on every PR, clean up the docstrings that feed it, and then the generator becomes a plain griffe→Jinja2 renderer.

Proposed decomposition into three stacked PRs, each independently valuable:

PR 1 (ruff)  →  PR 2 (docstring test + fixes)  →  PR 3 (= this PR, slimmed generator + tree)
   off main            stacks on PR 1                 rebase this branch after 1 & 2 land

---

PR 1: Adopt Ruff as linter + formatter (replaces Black + Flake8)

Goal: Ruff can lint and format docstrings (docstring code examples + a curated D set). Doing it first means all mechanical docstring reformatting happens once, before the tree is regenerated in PR 3.

Changes

- `pyproject.toml`: add `[tool.ruff]` (`line-length = 125`, `target-version = "py37"`), `[tool.ruff.lint]` at parity with `.flake8` (keep `E`/`W`/`F`, `ignore = ["E402", "F841", "F821"]`; `W503` is a formatter concern in Ruff, drop it), enable a curated docstring set via `[tool.ruff.lint.pydocstyle] convention = "google"` while ignoring noisy missing-docstring rules (`D100`–`D107`) and the manual-only ones (`D205` missing-blank-after-summary ×90, `D417` undocumented-param ×7 — these need real prose work, deferred to PR 2's docstring pass). Add `[tool.ruff.format] docstring-code-format = true`. Remove `[tool.black]`. - Delete `.flake8` (config now lives in `pyproject.toml`). - `requirements/dev_tools.txt`: replace `black` + `flake8` with a `ruff` pin (keep `mypy`). - `scripts/format.sh`: `black slack_bolt/ tests/` → `ruff format slack_bolt/ tests/`. - `scripts/lint.sh`: `flake8 slack_bolt/ && flake8 examples/` → `ruff check slack_bolt/ examples/`. - CI (`.github/workflows/ci-build.yml`): add a `ruff format --check` step to the existing `lint` job (CI enforces no formatter today — this closes that gap). - Apply the one-time mechanical result of `ruff format` + `ruff check --fix` (incl. the auto-fixable D-rules) across the repo. Large but mostly cosmetic; does **not** fence unfenced code — that's PR 2.

Verification: ./scripts/format.sh --no-install clean, ./scripts/lint.sh --no-install green, ./scripts/run_mypy.sh --no-install unchanged, full test suite passes.

---

PR 2: Docstring rendering test (griffe, latest-Python only) + source fixes

Goal: encode the invariants _check_mdx_hazards enforces — as a test that runs on every PR — and fix the docstrings that violate them. This is what lets PR 3's generator drop its defensive logic.

New tests/docstring/test_rendering.py: load the package with griffe exactly as the generator does — griffe.load("slack_bolt", search_paths=[REPO_ROOT], docstring_parser=griffe.Parser.google) — walk every documented object's docstring, and fail if any contains, outside a ``` fence:

• a line at column 0 starting with < (JSX) or import /export (ESM) — the _check_mdx_hazards invariant, checked at the source instead of post-render;
• a bare indented code block (blank line + ≥4-space indent) — the precondition that keeps _reflow_indented_code unnecessary.

Why griffe, not stdlib ast: it enumerates the same objects the generator documents (respecting __all__ re-export inlining) and parses docstrings the same way, so the test's scope == the generator's scope. The fence-aware hazard scanner can be lifted almost verbatim out of _check_mdx_hazards, so no logic is duplicated or left behind in the generator.

Latest-Python-only gating (required — griffe 2.x needs a modern Python and can't run on 3.7/3.8):

• Add griffe (pinned) to a new requirements/docs.txt (shared by the test and, in PR 3, the generator).
• Add a dedicated CI job docstring mirroring lint/typecheck: setup-python ${{ env.LATEST_SUPPORTED_PY }} (3.14) → pip install . -r requirements/docs.txt → pytest tests/docstring/.
• The 3.7–3.14 unittest matrix runs specific paths, never a blanket tests/, so it won't pick this up. Guard the module with pytest.importorskip("griffe") + a sys.version_info skip so the codecov job and local run_tests.sh (which default-collect tests/) skip it cleanly instead of erroring.

Source fixes (bundled here): fence any remaining unfenced code examples and fix any column-0 MDX hazards in slack_bolt/** docstrings so the new test is green — these are the docstring edits currently spread across this branch. Re-run ruff format after fencing.

Verification: pytest tests/docstring/ green on 3.14; skipped (not errored) where griffe is absent.

---

PR 3: Slim generator (griffe → Jinja2) + the migration (this PR)

Goal: rebase this branch onto main (after PR 1 + PR 2 land) and reduce it to the Markdown migration with the simplest generator, now that PR 2 guards the invariants.

Generator rewrite (scripts/generate_api_docs.py):

• Keep griffe as the loader/parser and the page-model building (_load_package, _iter_modules, _documented_members, _inlined_export_target, _doc_id/_doc_route, _write_pages).
• Move the imperative Markdown rendering (_render_object, _render_docstring, signature builders, _render_body) into Jinja2 templates under scripts/templates/ (mkdocstrings/quartodoc-style).
• Delete _reflow_indented_code (dead) and _check_mdx_hazards (now the PR 2 test).
• Keep MDX escaping as a small Jinja escape_mdx filter applied to prose only; signatures stay inside fenced blocks (MDX leaves </{ literal inside fences).
• Add jinja2 (pinned) to requirements/docs.txt; update scripts/generate_api_docs.sh to pip install -r requirements/docs.txt instead of the current unpinned pip install -U griffe (pins griffe).

Migration payload (already on this branch): the regenerated docs/english/reference/ tree, the _sidebar.json autogenerated Reference category, docs/english/reference_redirects.json, and deletion of the old docs/reference/**/*.html.

Guard the committed tree (new CI job): a docs-drift job on 3.14 that installs requirements/docs.txt, runs python scripts/generate_api_docs.py, and git diff --exit-code -- docs/english/reference — fails if the committed tree drifts from source.

Not byte-identical: because this is a Jinja2 rewrite, the tree differs cosmetically from the current output. Correctness is established by the PR 2 test (inputs), the docs-drift job (output matches source), and a Docusaurus build check rather than a byte diff.

---

Notes

• Order: PR 1 → PR 2 → PR 3. PR 2 stacks on PR 1 (both touch docstrings; PR 1 formats, PR 2 fences). PR 3 needs PR 2's test merged so the slimmed generator is safe.
• requirements/docs.txt is introduced in PR 2 (griffe) and extended in PR 3 (jinja2) — one pinned home for doc tooling that's currently pip install -U'd ad hoc.
• The docstring test and generator span sync and async modules automatically (griffe loads the whole package); just ensure PR 2's docstring source fixes cover async_* counterparts.