feat: add serial crash-safe mutation recovery

[gwokhou]

TL;DR

This PR makes document import fail more cleanly. If openkb add is interrupted or errors halfway through, OpenKB should no longer leave half-written KB files behind; it can recover the previous stable state before accepting the next mutation.

Summary

Adds crash-safe mutation recovery for serial KB mutations without bringing back the concurrent add pipeline from #104.

This PR keeps the scope focused on the first split-out piece from the abandoned PR: journaled snapshot/rollback for add and PageIndex Cloud import, plus the atomic writer prerequisites needed for hardlink-backed rollback.

What changed

• Added openkb/mutation.py with:

  • mutation snapshots
  • active/committed/rolled_back journals
  • rollback and recovery of interrupted mutations
  • bounded rollback retry
  • malformed journal handling
  • staged artifact publishing with atomic rename + EXDEV copy fallback

• Wired recovery into serial mutation paths:

  • openkb add <file> converts into staging, snapshots final KB paths, publishes, compiles, then commits via registry write + journal mark
  • openkb add --from-pageindex-cloud prepares cloud data read-only, snapshots doc-specific paths, writes/compiles/registers under the same recovery contract
  • exclusive KB lock acquisition drains pending journals before mutating

• Kept conversion serial while adding staging support:

  • convert_document(..., staging_dir=...) writes raw/source artifacts into isolated staging before commit
  • no parallel prepare hook or doc-name override path is introduced

• Made wiki writers atomic where hardlink snapshots depend on temp+replace semantics:

  • compiler summary/concept/entity/index writes
  • lint broken-link cleanup

Explicitly out of scope

This does not reintroduce the concurrency half of #104:

• no ThreadPoolExecutor
• no file_processing_jobs
• no jobs>1 directory add path
• no parallel prepare pipeline
• no batch doc-name reservation logic

Validation

• UV_CACHE_DIR=/tmp/uv-cache UV_PYTHON=3.13 uv run --extra dev pytest -q

  • 898 passed, 5 warnings

• UV_CACHE_DIR=/tmp/uv-cache UV_PYTHON=3.13 uv run --extra dev pytest tests/test_add_command.py::TestImportFromPageindexCloud tests/test_mutation.py -q

  • 25 passed

• UV_CACHE_DIR=/tmp/uv-cache UV_PYTHON=3.13 uv run --extra dev pytest tests/test_converter.py tests/test_add_command.py tests/test_mutation.py -q

  • 72 passed

• UV_CACHE_DIR=/tmp/uv-cache UV_PYTHON=3.13 uv run --extra dev ruff check openkb/converter.py openkb/mutation.py tests/test_mutation.py

  • passed

Notes

Full-project ruff and ty are not clean on current main; this PR fixes the new diagnostics introduced by the recovery changes but does not widen scope to existing lint/type debt.

[KylinMountain]

Solid PR — the snapshot/rollback design is sound and the hardlink backup contract holds (every writer into the hardlinked dirs is atomic temp+replace or append-only).

One gap worth closing before merge, since it's the PR's own goal: the URL path isn't crash-safe. openkb add <url> calls add_single_file(..., stage=False) (cli.py:964), so staging_dir=None and convert_document (cli.py:444) writes into the live KB before snapshot_paths() (cli.py:461). A crash mid-conversion leaves a partial wiki/sources/<doc>.md with no journal to roll back — exactly the orphan state the PR eliminates for the local/cloud paths. Could the URL path use stage=True and handle the dedup cleanup separately?

[gwokhou]

Solid PR — the snapshot/rollback design is sound and the hardlink backup contract holds (every writer into the hardlinked dirs is atomic temp+replace or append-only).

One gap worth closing before merge, since it's the PR's own goal: the URL path isn't crash-safe. openkb add <url> calls add_single_file(..., stage=False) (cli.py:964), so staging_dir=None and convert_document (cli.py:444) writes into the live KB before snapshot_paths() (cli.py:461). A crash mid-conversion leaves a partial wiki/sources/<doc>.md with no journal to roll back — exactly the orphan state the PR eliminates for the local/cloud paths. Could the URL path use stage=True and handle the dedup cleanup separately?

@KylinMountain Thanks for the CR, gap fixed🫡