Skip to content

Scan tree, snapshots and what grew since last time - #7

Merged
kingchenc merged 6 commits into
mainfrom
feat/scan-tree-snapshots
Sep 16, 2026
Merged

kingchenc merged 6 commits into
mainfrom
feat/scan-tree-snapshots

Conversation

@kingchenc

Copy link
Copy Markdown
Owner

Closes out the roadmap: "what grew since last time" is implemented end to end, drill-in/drill-up no longer re-walk the disk, and a handful of smaller gaps found while reading the code are fixed.

Core (diskghost-core)

  • ScanTree — one walk builds the whole directory tree in memory (arena + path index). report(path, top_n) returns the same ScanReport a fresh scan of that folder would, for the root or any sub-folder, with no further I/O. remove(path) takes a deleted file or folder out of the tree and fixes every ancestor's totals in place. scan_with_progress now runs on top of it (same output as before; measured on a 4.1 M-file / 489 k-dir tree: 68 s vs 70 s on main).
  • Snapshot + diff() — a snapshot records every directory's subtree totals plus the biggest files, as versioned JSON (SNAPSHOT_FORMAT). diff(old, new, at) reports grown / shrunk / added / removed immediate sub-folders of at (root or any sub-folder), with totals and file-count deltas. SnapshotError covers I/O, malformed JSON, foreign format, root mismatch and unknown folder.
  • ScanReport.cancelled — a cancelled walk is now distinguishable from a complete one (#[serde(default)], so old JSON still parses). FileEntry, DirSize, ScanReport gain Deserialize.
  • Windows on-disk sizeSetLastError(0) before GetCompressedFileSizeW, so a legitimate 0xFFFFFFFF low dword is no longer mistaken for a stale failure.
  • Tests: tree build / drill / remove / cancel, snapshot round-trip + format errors, diff at root and sub-folder, root mismatch, cancelled duplicate search, human_delta, UTC timestamp rendering (28 unit tests, was 13).

CLI

  • scan <dir> --save FILE writes a snapshot; --since FILE prints what changed since it (JSON: a diff object next to the report fields). New diff OLD NEW [--at DIR] [--top N] compares two saved snapshots. A cancelled scan prints a partial-result notice.
  • Integration tests for save → since → diff (human + JSON, --at, identical snapshots) and for every error path (8 tests, was 6).

GUI

  • Backend keeps the ScanTree of the last complete walk (with the options it was walked with); any folder below its root is answered from memory, refresh=true forces a walk. A clean delete updates the tree in place; a failed one or a real reclaim drops the cache. Every complete walk is stored as a snapshot under the app data dir (one history per root, 20 kept); list_snapshots / diff_since expose it. Cancelled walks are neither cached nor recorded.
  • Frontend: Since <date> picker + Show changes → total-delta stat, ± badge per folder row, treemap tiles tinted by growth/shrinkage (new folders dashed), drillable grown/shrunk/new/removed lists. Rescan button. Cancelled scans show a notice and a partial-result stat. Breadcrumb for a drive root now scans C:\ (was C:, which Windows resolves to the drive's working directory); UNC shares stay one segment. Duplicate action/sort labels are localised.
  • Pure helpers moved to gui/ui/lib.js with Node unit tests (10), run in CI.
  • Verified in a browser against a mocked Tauri bridge: scan → history → changes rendering, cancelled path, first-scan hint, rescan, delete → re-render, duplicates toolbar; no console errors.

Docs

README: features, CLI examples (incl. a cron/Task Scheduler one-liner), GUI paragraph, roadmap — drill-up/breadcrumbs ticked (it had shipped without the tick), release binaries described as checksummed + provenance-attested rather than "signed", signed GUI installers listed as the remaining open item.

- ScanTree: build the whole directory tree from one walk and serve a
  ScanReport for the root or any sub-folder from memory, so drilling in
  and back up never re-walks the disk. remove() drops a deleted path and
  fixes every ancestor's totals in place.
- Snapshot: persist every directory's subtree totals plus the biggest
  files as versioned JSON; diff() reports grown/shrunk/added/removed
  sub-folders at the root or at any sub-folder.
- ScanReport.cancelled tells callers a partial result apart from a
  complete one; FileEntry/DirSize/ScanReport now deserialize too.
- Windows on-disk size: reset the thread error state before
  GetCompressedFileSizeW so a legitimate 0xFFFFFFFF low dword is not
  mistaken for a stale failure.
- Tests: tree build/drill/remove/cancel, snapshot round-trip and diff,
  cancelled duplicate search, human_delta, timestamp rendering.
scan --save FILE writes a snapshot; --since FILE prints what grew,
shrank, appeared or vanished compared to it (in --json the diff is
nested under "diff" next to the report fields). diff OLD NEW compares
two saved snapshots, optionally --at a sub-folder. A cancelled scan is
labelled as a partial result.
scan_dir keeps the ScanTree of the last complete walk (with the options
it was walked with) and answers any folder below its root from memory;
refresh=true forces a new walk. A clean delete updates the tree in place
instead of re-walking; a failed one or a real reclaim drops the cache.
Every complete walk is stored as a snapshot under the app data dir (one
history per root, 20 kept); list_snapshots and diff_since expose that
history so the UI can show what grew since an earlier scan. Cancelled
walks are neither cached nor recorded.
- A "Since <date>" picker lists the folder's snapshot history (the one
  before the current scan preselected); Show changes adds a total-delta
  stat, a +/- badge per folder row, tints treemap tiles by growth or
  shrinkage (new folders dashed), and lists grown/shrunk/new/removed
  sub-folders, each drillable.
- Rescan forces a fresh walk; plain drill-in and drill-up use the cached
  tree and render instantly.
- A cancelled scan is labelled as a partial result instead of passing as
  complete.
- Breadcrumbs: a drive root now scans as C:\ (C: alone resolves to the
  drive's working directory); UNC shares stay one segment.
- Duplicate action and sort labels are localised like the rest.
- Pure helpers moved to lib.js with Node unit tests, run in CI.
Drill-up/breadcrumbs had been shipped without a tick; the release
binaries are checksummed and provenance-attested, not code-signed, so
the roadmap says that and lists signed GUI installers as still open.
… tie

On ext4 every file up to 4 KB occupies exactly one block, which made
the biggest-file order in drill_in_matches_a_fresh_scan arbitrary.
@kingchenc
kingchenc merged commit c98833d into main Sep 16, 2026
7 checks passed
@kingchenc
kingchenc deleted the feat/scan-tree-snapshots branch September 16, 2026 00:43
kingchenc added a commit that referenced this pull request Sep 16, 2026
Version bump only. Changes since 0.6.2 landed in #7.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant