Skip to content

fix(azcopy): keep archived backups out of latest-backup checks - #3

Merged
SakuraIsayeki merged 1 commit into
mainfrom
fix/azcopy-archive-catalogue
Sep 21, 2026
Merged

SakuraIsayeki merged 1 commit into
mainfrom
fix/azcopy-archive-catalogue

Conversation

@SakuraIsayeki

Copy link
Copy Markdown
Member

Archived historical Azure manifests currently prevent Backmaster from discovering a newer backup. The exporter downloads every manifest before selecting the latest epoch, and the same scan also breaks serial allocation and retention.

This change adds an online catalogue at catalogue/<created_epoch>/<backup_name>.json. Publication uploads the backup manifest after its payload, then the catalogue record in Hot storage. Freshness reads only the newest catalogue record; serial allocation and retention use listings without downloading archived manifests. Failed catalogue publication preserves the existing ready-stage retry flow.

The exporter now propagates listing failures, distinguishes failed downloads from invalid JSON, validates catalogue names/epochs, rejects conflicting backup-name reuse, and prevents stale catalogue records from counting toward minimum redundancy. Decimal serial allocation also handles 008/009 correctly.

Migration is explicit: upgrade all writers, exclude catalogue/ from lifecycle archival/deletion, then import the newest verified readable backup with catalogue-import NAME or run one forced backup with a unique name. Legacy unindexed backups are preserved with a retention warning; they are never silently deleted or scanned for freshness. The exporter reference, example configuration, and troubleshooting guide document rollout and failure recovery.

Validation:

  • All seven repository test scripts passed, including Debian packaging.
  • ShellCheck 0.11.0 and Bash syntax checks passed.
  • Regression coverage includes archived old manifests, latest-only reads, legacy import, listing failures with partial output, missing/corrupt newest records, upload/removal failures, stale catalogue entries, serial 009, and name collisions.
  • A core integration test verifies catalogue failure retains the ready stage, retry avoids a second prepare, and the next fresh run skips.
  • Azure operations were exercised with the repository's AzCopy mock; no live Azure deployment was performed.

@SakuraIsayeki
SakuraIsayeki merged commit c6c0838 into main Sep 21, 2026
2 checks passed
@SakuraIsayeki
SakuraIsayeki deleted the fix/azcopy-archive-catalogue branch September 21, 2026 08:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant