fix(azcopy): keep archived backups out of latest-backup checks - #3
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Archived historical Azure manifests currently prevent Backmaster from discovering a newer backup. The exporter downloads every manifest before selecting the latest epoch, and the same scan also breaks serial allocation and retention.
This change adds an online catalogue at
catalogue/<created_epoch>/<backup_name>.json. Publication uploads the backup manifest after its payload, then the catalogue record in Hot storage. Freshness reads only the newest catalogue record; serial allocation and retention use listings without downloading archived manifests. Failed catalogue publication preserves the existing ready-stage retry flow.The exporter now propagates listing failures, distinguishes failed downloads from invalid JSON, validates catalogue names/epochs, rejects conflicting backup-name reuse, and prevents stale catalogue records from counting toward minimum redundancy. Decimal serial allocation also handles 008/009 correctly.
Migration is explicit: upgrade all writers, exclude
catalogue/from lifecycle archival/deletion, then import the newest verified readable backup withcatalogue-import NAMEor run one forced backup with a unique name. Legacy unindexed backups are preserved with a retention warning; they are never silently deleted or scanned for freshness. The exporter reference, example configuration, and troubleshooting guide document rollout and failure recovery.Validation: