Skip to content

Empty sub-level chunk-map records are never pruned — autosave time grows unbounded (95% garbage after months of play) #1386

Description

@Sacfiritan

Summary

Records in the sub-level chunk-map region files (.slvlr) whose pointers list is empty are never pruned. They accumulate for the lifetime of a world — one per chunk that has ever contained a sub-level — and are re-read and re-written on every autosave.

On our server this grew to 4132 empty records out of 4340 total (95% garbage), and autosave of the overworld took up to 30 seconds, freezing the server every ~5 minutes. After removing only the empty records offline, the same save dropped to ~1 second.

Environment

  • Sable 2.0.3 (sable-neoforge-1.21.1-2.0.3.jar)
  • Create 6.0.10, Create: Aeronautics 1.3.0 (bundled)
  • Minecraft 1.21.1, NeoForge 21.1.233, dedicated server, Java 21
  • World age: several months of active multiplayer

Symptom

Every autosave the server thread stalls, then logs:

[Server thread/INFO] [dev.ryanhcode.sable.Sable/]: Saving sub-levels for level 'ServerLevel[world]'/minecraft:overworld
[Server thread/WARN] [net.minecraft.server.MinecraftServer/]: Can't keep up! Is the server overloaded? Running 28230ms or 564 ticks behind

Timing the gap between the overworld and the_nether save lines shows where the time goes — the overworld save is the expensive one; nether and end take ~0.5 s each:

18:46:03.011  Saving sub-levels ... overworld
18:46:27.999  Saving sub-levels ... the_nether     <- 25 seconds later
18:46:28.520  Saving sub-levels ... the_end

Across 52 consecutive autosaves: average 11.1 s, min 3.9 s, max 30.7 s.

The duration correlates with how many chunk-map records exist in the regions loaded around players, not with the number of live contraptions. Verified by cross-referencing player login coordinates with per-region record counts: saves were fast when the only player was in a sparse region, and slowest (10 s / 30.7 s) when players were in r.2.2 (257 records) and r.2.3 (168 records).

Data

Scanned every .slvlr in world/sublevels/ and decoded every record:

Total records 4340
Records with pointers empty 4132 (95 %)
Records that actually point to sub-levels 208
Distinct sub-levels with block data in .slvls 279
Region files consisting entirely of empty records 210 of 267

Distribution of pointers length: {0: 4130, 1: 162, 2: 34, 3: 9, 4: 3, 5: 1}.

So 95 % of the serialization work each autosave is spent on records that reference nothing.

On-disk format (as observed, for reference)

Both file types start with a 4096-byte allocation table: 1024 slots × 4 bytes = 3-byte big-endian offset + 1-byte length. The allocation unit differs:

  • .slvlr — unit 128 bytes, data starts at unit 32. max(offset+length) * 128 equals the file size exactly.
    Payload: 4-byte length + 1 compression byte (0) + gzip(NBT), and the NBT is a compound containing a single pointers TAG_Int_Array. Empty records decompress to a 19-byte NBT with pointers length 0.
  • .slvls — unit 4096 bytes. Payload same header shape, NBT contains plot { chunks, biome, plot_x, plot_z }, pose { position, orientation }, world_bounds, uuid.

(Noting the two different units because a third-party "repair" script circulating for this problem assumes 4096 bytes for .slvlr and consequently flags every record as corrupt — it would erase every ship on the server. Might be worth a warning somewhere.)

Measured effect of removing the empty records

Server stopped, only empty-pointers slots zeroed in the tables (.slvls untouched, file sizes unchanged, records with pointers preserved), server restarted:

Players online Overworld save — before after
~3 10.0 s 0.9 s / 1.9 s
~2 30.7 s 1.1 s / 1.6 s / 0.7 s

Average 11.1 s → ~1.2 s. Residual Can't keep up stalls dropped from 8–28 s to 2.0–3.5 s. No errors from Sable after the change, all contraptions loaded normally.

Suggested fix

When serializing the holding chunk-map, skip (or drop) entries whose pointer list is empty, so a chunk record disappears once the last sub-level leaves it. A prune on load would also clear existing worlds.

This is invisible on a fresh world and only shows up after months of play, which is probably why it hasn't been reported — the cost grows silently with every contraption that is ever moved or removed.

Notes

Happy to provide the full scan data, the region files, or test a patch against this world — it reproduces reliably here. Thanks for Sable, it's a remarkable piece of engineering.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions