[lumina] Release the vector file when FileBackedDataset fails to construct - #9481
[lumina] Release the vector file when FileBackedDataset fails to construct#9481PDGGK wants to merge 1 commit into
Conversation
…truct FileBackedDataset opens a RandomAccessFile and then validates the record size and allocates the read buffer, either of which can throw. A caller using try-with-resources never receives an object whose constructor threw, so nothing closes that handle. The enclosing writer already guards its own file this way; this brings the nested reader in line.
129aa4a to
3b81750
Compare
| this.readBuf.limit(0); // empty initially | ||
| this.phase = phase; | ||
| this.lastLoggedPercent = -1; | ||
| } catch (RuntimeException e) { |
There was a problem hiding this comment.
[P2] Also close the file when direct-buffer allocation throws an Error
The cleanup currently only handles RuntimeException, but the other post-open failure named in the PR, ByteBuffer.allocateDirect, can throw OutOfMemoryError. A direct-memory OOM can be caught by a higher-level task/runtime and leave this JVM alive; in that path the constructor still strands raf, which is exactly the resource-safety hole this change is intended to close. Please catch and rethrow RuntimeException | Error (or use the usual constructor-cleanup catch (Throwable) pattern) so every failure after the successful open releases the handle. The existing regression only exercises checkedRecordSize, so it does not cover this path.
Purpose
FileBackedDatasetopens the vector file and then does work that can fail:checkedRecordSizethrowsIllegalStateExceptionwhen a record does not fit the read buffer, andallocateDirectcan fail on its own. Both run after the file is open.Both call sites use try-with-resources:
but try-with-resources never receives an object whose constructor threw, so its
close()never runs and the handle is stranded.The enclosing writer already guards its own file this way at
:112, so this brings the nested reader in line.Tests
LuminaFileBackedDatasetCloseTest#testFailedConstructionReleasesTheFileruns 200 failed constructions with a dimension the buffer cannot hold, counting entries in/dev/fdbefore and after. It skips itself where that directory is not available.Reverting the change turns it red:
[200 failed constructions must not strand 200 descriptors].A note on how the assertion got there, since the obvious versions do not work. Asserting that the exception is raised passes either way. Letting the descriptors run out does not work either — the
RandomAccessFilefinalizer releases them under GC pressure, so twenty thousand leaked handles still never exhaust the limit and the test stays green with the bug in place. Counting open descriptors directly is what actually separates the two.mvn test -pl paimon-luminapasses; spotless and checkstyle clean.