fs: add atomic option to writeFile - #65754
Conversation
fff7adb to
aec6249
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #65754 +/- ##
========================================
Coverage 89.95% 89.96%
========================================
Files 757 757
Lines 258053 258231 +178
Branches 48934 48965 +31
========================================
+ Hits 232144 232319 +175
- Misses 16962 16978 +16
+ Partials 8947 8934 -13
🚀 New features to boost your workflow:
|
Writes go to a temp file next to the target and then get renamed over it, so a reader never sees a half written file. Refs: nodejs#49886 Signed-off-by: webdevelopersrinu <webdeveloper.srinu9@gmail.com>
aec6249 to
e6dfb9f
Compare
LiviaMedeiros
left a comment
There was a problem hiding this comment.
Thanks for the contribution! However, i don't think we should add such option in this form. It would cause more problems and footguns than it solves.
- it may fail due to directory permissions (i.e. when we are allowed to write to
/path/to/filebut not create new file in/path/to/ - it may leave orphaned temp files if process dies midway
- even if it doesn't, it would still pollute the directory with temp file until it's fully written and flushed
- it may fail if temp file gets locked (e.g. by indexing process or some antimalware scanner on windows)
- if may fail if filename with added suffix happens to be too long
- it will temporarily waste space in disk, having both old and new file on it
- it will change target file's inode (breaking connections if
nlink > 1) - the current implementation will break symlinks as well
- the current implementation does
flushimplicitly while technically these two are independent options (we don't have to force sync to make operation atomic on FS level) - the current implementation does not preserve file ownership
- it also won't preserve any additional metadata, xattr, ACL, etc.
I'd rather let users 'dance' around it in userland explicitly. In simplest form it's just one additional LoC per writeFile(), it's much clearer in terms of all implications and caveats, and it's much more configurable (e.g. app-wide directory for temp files, periodic cleanup, gitignore-friendly filenames, etc.)
Obviously an atomic option might be still welcome if implemented using FS capabilities (for example, utilizing RWF_ATOMIC on libuv side) that can not be accessed from userland.
| * `atomic` {boolean} If `true`, the data is written to a temporary file next | ||
| to `file`, flushed, and then renamed over `file`. A reader sees either the | ||
| old data or the new data, never a half-written file. The permissions of an | ||
| existing `file` are kept instead of `mode`. Cannot be used with a file | ||
| descriptor, a {FileHandle}, or a `flag` other than `'w'`. | ||
| **Default:** `false`. |
There was a problem hiding this comment.
This probably belongs to fs.writeFileSync rather than fs.writeSync
|
Fair points, thanks. You are right about the symlink, I checked it. Should I close this, or is the RWF_ATOMIC route worth trying? |
Refs: #49886
fs.writeFile can leave a half written file if the process dies in the middle.
People do the temp file and rename dance themselves, or install
write-file-atomic.
I added an
atomicoption that does it inside writeFile. It writes a tempfile next to the target, flushes it, then renames it over. A reader gets the
old data or the new data, never half. If a step fails I delete the temp file
and keep the original. It also keeps the old file's permissions. It works
like the
flushoption, and skips the writeFileSync utf8 fast path.I kept the parent directory fsync out of this one. It needs a binding change
since you cannot open a directory on Windows. I can add it in a follow up.
I added tests for sync, callback and promises, and for the failure paths.