Skip to content

fs: add atomic option to writeFile - #65754

Open
webdevelopersrinu wants to merge 1 commit into
nodejs:mainfrom
webdevelopersrinu:fs-atomic-write
Open

fs: add atomic option to writeFile#65754
webdevelopersrinu wants to merge 1 commit into
nodejs:mainfrom
webdevelopersrinu:fs-atomic-write

Conversation

@webdevelopersrinu

Copy link
Copy Markdown
Contributor

Refs: #49886

fs.writeFile can leave a half written file if the process dies in the middle.
People do the temp file and rename dance themselves, or install
write-file-atomic.

I added an atomic option that does it inside writeFile. It writes a temp
file next to the target, flushes it, then renames it over. A reader gets the
old data or the new data, never half. If a step fails I delete the temp file
and keep the original. It also keeps the old file's permissions. It works
like the flush option, and skips the writeFileSync utf8 fast path.

I kept the parent directory fsync out of this one. It needs a binding change
since you cannot open a directory on Windows. I can add it in a follow up.

I added tests for sync, callback and promises, and for the failure paths.

@nodejs-github-bot nodejs-github-bot added fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. labels Sep 3, 2026
@webdevelopersrinu
webdevelopersrinu force-pushed the fs-atomic-write branch 3 times, most recently from fff7adb to aec6249 Compare September 3, 2026 04:58
@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.02632% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.96%. Comparing base (6e7818e) to head (e6dfb9f).
⚠️ Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
lib/internal/fs/promises.js 91.89% 3 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##             main   #65754    +/-   ##
========================================
  Coverage   89.95%   89.96%            
========================================
  Files         757      757            
  Lines      258053   258231   +178     
  Branches    48934    48965    +31     
========================================
+ Hits       232144   232319   +175     
- Misses      16962    16978    +16     
+ Partials     8947     8934    -13     
Files with missing lines Coverage Δ
lib/fs.js 97.39% <100.00%> (+0.05%) ⬆️
lib/internal/fs/utils.js 96.13% <100.00%> (+0.09%) ⬆️
lib/internal/fs/promises.js 91.63% <91.89%> (+0.31%) ⬆️

... and 43 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Writes go to a temp file next to the target and then get renamed
over it, so a reader never sees a half written file.

Refs: nodejs#49886
Signed-off-by: webdevelopersrinu <webdeveloper.srinu9@gmail.com>

@LiviaMedeiros LiviaMedeiros left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution! However, i don't think we should add such option in this form. It would cause more problems and footguns than it solves.

  • it may fail due to directory permissions (i.e. when we are allowed to write to /path/to/file but not create new file in /path/to/
  • it may leave orphaned temp files if process dies midway
  • even if it doesn't, it would still pollute the directory with temp file until it's fully written and flushed
  • it may fail if temp file gets locked (e.g. by indexing process or some antimalware scanner on windows)
  • if may fail if filename with added suffix happens to be too long
  • it will temporarily waste space in disk, having both old and new file on it
  • it will change target file's inode (breaking connections if nlink > 1)
  • the current implementation will break symlinks as well
  • the current implementation does flush implicitly while technically these two are independent options (we don't have to force sync to make operation atomic on FS level)
  • the current implementation does not preserve file ownership
  • it also won't preserve any additional metadata, xattr, ACL, etc.

I'd rather let users 'dance' around it in userland explicitly. In simplest form it's just one additional LoC per writeFile(), it's much clearer in terms of all implications and caveats, and it's much more configurable (e.g. app-wide directory for temp files, periodic cleanup, gitignore-friendly filenames, etc.)


Obviously an atomic option might be still welcome if implemented using FS capabilities (for example, utilizing RWF_ATOMIC on libuv side) that can not be accessed from userland.

Comment thread doc/api/fs.md
Comment on lines +7210 to +7215
* `atomic` {boolean} If `true`, the data is written to a temporary file next
to `file`, flushed, and then renamed over `file`. A reader sees either the
old data or the new data, never a half-written file. The permissions of an
existing `file` are kept instead of `mode`. Cannot be used with a file
descriptor, a {FileHandle}, or a `flag` other than `'w'`.
**Default:** `false`.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This probably belongs to fs.writeFileSync rather than fs.writeSync

@webdevelopersrinu

Copy link
Copy Markdown
Contributor Author

Fair points, thanks. You are right about the symlink, I checked it.

Should I close this, or is the RWF_ATOMIC route worth trying?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants