Skip to content

metro-file-map: Watch directories as the node crawler lists them - #2108

Draft
robhogan wants to merge 1 commit into
pr2107from
pr2108
Draft

robhogan wants to merge 1 commit into
pr2107from
pr2108

Conversation

@robhogan

@robhogan robhogan commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Without Watchman, Watcher starts its backends before the crawl, so that nothing changed during the crawl is missed:

// Non-Watchman backends have no clock to subscribe `since`, so they must
// already be watching before the crawl begins, and their events buffered,
// for changes made while crawling not to be missed. Watchman backends are
// started by watch(), which subscribes `since` the crawl's clock to cover
// the same window. If the Watchman crawl falls back to the node crawler,
// there is no clock, so #crawl() starts non-Watchman backends before the
// node crawl instead.
if (!options.useWatchman) {
await this.#startBackendsBeforeCrawl();
}

For FallbackWatcher that means walking the tree once to watch it and again to crawl it. This lets the crawl do both: the crawler hands each directory to the watcher just before listing it, the order the watcher's own walk uses. Directories the crawl skips as ignored (eg node_modules without retainAllFiles) are handed over too, for the watcher to walk unless it ignores them as well.

This is internal to the built-in node crawler. A crawlerFactory crawler, a recrawl, a root added with addRoot and the node crawl after a failed Watchman crawl keep the watcher's own walk.

Benchmark

This takes another 28-30% off watch-mode startup on Linux and Windows. With the rest of this stack, watch-mode startup without Watchman goes from 1,887ms to 492ms (-74%) on Linux, and from 2,577ms to 525ms (-80%) on Windows. On macOS, which uses NativeWatcher, PRs 2-5 stay within 5% of #2104, which is noise: #2106 changes nothing there and measures +4.6%.

main #2107 This PR vs main vs #2107
Linux: Watch-mode start, warm cache 1,887ms 695ms 492ms -73.9% (-74.2% to -73.7%) -29.2% (-29.9% to -28.6%)
Windows: Watch-mode start, warm cache 2,577ms 727ms 525ms -79.6% (-79.7% to -79.5%) -27.7% (-28.1% to -27.4%)
Linux: Watch-mode start, cold cache 1,885ms 652ms 462ms -75.5% (-75.8% to -75.0%) -29.2% (-30.0% to -27.7%)
Windows: Watch-mode start, cold cache 2,562ms 701ms 493ms -80.8% (-80.8% to -80.6%) -29.6% (-29.9% to -29.3%)
Linux: Heap after a watch-mode start 31.7MB 28.1MB 28.1MB -11.3% (-11.3% to -11.3%) -0.1% (-0.1% to -0.1%)
Windows: Heap after a watch-mode start 31.7MB 28.2MB 28.2MB -11.3% (-11.3% to -11.3%) -0.0% (-0.0% to -0.0%)

Changelog

 - **[Performance]**: metro-file-map: Set up file watching during the crawl instead of in a separate walk before it, on Linux and Windows without Watchman

Test plan

New unit tests, and a watch-integration-test.js case for a directory the crawl skips but the watcher doesn't. Benchmark as in #2104.

@robhogan
robhogan added this pull request to stack #2109 October 25, 2026 15:22
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 25, 2026
@robhogan
robhogan force-pushed the pr2108 branch 2 times, most recently from b46060d to 7219809 Compare October 25, 2026 20:13
Without Watchman, `Watcher` starts its backends before the crawl, so that nothing changed during the crawl is missed:

https://github.com/react/metro/blob/56efab302200568c919ac1b3c28dacf8d84fc692/packages/metro-file-map/src/Watcher.js#L118-L127

For `FallbackWatcher` that means walking the tree once to watch it and again to crawl it. This lets the crawl do both: the crawler hands each directory to the watcher just before listing it, the order the watcher's own walk uses. Directories the crawl skips as ignored (eg `node_modules` without `retainAllFiles`) are handed over too, for the watcher to walk unless it ignores them as well.

This is internal to the built-in node crawler. A `crawlerFactory` crawler, a recrawl, a root added with `addRoot` and the node crawl after a failed Watchman crawl keep the watcher's own walk.

## Benchmark

This takes another 28-30% off watch-mode startup on Linux and Windows. With the rest of this stack, watch-mode startup without Watchman goes from 1,887ms to 492ms (-74%) on Linux, and from 2,577ms to 525ms (-80%) on Windows. On macOS, which uses `NativeWatcher`, PRs 2-5 stay within 5% of #2104, which is noise: #2106 changes nothing there and measures +4.6%.

|  | `main` | #2107 | This PR | vs `main` | vs #2107 |
|---|---|---|---|---|---|
| Linux: Watch-mode start, warm cache | 1,887ms | 695ms | 492ms | -73.9% (-74.2% to -73.7%) | -29.2% (-29.9% to -28.6%) |
| Windows: Watch-mode start, warm cache | 2,577ms | 727ms | 525ms | -79.6% (-79.7% to -79.5%) | -27.7% (-28.1% to -27.4%) |
| Linux: Watch-mode start, cold cache | 1,885ms | 652ms | 462ms | -75.5% (-75.8% to -75.0%) | -29.2% (-30.0% to -27.7%) |
| Windows: Watch-mode start, cold cache | 2,562ms | 701ms | 493ms | -80.8% (-80.8% to -80.6%) | -29.6% (-29.9% to -29.3%) |
| Linux: Heap after a watch-mode start | 31.7MB | 28.1MB | 28.1MB | -11.3% (-11.3% to -11.3%) | -0.1% (-0.1% to -0.1%) |
| Windows: Heap after a watch-mode start | 31.7MB | 28.2MB | 28.2MB | -11.3% (-11.3% to -11.3%) | -0.0% (-0.0% to -0.0%) |

## Changelog

```
 - **[Performance]**: metro-file-map: Set up file watching during the crawl instead of in a separate walk before it, on Linux and Windows without Watchman
```

## Test plan

New unit tests, and a `watch-integration-test.js` case for a directory the crawl skips but the watcher doesn't. Benchmark as in #2104.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant