[WIP][POC] Flip the subscription layer onto OnyxStore (atomic swap)

API-INTERNAL.md

@@ -9,6 +9,13 @@
 <dd><p>Minimum interval between disk-pressure alerts. One disk-pressure burst fails every queued operation
 with the identical error, so per-operation logging would amplify the very storm it reports.</p>
 </dd>
+<dt><a href="#NOT_DELIVERED">NOT_DELIVERED</a></dt>
+<dd><p>Sentinel for &quot;nothing delivered yet&quot; in <code>connect()</code>&#39;s per-subscription dedup. A Symbol
+can&#39;t collide with any real Onyx value, so the first <code>Object.is</code> check never matches and
+the initial fire runs even when a key&#39;s genuine first value is <code>undefined</code>. It only needs
+to be distinct from real values, not unique per subscription, so one module-level instance
+is reused by every connection.</p>
+</dd>
 </dl>
 
 ## Functions
@@ -17,6 +24,21 @@ with the identical error, so per-operation logging would amplify the very storm
 <dt><a href="#resetDiskPressureLogThrottle">resetDiskPressureLogThrottle()</a></dt>
 <dd><p>Test-only: clears the disk-pressure log throttle so each test observes its own alert.</p>
 </dd>
+<dt><a href="#trackPendingWrite">trackPendingWrite()</a></dt>
+<dd><p>Registers an in-flight write so <code>scheduleInitialFire</code> can wait for it. Returns the same
+promise so callers can wrap a write&#39;s return value inline. The write is removed from the
+pending set once it settles (success or failure).</p>
+</dd>
+<dt><a href="#whenWritesSettled">whenWritesSettled()</a></dt>
+<dd><p>Resolves once no write operations are in flight. Re-checks after each drain because a
+settling write can apply cache changes that spawn further writes (e.g. <code>Onyx.update</code>
+fans out into per-item merges), and those must be awaited too. Write failures are
+swallowed here: this only cares that writes have settled, not that they succeeded.</p>
+</dd>
+<dt><a href="#scheduleInitialFire">scheduleInitialFire()</a></dt>
+<dd><p>Defer a <code>Onyx.connect</code> callback&#39;s initial fire until writes issued in the same tick have
+applied, so it reads post-write cache.</p>
+</dd>
 <dt><a href="#getMergeQueue">getMergeQueue()</a></dt>
 <dd><p>Getter - returns the merge queue.</p>
 </dd>
@@ -62,33 +84,20 @@ The resulting collection will only contain items that are returned by the select
 to the values for those keys (correctly typed) such as <code>[OnyxCollection&lt;Report&gt;, OnyxEntry&lt;string&gt;]</code></p>
 <p>Note: just using <code>.map</code>, you&#39;d end up with <code>Array&lt;OnyxCollection&lt;Report&gt;|OnyxEntry&lt;string&gt;&gt;</code>, which is not what we want. This preserves the order of the keys provided.</p>
 </dd>
-<dt><a href="#storeKeyBySubscriptions">storeKeyBySubscriptions(subscriptionID, key)</a></dt>
-<dd><p>Stores a subscription ID associated with a given key.</p>
-</dd>
-<dt><a href="#deleteKeyBySubscriptions">deleteKeyBySubscriptions(subscriptionID)</a></dt>
-<dd><p>Deletes a subscription ID associated with its corresponding key.</p>
-</dd>
 <dt><a href="#getAllKeys">getAllKeys()</a></dt>
 <dd><p>Returns current key names stored in persisted storage</p>
 </dd>
-<dt><a href="#tryGetCachedValue">tryGetCachedValue()</a></dt>
-<dd><p>Tries to get a value from the cache. If the value is not present in cache it will return the default value or undefined.
-If the requested key is a collection, it will return an object with all the collection members.</p>
-</dd>
-<dt><a href="#keysChanged">keysChanged()</a></dt>
-<dd><p>When a collection of keys change, search for any callbacks matching the collection key and trigger those callbacks</p>
-</dd>
-<dt><a href="#keyChanged">keyChanged()</a></dt>
-<dd><p>When a key change happens, search for any callbacks matching the key or collection key and trigger those callbacks</p>
+<dt><a href="#notifyKey">notifyKey()</a></dt>
+<dd><p>Notify subscribers of a single-key write. Wrapper over <code>onyxSubscriptionManager.notifyKey()</code>
+that also performs LRU bookkeeping for eviction.</p>
 </dd>
-<dt><a href="#sendDataToConnection">sendDataToConnection()</a></dt>
-<dd><p>Sends the data obtained from the keys to the connection.</p>
-</dd>
-<dt><a href="#getCollectionDataAndSendAsObject">getCollectionDataAndSendAsObject()</a></dt>
-<dd><p>Gets the data for a given an array of matching keys, combines them into an object, and sends the result back to the subscriber.</p>
+<dt><a href="#notifyCollection">notifyCollection()</a></dt>
+<dd><p>Notify subscribers of a batch collection update. Wrapper over
+<code>onyxSubscriptionManager.notifyCollection()</code> that also performs LRU bookkeeping per
+changed member.</p>
 </dd>
 <dt><a href="#remove">remove()</a></dt>
-<dd><p>Remove a key from Onyx and update the subscribers</p>
+<dd><p>Remove a key from Onyx and update the subscribers.</p>
 </dd>
 <dt><a href="#retryOperation">retryOperation()</a></dt>
 <dd><p>Handles storage operation failures based on the error class (see lib/storage/errors.ts).
@@ -136,12 +145,6 @@ It will also mark deep nested objects that need to be entirely replaced during t
 <dt><a href="#doAllCollectionItemsBelongToSameParent">doAllCollectionItemsBelongToSameParent()</a></dt>
 <dd><p>Verify if all the collection keys belong to the same parent</p>
 </dd>
-<dt><a href="#subscribeToKey">subscribeToKey(connectOptions)</a> ⇒</dt>
-<dd><p>Subscribes to an Onyx key and listens to its changes.</p>
-</dd>
-<dt><a href="#unsubscribeFromKey">unsubscribeFromKey(subscriptionID)</a></dt>
-<dd><p>Disconnects and removes the listener from the Onyx key.</p>
-</dd>
 <dt><a href="#setWithRetry">setWithRetry(params, retryAttempt)</a></dt>
 <dd><p>Writes a value to our store with the given key.
 Serves as core implementation for <code>Onyx.set()</code> public function, the difference being
@@ -179,12 +182,46 @@ Retries on failure.</p>
 Minimum interval between disk-pressure alerts. One disk-pressure burst fails every queued operation
 with the identical error, so per-operation logging would amplify the very storm it reports.
 
+**Kind**: global constant  
+<a name="NOT_DELIVERED"></a>
+
+## NOT\_DELIVERED
+Sentinel for "nothing delivered yet" in `connect()`'s per-subscription dedup. A Symbol
+can't collide with any real Onyx value, so the first `Object.is` check never matches and
+the initial fire runs even when a key's genuine first value is `undefined`. It only needs
+to be distinct from real values, not unique per subscription, so one module-level instance
+is reused by every connection.
+
 **Kind**: global constant  
 <a name="resetDiskPressureLogThrottle"></a>
 
 ## resetDiskPressureLogThrottle()
 Test-only: clears the disk-pressure log throttle so each test observes its own alert.
 
+**Kind**: global function  
+<a name="trackPendingWrite"></a>
+
+## trackPendingWrite()
+Registers an in-flight write so `scheduleInitialFire` can wait for it. Returns the same
+promise so callers can wrap a write's return value inline. The write is removed from the
+pending set once it settles (success or failure).
+
+**Kind**: global function  
+<a name="whenWritesSettled"></a>
+
+## whenWritesSettled()
+Resolves once no write operations are in flight. Re-checks after each drain because a
+settling write can apply cache changes that spawn further writes (e.g. `Onyx.update`
+fans out into per-item merges), and those must be awaited too. Write failures are
+swallowed here: this only cares that writes have settled, not that they succeeded.
+
+**Kind**: global function  
+<a name="scheduleInitialFire"></a>
+
+## scheduleInitialFire()
+Defer a `Onyx.connect` callback's initial fire until writes issued in the same tick have
+applied, so it reads post-write cache.
+
 **Kind**: global function  
 <a name="getMergeQueue"></a>
 
@@ -284,70 +321,31 @@ to the values for those keys (correctly typed) such as `[OnyxCollection<Report>,
 Note: just using `.map`, you'd end up with `Array<OnyxCollection<Report>|OnyxEntry<string>>`, which is not what we want. This preserves the order of the keys provided.
 
 **Kind**: global function  
-<a name="storeKeyBySubscriptions"></a>
-
-## storeKeyBySubscriptions(subscriptionID, key)
-Stores a subscription ID associated with a given key.
-
-**Kind**: global function  
-
-| Param | Description |
-| --- | --- |
-| subscriptionID | A subscription ID of the subscriber. |
-| key | A key that the subscriber is subscribed to. |
-
-<a name="deleteKeyBySubscriptions"></a>
-
-## deleteKeyBySubscriptions(subscriptionID)
-Deletes a subscription ID associated with its corresponding key.
-
-**Kind**: global function  
-
-| Param | Description |
-| --- | --- |
-| subscriptionID | The subscription ID to be deleted. |
-
 <a name="getAllKeys"></a>
 
 ## getAllKeys()
 Returns current key names stored in persisted storage
 
 **Kind**: global function  
-<a name="tryGetCachedValue"></a>
-
-## tryGetCachedValue()
-Tries to get a value from the cache. If the value is not present in cache it will return the default value or undefined.
-If the requested key is a collection, it will return an object with all the collection members.
-
-**Kind**: global function  
-<a name="keysChanged"></a>
-
-## keysChanged()
-When a collection of keys change, search for any callbacks matching the collection key and trigger those callbacks
-
-**Kind**: global function  
-<a name="keyChanged"></a>
-
-## keyChanged()
-When a key change happens, search for any callbacks matching the key or collection key and trigger those callbacks
-
-**Kind**: global function  
-<a name="sendDataToConnection"></a>
+<a name="notifyKey"></a>
 
-## sendDataToConnection()
-Sends the data obtained from the keys to the connection.
+## notifyKey()
+Notify subscribers of a single-key write. Wrapper over `onyxSubscriptionManager.notifyKey()`
+that also performs LRU bookkeeping for eviction.
 
 **Kind**: global function  
-<a name="getCollectionDataAndSendAsObject"></a>
+<a name="notifyCollection"></a>
 
-## getCollectionDataAndSendAsObject()
-Gets the data for a given an array of matching keys, combines them into an object, and sends the result back to the subscriber.
+## notifyCollection()
+Notify subscribers of a batch collection update. Wrapper over
+`onyxSubscriptionManager.notifyCollection()` that also performs LRU bookkeeping per
+changed member.
 
 **Kind**: global function  
 <a name="remove"></a>
 
 ## remove()
-Remove a key from Onyx and update the subscribers
+Remove a key from Onyx and update the subscribers.
 
 **Kind**: global function  
 <a name="retryOperation"></a>
@@ -439,29 +437,6 @@ Validate the collection is not empty and has a correct type before applying merg
 Verify if all the collection keys belong to the same parent
 
 **Kind**: global function  
-<a name="subscribeToKey"></a>
-
-## subscribeToKey(connectOptions) ⇒
-Subscribes to an Onyx key and listens to its changes.
-
-**Kind**: global function  
-**Returns**: The subscription ID to use when calling `OnyxUtils.unsubscribeFromKey()`.  
-
-| Param | Description |
-| --- | --- |
-| connectOptions | The options object that will define the behavior of the connection. |
-
-<a name="unsubscribeFromKey"></a>
-
-## unsubscribeFromKey(subscriptionID)
-Disconnects and removes the listener from the Onyx key.
-
-**Kind**: global function  
-
-| Param | Description |
-| --- | --- |
-| subscriptionID | Subscription ID returned by calling `OnyxUtils.subscribeToKey()`. |
-
 <a name="setWithRetry"></a>
 
 ## setWithRetry(params, retryAttempt)

API.md

@@ -73,36 +73,40 @@ Connects to an Onyx key given the options passed and listens to its changes.
 This method will be deprecated soon. Please use `Onyx.connectWithoutView()` instead.
 
 **Kind**: global function  
-**Returns**: The connection object to use when calling `Onyx.disconnect()`.  
+**Returns**: The `Connection` handle to use when calling `Onyx.disconnect()`.  
 
 | Param | Description |
 | --- | --- |
 | connectOptions | The options object that will define the behavior of the connection. |
 | connectOptions.key | The Onyx key to subscribe to. |
 | connectOptions.callback | A function that will be called when the Onyx data we are subscribed changes. |
-| connectOptions.selector | This will be used to subscribe to a subset of an Onyx key's data. **Only used inside `useOnyx()` hook.**        Using this setting on `useOnyx()` can have very positive performance benefits because the component will only re-render        when the subset of data changes. Otherwise, any change of data on any property would normally        cause the component to re-render (and that can be expensive from a performance standpoint). |
 
 **Example**  
 ```ts
-const connection = Onyx.connectWithoutView({
+const connection = Onyx.connect({
     key: ONYXKEYS.SESSION,
     callback: onSessionChange,
 });
 ```
+
+For a collection root key, the callback fires with the entire frozen collection
+object whenever any member changes; signature `(collection, collectionKey)`.
+For any other key, the callback fires with the value at that key; signature
+`(value, key)`. Initial fire is deferred via `scheduleInitialFire` so it reads
+cache after any same-tick writes have applied.
 <a name="connectWithoutView"></a>
 
 ## connectWithoutView(connectOptions) ⇒
 Connects to an Onyx key given the options passed and listens to its changes.
 
 **Kind**: global function  
-**Returns**: The connection object to use when calling `Onyx.disconnect()`.  
+**Returns**: The `Connection` handle to use when calling `Onyx.disconnect()`.  
 
 | Param | Description |
 | --- | --- |
 | connectOptions | The options object that will define the behavior of the connection. |
 | connectOptions.key | The Onyx key to subscribe to. |
 | connectOptions.callback | A function that will be called when the Onyx data we are subscribed changes. |
-| connectOptions.selector | This will be used to subscribe to a subset of an Onyx key's data. **Only used inside `useOnyx()` hook.**        Using this setting on `useOnyx()` can have very positive performance benefits because the component will only re-render        when the subset of data changes. Otherwise, any change of data on any property would normally        cause the component to re-render (and that can be expensive from a performance standpoint). |
 
 **Example**  
 ```ts
@@ -111,6 +115,12 @@ const connection = Onyx.connectWithoutView({
     callback: onSessionChange,
 });
 ```
+
+For a collection root key, the callback fires with the entire frozen collection
+object whenever any member changes; signature `(collection, collectionKey)`.
+For any other key, the callback fires with the value at that key; signature
+`(value, key)`. Initial fire is deferred via `scheduleInitialFire` so it reads
+cache after any same-tick writes have applied.
 <a name="disconnect"></a>
 
 ## disconnect(connection)

README.md

@@ -9,7 +9,7 @@ Awesome persistent storage solution wrapped in a Pub/Sub library.
 - Onyx allows other code to subscribe to changes in data, and then publishes change events whenever data is changed
 - Anything needing to read Onyx data needs to:
     1. Know what key the data is stored in (for web, you can find this by looking in the JS console > Application > local storage)
-    2. Subscribe to changes of the data for a particular key or set of keys. React function components use the `useOnyx()` hook and non-React libs use `Onyx.connect()`.
+    2. Subscribe to changes of the data for a particular key or set of keys. React function components use the `useOnyx()` hook and non-React libs use `Onyx.connectWithoutView()`.
     3. Get initialized with the current value of that key from persistent storage (Onyx does this by calling `setState()` or triggering the `callback` with the values currently on disk as part of the connection process)
 - Subscribing to Onyx keys is done using a constant defined in `ONYXKEYS`. Each Onyx key represents either a collection of items or a specific entry in storage. For example, since all reports are stored as individual keys like `report_1234`, if code needs to know about all the reports (e.g. display a list of them in the nav menu), then it would subscribe to the key `ONYXKEYS.COLLECTION.REPORT`.
 
@@ -120,20 +120,20 @@ You should avoid arrays as much as possible. They do not work well with `merge()
 
 ## Subscribing to data changes
 
-To set up a basic subscription for a given key use the `Onyx.connect()` method.
+To set up a basic subscription for a given key outside of a React component use the `Onyx.connectWithoutView()` method. It returns a connection handle.
 
 ```javascript
 let session;
-const connectionID = Onyx.connect({
+const connection = Onyx.connectWithoutView({
     key: ONYXKEYS.SESSION,
     callback: (val) => session = val || {},
 });
 ```
 
-To teardown the subscription call `Onyx.disconnect()` with the `connectionID` returned from `Onyx.connect()`. It's recommended to clean up subscriptions anytime you are connecting from within a function to prevent memory leaks.
+To teardown the subscription call `Onyx.disconnect()` with the connection returned from `Onyx.connectWithoutView()`. It's recommended to clean up subscriptions anytime you are connecting from within a function to prevent memory leaks.
 
 ```javascript
-Onyx.disconnect(connectionID);
+Onyx.disconnect(connection);
 ```
 
 We can also access values inside React function components via the `useOnyx()` [hook](https://react.dev/reference/react/hooks). When the data changes the component will re-render.
@@ -204,7 +204,7 @@ export default App;
 
 * It is VERY important to NOT use empty string default values like `report.policyID || ''`. This results in the key returned to `useOnyx` as `policies_`, which subscribes to the ENTIRE POLICY COLLECTION and is most assuredly not what you were intending. You can use a default of `0` (as long as you are reasonably sure that there is never a policyID=0). This allows Onyx to return `undefined` as the value of the policy key, which is handled by `useOnyx` appropriately.
 
-It's also beneficial to use a [selector](https://github.com/Expensify/react-native-onyx/blob/main/API.md#connectmapping--number) with the mapping in case you need to grab a single item in a collection (like a single report action).
+It's also beneficial to use `useOnyx()`'s `selector` option in case you need to grab a single item in a collection (like a single report action).
 
 ## Collections
 
@@ -257,7 +257,7 @@ export default MyComponent;
 This will add a prop to the component called `allReports` which is an object of collection member key/values. Changes to the individual member keys will modify the entire object and new props will be passed with each individual key update. The prop doesn't update on the initial rendering of the component until the entire collection has been read out of Onyx.
 
 ```js
-Onyx.connect({key: ONYXKEYS.COLLECTION.REPORT}, callback: (allReports, collectionKey) => {...});
+Onyx.connectWithoutView({key: ONYXKEYS.COLLECTION.REPORT, callback: (allReports, collectionKey) => {...}});
 ```
 
 This will fire the callback once with the entire collection initially and later with an updated version of the collection when individual keys update.
@@ -289,7 +289,7 @@ function signOut() {
 ```
 
 ## Storage Providers
-`Onyx.get`, `Onyx.set`, and the rest of the API accesses the underlying storage
+`Onyx.set`, `Onyx.merge`, and the rest of the API accesses the underlying storage
 differently depending on the platform
 
 Under the hood storage access calls are delegated to a [`StorageProvider`](lib/storage/index.js)