fix(market-data): drain late router completions

Retain provider subscribe and unsubscribe completions across cancelled owner tasks, expose two-phase Router processing, and keep physical cleanup retryable until shutdown completes. Document the lifecycle in the English and Russian guides and cover late source-thread completions with Router and subscriber-base regressions.

Author: LimiNode

guides/api-and-header-contracts.md

@@ -253,7 +253,13 @@ Contract rules:
   provider handle and callback binding for cleanup. Check
   `failed_unsubscribe_count()` and call `retry_failed_unsubscribes()` from the
   owner loop; new routes through that provider are rejected until cleanup
-  succeeds. `shutdown()` makes a final best-effort attempt.
+  succeeds.
+- Router shutdown is two-phase. `shutdown()` stops new routes and user delivery,
+  retains tombstones for pending provider operations, and performs one immediate
+  `process()` pass. Keep the provider and owner loop alive and call Router
+  `process()` until `is_shutdown_complete()` becomes true. Late successful
+  subscribes are physically unsubscribed without user callbacks or replay;
+  failed cleanup remains retryable and prevents shutdown completion.
 - `register_provider()` adds a non-owning provider reference under a stable,
   application-assigned `MarketDataProviderId` and optional exact string aliases.
   Registration is a selection catalog only and does not bind live callbacks.
@@ -273,7 +279,9 @@ Contract rules:
   delivery instead of invoking subscriber code in the foreign source thread.
 - `BaseTradingPlatform::post_task()` is the standard adapter for using the
   platform TaskManager as that owner loop. Shutdown may cancel accepted tasks,
-  so Router and subscriber cleanup must be drained before stopping the platform.
+  so provider operation results are retained in Router state rather than owned
+  only by posted tasks. Router and subscriber cleanup must be drained before
+  stopping the platform.
 
 `MarketDataSubscriberBase` is optional convenience sugar over Router. A bot can
 derive from it, call protected `subscribe_ticks()`/`subscribe_bars()` from its

guides/market-data-router.md

@@ -172,8 +172,9 @@ if (router.failed_unsubscribe_count() != 0) {
 
 The original unsubscribe callback receives the failure once; internal retries
 do not resurrect the consumed public handle. Apply application-level backoff
-before retrying persistent failures. `shutdown()` makes one final best-effort
-cleanup attempt before releasing Router state.
+before retrying persistent failures. `shutdown()` retries cleanup entries that
+had already failed once, but a repeated failure keeps Router in its draining
+state until the application retries or applies a future explicit abandon policy.
 
 The optional subscription callback reports desired-state acceptance by the
 provider. It is not a transport-readiness callback:
@@ -340,19 +341,25 @@ transport is ready. Those stages are observed separately:
 
 If the subscriber expires before a posted subscribe command runs, the command is
 cancelled and its callbacks are not invoked. Once a configured dispatcher starts
-rejecting work during shutdown, new provider completions and deliveries are
-dropped rather than being invoked inline on the source thread. Tick, bar, and
-status deliveries need no physical cleanup and are discarded immediately.
-
-A successful subscribe completion reserves its concrete provider handle in
-Router state before the completion task is posted. The owner task must atomically
-claim that reservation before promoting the route from pending to active. If the
-task is rejected, the reservation remains with the pending route and Router
-quarantines that provider from new routes. If the task is accepted but later
-cancelled during owner shutdown, the reservation likewise remains available to
-`router.shutdown()`. In both cases shutdown unsubscribes the retained handle from
-the owner loop; neither the subscription callback nor subscriber callbacks run
-on the source thread.
+rejecting work during shutdown, new tick, bar, and status deliveries are dropped
+rather than being invoked inline on the source thread. Those deliveries need no
+physical cleanup and are discarded immediately.
+
+Subscribe and unsubscribe completions are recorded in Router state before an
+owner task is posted. A successful subscribe result reserves its concrete
+provider handle; the owner task atomically claims that reservation before
+promoting the route from pending to active. Rejected or later-cancelled owner
+tasks therefore do not own the only copy of a physical handle or completion
+result.
+
+`process()` advances Router-owned deferred lifecycle work on the owner thread.
+During normal operation posted completion tasks apply their transitions
+directly, so Router does not require a separate pump. After `shutdown()` starts
+draining, late provider completions remain in Router state and `process()` turns
+every late successful subscribe into a physical unsubscribe without invoking
+user callbacks or replay. It does not poll providers or transports; in manual
+platform mode call `platform.process()` first so the provider can produce its
+completion.
 
 All Router callbacks are serialized by the owner loop, but fields read directly
 from a different bot thread still require the bot's own mutex, atomics, or
@@ -368,18 +375,27 @@ provider and owner dispatcher
         outlives subscribers and posted cleanup work
 ```
 
-Use this shutdown order:
+`shutdown()` is an idempotent request to stop, not an unconditional assertion
+that every asynchronous provider operation finished before it returned. It
+performs one `process()` pass itself, so synchronous cleanup still completes in
+the call. Use this shutdown order:
 
 1. Stop producing new commands from bot threads.
 2. Request unsubscribe or destroy subscribers while the dispatcher still accepts
    work.
-3. Keep processing the owner loop until unsubscribe commands and any nested
-   provider completion callbacks are drained. Inspect
-   `failed_unsubscribe_count()` and retry according to the application policy
-   while the owner loop and providers are still available.
-4. Call `router.shutdown()` from the owner loop. It is idempotent, releases
-   provider callback slots, and requests unsubscribe for remaining active routes.
-5. Stop the platform/dispatcher and then destroy providers.
+3. Call `router.shutdown()` from the owner loop. New routes and user delivery
+   stop immediately; pending provider operations remain as cleanup tombstones.
+4. Keep the provider and owner loop running until
+   `router.is_shutdown_complete()` is true. In manual mode each host tick calls
+   `platform.process()` and then `router.process()`.
+5. Inspect `failed_unsubscribe_count()` and retry with application-level backoff
+   while the owner loop and providers remain available. Failed cleanup prevents
+   `is_shutdown_complete()` from becoming true.
+6. Stop the platform/dispatcher and then destroy providers.
+
+Applications that compose several process/shutdown modules should put this
+drain loop in their lifecycle supervisor rather than special-case Router in
+business code.
 
 Do not defer subscriber destruction until after the dispatcher is closed.
 `MarketDataSubscriberBase` normally posts remaining handles as one cleanup task;

guides/market-data-router.ru.md

@@ -172,8 +172,9 @@ if (router.failed_unsubscribe_count() != 0) {
 
 Исходный unsubscribe callback получает ошибку один раз; внутренний retry не
 восстанавливает уже использованный публичный handle. Для постоянных ошибок
-применяй backoff на уровне приложения. `shutdown()` делает последнюю
-best-effort попытку cleanup перед освобождением состояния Router.
+применяй backoff на уровне приложения. `shutdown()` повторяет cleanup entries,
+которые уже один раз завершились ошибкой, но повторная ошибка оставляет Router в
+состоянии draining до retry приложения или будущей явной abandon policy.
 
 Необязательный subscription callback сообщает о принятии desired state
 провайдером. Это не callback готовности транспорта:
@@ -341,19 +342,24 @@ tick batches, bar batches и status updates в этот loop. Боты испо
 
 Если subscriber уничтожен до выполнения posted subscribe command, команда
 отменяется, а её callbacks не вызываются. Когда настроенный dispatcher начинает
-отклонять работу во время shutdown, новые provider completions и deliveries
-отбрасываются, а не исполняются inline в source thread. Tick, bar и status
-deliveries не требуют физического cleanup и сразу удаляются.
-
-Успешный subscribe completion резервирует concrete provider handle в состоянии
-Router до постановки completion task в очередь. Owner task должен атомарно
-забрать эту reservation перед переводом маршрута из pending в active. Если задача
-отклонена, reservation остаётся у pending маршрута, а Router помещает provider в
-карантин для новых routes. Если задача принята, но позднее отменена во время
-shutdown owner loop, reservation также остаётся доступной для
-`router.shutdown()`. В обоих случаях shutdown отписывает сохранённый handle из
-owner loop; ни subscription callback, ни callbacks subscriber не выполняются в
-source thread.
+отклонять работу во время shutdown, новые tick, bar и status deliveries
+отбрасываются, а не исполняются inline в source thread. Эти deliveries не
+требуют физического cleanup и сразу удаляются.
+
+Subscribe и unsubscribe completions записываются в состояние Router до
+постановки owner task. Успешный subscribe result резервирует concrete provider
+handle; owner task атомарно забирает reservation перед переводом route из pending
+в active. Поэтому отклонённая или позднее отменённая owner task не владеет
+единственной копией physical handle или completion result.
+
+`process()` продвигает принадлежащую Router deferred lifecycle work в owner
+thread. При обычной работе posted completion tasks применяют transitions
+напрямую, поэтому Router не требует отдельной прокачки. После начала draining
+через `shutdown()` поздние provider completions остаются в состоянии Router, а
+`process()` превращает каждый поздний успешный subscribe в physical unsubscribe
+без user callbacks и replay. Метод не опрашивает providers или transports; в
+ручном режиме платформы сначала вызывай `platform.process()`, чтобы provider
+смог сформировать completion.
 
 Все callbacks Router сериализованы owner loop, но поля, которые напрямую читает
 другой поток бота, всё равно требуют собственного mutex, atomics или очереди
@@ -369,18 +375,26 @@ provider и owner dispatcher
         который живёт дольше subscribers и posted cleanup work
 ```
 
-Используй такой порядок остановки:
+`shutdown()` — идемпотентный запрос остановки, а не безусловное утверждение, что
+все асинхронные provider operations завершились до возврата метода. Он сам
+выполняет один проход `process()`, поэтому синхронный cleanup по-прежнему
+завершается внутри вызова. Используй такой порядок остановки:
 
 1. Прекрати создавать новые команды в потоках ботов.
 2. Запроси unsubscribe или уничтожь subscribers, пока dispatcher принимает
    работу.
-3. Продолжай обработку owner loop, пока не завершатся unsubscribe commands и
-   вложенные provider completion callbacks. Проверь
-   `failed_unsubscribe_count()` и выполни retry по политике приложения, пока
-   owner loop и providers ещё доступны.
-4. Вызови `router.shutdown()` из owner loop. Метод идемпотентен, освобождает
-   callback slots провайдера и запрашивает unsubscribe оставшихся маршрутов.
-5. Останови platform/dispatcher, затем уничтожай providers.
+3. Вызови `router.shutdown()` из owner loop. Новые routes и user delivery
+   прекращаются сразу; pending provider operations остаются cleanup tombstones.
+4. Оставь provider и owner loop работающими, пока
+   `router.is_shutdown_complete()` не вернёт true. В ручном режиме каждый host
+   tick вызывает `platform.process()`, а затем `router.process()`.
+5. Проверяй `failed_unsubscribe_count()` и выполняй retry с backoff приложения,
+   пока owner loop и providers доступны. Failed cleanup не позволяет
+   `is_shutdown_complete()` стать true.
+6. Останови platform/dispatcher, затем уничтожай providers.
+
+Если приложение объединяет несколько process/shutdown modules, этот drain loop
+должен находиться в lifecycle supervisor, а не в Router-specific business code.
 
 Не откладывай уничтожение subscriber до момента, когда dispatcher уже закрыт.
 Обычно `MarketDataSubscriberBase` отправляет оставшиеся handles одной cleanup