Skip to content

fix: guard coroutine scopes against crashes#1094

Open
jvsena42 wants to merge 4 commits into
masterfrom
fix/harden-coroutine-error-handling
Open

fix: guard coroutine scopes against crashes#1094
jvsena42 wants to merge 4 commits into
masterfrom
fix/harden-coroutine-error-handling

Conversation

@jvsena42

@jvsena42 jvsena42 commented Jul 17, 2026

Copy link
Copy Markdown
Member

Relates to #1093 , #986

This PR:

  1. Adds a shared exception-handling convention so every long-lived background scope logs uncaught errors instead of crashing the app
  2. Fixes background task cancellation being reclassified as a generic error
  3. Adds a process-level uncaught-exception logger as a last-resort backstop

Description

A recent crash came from an uncaught error escaping a background coroutine. None of the app's long-lived scopes had an exception handler, so any unhandled throw in a background task reached the system's default handler and killed the process — often in a restart loop.

This introduces a single helper used across the repositories, services, and background workers so that every long-lived scope shares the same behavior: an uncaught error is logged with its owning tag and the scope stays alive, rather than taking the app down. The same handler is applied to the shared base scope used by the Lightning service and the keychain.

Background queue work now preserves coroutine cancellation instead of wrapping it into a generic error, so structured cancellation keeps working as intended.

Finally, a global uncaught-exception logger is installed at startup that records anything still slipping through before delegating to the platform's default handler, so normal crash behavior is preserved while we gain a log trail.

This is the safety net for the broader hardening tracked in #1093. Guarding the specific background bodies so a failure is handled meaningfully instead of only logged remains a follow-up.

Preview

N/A — no UI changes.

QA Notes

Manual Tests

  • 1. regression: Cold start with an existing wallet → let startup finish: node starts, balances and widgets load, app does not crash.
  • 2. regression: Background the app during sync, then foreground: sync resumes, no crash.

Automated Checks

  • Unit tests added: CoroutineScopesTest.kt asserts a scope keeps running after an uncaught child exception (verified failing without the handler, passing with it).
  • Unit tests added: ServiceQueueTest.kt asserts background work rethrows cancellation unchanged and wraps other failures in AppError.
  • CI: standard compile, unit test, and detekt checks run by the PR bot.

Runtime demonstration (crash simulation)

Verified the two runtime guards on a device (to.bitkit.dev, dev/regtest) by
temporarily injecting uncaught crashes (not committed — reverted after each
run), building & installing the dev build, and driving the app with the journeys
below via the android CLI. Each case shows the injected diff, the journey, and
the resulting logcat + persisted device log (files/logs/…).

Case A — background coroutine crash is contained (scope handler)

A child launched in a long-lived appScope throws; the app must stay alive
and the throw must be logged with the owning tag.

Injected diff (reverted):

--- a/app/src/main/java/to/bitkit/repositories/WalletRepo.kt
+++ b/app/src/main/java/to/bitkit/repositories/WalletRepo.kt
     init {
+        // CRASH-SIM (do not commit): proves appScope loggingExceptionHandler contains uncaught throws
+        repoScope.launch { error("CRASH-SIM: uncaught exception in WalletRepo background scope") }
         repoScope.launch {
             lightningRepo.nodeEvents.collect { event ->

Journey:

<journey name="scope handler contains background crash">
  <description>Precondition: onboarded dev wallet; WalletRepo background scope throws on startup.</description>
  <actions>
    <action>Launch the Bitkit dev app</action>
    <action>Verify the wallet home screen is visible with the total balance and the activity list</action>
    <action>Verify the app process is still running (it did not crash)</action>
  </actions>
</journey>

Result: PASSED — home screen rendered fully, process stayed alive (pid 9270).

logcat + persisted log (files/logs/bitkit_2026-07-20_17-56-47.log):

2026-07-20 17:56:47.533 ERROR [CoroutineExceptionHandler.kt:50]  Uncaught coroutine exception [IllegalStateException='CRASH-SIM: uncaught exception in WalletRepo background scope'] - WalletRepo
java.lang.IllegalStateException: CRASH-SIM: uncaught exception in WalletRepo background scope
    at to.bitkit.repositories.WalletRepo$1.invokeSuspend(WalletRepo.kt:85)

The throw is logged under the owning tag (WalletRepo); startup proceeds and the
UI loads — the scope survives instead of crashing the process.

Case B — startup uncaught crash is recorded then delegated (backstop / P2)

An uncaught throw on a background thread during early startup must be written to
the log file
(the P2 fix) and then follow normal crash behavior.

Injected diff (reverted), on top of the 5949a023c reorder:

--- a/app/src/main/java/to/bitkit/App.kt
+++ b/app/src/main/java/to/bitkit/App.kt
         Env.initAppStoragePath(filesDir.absolutePath)
         installUncaughtExceptionLogger()
+        // CRASH-SIM (do not commit): P2 — genuine uncaught crash during early startup. Because the
+        // handler is installed after initAppStoragePath, the Logger delegate has a valid session
+        // log-file path, so the backstop records this crash before delegating to the default handler.
+        Thread({ throw RuntimeException("CRASH-SIM: uncaught startup crash on background thread") }, "crash-sim").start()
         SingletonImageLoader.setSafe { imageLoader }

Journey:

<journey name="startup backstop records uncaught crash">
  <description>Precondition: dev build; a background thread throws during App.onCreate.</description>
  <actions>
    <action>Launch the Bitkit dev app</action>
    <action>Verify the process crashes (normal crash behavior is preserved)</action>
    <action>Verify the session log file recorded an "Uncaught exception on thread" entry</action>
  </actions>
</journey>

Result: PASSED — backstop logged the crash, then the default handler killed
the process; the entry is present in the session log file.

logcat:

E AndroidRuntime: FATAL EXCEPTION: crash-sim
E AndroidRuntime: Process: to.bitkit.dev, PID: 9664
E AndroidRuntime: java.lang.RuntimeException: CRASH-SIM: uncaught startup crash on background thread
E AndroidRuntime:     at to.bitkit.App.onCreate$lambda$0(App.kt:43)
E APP:            Uncaught exception on thread 'crash-sim' [RuntimeException='CRASH-SIM: uncaught startup crash on background thread'] - App
I ActivityManager: Process to.bitkit.dev (pid 9664) has died: fg  TOP

Persisted log (files/logs/bitkit_2026-07-20_18-00-48.log) — the P2 proof:

2026-07-20 18:00:48.478 INFO  [Env.kt:199]  App storage path: /data/user/0/to.bitkit.dev/files
2026-07-20 18:00:48.480 ERROR [App.kt:54]   Uncaught exception on thread 'crash-sim' [RuntimeException='CRASH-SIM: uncaught startup crash on background thread'] - App

The App storage path: … line (from initAppStoragePath) is written before
the crash entry, confirming the reorder: the storage path — and therefore the
Logger delegate's session file — is initialized before the backstop fires, so the
early-startup crash is captured on disk instead of dropped.

Out of scope (P1, tracked in #1093): the failed child coroutine itself still
terminates — a long-lived collector that throws stops observing until restart.
This PR contains and logs the failure; per-body recovery/relaunch is the
follow-up.

@jvsena42 jvsena42 self-assigned this Jul 17, 2026
ovitrif
ovitrif previously approved these changes Jul 18, 2026

@ovitrif ovitrif left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did an early untested review. LGTM 🫡

@jvsena42 jvsena42 added this to the 2.5.0 milestone Jul 20, 2026
@jvsena42
jvsena42 marked this pull request as ready for review July 20, 2026 17:46
@greptile-apps

greptile-apps Bot commented Jul 20, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds shared crash handling for long-lived coroutine scopes. The main changes are:

  • Shared supervised scopes with tagged exception logging.
  • Cancellation-preserving error handling in ServiceQueue.
  • A process-level uncaught-exception logger.
  • Tests for scope isolation and cancellation propagation.

Confidence Score: 4/5

Long-lived background observers can stop permanently after a handled exception, so that path needs a fix before merging.

  • The parent scope survives, but the failed child coroutine does not restart.
  • A failed connectivity observer can leave synchronization and balances stale.
  • Startup failures before storage initialization can bypass the new crash log.
  • ServiceQueue cancellation propagation matches the intended behavior.

app/src/main/java/to/bitkit/async/CoroutineScopes.kt, app/src/main/java/to/bitkit/repositories/LightningRepo.kt, and app/src/main/java/to/bitkit/App.kt

Important Files Changed

Filename Overview
app/src/main/java/to/bitkit/async/CoroutineScopes.kt Adds shared supervised scopes, but failed long-lived child jobs are logged without being restarted.
app/src/main/java/to/bitkit/repositories/LightningRepo.kt Moves long-lived observers and retry work to the shared handler, exposing permanent child-job termination after an error.
app/src/main/java/to/bitkit/App.kt Installs the global logger before logger storage initialization, leaving an early-startup diagnostic gap.
app/src/main/java/to/bitkit/async/ServiceQueue.kt Preserves coroutine cancellation while continuing to wrap ordinary failures in AppError.
app/src/main/java/to/bitkit/async/BaseCoroutineScope.kt Adds tagged exception handling to the existing supervised base scope.

Reviews (1): Last reviewed commit: "Merge branch 'master' into fix/harden-co..." | Re-trigger Greptile

Comment thread app/src/main/java/to/bitkit/async/CoroutineScopes.kt
Comment thread app/src/main/java/to/bitkit/App.kt Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants