UTXO missing from wallet after a restart and an unwanted local force-close #9200

Activity

1. nsabedra27-glitch commented on Jun 10, 2026

   

   nsabedra27-glitch

   on Jun 10, 2026

   More actions

   It wouldn't ever show my current bitcoin address
2. sangbida commented on Nov 5, 2026

   

   sangbida

   on Nov 5, 2026

   Collaborator

   More actions

   Thanks for such a detailed report!

   Your logs show the peer had a newer commitment than your node (they have a future one), so your node waited for them to close, and they closed with a commitment yours had never seen (Unknown commitment #1457929). In that case we can only recover the to_remote output (your 9922922 sat). The 215289 sat is a pending HTLC output; spending it needs the peer's per-commitment point for that commitment, which we didn't keep.

   The onchain_fee entry is the bookkeeper balancing the account, not a real fee: that output is still unspent at 5bf58ab7…de4b:2, the HTLC has long expired, and the peer never claimed it, so it's yours. Rescans can't find it because it was never a wallet address.

   Are you able to reach out to the operator of peer node for the per-commitment point of commitment #1457929 and the payment_hash and cltv_expiry of the 215289 sat HTLC. Unfortunately, there isn't an inbuilt tool at the moment to recover the funds yet but we could write a script for you to try recover these funds.

   This is quite strange for this to happen on a simple restart, did you have to restore from a backup?
3. sangbida

   self-assigned this

   on Nov 6, 2026
4. micoq commented on Nov 7, 2026

   

   micoq

   on Nov 7, 2026

   Last edited by micoq

   Author

   More actions

   Thank you for your response and the explanations!

   There was no data recovery (and I know that's a very bad idea!), just a reboot of the node.
   The shutdown was clean, and there was no data corruption at the file system (or block) level. The node directory (with its SQLite database) is on a RAID 1 array that was intact. I rechecked the entire array after noticing the problem and there was no corruption.

   I will try to contact the node owner.

   Edit: Thanks to the peer node owner, here are the required values:

   • per-commitment point: 0381c3d143f665a949eecc8b02fceffa7774a33af80dbc5a3b484fa5fa6f6e3d67
   • payment_hash: e0258417c9514babeb6d9d5f3db5db9fec81755d81da93eb3690e24407c430ec
   • cltv_expiry: 952295

   These values seem correct: I was able to reconstruct the witness script and its hash is equal to the ScriptPubKey of the UTXO.

   Now, I just need a little help so I can build the final transaction and sign it correctly.
5. sangbida commented on Nov 12, 2026

   

   sangbida

   on Nov 12, 2026

   Collaborator

   More actions

   Hey @micoq! It’s great that you were able to recover all of the information and verify that the witness script matches the stranded output.
   I’ve added a recoverhtlc command to lightning-hsmtool so you can build and sign the recovery transaction, and so other people can use it if they end up in the same situation.

   The PR is here:
   #9730

   The patch can be downloaded directly from:
   https://github.com/ElementsProject/lightning/pull/9730.patch

   The branch is here:
   https://github.com/sangbida/lightning/tree/hsmtool/recover-htlc

   Please note that this PR has not been reviewed or merged yet and the command will sign a transaction spending real funds.

   You do not need to replace or upgrade your running lightningd. lightning-hsmtool is standalone and does not connect to your node.

   If you installed CLN from a binary package and do not already have a source tree, you can build just the tool separately:

   git clone --recurse-submodules \
       --branch hsmtool/recover-htlc \
       https://github.com/sangbida/lightning.git lightning-recoverhtlc
   cd lightning-recoverhtlc
   
   ./configure
   
   make -j"$(nproc)" tools/lightning-hsmtool
   

   This produces tools/lightning-hsmtool. You do not need to install the build or run the lightningd it creates.
   I would use copies of your database and hsm_secret while doing this. Please never send your hsm_secret to me or anyone else.
   You still need the channel DBID and the peer’s HTLC basepoint. They should be in the surviving channel row. You can read them from a copy of the database with:

   sqlite3 /path/to/copied-lightningd.sqlite3 \
     "SELECT id, lower(hex(htlc_basepoint_remote))
        FROM channels
       WHERE id = 11240;"

   Please check the ID rather than assuming 11240 from the log is correct.

   Then create a recovery.json file like this:

   {
     "version": 1,
     "network": "bitcoin",
     "peer_id": "03a93b87bf9f052b8e862d51ebbac4ce5e97b5f4137563cd5128548d7f5978dda9",
     "channel_dbid": <channel DBID>,
     "remote_htlc_basepoint": "<htlc_basepoint_remote>",
     "commitment_point": "0381c3d143f665a949eecc8b02fceffa7774a33af80dbc5a3b484fa5fa6f6e3d67",
     "commitment_tx": "<raw hex of 5bf58ab7e136d74d837472eaa907a5b3c508e4dfd8a20063c4814ebdddc1de4b>",
     "feerate_perkw": 2500,
     "htlcs": [
       {
         "direction": "out",
         "payment_hash": "e0258417c9514babeb6d9d5f3db5db9fec81755d81da93eb3690e24407c430ec",
         "cltv_expiry": 952295
       }
     ]
   }

   Run it with a fresh Bitcoin address that you control:

   ./tools/lightning-hsmtool recoverhtlc \
       recovery.json \
       <destination-address> \
       /path/to/copied-hsm_secret

   The tool checks that the reconstructed HTLC script actually matches the commitment output before signing. It does not broadcast anything or print private keys.

   It will print the signed transaction and commands for checking it with decoderawtransaction and testmempoolaccept. Please inspect those results before broadcasting it with sendrawtransaction.

   The alternative is to wait until this has been reviewed, merged and included in a release, but I don’t want you to have to leave the funds stranded while waiting. If you would prefer, send me the DBID, remote HTLC basepoint and raw commitment transaction—none of those are secret—and I can help check the bundle before you run it.

   Before running anything, please independently verify every value in the recovery bundle against your own database, logs and the on-chain transaction. In particular, please verify:

   • the peer ID and channel DBID;
   • htlc_basepoint_remote
   • the per-commitment point supplied by the peer;
   • the raw commitment transaction;
   • that this was an HTLC you offered ("direction": "out")
   • the payment hash and CLTV expiry;
   • the destination address and feerate.

   Please do not rely only on the values or interpretation in this comment. Read through the PR and the tool output yourself, and inspect the signed transaction with decoderawtransaction and testmempoolaccept before broadcasting it. If anything does not match what you expect, stop rather than broadcasting.