osslsigncode fails to verify correctly signed Microsoft msi package

[svassilev]

Current Behavior

osslsigncode computes the wrong Authenticode content digest for MSI files that
have a \x05MsiDigitalSignatureEx (DSE) stream and at least one stream whose
name sorts before it. Such files fail verify even though Windows/signtool
accept them. Affects real Microsoft-signed packages.

Example:

svassilev@LAPTOP-9G3Y694:/mnt/c/code/osslsigncode/build$ ./osslsigncode verify /mnt/d/MsiRepo/WindowsInstaller/25dfc7c.msi

Signature Index: 0  (Primary Signature)

Message digest algorithm         : SHA256
Current MsiDigitalSignatureEx    : E610ADA5DABECD1E49E4CA2E6A33170517950603147A04F8D6BD44AB9F09F1B1
Calculated MsiDigitalSignatureEx : E610ADA5DABECD1E49E4CA2E6A33170517950603147A04F8D6BD44AB9F09F1B1
Current DigitalSignature         : 49629B77203525B1B10ED688CB0C1D9E612DCCA5F9E22A97AC437CFE0AB90BDB
Calculated DigitalSignature      : 48147FA6E95001D96C00FA3CDB5FA5670E0BCCF9470633DB95003E6AC9DD05A0     MISMATCH!!!

Signature verification: failed

Number of verified signatures: 1
Failed

Note the mismatch - the calculated DigitalSignature is in fact incorrect.

Expected Behavior

The file is correctly signed. Here is the output from SysInternal's sigcheck64 tool:

C:\>\Tools\sigcheck\sigcheck64.exe /i d:\MsiRepo\WindowsInstaller\25dfc7c.msi

Sigcheck v2.90 - File version and signature viewer
Copyright (C) 2004-2022 Mark Russinovich
Sysinternals - www.sysinternals.com

d:\msirepo\windowsinstaller\25dfc7c.msi:
        Verified:       Signed
        File date:      8:02 PM 4/7/2026
        Signing date:   11:35 PM 1/26/2026
        Catalog:        d:\msirepo\windowsinstaller\25dfc7c.msi
        Signers:
           Microsoft Corporation
                Cert Status:    This certificate or one of the certificates in the certificate chain is not time valid.
                Valid Usage:    1.3.6.1.4.1.311.61.6.1, Code Signing
                Cert Issuer:    Microsoft Windows Code Signing PCA 2024
                Serial Number:  33 00 00 00 86 E7 71 94 B9 4D FF 09 FB 00 00 00 00 00 86
                Thumbprint:     F6EECCC7FF116889C2D5466AE7243D7AA7698689
                Algorithm:      sha384RSA
                Valid from:     11:24 AM 5/8/2025
                Valid to:       11:24 AM 5/6/2026
           Microsoft Windows Code Signing PCA 2024
                Cert Status:    Valid
                Valid Usage:    All
                Cert Issuer:    Microsoft Root Certificate Authority 2010
                Serial Number:  33 00 00 00 1C 48 9F 81 DF A1 B0 B7 77 00 00 00 00 00 1C
                Thumbprint:     D30F05F637E605239C0070D1EA9860D434AC2A94
                Algorithm:      sha384RSA
                Valid from:     2:36 PM 8/8/2024
                Valid to:       3:04 PM 6/23/2035
           Microsoft Root Certificate Authority 2010
                Cert Status:    Valid
                Valid Usage:    All
                Cert Issuer:    Microsoft Root Certificate Authority 2010
                Serial Number:  28 CC 3A 25 BF BA 44 AC 44 9A 9B 58 6B 43 39 AA
                Thumbprint:     3B1EFD3A66EA28B16697394703A72CA340A05BD5
                Algorithm:      sha256RSA
                Valid from:     2:57 PM 6/23/2010
                Valid to:       3:04 PM 6/23/2035
        Counter Signers:
           Microsoft Time-Stamp Service
                Cert Status:    Valid
                Valid Usage:    Timestamp Signing
                Cert Issuer:    Microsoft Time-Stamp PCA 2010
                Serial Number:  33 00 00 02 16 D6 93 CE FB 93 1F ED E1 00 01 00 00 02 16
                Thumbprint:     E944CB9EA86A101D99D697C7C1614F89007B593B
                Algorithm:      sha256RSA
                Valid from:     11:48 AM 8/14/2025
                Valid to:       11:48 AM 11/13/2026
           Microsoft Time-Stamp PCA 2010
                Cert Status:    Valid
                Valid Usage:    Timestamp Signing
                Cert Issuer:    Microsoft Root Certificate Authority 2010
                Serial Number:  33 00 00 00 15 C5 E7 6B 9E 02 9B 49 99 00 00 00 00 00 15
                Thumbprint:     36056A5662DCADECF82CC14C8B80EC5E0BCC59A6
                Algorithm:      sha256RSA
                Valid from:     11:22 AM 9/30/2021
                Valid to:       11:32 AM 9/30/2030
           Microsoft Root Certificate Authority 2010
                Cert Status:    Valid
                Valid Usage:    All
                Cert Issuer:    Microsoft Root Certificate Authority 2010
                Serial Number:  28 CC 3A 25 BF BA 44 AC 44 9A 9B 58 6B 43 39 AA
                Thumbprint:     3B1EFD3A66EA28B16697394703A72CA340A05BD5
                Algorithm:      sha256RSA
                Valid from:     2:57 PM 6/23/2010
                Valid to:       3:04 PM 6/23/2035
        Company:        n/a
        Description:    n/a
        Product:        n/a
        Prod version:   n/a
        File version:   n/a
        MachineType:    n/a

Steps To Reproduce & Observed Output

1. Signing with osslsigncode

2. Verification with osslsigncode

svassilev@LAPTOP-9G3Y694:/mnt/c/code/osslsigncode/build$ ./osslsigncode verify /mnt/d/MsiRepo/WindowsInstaller/25dfc7c.msi

Signature Index: 0  (Primary Signature)

Message digest algorithm         : SHA256
Current MsiDigitalSignatureEx    : E610ADA5DABECD1E49E4CA2E6A33170517950603147A04F8D6BD44AB9F09F1B1
Calculated MsiDigitalSignatureEx : E610ADA5DABECD1E49E4CA2E6A33170517950603147A04F8D6BD44AB9F09F1B1
Current DigitalSignature         : 49629B77203525B1B10ED688CB0C1D9E612DCCA5F9E22A97AC437CFE0AB90BDB
Calculated DigitalSignature      : 48147FA6E95001D96C00FA3CDB5FA5670E0BCCF9470633DB95003E6AC9DD05A0     MISMATCH!!!

Signature verification: failed

Number of verified signatures: 1
Failed

3. Signing / verification with Windows signtool (if applicable)

N/A

Environment

ALL - the algorithm for digest calculation makes an incorrect assumption.

Versions

• osslsigncode built from:
  • [ x] upstream master
  • upstream release (tag):
  • distribution package (name and version):
• openssl version -a
• osslsigncode --version

Files

• unsigned file
• file signed with osslsigncode
• [x ] file signed with signtool or the other tool (for comparison)
• certificate chain used for verification (PEM format)

Anything else

Root cause
For the "Ex" signature, osslsigncode treats the pre-hash as a mandatory prefix:
msi_digest_calc_bio() prepends it via msi_calc_MsiDigitalSignatureEx(), and
msi_hash_dir() then skips both \x05DigitalSignature and
\x05MsiDigitalSignatureEx. So the pre-hash is always hashed at position 0.

The two constructions produce the same digest only when the DSE stream happens
to be the first hashed stream. As soon as any stream sorts before it (i.e. a
name whose first byte < 0x05), the pre-hash lands in the wrong slot and the
digest diverges.

Suggested Fix
If DSE is present:
: do not prepend the pre-hash, and do not skip
\x05MsiDigitalSignatureEx in the directory walk — hash it as a normal stream
in sorted order. msi_calc_MsiDigitalSignatureEx() is still needed on verify to
check that the stored DSE stream equals the recomputed metadata pre-hash, but
not to feed the content hash.

25dfc7c.zip