fix(os): normalize the kernel setup header to QEMU's layout, not zeros - #1229
Merged
Merged
Conversation
kvinwang
force-pushed
the
fix/ovmf-qemu-legacy-setup-header
branch
3 times, most recently
from
September 18, 2026 06:13
db6b55b to
a0bc34d
Compare
Signed-off-by: Kevin Wang <wy721@qq.com>
kvinwang
force-pushed
the
fix/ovmf-qemu-legacy-setup-header
branch
from
September 18, 2026 06:59
a0bc34d to
3f451c3
Compare
Signed-off-by: Kevin Wang <wy721@qq.com>
kvinwang
changed the title
0x40 is XLF_5LEVEL_ENABLED. All three implementations of the setup-header layout tested it as XLF_CAN_BE_LOADED_ABOVE_4G, so a kernel built without 5-level paging would get the 0x37ffffff initrd ceiling where QEMU raises it to 4G, putting the initrd about 1.1 GiB from where QEMU puts it and breaking the RTMR[1] prediction an earlier release's verifier or KMS makes. Every kernel dstack has shipped sets both bits, so no released image changes. A kernel loaded low is now refused rather than normalized: QEMU puts its command line at 0x9a000 - cmdline_size, which depends on the command line the host passes, so there is no single value to pin. The firmware leaves such a header as served. The cross-check test no longer restates what the script writes. It derives the written fields and their widths from both implementations and compares them, matches dstack-mr's constants where it computes the layout rather than anywhere in the file, and covers the ceiling every shipped kernel actually takes -- which the previous vector, with xloadflags = 0, never reached. Signed-off-by: Kevin Wang <wy721@qq.com>
The rationale paragraph carried over from the zeroing version said the field set comes from the boot protocol rather than from QEMU's behavior, that every field written is typed `write` in Documentation/arch/x86/boot.rst, and that fields typed `modify` are left alone. Two of those are false: loadflags is typed `modify (obligatory)` and is written, and the set is now exactly what QEMU fills in, with the write-typed fields QEMU never touches deliberately dropped. A reader following the paragraph would either drop the CAN_USE_HEAP write or add hardware_subarch and setup_data back, desyncing the script from the firmware. It appeared verbatim in the script, the image README, the security model and the OVMF patch's commit message; all four now say what the code does, including that only the CAN_USE_HEAP bit of loadflags is set, because the boot protocol assigns that bit to the boot loader. They also now say what this is: one fixed header, chosen because dstack-mr already computes it, not a reimplementation of QEMU's loader. Zeros would serve the measurement goal equally well; reproducing a header QEMU once wrote is what lets an earlier release's verifier or KMS keep measuring these images. The README's guest-memory note had its appositives attached to the allowed range rather than its complement, so it read as naming the sizes to avoid. It now names both sides. The HEADER_END_OFFSET comment still justified 0x258 as one past the last field touched, which was true when setup_data was zeroed; nothing above 0x238 is read now, so the constant is stated as what it is, the end of the protocol 2.09 header kept as a minimum length. Also joins a paragraph broken mid-sentence in the README, rewraps a comment broken mid-phrase and a 106-character docstring line. No behavior change. The OVMF patch's diff is untouched -- only the commit message above it changes -- so the file it applies to is byte-identical and the firmware does not move. Signed-off-by: Kevin Wang <wy721@qq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
#1189 made RTMR[1] independent of the host's QEMU version by normalizing the setup-header fields a boot loader owns, in both the shipped kernel and OVMF. The mechanism is right; the value it normalizes to is not. Zeros are a layout no released verifier or KMS can reproduce: the
dstack-mrembedded in a 0.5.x KMS predateskernel_header_normalized, so it predicts QEMU's patched header and rejects the image withRTMR1 mismatch.KMS.GetAppKeyruns the same check, so an un-upgraded 0.5.x KMS with the defaultcore.image.verify = truerefuses keys to any CVM on a post-#1189 image, and a root-key handover out of a 0.5.x cluster cannot complete at all.Change
Normalize to the layout QEMU <= 10.1 writes instead of to zeros. Nothing else moves: same two halves, same file names, same
kernel_header_normalizeddeclaration and its wire format, no change todstack-mr,dstack-typesor the verifier.os/image/normalize-kernel-header.pywritestype_of_loader,loadflags.CAN_USE_HEAP,heap_end_ptr,cmd_line_ptr,ramdisk_imageandramdisk_sizeinstead of clearing them, and takes the shipped initrd, whose size decidesramdisk_image.0007-OvmfPkg-QemuKernelLoaderFsDxe-normalize-setup-header.patchwrites the same values, reading the initrd size from fw_cfg.assemble.shpasses the initrd; its comment and the two docs sections say which layout is normalized to.os/tests/test-kernel-header-normalization.shkeeps checking that both sides agree, now on values rather than on a zeroed field table, and additionally thatdstack-mrspells the same constants.RTMR[1] stays the plain Authenticode hash of the
bzImageinsha256sum.txt, on every QEMU version and at every guest memory size — the fields written follow from the kernel file and the initrd size, and the initrd address QEMU derives from guest RAM is fixed at the value it resolves to for a guest with 2 GiB or more below 4G.What QEMU's layout buys is the third implementation:
dstack-mr'spatch_kernelalready computes it for any image that does not declare the flag, so a verifier or KMS from an earlier release can still measure these images. Recomputing it needs the guest RAM size, which limits such a verifier to guests with exactly 2 GiB or at least 2816 MiB — the one range QEMU places the initrd differently in, and the range the no-image-download path already excludes. Images that declare the flag are unaffected by that range, and images built between #1189 and this change keep their zeroed header, their declaration and their measurement.Testing
Local:
os/tests/test-kernel-header-normalization.sh(run byos/mkosi/build.sh lint),reuse lint,prek run. The script's output on a synthetic kernel matchesdstack-mr'spatch_kernelbyte for byte.On
tdxlab(physical TDX, QEMU 8.2.2), against mkosi images rebuilt from this branch, with the shipped kernel carrying this layout and the firmware writing it again:tc-kms-upgrade-005PASS — 0.5.8 and 0.5.11 source KMS nodes, with image verification on, accept a 0.6.0 target in bothliteandautovariants; 0.5.4 still refuses on its certificate envelope. This is the case that fails onnext.tc-gos-platform-002PASS — a CVM on an image with the new firmware boots and provisions sealing keys on physical TDX.tc-ver-input-plat-004,tc-ver-tools-001,tc-ver-tools-002PASS. The shareddstack-mrmatrix passes all 37 rows, among themshipped-canonical-kernel-matches-recomputed-layout: a kernel that ships this layout and declares the flag measures the same RTMR[1] as the same image without the declaration, whichdstack-mrrecomputes from QEMU's behavior. That equality is the compatibility this change exists for.A live boot on QEMU >= 10.2 could not be done here: the 10.2.50 build on the lab host is refused by KVM (
vm-type TDX not supported by KVM), since the host kernel exposes the older TDX ABI. What 10.2 changes is only whether QEMU pre-wrote those fields; OVMF writes them unconditionally from inputs 10.2 still serves, and the test covers that a header QEMU already patched for a different guest size normalizes to the same bytes.normalizedis kept as the name: it says the header is forced into one fixed form, which is what happens, and the form is now named wherever the word appears. Thekernel_header_normalizeddeclaration and its wire format are untouched.The acceptance cases quoted above live in #841, whose required-test list referenced a
dstack-mrtest from an earlier revision of this branch; that is fixed there.