DF-5556 — vkernel npxsave() missing FPU security wipe: previous FP owner's XMM registers are handed to the next exec'd image (cross-process/cross-exec FP state leak)

What

sys/platform/vkernel64/x86_64/npx.c:248-258 — the vkernel's npxsave() saves the outgoing thread's FP state and calls fninit(), but never wipes the SSE register file:

void
npxsave(union savefpu *addr)
{
    crit_enter();
    fpusave(addr, 0);
    mdcpu->gd_npxthread = NULL;
    fninit();                       /* x87 only — XMM0-15 keep old data */
    crit_exit();
}

pc64's twin (sys/platform/pc64/x86_64/npx.c:303-317) got the security wipe years ago:

    fninit();
    fpurstor(&md->gd_zerofpu, npx_xcr0_mask);   /* security wipe */

with union savefpu gd_zerofpu (BSS-zeroed) in pc64's mdglobaldata (sys/platform/pc64/include/globaldata.h:72). The vkernel copy never got it — sys/platform/vkernel64/include/globaldata.h has no gd_zerofpu at all.

Why it leaks (mechanism, all on this file / its platform)

The vkernel multiplexes its user processes inside ONE host thread, and go_user() (sys/platform/vkernel64/x86_64/trap.c:1371-1377) does an eager FP switch on every entry to emulated user mode:

    if (mdcpu->gd_npxthread != curthread) {
        if (mdcpu->gd_npxthread)
            npxsave(mdcpu->gd_npxthread->td_savefpu);
        npxdna(tf);
    }

For a thread whose TDF_USINGFP is clear (any freshly exec'd image), npxdna() (npx.c:187-191) calls npxinit(), whose fpusave(curthread->td_savefpu, 0) (npx.c:145) is an fxsave of the live registers — the previous FP owner's leftovers — followed by fpurstor() of the same seeded image (npxdna, npx.c:222). Nothing in the chain ever clears XMM0-15, and the vkernel kernel itself is compiled -mno-sse so kernel code never clobbers them either.

Net effect: whatever a process leaves in XMM0-15 survives process boundaries — through exit() (npxexit only saves, npx.c:154-159) and, deterministically, through the fork+exec of the next image (the child's first user-mode entry seeds from the live registers; cpu_fork() at sys/platform/vkernel64/x86_64/vm_machdep.c:110-112 saves the parent's FP via the same non-wiping npxsave, and nothing between that and the child's first entry touches XMM).

FP-resident secrets are the classic target: AES-NI round-key schedules (openssl keeps expanded keys in XMM), bignum limbs (MPIR/GMP), hashed key material — exactly the data class that OSes wipe FP registers for (this is why pc64 has the wipe; FreeBSD fixed the same class in SA-05:14/netisr advisory history).

Verified (this pack)

Inside a vkernel (kernel.debug -m 512m -r /vkroot.img -n 1, built from the audited tree, on the DragonFly QEMU guest):

• victim loads a distinct 128-bit marker (hi=0xDF555600DEADBEEF, lo=register index) into XMM0-15, then fork()+execl()s the attacker (a different program image).
• attacker is a crt-free static binary whose _start reads XMM0-15 as its very first instructions and writes raw hex to stdout.
• Result (run.log), 3/3 runs: 15 of 16 registers contain the victim's exact marker bytes (xmm0 is clobbered by the exec path; xmm1-15 leak): X01 df555600deadbeef 0000000000000001 X02 df555600deadbeef 0000000000000002 ... X0f df555600deadbeef 000000000000000f
• Control run (attacker alone, no victim): all registers zero.
• Cross-process probe variant (prober: unrelated process fork+exec'ing readouts while an independent victim spins): no markers — the prober's own user-mode re-entries run npxdna() which fxrstor's the prober's own saved state over the live registers before each child's first entry. The leak channel therefore requires the victim (or a victim-descended exec chain) to be the last FP owner before the probe's first user entry — which is exactly the shell/mail-client/crypto-app-that-execs shape, and any attacker controlling a program the victim execs, or racing the victim's exit with an exec probe (no FP-using intermediary scheduled in between).
• Fix validation: applying fix.diff (the pc64 wipe ported to the vkernel), rebuilding VKERNEL64 and re-running the exact PoC: 0/16 markers, registers read back zero (run.fixed.log).

Fix

Port pc64's wipe (fix.diff): add union savefpu gd_zerofpu; to vkernel64's mdglobaldata and fpurstor(&mdcpu->gd_zerofpu, 0); after fninit() in npxsave().

Build / Run

See build.sh / build_image.sh / run.sh (rig adapted from the proven DF-5337/DF-5538 vkernel packs; needs the DF-5474 SYSTEM_LD workaround to link VKERNEL64).

DF-5556 — VERDICT

status: reproduced (leak) · fix validated · impact: cross-process/cross-exec FP-register info leak inside a vkernel

What was run

Rig (from the proven DF-5337/DF-5538 packs): DragonFly QEMU guest → /usr/obj/usr/src/sys/VKERNEL64/kernel.debug -m 512m -r /vkroot.img -n 1 -e init_path=/bin/vkinit (single vCPU so all emulated processes share one hardware FPU; VKERNEL64 linked with the DF-5474 --defsym __build_id_* workaround).

• victim (normal static binary): loads hi=0xDF555600DEADBEEF, lo=reg_index into XMM0..15, then fork()+execl("/root/attacker").
• attacker (static, -nostartfiles, raw syscalls): _start's first instructions read XMM0..15 and write raw hex to stdout.
• control: attacker alone; prober variant: unrelated process fork+exec'ing readouts next to a spinning victim.

Baseline (unfixed vkernel)

run.log (raw vkernel console; leak_sample.txt extracted):

# /root/victim fork; echo RUNA1_RC=$?; sleep 2
X00 0000000000000000 0000000000000000      <- clobbered by execl()'s libc SSE
X01 df555600deadbeef 0000000000000001      <- victim's marker
X02 df555600deadbeef 0000000000000002
...
X0f df555600deadbeef 000000000000000f

3/3 runs: 15/16 registers carry the victim's exact 128-bit markers (xmm1-15; xmm0 is overwritten by the libc execl path of the victim's own child — the child runs victim-image libc between fork and exec). Control run: 16/16 zeros. Cross-process prober variant: 0 markers — the prober's own user-mode re-entries (go_user→npxdna restores the prober's saved state) overwrite the live registers before each child's first entry; the channel therefore needs the victim to be the last FP owner before the fresh image's first user entry (the fork+exec chain of RUN A proves it deterministically; the exit-then-exec and victim-execs-attacker-controlled-program shapes are the real-world versions).

Root cause (code)

• sys/platform/vkernel64/x86_64/npx.c:248-258 — npxsave() does fxsave+fninit() and no XMM wipe (pc64 twin wipes via fpurstor(&md->gd_zerofpu) at sys/platform/pc64/x86_64/npx.c:314; vkernel64's mdglobaldata has no gd_zerofpu at all, sys/platform/vkernel64/include/globaldata.h:66-84).
• sys/platform/vkernel64/x86_64/trap.c:1371-1377 — go_user() does an eager FP handoff on every entry to emulated user mode.
• sys/platform/vkernel64/x86_64/npx.c:187-191,145 — a fresh image's first entry (TDF_USINGFP clear) runs npxinit() whose fpusave() is an fxsave of the live registers → the previous owner's XMM data becomes the new image's initial FP state (registers and its td_savefpu).
• sys/platform/vkernel64/x86_64/npx.c:154-159 — npxexit() likewise only saves; a dying process's XMM state stays live for the next first-use thread.

The vkernel kernel itself is compiled -mno-sse -msoft-float, so no kernel code clobbers XMM between owner and victim-image — the channel is quiet and deterministic.

Fix validation

fix.diff (pc64's wipe ported: union savefpu gd_zerofpu in vkernel64 mdglobaldata + fpurstor(&mdcpu->gd_zerofpu, 0) after fninit() in npxsave()), applied to the guest /usr/src, make nativekernel KERNCONF=VKERNEL64 (RC=0, npx.o rebuilt, fix-build.log), exact same PoC re-run:

RUN A ×3 : markers = 0/16, 0/16, 0/16   (run.fixed.log)
control   : 0/16 (unchanged)

Bad behavior gone; the fork-path cpu_fork() → npxsave() (and every switch-out) now zeroes the live register file after saving.

PoC quirks (honesty notes)

• The attacker's in-binary LEAK_REGS= counter printed 0 despite the raw hex lines showing markers — an unexplained miscompilation/artifact of the crt-free binary (the disassembled compare at 0x400182 cmp %rbp,%r13 looks correct; the sibling IDX= counter for the low halves worked). The canonical count is taken externally from the raw dump lines (leak_sample.txt / analysis in the run logs): 15/16 × 3.
• Early PoC iterations crashed for environment reasons, each fixed and documented: libc %fs:0 errno path without crt (switched to raw syscalls); vkernel initial %rsp sits under the stack top page (switched to scalar-per-register output with a tiny frame); victim dumps went to the pipe instead of the console (moved to stderr); the readout must happen at _start (first user entry), not after a blocking syscall — the first entry is exactly where TDF_USINGFP gets set and the seed happens.

Attempts

7 distinct guest iterations (build system fix, 2 PoC crashes, instrumented victim proving load+syscall survival, readout-point fix, final baseline, final fixed run).

Impact ceiling

In-vkernel cross-process disclosure of FP-register-resident secrets (AES-NI round keys, bignum limbs) — up to 256 bytes per observed handoff (XMM1-15 in the repro). Not kernel-memory disclosure; requires the victim to be the last FP owner before the attacker image's first user entry (deterministic via the victim's own fork+exec; otherwise a race). Severity: Medium (kernleak bucket), same class as DF-0938/DF-1056 (uninit/leftover FP state exposure) and the pc64 wipe exists precisely to prevent it.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.