MSI/MSI-X allocation count validated only by INVARIANTS-only KASSERTs β OOB write past caller's array, irqmap row-crossing writes and IPI IDT-gate smash on production kernels
Summary
msi_alloc/msi_release validate count/cpuid only with KASSERTs that compile to NOTHING on production kernels (stock GENERIC has no INVARIANTS): count>32 returns rc=0 and writes past the caller's int irqs[32] (on-stack in the reference pci.c caller = stack smash); count>=128 (PCI MMC allows 128) crosses a full ioapic_irqmaps row and msi_setup overwrites the IPI IDT gates + past the 256-entry IDT -> next IPI jumps to garbage. Reproduced on a guest-built production-behavior kernel: canary OOB write proven (32/96 words) and count=128 died 'Stopped at 0x6970695f' (RIP = '_ipi' string bytes loaded via the OOB msi_intr read into an IPI gate). Fix validated (EINVAL, zero canary writes, legal count=1 unaffected). Privileged-driver trigger. Runtime-validation fix.
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-5351 Β· 13 files| File | Type | Description | Size | |
|---|---|---|---|---|
| msiabi_probe.c | β | 3.1 KB | view raw | |
| msiabi_row128.c | β | 2.4 KB | view raw | |
| hello_kld.c | β | 516 B | view raw | |
| probe_nocrash.c | β | 1.1 KB | view raw | |
| build.sh | β | 119 B | view raw | |
| run.sh | β | 164 B | view raw | |
| run.log | β | 1.3 KB | view raw | |
| panic_row128.txt | β | 1.3 KB | view raw | |
| run_fixed.log | β | 532 B | view raw | |
| env.txt | β | 988 B | view raw | |
| VERDICT.md | β | 6.3 KB | β raw | |
| fix.diff | β | 1.1 KB | view raw | |
| verdict.json | β | 4.4 KB | view raw |
DF-5351 VERDICT β reproduced (panic + proven OOB write primitive), driver-facing MSI ABI, Low severity
Bottom line
REPRODUCED on the guest, on a kernel built exactly like production
(X86_64_GENERIC minus options INVARIANTS; stock GENERIC ships without
INVARIANTS too): ioapic_abi_msi_alloc_intern() performs no runtime
validation of the driver-supplied count. A KLD (the driver-facing ABI,
identical in reach to PCIB_ALLOC_MSI β nexus_alloc_msi β
MachIntrABI.msi_alloc) passing count=64 gets a clean success return and
32 kernel words written past the caller's 32-int contract array
(ioapic_abi.c:1295 intrs[j] = intr; the reference caller's array is the
on-stack int irqs[32] at sys/bus/pci/pci.c:1952/2072). count=128 (the
PCI-spec maximum a device may advertise via MMC=7) additionally makes the
loop write/read ioapic_irqmaps[cpuid][192..255] β one full row past the
192-entry per-CPU row β and call msi_setup(intr=192..255), whose
msi_intr[192..255] OOB read (msi.c:433) feeds garbage "handlers" into
setidt() over the IPI gate range: the machine died with RIP
0x6970695f (ASCII "_ipi", a rodata string fragment loaded as an
instruction pointer). Fix validated: with runtime count validation both
counts return EINVAL (22), no canary writes, no crash, and legal
allocations (count=1 msix) keep working.
How it was reproduced (baseline, unfixed production-behaviour kernel)
RUN 1 β safe canary demo (full output in run.log):
msiabi_probe: calling msi_alloc(count=64, cpu0) (contract max is 32) msiabi_probe: msi_alloc rc=0 msiabi_probe: zone[0..7] = 128 129 130 131 132 133 134 135 msiabi_probe: 32/96 canary words past zone[31] overwritten (32 expected ...) msiabi_probe: zone[32]=160 <-- OOB write past 32-int contract msiabi_probe: msi_release(64) done <- guest healthy afterwards
zone is a 128-int buffer of which only zone[0..31] are inside the ABI
contract; the kernel wrote zone[32..63]. In the real caller this is the
kernel stack. (KASSERT at ioapic_abi.c:1256 would have rejected count=64 β
on an INVARIANTS kernel only.)
RUN 2 β count=128 crasher (full output in panic_row128.txt):
msiabi_row128: msi_alloc(count=128, cpu5) ... msiabi_row128: rc=0 msiabi_row128: zone[0..3] = 12Stopped at 0x6970695f: kernel: type 12 trap, code=0 db>
rc=0 proves the scan accepted a 128-wide "block" spanning ioapic_irqmaps[5][128..191] plus row [6][0..63] (the next CPU's row β the alloc ran on cpu ncpus-1 whose successor row is all-zero/UNUSED). The kprintf of zone[0..3] was cut mid-number because the msi_setup() calls inside the alloc had already installed OOB-read garbage into the IPI IDT gates and an IPI fired before the print finished. RIP 0x6970695f = bytes 5f 69 70 69 = "_ipi" β a substring of kernel rodata (e.g. "send_ipiq"), i.e. a value read from beyond the 192-entry msi_intr[] table used as a function pointer. Fully consistent with the static analysis of ioapic_abi.c:1269-1305 + msi.c:431-435.
Primitive characterization (why panic, not uid0)
The write primitive is: for each j in [0,count): intrs[j] = i + j and ioapic_irqmaps[cpuid][i+j].{im_type,im_msi_base} = {4|5, i} β linear, values constrained to small ints and intr numbers. The strongest consequence (countβ₯128) is the IDT/IPI-gate smash with semi-controlled (kernel-data-derived) target RIPs and the row-crossing irqmap corruption. Triggering requires a driver calling the MSI ABI with a bad count β i.e. already-privileged code (kldload) or a malicious PCIe device paired with a driver that forwards the device-reported message count. No unprivileged entry point exists (verified: no sysctl/ioctl/dev-node surface in this file; the only untrusted-input path into the file is the boot-time hw.ioapic.msi_start tunable, already covered by DF-1037). Hence Low severity / dos impact β but the primitive is real memory corruption past an attacker-influenced array bound, not a mere assert.
Fix validation (fix.diff, applied to guest /usr/src, incremental rebuild)
Baseline reproduced (runs 1+2 above) β apply fix.diff (runtime
count/powerof2/cpuid checks returning EINVAL/early-return in
msi_alloc_intern/msi_release_intern) β rebuild X86_64_NOINV (#1 08:38:15)
β reboot β identical PoCs (run_fixed.log):
msiabi_probe: msi_alloc rc=22 (EINVAL), "alloc failed, no canary test",
SAFE PHASE COMPLETE, msix count=1 round trip still OK
msiabi_row128: rc=22, SURVIVED β no crash, guest healthy
Bad behavior gone; no regression on legal allocations.
What I had to fix vs. a naive PoC (poc_changes)
- DFly master's
DECLARE_MODULE(name, data, sub, order)takes a moduledata_t, not a modevent handler (sys/sys/module.h:120-123) β the natural FreeBSD-style invocation crashes in module_register (took a DDB backtrace over a serial socket to find; guest plumbing, not the finding). - Modules must be built with the in-tree recipe (
SYSDIR?=${.CURDIR}/../.. .include "${SYSDIR}/conf/kmod.mk"+SRCS+= device_if.h bus_if.h); themake -f kmod.mk KMOD=β¦override form produces unloadable .ko on this guest.- The count=128 crasher must run on cpu
ncpus-1(its successor map row is the un-fixup'd zero row), hencencpus - 1in the module. - The legacy-range scan: on this QEMU guest irqs 19-23 are legacy level/low maps on some CPU; the DF-5352 trigger therefore loops msix allocations until irq >= 24 (relevant to the sibling finding).
Kernel references
- sys/platform/pc64/apic/ioapic_abi.c:1256-1257 (KASSERT-only count check)
- sys/platform/pc64/apic/ioapic_abi.c:1269-1305 (scan loop, intrs[j]=intr, msi_setup)
- sys/platform/pc64/apic/ioapic_abi.c:1321-1330 (release count check + intrs[i] read)
- sys/sys/systm.h:94-122 (KASSERT/KKASSERT compiled out without INVARIANTS)
- sys/bus/pci/pci.c:1952,2072 (caller's 32-int stack arrays)
- sys/platform/pc64/x86_64/msi.c:431-435 (msi_setup: msi_intr[intr] + setidt)
- sys/config/GENERIC (no INVARIANTS) vs sys/config/X86_64_GENERIC:56
Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.
Fix verification
fixedfix.diff applied to guest /usr/src/sys/platform/pc64/apic/ioapic_abi.c, kernel rebuilt (X86_64_NOINV #1 08:38:15) and rebooted; identical PoCs re-run: msi_alloc(count=64) and (count=128) both return EINVAL(22), zero canary words overwritten, no crash, and the legal count=1 MSI-X round trip still succeeds (run_fixed.log). Bad behavior gone, no regression.
findings/poc/DF-5351/run_fixed.log; fix.diff
Confirmed kernel references
- sys/platform/pc64/apic/ioapic_abi.c:1256
- sys/platform/pc64/apic/ioapic_abi.c:1269
- sys/platform/pc64/apic/ioapic_abi.c:1295
- sys/platform/pc64/apic/ioapic_abi.c:1296
- sys/platform/pc64/apic/ioapic_abi.c:1321
- sys/platform/pc64/apic/ioapic_abi.c:1330
- sys/sys/systm.h:94
- sys/bus/pci/pci.c:1952
- sys/platform/pc64/x86_64/msi.c:433
Detail
Exploit chain
Driver (KLD, i.e. already-privileged code, or malicious PCIe device whose MMC field advertises 128 messages plus a driver forwarding pci_msi_count()) -> MachIntrABI.msi_alloc(intrs, count>32, cpuid) -> intrs[j]=intr OOB write past caller array (kernel stack in pci.c) -> for count>=128: ioapic_irqmaps row-crossing writes + msi_setup(192..255) -> msi_intr[] OOB read + setidt() IPI-gate smash -> next IPI jumps to garbage RIP (observed 0x6970695f) -> crash/panic. No unprivileged entry point exists (no sysctl/ioctl/dev surface; boot-tunable path is DF-1037), so impact is dos-class, not uid0.
Evidence (decisive lines)
run.log (canary: 'zone[32]=160 <-- OOB write past 32-int contract', '32/96 canary words past zone[31] overwritten'); panic_row128.txt ('msi_alloc(count=128, cpu5) ... rc=0' then 'Stopped at 0x6970695f: kernel: type 12 trap'); run_fixed.log ('msi_alloc rc=22' both counts, no crash, msix count=1 round trip OK); VERDICT.md full narrative
PoC changes
Fixed DECLARE_MODULE to the moduledata_t form (DFly master signature); in-tree kmod build recipe (SYSDIR relative + device_if.h/bus_if.h as generated SRCS); row128 crasher targets cpu ncpus-1 so the crossed row is the all-zero successor row; count=64 canary buffer is 128 ints with only [0..31] inside the ABI contract
Verified recommended fix
Validate count/cpuid at run time in ioapic_abi_msi_alloc_intern (return EINVAL for count<=0, >32 or non-power-of-2) and in ioapic_abi_msi_release_intern (early return), replacing INVARIANTS-only KASSERTs as the sole guard
Verdict
Reproduced on a production-behaviour kernel (X86_64_GENERIC minus INVARIANTS; stock GENERIC ships without INVARIANTS): ioapic_abi_msi_alloc_intern() performs no runtime validation of the driver-supplied count, so msi_alloc(count=64) returns success and writes 32 words past the caller's 32-int contract array (intrs[j]=intr, ioapic_abi.c:1295; reference caller uses on-stack int irqs[32], pci.c:1952), and msi_alloc(count=128) additionally reads/writes ioapic_irqmaps[cpuid][192..255] one row past the 192-entry per-CPU row and feeds OOB-read msi_intr[192..255] values into setidt() over the IPI gates - the guest died with RIP=0x6970695f ('_ipi' string bytes) proving the IDT smash. Fix (runtime count validation returning EINVAL) makes both counts fail cleanly with no corruption and no crash while legal allocations keep working.
No comments yet.