vmxnet3_rxq_eof() RX completion walk has no work bound: interrupt-thread livelock holding the RXQ lock under a remote line-rate flood (DF-3761/3778 family, vmx(4) instance)
Summary
The for(;;) walk exits only when the guest catches up with the host's completion deposits (gen mismatch) or loses IFF_RUNNING; each iteration costs an mbuf+cluster allocation, a dmamap load and a full if_input - strictly more than the host spends per completion. It runs on the interrupt path holding vxrxq_lock, and the RX interrupt is re-enabled only after the walk returns. A remote line-rate small-packet flood via the host backend keeps the ring perpetually non-empty -> interrupt-thread livelock / whole-interface wedge. vmxnet3 has no process_limit knob at all (unlike alc/msk). Not exercisable on the KVM audit guest (the claim is about host production vs guest drain rates) - honest inconclusive; family mechanism already proven for DF-3761/3778. Fix: budget + save/restore chain state + re-arm (advisory diff in pack).
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-3782 Β· 3 files| File | Type | Description | Size | |
|---|---|---|---|---|
| README.md | β | 2.1 KB | β raw | |
| VERDICT.md | β | 2.7 KB | β raw | |
| fix.diff | β | 1.3 KB | view raw |
DF-3782 β vmxnet3_rxq_eof() RX completion walk has no work bound (DF-3761/3778 family)
What
vmxnet3_rxq_eof() (sys/dev/virtual/vmware/vmxnet3/if_vmx.c:2201-2328) runs
for (;;) {
rxcd = &rxc->vxcr_u.rxcd[rxc->vxcr_next];
if (rxcd->gen != rxc->vxcr_gen)
break; /* the ONLY exit beside IFF_RUNNING loss */
...vmxnet3_newbuf() (mbuf alloc + dmamap load) + if_input()...
}
with no iteration budget. It is called from the interrupt path
(vmxnet3_legacy_intr if_vmx.c:2352-2354; vmxnet3_rxq_intr if_vmx.c:2397-2399
on MSI-X platforms) holding the RX queue lock, and the RX interrupt is only
re-enabled after the walk returns (if_vmx.c:2361, 2401). While the host
(nominally fed by the physical network) keeps the completion ring populated
faster than the guest drains it β each iteration costs an mbuf cluster
allocation, a DMA map load and a full if_input() β the walk never exits:
interrupt-thread livelock holding vxrxq_lock, exactly the DF-3761 (alc) /
DF-3778 (jme) family. A remote line-rate small-packet flood through the
host's vmxnet3 backend is the trigger; the whole vCPU services RX forever
and nothing else on that interrupt thread progresses.
vmxnet3 has no process_limit-style knob at all (unlike alc/msk), so there
is no operator mitigation.
Status on this audit guest
The KVM/QEMU guest has no vmxnet3 device, so the livelock cannot be exercised end-to-end here. The mechanism is source-certain (the loop, its sole exit conditions, the lock, and the interrupt re-enable ordering are all plain in if_vmx.c), and it is the same mechanism already proven on this guest for the alc(4)/jme(4) instances of the family (DF-3761/DF-3778). Verdict: inconclusive (device-triggerable, not testable on this guest), confidence in the analysis: high.
Suggested fix (fix.diff, advisory)
Bound the walk with a budget; when the budget is exhausted, save the chain state (as the gen-mismatch exit already does, if_vmx.c:2204-2205) and re-arm processing via a short callout instead of spinning. Alternatively defer to a netisr as other DFly drivers do.
DF-3782 VERDICT β INCONCLUSIVE on this guest β impact (analysis): dos / RX livelock
Analysis (source-certain)
- The walk
for (;;)at if_vmx.c:2201 has exactly two exits: a generation mismatch on the next completion descriptor (i.e. the guest caught up with everything the host deposited) or losing IFF_RUNNING mid-walk (if_vmx.c:2311-2312, only checked at packet boundaries). - Per iteration the guest pays
vmxnet3_newbuf()(m_getjcl + bus_dmamap load) and a full(*ifp->if_input)()with the RXQ lock dropped/re-taken (if_vmx.c:2257, 2169-2171) β strictly more work than the host spends to deposit a completion (a DMA write + doorbell). - Interrupt re-enable happens only after the walk returns:
vmxnet3_legacy_intr()if_vmx.c:2361 /vmxnet3_rxq_intr()if_vmx.c:2401. So while the ring stays non-empty, the interrupt thread never yields the CPU to anything at lower priority, andvxrxq_lockis held across the whole walk except inside if_input. - Consequence under a remote line-rate small-packet flood (delivered by the host backend): interrupt-thread livelock / softnet livelock; queue stops, watchdog cannot run (it needs the core lock path via vmxnet3_tick, which still functions, but the rxq never drains). Identical shape to DF-3761 (alc) and DF-3778 (jme), both already established on this project.
Why not exercised here
Requires a vmxnet3 paravirtual NIC served by a host that can sustain
completion production; the audit guest is KVM/QEMU with virtio only. A
replica module could demonstrate an unbounded loop trivially, but that
would prove nothing beyond the visible for(;;) β the security claim is
about host/guest production-vs-drain rates, which needs the real device.
Honest status: inconclusive, analysis confidence high, family precedent
(DF-3761/DF-3778) reproduced on this very guest.
Fix
Budget bound + deferred continuation (fix.diff, advisory). Note the subtlety: simply breaking early can strand already-deposited completions if the device does not re-raise the RX interrupt for them, so the fix must re-arm (short callout / netisr deferral), not just cap the loop.
Kernel refs
sys/dev/virtual/vmware/vmxnet3/if_vmx.c:2201-2328 (walk), 2352-2361 (legacy intr: lock, eof, then re-enable), 2397-2401 (rxq intr), 2169-2171 (lock drop around if_input), 2257 (newbuf cost per iteration).
Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.
Fix verification
not_testableNot testable on this guest (no vmxnet3 device; no baseline repro possible either).
VERDICT.md analysis; family mechanism (DF-3761/3778) proven on this guest previously.
Confirmed kernel references
Detail
Evidence (decisive lines)
VERDICT.md cites the loop, its exits, the lock and the intr re-enable ordering (if_vmx.c:2201-2328, 2361, 2401); family precedent DF-3761/DF-3778 reproduced on this guest in earlier runs.
PoC changes
No PoC run: device-triggerable finding, no vmxnet3 NIC on the KVM guest; a replica loop would not honestly model host-vs-guest production rates.
Verified recommended fix
Bound the RX completion walk with a budget and re-arm via a short callout/netisr when the budget is exhausted (see fix.diff)
Verdict
vmxnet3_rxq_eof() walks the RX completion ring with no work bound (for(;;) whose only exits are catching up with the host's deposits or losing IFF_RUNNING) while holding the RXQ lock on the interrupt path, with the interrupt re-enabled only after the walk returns (if_vmx.c:2201-2328, 2352-2361, 2397-2401). Source-certain analysis; identical mechanism to DF-3761 (alc) / DF-3778 (jme), both reproduced on this project's guest. Not exercisable on the KVM audit guest (no vmxnet3 device; the claim is about host production vs guest drain rates, which a replica cannot honestly simulate). Remote line-rate flood -> interrupt-thread livelock -> interface/CPU DoS.
No comments yet.