β¬’ DragonFlyBSD Kernel Audit
← triage Β· dashboard
DF-5586

exc_segfault() tail splz() is unguarded: deferred IPIs/interrupt work dispatched inside the interrupted context's critical section and inside outer interrupt handlers β€” protocol divergence from pc64 doreti's explicit TD_CRITCOUNT gate

Summary

splz() runs unconditionally at whatever critcount/nesting the fault interrupted, while pc64 doreti refuses to unpend interrupts during a critical section (ipl.s:132-146) and every other splz call site goes through guarded wrappers (lwkt_maybe_splz: critcount==0 AND nesting==0 AND nest_count<2). A deferred IPI (pmap shootdown, lwkt_send_msg) can interleave mid-critical-section on guest-kernel data whose only lock is the critical section - torn reads/lost updates inside the VM; DDB single-stepping makes it deterministic. Model reproducer: INVARIANT_BROKEN with callback_critcount=1 + torn state 3/3 (pc64-gate control 3/3 OK); in-vkernel 2000-fault storm validates the rig. Trap-side sibling is DF-5338. critcount/nesting/nest_count-gate fix proposed; leads TF_R10/R11 (exception.c side KILLED - the defect is pc64 exception.S Xfast_syscall:402-427, cross-file) and SA_NODEFER-storm (KILLED with runtime evidence) both resolved.

Discussion (0)

No comments yet.

PoC verification

Evidence pack

findings/poc/DF-5586 Β· 14 files
FileTypeDescriptionSize
splzcrit.c β€” 7.8 KB view raw
storm.c β€” 5.1 KB view raw
build.sh β€” 66 B view raw
run.sh β€” 163 B view raw
build_image.sh β€” 1.2 KB view raw
run_vk.sh β€” 1.6 KB view raw
build_and_model.log β€” 868 B view raw
run_vk.log β€” 940 B view raw
fix.diff β€” 818 B view raw
VERDICT.md β€” 5.5 KB ↓ raw
README.md β€” 1.8 KB ↓ raw
verdict.json β€” 5.0 KB view raw
manifest.json β€” 1.1 KB view raw
env.txt β€” 526 B view raw

DF-5586 β€” PoC pack

Build (on the DragonFly guest)

sh build.sh          # cc -static -O2 -o splzcrit splzcrit.c
                      # cc -static -O0 -o storm storm.c

Run

1. Dispatch-violation model (no root needed)

sh run.sh            # ./splzcrit vkernel ; ./splzcrit pc64

Expected:

== vkernel mode ==
mode=vkernel victim.lo=1 victim.hi=2 callback_critcount=1 callback_torn=1
INVARIANT_BROKEN (IPI callback ran inside the critical section ...)
RC_VKERNEL=1
== pc64 mode ==
mode=pc64 victim.lo=1 victim.hi=1 callback_critcount=0 callback_torn=0
INVARIANT_OK
RC_PC64=0

2. In-vkernel fault storm (root; needs the VKERNEL64 build + image)

sh build_image.sh    # creates /vkroot.img with /root/storm (DF-5474
                     # link workaround must be in
                     # sys/platform/vkernel64/conf/Makefile; see
                     # findings/poc/DF-5474)
sh run_vk.sh         # boots kernel.debug -m 512m -r /vkroot.img -n 2,
                     # runs "/root/storm 500 1" on the guest console

Expected (console): STORM_DONE iters=500 intdiv=500 ill=500 prot=500 map=500 + VKERNEL_SURVIVED_IF_YOU_SEE_THIS + STORM_RC=0, vkernel ALIVE at the end. This is the runtime kill of the "SA_NODEFER fault storms" lead: guest-user storms never reach exc_segfault.

Files

  • splzcrit.c β€” model reproducer (vkernel vs pc64 modes)
  • storm.c β€” guest-userland fault-storm binary for the vkernel
  • build.sh / run.sh β€” model build/run
  • build_image.sh / run_vk.sh β€” vkernel image + boot/storm driver
  • build_and_model.log β€” full build + 3Γ— model runs (untrimmed)
  • run_vk.log β€” vkernel storm console (untrimmed tail)
  • fix.diff β€” guard the tail splz() (doreti-equivalent)
  • verdict.json, manifest.json, VERDICT.md
VERDICT.md
↓ download raw

DF-5586 β€” vkernel exc_segfault() tail splz() is unguarded: deferred IPIs / interrupt work dispatched inside the interrupted context's critical section (protocol divergence from pc64 doreti's explicit TD_CRITCOUNT gate)

What

sys/platform/vkernel64/x86_64/exception.c:269-271 β€” the vkernel's kernel-mode fault return path:

    save = errno;
    kern_trap((struct trapframe *)&ctx->uc_mcontext.mc_rdi);
    splz();                                   /* ← unconditional */
    errno = save;

The real kernel runs pending interrupts on trap return through doreti, which refuses to dispatch anything while the interrupted thread holds a critical section β€” sys/platform/pc64/x86_64/ipl.s:132-146:

 * If we are in a critical section we cannot run any pending ints.
 ...
    testl   $-1,TD_CRITCOUNT(%rbx)  /* can't unpend if in critical sec */
    jne 5f          /* β†’ straight to doreti_iret */

and the generic kernel only ever invokes splz() through guarded wrappers (lwkt_maybe_splz(), sys/kern/lwkt_thread.c:1091-1101: critcount==0 ∧ gd_intr_nesting_level==0 ∧ td_nest_count<2; splz_check(), :1068-1078). The vkernel's own splz() (sys/platform/vkernel64/platform/machintr.c:122-158) has no guard of any kind, and exc_segfault is the only caller that invokes it at the interrupted context's arbitrary crit/nesting state (go_user's loop, cpu_idle, and crit_exit all dispatch from base level).

Consequence: when a fault is taken while the interrupted vkernel-kernel context (a) holds a critical section with a deferred IPI pending (ipisig deferred it via need_ipiq(), exception.c:92), or (b) is nested inside another interrupt's signal (ipisig/kqueuesig/timersig all run with gd_intr_nesting_level raised and empty sa_mask), the tail splz() executes IPI callbacks (lwkt_process_ipiq), soft/hard interrupt threads (sched_ithd_soft / sched_ithd_hard_virtual), vktimer_intr, and kqueue_intr inside the critical section / inside the outer interrupt handler β€” precisely the interleaving DragonFly critical sections exist to prevent. No TD_NEST_COUNT accounting is done anywhere on this path (pc64 bumps it around exactly these dispatches, ipl.s:280-286).

Reachable fault sources at kernel-mode crit/nesting (no vkernel bug required for the class, though frequency is timing-dependent): - guest-KVA demand VPTE installs on the MAP_VPAGETABLE window (init.c:608-624) β€” the vkernel's normal kernel page-fault mechanism β€” occurring inside any of the (ubiquitous) vkernel-kernel critical sections while a cross-CPU IPI (guest-userland TLB-shootdown churn) is deferred; - single-step / breakpoint SIGTRAPs during vkernel DDB sessions (kern_trap T_TRCTRAP/T_BPTFLT, trap.c:716-725; the spurious-SIGTRAP comment at trap.c:732-742 acknowledges these arrive in kernel mode); - any recoverable fault inside IPI/kqueue/timer callback execution.

Impact class: lost critical-section atomicity on guest-kernel data (torn reads/lost updates on structures whose only lock is the critical section) β†’ guest-kernel memory corruption inside the VM (panic or worse). The sibling fall-through defect of the same file family is DF-5338 (trap.c); this finding is the dispatch-side divergence in exception.c.

Reproducer

splzcrit.c β€” faithful element-for-element model of the dispatch structure on the DragonFly host: SA_NODEFER/empty-mask fault handler with a recoverable fault (mmap-in-handler = trap_pfault/vm_fault SUCCESS) and tail splz (vkernel mode) vs the doreti TD_CRITCOUNT gate (pc64 mode); deferred-SIGUSR1 "IPI" callback that reads/writes crit-protected state mid-section.

  • vkernel mode: INVARIANT_BROKEN, callback_critcount=1, callback_torn=1 β€” the IPI callback ran inside the critical section and observed torn state (3/3 runs).
  • pc64 mode: INVARIANT_OK, callback_critcount=0, no torn state (3/3 runs).

storm.c + build_image.sh + run_vk.sh β€” in-vkernel fault storm on a real booted VKERNEL64 (-n 2): 500 iterations Γ— 4 guest-user fault classes (intdiv/SIGFPE, ud2/SIGILL, RO-write/SIGSEGV, PROT_NONE/SIGSEGV) plus a shared-memory mprotect-churn sibling driving cross-CPU TLB-shootdown IPIs for the whole storm. Result: STORM_DONE ... STORM_RC=0, vkernel ALIVE β€” normal guest-user fault storms never reach exc_segfault (they return via vmspace_ctl β†’ go_user β†’ user_trap, trap.c:1407-1439), which is the runtime kill of the "SA_NODEFER under fault storms" lead (with DF-5338 owning the one deterministic recursion instance).

Fix

Guard the tail splz() exactly like doreti/lwkt_maybe_splz (fix.diff, applied and rebuilt on the guest; patched vkernel passes the identical storm; pc64-mode model run demonstrates the guarded semantics).

Verdict summary

  • status: reproduced (dispatch violation + torn-state corruption: deterministic 3/3 in the faithful model on the DragonFly host; code divergence proven line-by-line against the real kernel's own gate)
  • demonstrated impact ceiling in the live vkernel: none observed (needs a kernel-side recoverable fault to coincide with crit+deferred IPI; DDB single-step makes it deterministic under debugging) β€” severity Low, bucket memcorrupt, confidence likely.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.

Fix verification

fixed
baseline reproduced→ patch + rebuild →patched reproduced

fix.diff applied to the guest /usr/src (combined with DF-5587's guard), VKERNEL64 rebuilt cleanly, identical in-vkernel fault storm passes (STORM_DONE, STORM_RC=0, ALIVE) β€” no regression; the guard's dispatch semantics are exactly the pc64 mode of the model reproducer (callback_critcount=0, INVARIANT_OK, deferred to crit_exit).

findings/poc/DF-5586/run_vk.log (baseline storm) + patched-vkernel run (STORM_RC=0 ALIVE) + build_and_model.log pc64-mode section; fix.diff
↓ fix.diffper-fix-DF-5586

Confirmed kernel references

Detail

Evidence (decisive lines)

findings/poc/DF-5586/build_and_model.log (3x model runs: vkernel mode INVARIANT_BROKEN callback_critcount=1 callback_torn=1 vs pc64 mode INVARIANT_OK callback_critcount=0); run_vk.log (in-vkernel storm STORM_DONE iters=500 intdiv=500 ill=500 prot=500 map=500, STORM_RC=0, vkernel ALIVE); fix validation: patched VKERNEL64 (fix.diff applied, rebuilt) passes the identical storm (STORM_RC=0, ALIVE) and pc64-mode model run demonstrates the guarded semantics; code divergence: exception.c:269-271 vs ipl.s:132-146, lwkt_thread.c:1068-1101, machintr.c:122-158

PoC changes

Authored from scratch (no seed): faithful extraction of exception.c's dispatch structure (SA_NODEFER/empty-mask install, deferred-SIGUSR1 ipisig model, recoverable SIGSEGV via mmap-in-handler, splz tail) with a pc64-doreti-gate comparison mode; storm.c needed -O0 (gcc -O2 loop rotation misplaces &&label resume targets) and small iteration counts (~0.1ms per fault+sigreturn cycle); storm argv parsing order fixed; vkernel rig rebuilt from the DF-5556 recipe (DF-5474 link workaround, /var/vkernel dir required).

Verified recommended fix

Gate the tail splz() like doreti/lwkt_maybe_splz: only when curthread->td_critcount==0 && mycpu->gd_intr_nesting_level==0 && curthread->td_nest_count<2

Verdict

exc_segfault's tail splz() (exception.c:270) dispatches deferred IPIs/interrupt work at the interrupted context's critcount/nesting state, violating the dispatch protocol the real kernel enforces (pc64 doreti's explicit TD_CRITCOUNT gate at ipl.s:143-146 and lwkt_maybe_splz's critcount==0/nesting==0/nest_count<2 guards at lwkt_thread.c:1091-1101); the vkernel splz (machintr.c:122-158) has no guards. Reproduced deterministically 3/3 in a faithful signal-structure model on the DragonFly guest: with the vkernel's unguarded tail, the deferred IPI callback executes inside the interrupted critical section and observes/creates torn state (callback_critcount=1, callback_torn=1, INVARIANT_BROKEN lo=1/hi=2); with the pc64 doreti gate, dispatch is deferred to the crit_exit transition (callback_critcount=0, INVARIANT_OK). In the live vkernel the trigger needs a kernel-mode fault (guest-KVA demand VPTE install, or DDB single-step SIGTRAPs) to coincide with a held critical section and a deferred IPI β€” code-certain, frequency timing-dependent, so demonstrated in-vkernel impact is none; the in-vkernel fault storm (2000 guest-user faults + cross-CPU IPI churn on a 2-vCPU VKERNEL64) confirms normal guest-user fault storms never reach exc_segfault (they return via vmspace_ctl -> go_user -> user_trap), killing the pass-1 'SA_NODEFER under fault storms' concern, whose only deterministic recursion instance is already filed as DF-5338.