DF-3717 / verdict.json
{ "finding_id": "DF-3717", "status": "inconclusive", "reproduced": 0, "impact": "none", "confidence": "certain", "verdict": "Mechanism PROVEN on the guest kernel by deterministic replica of if_mxge.cs exact callout/serializer teardown protocol; mxge-specific runtime hardware-gated (inconclusive per family convention, cf. DF-3715). mxge_tick (sys/dev/netif/mxge/if_mxge.c:3712-3750) is a self-re-arming MP callout (armed :4445, re-armed :3747 inside the held lock) whose FIRST act is a BLOCKING lwkt_serialize_enter of sc->main_serialize (:3720); mxge_detach runs ifnet_serialize_all (:4463, main_serialize is serializes[0] :4133) and then SYNCHRONOUS callout_stop(&sc->co_hdl) (:4468) while holding that serializer. DragonFly callout_stop sleeps on costp (sys/kern/kern_timeout.c:920 via :1091 sync=1) until the INPROG callback completes; the callback sleeps in lwkt_serialize_enter (sys/kern/lwkt_serialize.c:84-90, wchan slize) until the stopper releases the serializer it holds. AB-BA wedge. Kernel-module replica on the guest: shipped shape -> callout_stop NEVER returns, ps shows the stopper D1 costp while softclock N sits B2 slize (both halves of the deadlock captured live; that CPUs callout delivery dead); fix shape (callout_stop_async under the serializer + the drivers existing callout_terminate :4474 after release) returns cleanly with pending=0 in three independent runs - the recommended fix is validated on-guest. Trigger on real hardware: any mxge_detach (kldunload, devd/hotplug removal) racing the hz/2 tick.", "exploit_chain": "", "evidence": "findings/poc/DF-3717/run.log (decisive run: serial-console [stop] ... IF THIS IS THE LAST mxge_replica LINE then silence; second-session ps -1 B2 slize (softclock 2) + 859 D1 costp sysctl kern.mxgereplica.stop=1; fix-shape returned; pending=0 ran=9 FIX_RC=0); build.log; VERDICT.md (incl. v1/v2 harness-incident honesty record); fix.diff (git-apply clean; callout_stop -> callout_stop_async at :4468)", "kernel_refs": [ "sys/dev/netif/mxge/if_mxge.c:3720", "sys/dev/netif/mxge/if_mxge.c:3747", "sys/dev/netif/mxge/if_mxge.c:4445", "sys/dev/netif/mxge/if_mxge.c:4463", "sys/dev/netif/mxge/if_mxge.c:4468", "sys/dev/netif/mxge/if_mxge.c:4474", "sys/dev/netif/mxge/if_mxge.c:4133", "sys/kern/kern_timeout.c:920", "sys/kern/kern_timeout.c:1091", "sys/kern/lwkt_serialize.c:84" ], "poc_changes": "No seed PoC. Authored mxge_callout_replica.ko (out-of-tree module replicating the mxge_tick/mxge_detach protocol: blocking-enter-first callback, self re-arm inside the lock, sync stop under the lock) because the guest has no Myricom hardware; two harness iterations were needed (v1 wrongly re-armed a terminated callout -> unrelated DID_INIT panic; v2s tsleep-timeout poll froze on the parked softclock) - both documented; final replica uses two independent callouts and a bounded busy-wait observer.", "attempts": 3, "guest_uname": "DragonFly dfbsd 6.5-DEVELOPMENT DragonFly 6.5-DEVELOPMENT #0: Thu Jul 2 06:02:54 UTC 2026 root@dfbsd:/usr/obj/usr/src/sys/X86_64_GENERIC x86_64", "runtime_sec": 900, "guest_dirty": 0, "build_cmd": "cd /root/kmod && make", "run_cmd": "vm.sh run_root kldload /root/kmod/mxge_callout_replica.ko; sysctl kern.mxgereplica.armfix=1; sleep 2; sysctl kern.mxgereplica.fix=1; sysctl kern.mxgereplica.arm=1; sleep 2; (sysctl kern.mxgereplica.stop=1 >/dev/null 2>&1 &)", "code_hash": "39237ad017a618cce92fcece7969f16d5cdaaaef", "notes": "Family-convention classification (no mxge hardware on guest): inconclusive/none/certain, mechanism guest-proven as for DF-3715. The wedge kills the detaching thread AND one CPUs softclock thread (all callouts/tsleep-timeouts on that CPU stop) - permanent system degradation. Guest reset with-src after capture; /root/kmod removed.", "recommended_fix": "Replace callout_stop(&sc->co_hdl) at if_mxge.c:4468 with callout_stop_async(); the existing callout_terminate() at :4474 (already outside ifnet_serialize_all) then drains synchronously and safely (validated on-guest by the fix-shape replica run).", "fix_status": "not_testable", "fix_baseline_reproduced": 0, "fix_patched_reproduced": 0, "fix_verdict": "fix.diff authored against the read-only sys/ tree (git apply --check clean); the fix SHAPE (stop_async under serializer + terminate after release) was validated on the guest via the replica fix-test (returned, pending=0, no deadlock), but the mxge kernel itself cannot be diff-built without hardware.", "fix_evidence": "findings/poc/DF-3717/fix.diff; run.log fix-shape section" } |