DF-3906 / verdict.json
{ "finding_id": "DF-3906", "status": "inconclusive", "reproduced": 0, "impact": "none", "confidence": "certain", "verdict": "Mechanism PROVEN on the guest kernel on the FIRST attempt by a deterministic replica of if_xl.cs exact callout/serializer protocol; xl-specific runtime hardware-gated (no 3Com 3c90x NIC on the audit guest) -> inconclusive per family convention. xl_stats_update (sys/dev/netif/xl/if_xl.c:2372) is the callback of a PLAIN (non-MPSAFE) callout (callout_init at :1320 -> callback executes under the Giant MP lock, kern_timeout.c:583) and does a BLOCKING lwkt_serialize_enter(ifp->if_serializer) at :2377, then self re-arms at :2429 inside the serialized body (armed from xl_init :2918, 1 Hz); xl_stop() tears the callout down with the synchronous callout_stop() at :3169 while holding that same serializer (ASSERT_SERIALIZED :3142; callers: xl_detach :1573, xl_ioctl :3065/:3062, xl_init :2712 via xl_ifmedia_upd :2955 and watchdog :3125, xl_shutdown :3212, xl_suspend :3222); DragonFly callout_stop synchronously waits for an INPROG callback (kern_timeout.c:1091 -> :859-932, ssleep costp at :920). A stats tick dequeued and parked on the serializer therefore deadlocks against the stopper (AB-BA). Kernel-module replica on the guest: with the xl-shaped handler parked, callout_stop NEVER returned - stopper observed D2/wchan costp (pid 1104), softclock 4 observed B4/wchan slize holding the Giant MP lock (non-MPSAFE callout, aggravation shared with dc DF-3850 and sk DF-3903), serial console shows fired=2 ran=1 then the terminal marker line, and the wedge survived a clean shutdown attempt (vm.sh down had to SIGKILL QEMU). 10th member of the DF-3715/3738/3760/3776/3806/3850/3903 family, now live-proven 8 times across 8 drivers with zero false triggers. Fix-shape validation on a fresh guest with the exact fix.diff protocol (lwkt_serialize_try + 1-tick retry): callout_stop returned with pending=0 under both interleave shapes, including cross-CPU contention (fired=312 ran=5 retry=307, 0 parks) - deadlock eliminated.", "exploit_chain": "", "evidence": "findings/poc/DF-3906/VERDICT.md; run.log (D2 costp stopper + B4 slize softclock 4); console_markers.txt (fired=2 ran=1 + terminal marker line from dfbsd-qemu/boot.log); build_shared.log (in DF-3903 pack; modules built clean under -Werror); fix_validation.log (fixed shape: callout_stop RETURNED, pending=0, 312 fires/307 retries/0 parks); env.txt (guest uname); fix.diff (try-enter + 1-tick retry for xl_stats_update)", "kernel_refs": [ "sys/dev/netif/xl/if_xl.c:1320", "sys/dev/netif/xl/if_xl.c:2377", "sys/dev/netif/xl/if_xl.c:2429", "sys/dev/netif/xl/if_xl.c:2918", "sys/dev/netif/xl/if_xl.c:3142", "sys/dev/netif/xl/if_xl.c:3169", "sys/dev/netif/xl/if_xl.c:1573", "sys/dev/netif/xl/if_xl.c:2712", "sys/dev/netif/xl/if_xl.c:3212", "sys/kern/kern_timeout.c:583", "sys/kern/kern_timeout.c:920", "sys/kern/kern_timeout.c:1091", "sys/kern/lwkt_serialize.c:105" ], "poc_changes": "No seed PoC. Authored xl_callout_replica.ko / xl_callout_replica_fixed.ko (out-of-tree kmodules replicating the if_xl.c callout protocol, including the plain/non-MPSAFE callout_init at :1320) because the guest has no 3c90x hardware; the deadlock is in the generic protocol xl uses, so the replica exercises the identical kernel paths.", "attempts": 1, "guest_uname": "DragonFly dfbsd 6.5-DEVELOPMENT DragonFly 6.5-DEVELOPMENT #0: Thu Jul 2 06:02:54 UTC 2026 root@dfbsd:/usr/obj/usr/src/sys/X86_64_GENERIC x86_64", "runtime_sec": 1200, "guest_dirty": 0, "build_cmd": "cd /root/kmod && make (KMOD=xl_callout_replica SRCS=xl_callout_replica.c; then xl_callout_replica_fixed)", "run_cmd": "vm.sh run_root kldload /root/kmod/xl_callout_replica.ko; sysctl kern.xlreplica_arm=1; sleep 1; (sysctl kern.xlreplica_stop=1 >/dev/null 2>&1 &) then ps from a second session", "code_hash": "2c1195cceb8a6eb2cf5ffea7ca2741039945bef9a3d39d1c8c82e20f0e785a43", "notes": "10th instance of the DF-3715 AB-BA callout family (em, emx, alc, jme, msk, dc, sk, xl). Aggravation: xl_stat_timer is non-MPSAFE (parked softclock holds the Giant MP lock - the wedge defeated the guests clean shutdown). Triggers on xl hardware: ifconfig xlX down/up, media change while running, watchdog reinit, detach/kldunload, suspend/resume, shutdown. RX-length (DF-3604 family) checked and NOT present: xl_rxeof bounds total_len to XL_MAX_FRAMELEN 1522 < 2046 posted bytes at :1973. Guest reset to with-src snapshot after verification.", "recommended_fix": "Make xl_stats_update() non-blocking: lwkt_serialize_try() with a 1-tick callout_reset retry (validated live at protocol level).", "fix_status": "inconclusive", "fix_baseline_reproduced": 0, "fix_patched_reproduced": 0, "fix_verdict": "fix.diff authored against the read-only sys/ tree: xl_stats_update uses lwkt_serialize_try + 1-tick retry. Validated at protocol level on the guest with xl_callout_replica_fixed.ko (callout_stop returns, pending==0, 312 fires / 307 retries / 0 parks while the serializer was held 3s from another CPU). Driver-level validation impossible on this guest (no 3c90x NIC).", "fix_evidence": "findings/poc/DF-3906/fix.diff; findings/poc/DF-3906/fix_validation.log" } |