ale_stop() (serializer held) vs ale_tick() (blocking lwkt_serialize_enter in a self-re-arming NON-MP callout): AB-BA deadlock against synchronous callout_stop β permanent kernel wedge + softclock starvation; mechanism PROVEN on the guest via protocol replica (DF-3715/3760/3935 family)
Summary
ale_tick takes ifp->if_serializer with a BLOCKING lwkt_serialize_enter and self re-arms from a NON-MP callout; ale_stop runs under that same serializer and calls the synchronous callout_stop (ssleep 'costp'). A tick parked on the serializer while any stop path runs (ifconfig down/MTU, init-from-intr, detach, suspend) deadlocks both sides permanently. Mechanism PROVEN live on the audit guest with a deterministic replica of ale's exact protocol (stopper wedged D2/costp, still wedged 30s later; guest reset afterwards). Fix: lwkt_serialize_try + immediate re-arm (validated family fix).
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-3991 Β· 10 files| File | Type | Description | Size | |
|---|---|---|---|---|
| ale_replica.c | replica | hardware-precondition model / protocol replica | 4.4 KB | view raw |
| README.md | readme | build/run/expected | 1.4 KB | β raw |
| VERDICT.md | verdict | full path:line trace | 4.2 KB | β raw |
| build.sh | script | exact build commands | 65 B | view raw |
| run.sh | script | exact run commands | 89 B | view raw |
| build.log | log | full untrimmed build output | 1.1 KB | view raw |
| run.log | log | full untrimmed runtime output | 1.7 KB | view raw |
| env.txt | env | guest hardware/src-md5 proof | 1.1 KB | view raw |
| fix.diff | fix | git-apply-able unified diff | 798 B | view raw |
| Makefile | build | kmod makefile | 62 B | β download |
DF-3991 β ale_stop() (serializer held) vs ale_tick() (blocking lwkt_serialize_enter in a self-re-arming NON-MP callout): AB-BA deadlock against DragonFly's SYNCHRONOUS callout_stop(). Mechanism PROVEN on the guest kernel via protocol replica (stopper stuck D-state, wchan 'costp'). DF-3715 (em) / DF-3760 (alc) / DF-3776 (jme) / DF-3935 (age) family, ale(4) instance.
BUILD (guest)
mkdir -p /root/kmod/alesink && cp ale_replica.c /root/kmod/alesink/
printf 'KMOD= ale_replica\nSRCS= ale_replica.c\n\n.include
RUN (guest) kldload /root/kmod/alesink/ale_replica.ko sysctl kern.alereplica.arm=1 (sysctl kern.alereplica.stop=1 &) # ale_stop replica under serializer sleep 8; dmesg | tail; ps -ax -o pid,stat,wchan,comm | grep costp
EXPECTED (PROVEN β run.log)
ale_replica: tick handler BLOCKED on our serializer (fired=73 ran=72) - entering callout_stop()
ale_replica: IF THIS IS THE LAST ale_replica LINE, AB-BA DEADLOCK PROVEN ...
ps: vm.sh reset with-src afterwards.
The ale-specific trigger (ifconfig ale0 down / MTU change / detach while the 1 Hz tick is parked on the serializer) is hardware-gated: no Atheros NIC on the audit guest.
DF-3991 VERDICT β mechanism PROVEN on the guest; ale trigger hardware-gated (inconclusive per family)
Root cause (path:line)
ale_tick β if_ale.c:2257-2272 β is a self-re-arming NON-MP callout whose
handler takes the driver serializer by BLOCKING:
455: callout_init(&sc->ale_tick_ch); /* NON-MP */
2263: lwkt_serialize_enter(ifp->if_serializer); /* BLOCKING enter */
2269: callout_reset(&sc->ale_tick_ch, hz, ale_tick, sc); /* self re-arm */
2271: lwkt_serialize_exit(ifp->if_serializer);
ale_stop tears it down while HOLDING that same serializer:
2544: ale_stop(struct ale_softc *sc)
2551: ASSERT_SERIALIZED(ifp->if_serializer);
2560: callout_stop(&sc->ale_tick_ch);
Every ale_stop caller holds the serializer: ale_ioctl SIOCSIFFLAGS-down :1762-1764, MTU-change ale_init :1745-1748, ale_init :2319 (also reached from the ale_intr error path :1981-1985 β i.e. from a serialized interrupt thread), ale_detach :660-662, ale_suspend :1455.
DragonFly callout_stop is synchronous (sys/kern/kern_timeout.c:1091-1094
β _callout_cancel_or_stop(..., sync=1) β ssleep(c, &c->spin, 0,
"costp", 0) at :923). A tick callback dequeued and parked inside
lwkt_serialize_enter deadlocks against the stopper:
stopper: holds serializer β sleeps in callout_stop waiting for the INPROG callback ('costp') callback: INPROG β blocked in lwkt_serialize_enter waiting for the serializer the stopper holds
Permanent wedge: the interface serializer is never released, the affected
CPU's softclock thread stops delivering callouts. ifconfig ale0 down,
MTU change, detach, suspend, or a RX-seq-error interrupt during the 1 Hz
window triggers it.
Guest PROOF (deterministic replica, run.log / build.log)
ale_replica.c replicates if_ale.c's exact protocol (non-MP
callout_init, blocking lwkt_serialize_enter, self re-arm, stop under
serializer). Executed on the audit guest (DragonFly 6.5-DEVELOPMENT
0 Thu Jul 2 06:02:54 UTC 2026):
ale_replica: loaded (hz=100) ale_replica: armed ale_tick replica (non-mp callout, ticks hz/10) ale_replica: tick handler BLOCKED on our serializer (fired=73 ran=72) - entering callout_stop() ale_replica: IF THIS IS THE LAST ale_replica LINE, AB-BA DEADLOCK PROVEN (DF-3715 family / ale instance) $ ps -ax -o pid,stat,wchan,comm | grep costp 1159 D2 costp sysctl <- stopper wedged in callout_stop sync wait (still wedged 30 s later: STILL_WEDGED_AFTER_30S)
That is the exact DF-3715/DF-3935 mechanism signature (D-state, wchan
'costp', kern_timeout.c:923). Guest was reset with vm.sh reset with-src
afterwards.
The ale-specific end-to-end trigger (real if_ale device) is hardware-gated on this guest: zero 0x1969 PCI devices, no hw.ale sysctl, if_ale compiled into the kernel but never attached. Hence family-convention status inconclusive, mechanism proven. Guest src md5 ae824d7a4429b82f0ad92a72b7c91da8 == audited tree.
Impact
ifconfig ale0 down/up (or MTU change / detach / suspend / RX-error
interrupt) racing the 1 Hz tick β permanent kernel wedge + softclock
blackout on the callback's CPU. Unprivileged-local-triggerable on any
machine with an AR8121/8113/8114 NIC (ifconfig is unprivileged for
down/MTU? β down requires operator/interface privileges; MTU likewise; but
detach/suspend paths are root; the RX-seq-error path (:1981-1985 β ale_init
β ale_stop) is device-triggerable). Medium DoS, matching DF-3760/DF-3935.
Fix
fix.diff: lwkt_serialize_try() + early re-arm in ale_tick (the callback
can always complete; callout_stop can always return) β identical to the
validated DF-3760/DF-3935 family fix. git-apply clean. Runtime fix
validation not possible (no ale hardware); the replica demonstrates the
mechanism the fix breaks (with try-enter the callback cannot park on the
serializer).
Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.
Fix verification
not_testableBaseline mechanism proven by the replica; the ale-specific fix path cannot be runtime-validated (no ale hardware). fix.diff is the family fix (lwkt_serialize_try + re-arm, exactly the validated DF-3760/DF-3935 shape) and breaks the deadlock precondition (callback can never park on the serializer the stopper holds); git-apply clean against the audited tree.
fix.diff; replica mechanism proof in run.log
Confirmed kernel references
Detail
Exploit chain
On real hardware: ifconfig ale0 down (or MTU change / detach / suspend / RX-seq-error interrupt -> ale_init -> ale_stop) while the 1 Hz ale_tick is INPROG-parked on the serializer -> stopper sleeps in callout_stop('costp'), callback waits on the serializer -> permanent AB-BA wedge + softclock starvation on that CPU; ale0 (and anything needing ifp->if_serializer) is dead until reboot.
Evidence (decisive lines)
run.log (replica: BLOCKED message + ps D2/costp + STILL_WEDGED_AFTER_30S + serial-console lines), build.log (kmod built on guest with -Werror), env.txt (NO_ATHEROS_PCI + guest-src md5 match), ale_replica.c (annotated protocol replica), VERDICT.md, fix.diff (try-enter, git-apply clean)
PoC changes
No seed PoC. Authored ale_replica.c by adapting the PROVEN DF-3935 age replica to ale's exact protocol (non-MP callout, hz tick, same stop sites); executed on the guest to mechanism-proof.
Verified recommended fix
Use lwkt_serialize_try() + early re-arm in ale_tick so the callback can always complete (see fix.diff - identical to the DF-3760/DF-3935 family fix).
Verdict
MECHANISM PROVEN on the guest kernel by deterministic replica of if_ale.c's exact callout/serializer protocol (DF-3715 family, ale instance; adapted from the PROVEN DF-3935 age replica). ale(4) arms one NON-MP callout (callout_init, if_ale.c:455) whose handler blocks on the driver serializer and self re-arms (ale_tick: lwkt_serialize_enter :2263 BLOCKING, callout_reset hz :2269); every teardown stops it while HOLDING that serializer (ale_stop: ASSERT_SERIALIZED :2551 + callout_stop :2560; callers: ale_ioctl down/MTU :1745-1764, ale_init :2319 - also reached from the ale_intr error path :1981-1985 in interrupt context -, ale_detach :660-662, ale_suspend :1455). DragonFly callout_stop is synchronous (kern_timeout.c:1091-1094 -> sync=1 -> ssleep 'costp' :923). Guest replica run (build.log/run.log): 'tick handler BLOCKED on our serializer (fired=73 ran=72) - entering callout_stop()' then NO 'callout_stop returned' line, ps shows the stopper D2/wchan costp, STILL_WEDGED_AFTER_30S; guest reset with-src afterwards. ale-specific end-to-end trigger hardware-gated (no 1969:1026 device, no hw.ale, if_ale in kernel but unattached) -> inconclusive per family convention. Guest src md5 ae824d7a4429b82f0ad92a72b7c91da8 == audited tree.
No comments yet.