β¬’ DragonFlyBSD Kernel Audit
← triage Β· dashboard
DF-3991

ale_stop() (serializer held) vs ale_tick() (blocking lwkt_serialize_enter in a self-re-arming NON-MP callout): AB-BA deadlock against synchronous callout_stop β€” permanent kernel wedge + softclock starvation; mechanism PROVEN on the guest via protocol replica (DF-3715/3760/3935 family)

Summary

ale_tick takes ifp->if_serializer with a BLOCKING lwkt_serialize_enter and self re-arms from a NON-MP callout; ale_stop runs under that same serializer and calls the synchronous callout_stop (ssleep 'costp'). A tick parked on the serializer while any stop path runs (ifconfig down/MTU, init-from-intr, detach, suspend) deadlocks both sides permanently. Mechanism PROVEN live on the audit guest with a deterministic replica of ale's exact protocol (stopper wedged D2/costp, still wedged 30s later; guest reset afterwards). Fix: lwkt_serialize_try + immediate re-arm (validated family fix).

Discussion (0)

No comments yet.

PoC verification

Evidence pack

findings/poc/DF-3991 Β· 10 files
FileTypeDescriptionSize
ale_replica.c replica hardware-precondition model / protocol replica 4.4 KB view raw
README.md readme build/run/expected 1.4 KB ↓ raw
VERDICT.md verdict full path:line trace 4.2 KB ↓ raw
build.sh script exact build commands 65 B view raw
run.sh script exact run commands 89 B view raw
build.log log full untrimmed build output 1.1 KB view raw
run.log log full untrimmed runtime output 1.7 KB view raw
env.txt env guest hardware/src-md5 proof 1.1 KB view raw
fix.diff fix git-apply-able unified diff 798 B view raw
Makefile build kmod makefile 62 B ↓ download
README.md readme build/run/expected
↓ download raw

DF-3991 β€” ale_stop() (serializer held) vs ale_tick() (blocking lwkt_serialize_enter in a self-re-arming NON-MP callout): AB-BA deadlock against DragonFly's SYNCHRONOUS callout_stop(). Mechanism PROVEN on the guest kernel via protocol replica (stopper stuck D-state, wchan 'costp'). DF-3715 (em) / DF-3760 (alc) / DF-3776 (jme) / DF-3935 (age) family, ale(4) instance.

BUILD (guest) mkdir -p /root/kmod/alesink && cp ale_replica.c /root/kmod/alesink/ printf 'KMOD= ale_replica\nSRCS= ale_replica.c\n\n.include \n' \ > /root/kmod/alesink/Makefile cd /root/kmod/alesink && make (build.log)

RUN (guest) kldload /root/kmod/alesink/ale_replica.ko sysctl kern.alereplica.arm=1 (sysctl kern.alereplica.stop=1 &) # ale_stop replica under serializer sleep 8; dmesg | tail; ps -ax -o pid,stat,wchan,comm | grep costp

EXPECTED (PROVEN β€” run.log) ale_replica: tick handler BLOCKED on our serializer (fired=73 ran=72) - entering callout_stop() ale_replica: IF THIS IS THE LAST ale_replica LINE, AB-BA DEADLOCK PROVEN ... ps: D2 costp sysctl (still wedged 30s later) NOTE: this run wedges the guest; vm.sh reset with-src afterwards.

The ale-specific trigger (ifconfig ale0 down / MTU change / detach while the 1 Hz tick is parked on the serializer) is hardware-gated: no Atheros NIC on the audit guest.

VERDICT.md verdict full path:line trace
↓ download raw

DF-3991 VERDICT β€” mechanism PROVEN on the guest; ale trigger hardware-gated (inconclusive per family)

Root cause (path:line)

ale_tick β€” if_ale.c:2257-2272 β€” is a self-re-arming NON-MP callout whose handler takes the driver serializer by BLOCKING:

455:  callout_init(&sc->ale_tick_ch);              /* NON-MP */
2263: lwkt_serialize_enter(ifp->if_serializer);    /* BLOCKING enter */
2269: callout_reset(&sc->ale_tick_ch, hz, ale_tick, sc);   /* self re-arm */
2271: lwkt_serialize_exit(ifp->if_serializer);

ale_stop tears it down while HOLDING that same serializer:

2544: ale_stop(struct ale_softc *sc)
2551: ASSERT_SERIALIZED(ifp->if_serializer);
2560: callout_stop(&sc->ale_tick_ch);

Every ale_stop caller holds the serializer: ale_ioctl SIOCSIFFLAGS-down :1762-1764, MTU-change ale_init :1745-1748, ale_init :2319 (also reached from the ale_intr error path :1981-1985 β€” i.e. from a serialized interrupt thread), ale_detach :660-662, ale_suspend :1455.

DragonFly callout_stop is synchronous (sys/kern/kern_timeout.c:1091-1094 β†’ _callout_cancel_or_stop(..., sync=1) β†’ ssleep(c, &c->spin, 0, "costp", 0) at :923). A tick callback dequeued and parked inside lwkt_serialize_enter deadlocks against the stopper:

stopper: holds serializer β†’ sleeps in callout_stop waiting for the INPROG callback ('costp') callback: INPROG β†’ blocked in lwkt_serialize_enter waiting for the serializer the stopper holds

Permanent wedge: the interface serializer is never released, the affected CPU's softclock thread stops delivering callouts. ifconfig ale0 down, MTU change, detach, suspend, or a RX-seq-error interrupt during the 1 Hz window triggers it.

Guest PROOF (deterministic replica, run.log / build.log)

ale_replica.c replicates if_ale.c's exact protocol (non-MP callout_init, blocking lwkt_serialize_enter, self re-arm, stop under serializer). Executed on the audit guest (DragonFly 6.5-DEVELOPMENT

0 Thu Jul 2 06:02:54 UTC 2026):

ale_replica: loaded (hz=100)
ale_replica: armed ale_tick replica (non-mp callout, ticks hz/10)
ale_replica: tick handler BLOCKED on our serializer (fired=73 ran=72) - entering callout_stop()
ale_replica: IF THIS IS THE LAST ale_replica LINE, AB-BA DEADLOCK PROVEN (DF-3715 family / ale instance)

$ ps -ax -o pid,stat,wchan,comm | grep costp
1159  D2  costp  sysctl            <- stopper wedged in callout_stop sync wait
(still wedged 30 s later: STILL_WEDGED_AFTER_30S)

That is the exact DF-3715/DF-3935 mechanism signature (D-state, wchan 'costp', kern_timeout.c:923). Guest was reset with vm.sh reset with-src afterwards.

The ale-specific end-to-end trigger (real if_ale device) is hardware-gated on this guest: zero 0x1969 PCI devices, no hw.ale sysctl, if_ale compiled into the kernel but never attached. Hence family-convention status inconclusive, mechanism proven. Guest src md5 ae824d7a4429b82f0ad92a72b7c91da8 == audited tree.

Impact

ifconfig ale0 down/up (or MTU change / detach / suspend / RX-error interrupt) racing the 1 Hz tick β†’ permanent kernel wedge + softclock blackout on the callback's CPU. Unprivileged-local-triggerable on any machine with an AR8121/8113/8114 NIC (ifconfig is unprivileged for down/MTU? β€” down requires operator/interface privileges; MTU likewise; but detach/suspend paths are root; the RX-seq-error path (:1981-1985 β†’ ale_init β†’ ale_stop) is device-triggerable). Medium DoS, matching DF-3760/DF-3935.

Fix

fix.diff: lwkt_serialize_try() + early re-arm in ale_tick (the callback can always complete; callout_stop can always return) β€” identical to the validated DF-3760/DF-3935 family fix. git-apply clean. Runtime fix validation not possible (no ale hardware); the replica demonstrates the mechanism the fix breaks (with try-enter the callback cannot park on the serializer).

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.

Fix verification

not_testable
baseline no→ patch + rebuild →patched clean

Baseline mechanism proven by the replica; the ale-specific fix path cannot be runtime-validated (no ale hardware). fix.diff is the family fix (lwkt_serialize_try + re-arm, exactly the validated DF-3760/DF-3935 shape) and breaks the deadlock precondition (callback can never park on the serializer the stopper holds); git-apply clean against the audited tree.

fix.diff; replica mechanism proof in run.log
↓ fix.diffper-fix-DF-3991

Confirmed kernel references

Detail

Exploit chain

On real hardware: ifconfig ale0 down (or MTU change / detach / suspend / RX-seq-error interrupt -> ale_init -> ale_stop) while the 1 Hz ale_tick is INPROG-parked on the serializer -> stopper sleeps in callout_stop('costp'), callback waits on the serializer -> permanent AB-BA wedge + softclock starvation on that CPU; ale0 (and anything needing ifp->if_serializer) is dead until reboot.

Evidence (decisive lines)

run.log (replica: BLOCKED message + ps D2/costp + STILL_WEDGED_AFTER_30S + serial-console lines), build.log (kmod built on guest with -Werror), env.txt (NO_ATHEROS_PCI + guest-src md5 match), ale_replica.c (annotated protocol replica), VERDICT.md, fix.diff (try-enter, git-apply clean)

PoC changes

No seed PoC. Authored ale_replica.c by adapting the PROVEN DF-3935 age replica to ale's exact protocol (non-MP callout, hz tick, same stop sites); executed on the guest to mechanism-proof.

Verified recommended fix

Use lwkt_serialize_try() + early re-arm in ale_tick so the callback can always complete (see fix.diff - identical to the DF-3760/DF-3935 family fix).

Verdict

MECHANISM PROVEN on the guest kernel by deterministic replica of if_ale.c's exact callout/serializer protocol (DF-3715 family, ale instance; adapted from the PROVEN DF-3935 age replica). ale(4) arms one NON-MP callout (callout_init, if_ale.c:455) whose handler blocks on the driver serializer and self re-arms (ale_tick: lwkt_serialize_enter :2263 BLOCKING, callout_reset hz :2269); every teardown stops it while HOLDING that serializer (ale_stop: ASSERT_SERIALIZED :2551 + callout_stop :2560; callers: ale_ioctl down/MTU :1745-1764, ale_init :2319 - also reached from the ale_intr error path :1981-1985 in interrupt context -, ale_detach :660-662, ale_suspend :1455). DragonFly callout_stop is synchronous (kern_timeout.c:1091-1094 -> sync=1 -> ssleep 'costp' :923). Guest replica run (build.log/run.log): 'tick handler BLOCKED on our serializer (fired=73 ran=72) - entering callout_stop()' then NO 'callout_stop returned' line, ps shows the stopper D2/wchan costp, STILL_WEDGED_AFTER_30S; guest reset with-src afterwards. ale-specific end-to-end trigger hardware-gated (no 1969:1026 device, no hw.ale, if_ale in kernel but unattached) -> inconclusive per family convention. Guest src md5 ae824d7a4429b82f0ad92a72b7c91da8 == audited tree.