DF-4109 — tl(4): callout AB-BA deadlock — tl_stats_update blocks on the serializer tl_stop stops the callout under

What this is

sys/dev/netif/tl/if_tl.c arms sc->tl_stat_timer with a plain callout_init() (if_tl.c:1197) — MP-safe on this kernel (sys/kern/kern_timeout.c:770) — and re-arms it every hz from tl_init (if_tl.c:1981). The callback, tl_stats_update() (if_tl.c:1666-1674), takes the driver serializer by blocking (lwkt_serialize_enter(ifp->if_serializer), if_tl.c:1671 → parks in tsleep "slize", sys/kern/lwkt_serialize.c:227-237) and re-arms itself inside the serializer (if_tl.c:1722).

tl_stop() (if_tl.c:2121) calls callout_stop(&sc->tl_stat_timer) while its caller holds that same serializer (tl_stop is reached under the serializer from tl_ioctl if_tl.c:2062, tl_init :1916, tl_watchdog :2103, tl_detach :1258, and tl_intvec_adchk :1571 in the ISR).

DragonFly's callout_stop() synchronously waits for an in-progress callback (_callout_cancel_or_stop(..., sync=1), ssleep "costp", sys/kern/kern_timeout.c:1091, 920). A callback that was dequeued and is parked on the stopper's serializer therefore deadlocks against it: both threads wedge permanently, the interface serializer is never released, and the parked softclock thread stops delivering all callouts for its CPU. Same mechanism as DF-3715 (em(4)), PROVEN there on this guest; re-proven here with the exact tl shape.

Build / run (verification on the audit guest)

# guest has no tl(4) hardware (vtnet0 lo0 only), so the driver protocol
# is replicated verbatim by an out-of-tree module: plain callout_init,
# hz period, BLOCKING lwkt_serialize_enter, re-arm inside the serializer,
# stopper holds the serializer and calls callout_stop.

scp tlnge_replica.c Makefile root@guest:/root/kmod/
vm.sh run_root 'cd /root/kmod && make'                       # build.log
vm.sh run_root 'kldload /root/kmod/tlnge_replica.ko; sysctl kern.tlreplica.arm=1; sleep 3; \
                (sysctl kern.tlreplica.stop_blk=1 >/dev/null 2>&1 &); sleep 6; \
                dmesg | grep "replica\[tl\]" | tail -3; \
                ps -o pid,stat,wchan,command -ax | grep stop_blk | grep -v grep'

Expected / observed

• dmesg ends with replica[tl]: handler PARKED on our serializer (fired=4 ran=3) - entering callout_stop() then the marker line — and nothing after (permanent).
• stopper thread stuck D1/D5, wchan costp (kern_timeout.c:920).
• Negative control (stop_plain: callout_stop without the serializer) returns immediately with pending=0.
• Fix-shape control (stop_fix: try-enter + 1-tick retry, the proposed fix): callout_stop under the serializer returns, pending=0, firing fully stalled — the fix pattern is validated on this kernel.
• On real tl(4) hardware the trigger is ifconfig tl0 down (or MTU change / detach / chip adapter-check) racing the 1-second stats tick.

Trigger on hardware

ifconfig tl0 down / ifconfig tl0 mtu 1500-style reinit / kldunload if_tl / adapter-check interrupt, while tl_stat_timer fires (once per second). Narrow per-call window, but the interface cycles on every down/up and the cost is a permanent system wedge.

DF-4109 — VERDICT

Finding: tl(4) tl_stats_update callout callback blocks on the driver serializer (lwkt_serialize_enter, sys/dev/netif/tl/if_tl.c:1671) while tl_stop() calls the synchronous callout_stop(&sc->tl_stat_timer) (if_tl.c:2121) holding that same serializer — AB-BA deadlock, the DF-3715 (em(4)) family.

Status: inconclusive per the family convention (target hardware absent — guest has vtnet0 lo0 only, no ThunderLAN PCI device), with the mechanism PROVEN deterministically on the guest kernel by a replica module that reproduces the exact tl protocol:

• callout created with plain callout_init() (as if_tl.c:1197; MPSAFE on this kernel per sys/kern/kern_timeout.c:770, so the callback is allowed to block and parks the softclock thread),
• period hz (as if_tl.c:1981/1722),
• callback does a blocking lwkt_serialize_enter (if_tl.c:1671 → parks in tsleep "slize", sys/kern/lwkt_serialize.c:227-237) and re-arms itself inside the serializer (if_tl.c:1722),
• stopper takes the serializer (as every tl_stop caller does: tl_ioctl if_tl.c:2062, tl_init :1916, tl_watchdog :2103, tl_detach :1258, tl_intvec_adchk :1571), waits for the handler to park, then callout_stop() (if_tl.c:2121 → kern_timeout.c:1091 sync wait, ssleep "costp" at :920).

Observed on the guest (DragonFly 6.5-DEVELOPMENT x86_64):

replica[tl]: handler PARKED on our serializer (fired=4 ran=3) - entering callout_stop()
replica[tl]: IF THIS IS THE LAST replica LINE, AB-BA DEADLOCK IS PROVEN
1603 D1  costp  sysctl kern.tlreplica.stop_blk=1      <- never returns

• callout_stop() never returned (re-checked +10 s later; still parked).
• Negative control: callout_stop() without the serializer returns with pending=0 — module logic sound.
• Fix-shape control: the proposed fix pattern (lwkt_serialize_try
• 1-tick retry — the em_txgc_timer pattern) is fully stopped by callout_stop() under a held serializer on this exact kernel: fix-shape callout_stop RETURNED ... RESULT pending=0 fired-stalled=YES.
• Collateral observed on wedged guests: subsequent sysctl access hangs (the wedged sysctl writer holds the sysctl write lock), and clean shutdown never completes (both post-wedge resets force-killed QEMU) — on hardware this is an un-shutdown-able system.

Why hardware-gated: tl(4) attaches only to TI/Compaq/Olicom ThunderLAN PCI IDs (if_tl.c:227-257); this guest has none, so the in-driver race itself cannot be entered on the guest. The protocol the driver uses, however, is fully generic and was exercised verbatim.

Exploit chain / impact: not a corruption primitive. A privileged local user (ifconfig down/up, MTU change, kldunload if_tl) racing the 1-second stats tick permanently wedges: the calling thread, one softclock thread (callout starvation for that CPU), the interface serializer (any later ifnet consumer blocks), and — as observed — the sysctl subsystem.

Fix: tl_stats_update() must not block: lwkt_serialize_try() with a 1-tick callout_reset retry (fix.diff). Mechanism of the fix validated on the guest (fix-shape control above); the patched driver itself cannot be exercised without hardware → fix_status not_tested.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.