DF-4218 — ae(4) tick callout vs ae_stop(): synchronous callout_stop under the serializer = AB-BA wedge (DF-3715 family)

sys/dev/netif/ae/if_ae.c — pass 2 (GLM 5.3). DF-3715 (em, PROVEN) / DF-3760 (alc) / DF-3776 (jme) / DF-3935 (age, PROVEN) / DF-3991 (ale, PROVEN) family instance for ae(4).

Build

scp -F dfbsd-qemu/config findings/poc/DF-4218/ae_replica.c dfbsd:/tmp/
vm.sh run_root 'mkdir -p /root/kmod/aesink && cp /tmp/ae_replica.c /root/kmod/aesink/ && \
  printf "KMOD= ae_replica\nSRCS= ae_replica.c\n\n.include <bsd.kmod.mk>\n" > /root/kmod/aesink/Makefile && \
  cd /root/kmod/aesink && make'

Builds clean with -Werror (full log in build.log).

Run (WEDGES THE GUEST BY DESIGN — reset with vm.sh reset with-src afterwards)

vm.sh run_root 'kldload /root/kmod/aesink/ae_replica.ko && sysctl kern.aereplica.arm=1'
timeout 90 vm.sh run_root '(sysctl kern.aereplica.stop=1 &) ; sleep 8; dmesg | tail -4; ps -ax -o pid,stat,wchan,comm | grep costp'
sleep 30
vm.sh run_root 'ps -ax -o pid,stat,wchan,comm | grep costp; echo STILL_WEDGED_AFTER_30S'
vm.sh reset with-src

Expected output

ae_replica: tick handler BLOCKED on our serializer (fired=148 ran=147) - entering callout_stop()
ae_replica: IF THIS IS THE LAST ae_replica LINE, AB-BA DEADLOCK PROVEN (DF-3715 family / ae instance)
  1135 D4     costp    sysctl
STILL_WEDGED_AFTER_30S

No callout_stop returned line ever appears: the stopper (the sysctl thread, playing ae_stop's role while holding the serializer) sleeps in costp (kern_timeout.c:920 ssleep inside the synchronous callout_stop, kern_timeout.c:1091) and the ae_tick replica — a NON-MP callout callback parked in lwkt_serialize_enter (if_ae.c:995 shape) holding the mplock — can never acquire the serializer. AB-BA, permanent.

Observed live on the audit guest 2026-09-18; serial console shows the BLOCKED line as the last ae_replica output; guest was reset with-src afterwards.

DF-4218 — ae(4): the 1 Hz ae_tick() callout (non-MP, self-re-arming, blocking lwkt_serialize_enter) deadlocks against every ae_stop() path, which calls the synchronous callout_stop() while holding that same serializer. DF-3715 family instance (em/alc/jme/age/ale), proven live on the audit guest by a deterministic replica of if_ae.c's exact protocol.

MECHANISM PROVEN ON THE GUEST (run.log): 'ae_replica: tick handler BLOCKED on our serializer (fired=148 ran=147) - entering callout_stop()' ps: pid 1135 stat D4 wchan costp (the stopper thread, still wedged 30+ s later; STILL_WEDGED_AFTER_30S). No "callout_stop returned" line ever printed; the serial console's last ae_replica line is the BLOCKED message. The parked NON-MP callback also holds the mplock, degrading the whole system (follow-on ssh commands timed out) — same partial-degradation signature as the PROVEN DF-3991 ale run on this same guest kernel. Guest was reset with-src afterwards (verified clean).

WHY IT IS ae(4) AND NOT JUST THE REPLICA (path:line): * callout_init(&sc->ae_tick_ch) if_ae.c:1262 — plain (non-MPSAFE) callout: _callout_init sets only CALLOUT_DID_INIT (kern_timeout.c:760-764), so the callback runs holding the mplock (kern_timeout.c:580-585). * ae_tick() if_ae.c:988-999 lwkt_serialize_enter(ifp->if_serializer); :995 (BLOCKING: atomic_intr_cond_enter -> lwkt_serialize_sleep, sys/kern/lwkt_serialize.c:105-113) mii_tick(mii); callout_reset(&sc->ae_tick_ch, hz, ae_tick, sc); :997 (self re-arm; also armed by ae_init :965) lwkt_serialize_exit(ifp->if_serializer); :998 * ae_stop() if_ae.c:219-262 ASSERT_SERIALIZED(ifp->if_serializer); :225 callout_stop(&sc->ae_tick_ch); :232 callout_stop is SYNCHRONOUS: kern_timeout.c:1091-1094 -> _callout_cancel_or_stop(..., sync=1) -> ssleep(c, ..., "costp", 0) at kern_timeout.c:920 — the stopper sleeps holding the serializer. * Every ae_stop caller holds the serializer: ae_ioctl SIOCSIFFLAGS "down" :1220-1221; ae_init :799 — itself reached from ae_intr's DMAR/DMAW-timeout / PHY-linkdown path :770-775 IN INTERRUPT CONTEXT; ae_watchdog :983; ae_detach :1417; ae_suspend :1786; ae_shutdown :1801.

AB-BA: stopper holds serializer -> sleeps in costp waiting for the INPROG callback; callback is INPROG -> parked in lwkt_serialize_enter waiting for the serializer the stopper holds (and holds the mplock while parked). Neither side progresses; the interface serializer is never released; the affected CPU's softclock stops delivering callouts.

IMPACT: permanent local DoS (interface + calling thread wedge, mplock held => system-wide degradation). Trigger: race the 1 Hz tick against any stop path (ifconfig down/up repeated, suspend, detach, or the intr-error reinit). No memory corruption, no privilege path.

WHY NOT end-to-end on the ae driver itself: no Attansic L2 NIC on the audit guest (env.txt: NO_ATTANSIC_PCI_1969; if_ae not in X86_64_GENERIC; guest /usr/src md5 429ba8f801876d3e7a39eca15612f3ed == audited tree). The replica reproduces if_ae.c's protocol character-for-character (same callout flavor, same blocking enter, same self re-arm, same stop-under-serializer) on the same kernel's real callout/serializer machinery — the same evidence bar the family set in DF-3991.

FIX: family fix — try-enter + immediate re-arm in ae_tick so the callback can always complete (fix.diff; validated shape from DF-3760/DF-3935/DF-3991).

VERDICT: mechanism PROVEN live; ae-specific end-to-end hardware-gated. Status: inconclusive (family convention), confidence certain. Guest reset with-src after the proof (verified).

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.