DF-3935 — age: age_stop() (serializer held) vs age_tick (blocking serializer enter) — AB-BA callout deadlock, DF-3715 family instance

Build / Run — MECHANISM PROVEN ON THE AUDIT GUEST (replica)

The deadlock protocol does not depend on the NIC; it was replicated exactly (age_replica.c in this pack: plain callout_init as if_age.c:517, callback lwkt_serialize_enter + self callout_reset as if_age.c:2194-2199, stopper holding the serializer calling callout_stop as if_age.c:2534):

build: see build.log (guest cc 8.3, -Werror, BUILD_RC=0) run: kldload age_replica.ko sysctl kern.agereplica.arm=1 (sysctl kern.agereplica.stop=1 &) expected: "IF THIS IS THE LAST age_replica LINE, AB-BA DEADLOCK PROVEN" then silence; ps shows the stopper stuck D5 wchan costp

observed (run.log): exactly that — age_replica: tick handler BLOCKED on our serializer (fired=14 ran=13) - entering callout_stop() [never returns] ps: 1001 D5 costp sysctl (immediately AND 30s later) the wedge then degraded the guest enough to kill the ssh session.

On real age(4) hardware the same interleaving is entered by any age_stop() caller holding the serializer while age_tick is INPROG: ifconfig age0 down/up, SIOCSIFMTU on a running interface (age_init → age_stop, if_age.c:2249), kldunload/detach (:686-688), suspend (:1393-1394).

See VERDICT.md.

DF-3935 verdict narrative

Reproduced?

Mechanism PROVEN on the guest kernel by deterministic replica of if_age.c's exact callout/serializer protocol (same method as the family-proven DF-3715/em instance); the age-specific runtime trigger is hardware-gated (no L1 NIC in the guest) → filed inconclusive per family convention.

Root cause chain (path:line)

• age(4) arms one NON-MP callout: if_age.c:517 callout_init(&sc->age_tick_ch); (contrast em(4)'s callout_init_mp — this replica matches the non-mp kind).
• Its handler takes the serializer by hand, blocking, and re-arms itself: if_age.c:2194 lwkt_serialize_enter(ifp->if_serializer); if_age.c:2197 callout_reset(&sc->age_tick_ch, hz, age_tick, sc); if_age.c:2199 lwkt_serialize_exit(ifp->if_serializer);
• Every teardown path stops it while HOLDING that serializer: if_age.c:2524 ASSERT_SERIALIZED(ifp->if_serializer); if_age.c:2534 callout_stop(&sc->age_tick_ch); callers: age_ioctl SIOCSIFFLAGS (:1673-1674), age_init (:2242+:2249), age_detach (:686-688), age_suspend (:1393-1394).
• DragonFly's callout_stop is synchronous: kern_timeout.c:1074-1077 → _callout_cancel_or_stop(..., sync=1) → ssleep(c, &c->spin, 0, "costp", 0) at kern_timeout.c:923, waiting for the INPROG callback to finish.
• AB-BA: a callback dequeued and blocked in lwkt_serialize_enter (:2194) waits for the serializer the stopper holds; the stopper sleeps in callout_stop ('costp') waiting for that callback. Neither progresses.

Replica proof (this run, guest 6.5-DEVELOPMENT of 2026-07-02)

age_replica: loaded (hz=100)
age_replica: armed age_tick replica (non-mp callout, ticks hz/10)
age_replica: tick handler BLOCKED on our serializer (fired=14 ran=13) - entering callout_stop()
age_replica: IF THIS IS THE LAST age_replica LINE, AB-BA DEADLOCK PROVEN (DF-3715 family / age instance)
[silence — callout_stop never returned]
$ ps -ax -o pid,stat,wchan,comm | grep costp
1001 D5     costp    sysctl            (still D5/costp 30s later)

Additionally the wedge cascaded: the observing ssh session itself timed out (callback CPU softclock starvation), and the guest could not complete a clean vm.sh down afterwards (qemu had to be killed before snapshot revert) — i.e., not merely two stuck threads but system-wide degradation, matching the DF-3715 impact analysis ("the affected CPU's softclock thread stops delivering all callouts").

Triggers on real hardware

ifconfig age0 down / up, MTU change on a running age interface, kldunload if_age / device detach, suspend — colliding with the 1-second tick. Narrow per-call window, repeatable; admin-gated (as with the em instance).

Impact

Permanent wedge of the calling thread + a softclock thread + the interface serializer; progressive system degradation. DoS, Medium (family-consistent).

Fix

Same shape as the em fix: try-enter in age_tick with a 1-tick retry (fix.diff) so the callback can always complete and callout_stop can always return.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.