DF-4051 — nfe_stop() vs nfe_tick() callout AB-BA deadlock (nfe(4) instance of the DF-3715/4025/4046 family)

What

sys/dev/netif/nfe/if_nfe.c:

• nfe_attach arms one callout with plain non-MP callout_init(&sc->sc_tick_ch) (if_nfe.c:636).
• nfe_tick (if_nfe.c:2181-2193) blocks acquiring the driver serializer (lwkt_serialize_enter, :2187), does mii_tick, then re-arms itself UNCONDITIONALLY while holding it (callout_reset(&sc->sc_tick_ch, hz, nfe_tick, sc), :2190 — no IFF_RUNNING gate).
• nfe_stop (if_nfe.c:1585-1640) issues the synchronous callout_stop(&sc->sc_tick_ch) (:1593) as its first action after ASSERT_SERIALIZED (:1591) — i.e. while holding that serializer.
• DragonFly callout_stop() is synchronous (sys/kern/kern_timeout.c:1091-1093 → _callout_cancel_or_stop(..., sync=1); the ssleep "costp" wait at kern_timeout.c:920): it waits for an INPROG callback to finish.

A nfe_tick dequeued by a softclock thread and parked in lwkt_serialize_enter therefore deadlocks against the stopper: both threads wedge permanently, the interface serializer is never released (every later ioctl / if_start on nfeX blocks), and the affected softclock stops delivering all callouts on that CPU.

Reachable from every down/detach/reinit path racing the 1 Hz tick: nfe_detach :669-672, nfe_shutdown :705-708, nfe_suspend :716-719, nfe_ioctl SIOCSIFFLAGS :1025-1027, and nfe_init :1446 (nfe_stop is its first statement) — itself reachable from SIOCSIFMTU :1005-1007, SIOCSIFCAP :1049-1051, nfe_resume :730-732, nfe_watchdog :1431.

Reproduce (guest, root)

scp -F dfbsd-qemu/config nfereplica.c Makefile dfbsd:/root/nfereplica/
ssh -F dfbsd-qemu/config dfbsd 'cd /root/nfereplica && make'
kldload /root/nfereplica/nfereplica.ko
sysctl kern.nfereplica.arm=1      # arm the nfe_tick replica (1-tick period)
sleep 2
(sysctl kern.nfereplica.stop=1 &) # nfe_stop shape: NEVER returns
sleep 6; dmesg | grep nfereplica | tail -4
ps -o pid,stat,wchan,command -ax | grep -E "stop=1|softclock" | grep -v grep

Expected output (observed, run.log + serial_console.txt)

nfereplica: nfe_tick parked on our serializer (fired=201 ran=200) - entering callout_stop()
nfereplica: IF THIS IS THE LAST nfereplica LINE, AB-BA DEADLOCK IS PROVEN
...
   974 D2     costp    sysctl kern.nfereplica.stop=1     <- kern_timeout.c:920
    -1 B3     slize    (softclock 3)                     <- lwkt_serialize_enter

The stopper sits in costp across ≥2 samples minutes apart (permanent), the console ends at the marker line, and the guest cannot shut down cleanly — vm.sh reset with-src is the documented recovery.

Hardware gating / family convention

The guest has no NVIDIA nForce MCP NIC (pciconf -l shows none; QEMU does not emulate forcedeth), so the in-driver instance cannot be triggered at runtime; per the DF-4025/DF-4046 family convention the persisted status is inconclusive with the mechanism PROVEN by this deterministic replica of if_nfe.c's exact callout/serializer protocol.

DF-4051 VERDICT — nfe_stop() vs nfe_tick() callout AB-BA (mechanism PROVEN on guest; nfe runtime hardware-gated)

Reproduced?

Mechanism: PROVEN on the guest kernel by a deterministic replica (nfereplica.ko) of if_nfe.c's exact callout/serializer protocol — plain non-MP callout_init (if_nfe.c:636), tick handler that BLOCKS in lwkt_serialize_enter (:2187) and re-arms itself unconditionally while holding the serializer (:2190), and a stop that issues the synchronous callout_stop() under the serializer as its first action (:1591-:1593).

nfe runtime: hardware-gated (guest has no nForce MCP NIC, env.txt; QEMU does not emulate forcedeth), so per family convention (DF-4025, DF-4046) the persisted status is inconclusive.

Observed on the guest (run.log + serial_console.txt)

1. Module built: BUILD_RC=0 (build.log, -Werror clean).
2. Armed; the tick fired 201 times, ran 200 (re-arm loop working, period 1 tick vs the real driver's hz).
3. nfe_stop-shaped stop under the serializer: the 201st firing parked in lwkt_serialize_enter; console ends at IF THIS IS THE LAST nfereplica LINE, AB-BA DEADLOCK IS PROVEN — callout_stop() never returned.
4. Cross-CPU AB-BA visible in ps: stopper pid 974 D2 wchan costp (ssleep(c, &c->spin, 0, "costp", 0), sys/kern/kern_timeout.c:920, reached from callout_stop → _callout_cancel_or_stop(..., sync=1) at kern_timeout.c:1091-1093) vs (softclock 3) blocked at wchan slize (lwkt_serialize_enter) — each holds what the other needs, forever. Two samples minutes apart show the identical pair (permanent).
5. An early replica revision (v1, tsleep-based park-wait) wedged even harder: the stopper's tsleep timeout is delivered by the same softclock the tick parks, so the stopper starved in its own wait (D4/nfetkwt while softclock 4/slice) — additional evidence of the CPU-wide callout starvation this protocol causes. v2 (DELAY-based park-wait, shipped here) isolates the true block point at callout_stop()/'costp'.
6. The observing ssh session's stream froze mid-script (its shell lived on the wedged CPU's timers); the guest could not shut down cleanly — vm.sh reset with-src recovered it (healthy afterwards, env.txt).

Root cause (path:line)

• if_nfe.c:636 — callout_init(&sc->sc_tick_ch) (non-MP variant).
• if_nfe.c:2181-2193 — nfe_tick: lwkt_serialize_enter at :2187 (BLOCKING acquire), mii_tick :2189, then unconditional callout_reset(&sc->sc_tick_ch, hz, nfe_tick, sc) at :2190 under the serializer (no IFF_RUNNING gate), exit at :2192.
• if_nfe.c:1585-1640 — nfe_stop: ASSERT_SERIALIZED :1591 then callout_stop(&sc->sc_tick_ch) :1593 as the FIRST teardown action while holding the serializer.
• Callers of nfe_stop all hold the serializer: nfe_detach :669-672, nfe_shutdown :705-708, nfe_suspend :716-719, nfe_ioctl SIOCSIFFLAGS :1025-1027, nfe_init :1446 (first statement; reachable from SIOCSIFMTU :1005-1007, SIOCSIFCAP :1049-1051, nfe_resume :730-732, nfe_watchdog :1431).
• sys/kern/kern_timeout.c:1091-1093 — synchronous callout_stop (_callout_cancel_or_stop(..., sync=1)), :920 — the costp wait.

Threat

Local DoS: any ifconfig down/up/MTU/cap-change (or detach/suspend/watchdog) on an nfe(4) interface racing the 1 Hz tick wedges two kernel threads permanently and starves one CPU's callout subsystem (system-wide timer starvation for threads on that CPU). No privilege boundary crossed.

Fix

lwkt_serialize_try + retry pattern (identical to the accepted DF-3715 / DF-4046 family fix): never block in the tick handler.

Fix validation

Not testable in-driver on this guest (no nfe hardware); the fix.diff applies the same try-enter shape proven to unload cleanly in the DF-3715/DF-4046 negative controls.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.