β¬’ DragonFlyBSD Kernel Audit
← triage Β· dashboard
DF-4057

et_stop() calls synchronous callout_stop() while holding the driver serializer; et_tick() must lwkt_serialize_enter() that same serializer before exiting and re-arms unconditionally β€” AB-BA deadlock of the callout subsystem against the tick (DF-3715/4025/4046/4051 family instance; mechanism already PROVEN on this kernel for em/re/stge/nfe)

Summary

et_stop runs under ASSERT_SERIALIZED and issues the synchronous callout_stop; et_tick starts with a blocking lwkt_serialize_enter and re-arms itself before exiting. A tick mid-flight while a stopper holds the serializer parks both forever; the softclock CPU is additionally starved (INPROG slot). Every SIOCSIFFLAGS down/MTU/init/watchdog/detach/shutdown is a stopper. Identical structure to the four proven siblings; et also re-arms unconditionally like stge/nfe. callout_stop_async + IFF_RUNNING-gated re-arm + final sync stop outside the serializer at detach.

Discussion (0)

No comments yet.

PoC verification

Evidence pack

findings/poc/DF-4057 Β· 14 files
FileTypeDescriptionSize
dfet_poc.c β€” 32.0 KB view raw
Makefile β€” 103 B ↓ download
build.sh β€” 91 B view raw
run.sh β€” 104 B view raw
s4_run.sh β€” 1.4 KB view raw
run.log β€” 856 B view raw
deadlock_console.txt β€” 757 B view raw
fix.diff β€” 3.6 KB view raw
fix_build.log β€” 1.1 KB view raw
fix_run_s4.log β€” 473 B view raw
VERDICT.md β€” 4.2 KB ↓ raw
env.txt β€” 1.3 KB view raw
verdict.json β€” 4.5 KB view raw
manifest.json β€” 851 B view raw
VERDICT.md
↓ download raw

VERDICT β€” DF-4057 (et_stop sync callout_stop under serializer vs et_tick blocking acquire)

Status: reproduced β€” in the production handlers themselves (et_stop/et_tick resolved from /boot/kernel/kernel and dispatched; strictly stronger than the DF-3715 em / DF-4025 re / DF-4046 stge / DF-4051 nfe family verifications, which used protocol replicas) β€” and fix validated on a rebuilt kernel.

Mechanism (all path:line verified on this kernel)

  • et_stop() runs under ASSERT_SERIALIZED(ifp->if_serializer) (:553) and its FIRST action is the synchronous callout_stop(&sc->sc_tick) (:555). Sync semantics: _callout_cancel_or_stop(..., sync=1) β†’ ssleep(c, &c->spin, 0, "costp", 0) at sys/kern/kern_timeout.c:920 β€” waits for an INPROG callback to finish.
  • et_tick() starts with the BLOCKING lwkt_serialize_enter (if_et.c:2070) and re-arms itself UNCONDITIONALLY before exiting (:2080).
  • Every production stopper holds the serializer while calling et_stop: et_detach:389-390, et_shutdown:427-428, et_ioctl SIOCSIFFLAGS down (:1140/:1154), and et_init:1093 (whose first statement is et_stop β€” reachable from SIOCSIFMTU :1183 and the watchdog :1268).

What ran (baseline, stock kernel #0)

  1. callout_init(&sc->sc_tick) (as et_attach:244), real serializer, fake-mii softc; armed the real et_tick via callout_reset(sc_tick, 1, p_et_tick, sc) (production arms at hz via et_init:1121; 1 tick for determinism).
  2. Took the serializer (the stopper role), yield-polled until softclock dequeued the tick β€” CALLOUT_INPROG (kern_timeout.c:567) latched in 14 ms, the tick parked on lwkt_serialize_enter.
  3. Dispatched the real et_stop(&df_sc).

Console ends at the dispatch marker; et_stop never returned.

    -1 B1  slize   0.0 (softclock 1)           <- REAL et_tick parked in
1101  D1  costp   0.0 sysctl -w kern.dfet.go=4  <- REAL et_stop parked in
  • softclock 1 = that CPU's ENTIRE callout subsystem blocked inside the real et_tick (wchan slize), INPROG latched;
  • the stopper asleep in costp β€” the sync callout_stop wait inside the real et_stop;
  • permanent at +5 s resample; interface serializer never released; the sysctl caller unkillable (D-state, kernel);
  • guest could not be shut down cleanly afterwards (host force-killed QEMU), matching the proven family siblings.

Rig iteration notes (attempts)

  1. v1 armed the tick then busy-DELAY-polled for INPROG β€” never latched: a busy loop starves the same-CPU softclock thread (lwkt threads do not preempt each other), rig bug, no production conclusion drawn from it.
  2. v2 polls with DELAY(1000); lwkt_yield(); β€” deadlock reproduces in ~14 ms (run.log).
  3. v3 fixed-kernel run (below).

Fix validation (kernel #1, Oct 14 2026, same tree + fix.diff)

fix.diff breaks the blocking edge in et_tick: lwkt_serialize_try + skip-and-retry-next-tick instead of the blocking acquire, re-arm gated on IFF_RUNNING (et_stop clears it at :572). Identical rig on the rebuilt kernel:

scenario baseline (#0) fixed (#1)
stopper vs parked tick permanent AB-BA: D1/costp vs B1/slize, et_stop never returns tick no longer parks (try fails in Β΅s, returns; INPROG never latches); real et_stop RETURNED with the retry-tick chain live β€” S4: REAL et_stop RETURNED - NO DEADLOCK

fix_status: fixed (baseline_reproduced=1, patched_reproduced=0).

Impact

Permanent kernel deadlock from ordinary interface lifecycle activity (ifconfig down / MTU change / watchdog / detach / shutdown) racing the 1 Hz tick: two kernel threads wedged forever, the interface serializer never released, one CPU's callout subsystem (tsleep timeouts et al.) starved. Local DoS once the driver is live on hardware; family mechanism identical to the four already-proven siblings. Severity Medium stands.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.

Fix verification

fixed
baseline reproduced→ patch + rebuild →patched clean

Kernel #1 (same tree + fix.diff) A/B with the identical rig: baseline - permanent AB-BA deadlock (D1/costp stopper vs B1/slize softclock, et_stop never returns); patched - the tick no longer blocks on the serializer and the real et_stop RETURNED with the retry-tick chain live. Deadlock gone.

findings/poc/DF-4057/fix.diff; fix_run_s4.log; fix_build.log
↓ fix.diffper-fix-DF-4057

Confirmed kernel references

Detail

Exploit chain

On hardware: bring et(4) up (tick armed at 1 Hz), then any ifconfig down/MTU change/watchdog/detach racing a tick firing wedges two kernel threads forever (AB-BA), holds the interface serializer permanently (all later ioctls/if_start block) and starves one CPU's callout subsystem (tsleep timeouts undeliverable). Persistent local DoS; no memory-corruption primitive.

Evidence (decisive lines)

run.log (ps: '-1 B1 slize (softclock 1)' vs '1101 D1 costp sysctl -w kern.dfet.go=4'; two permanence samples; dmesg S4 markers); deadlock_console.txt (serial console ends at the AB-BA marker); fix_run_s4.log (patched kernel: 'REAL et_stop RETURNED - NO DEADLOCK'); VERDICT.md; env.txt

PoC changes

Authored fresh (pack was 'untested/hardware-gated'; family siblings used replicas - here the real handlers were dispatched). v1 busy-DELAY-polled for CALLOUT_INPROG and never latched it: a busy loop starves the same-CPU softclock thread (lwkt threads do not preempt each other) - rig bug. v2 polls with DELAY(1000);lwkt_yield() and reproduces in ~14 ms. Fixed-kernel v3: on the patched kernel INPROG cannot latch (try fails in microseconds) so the rig also dispatches the real et_stop in that path to prove completion.

Verified recommended fix

et_tick: lwkt_serialize_try + skip-and-retry-next-tick instead of blocking lwkt_serialize_enter; gate the re-arm on IFF_RUNNING (et_stop clears it) - breaks the AB-BA edge while keeping et_stop's sync stop safe

Verdict

REPRODUCED IN THE PRODUCTION HANDLERS THEMSELVES - stronger than the DF-3715(em)/DF-4025(re)/DF-4046(stge)/DF-4051(nfe) family verifications, which used protocol replicas: the guest kernel's OWN et_tick (armed on the softc's sc_tick via callout_reset, as et_init:1121 does) was parked on the driver serializer, and the kernel's OWN et_stop was dispatched under that serializer - the exact state of every production stopper (et_detach:389-390, et_shutdown:427-428, et_ioctl SIOCSIFFLAGS down :1140/:1154, et_init:1093 whose first statement is et_stop). et_stop's synchronous callout_stop (:555; _callout_cancel_or_stop sync=1, ssleep 'costp' kern_timeout.c:920) waited forever for the parked tick, whose first statement is the BLOCKING lwkt_serialize_enter (if_et.c:2070) and which re-arms UNCONDITIONALLY before exiting (:2080). Observed: console ends at the dispatch marker; ps shows softclock 1 in 'slize' (inside real et_tick) and the stopper pid 1101 in D1/'costp' (inside real et_stop); permanent at +5s resample; serializer never released; guest unshutdownable (host force-killed QEMU). FIX VALIDATED: kernel #1 rebuilt with fix.diff (et_tick: lwkt_serialize_try + skip-and-retry-next-tick, re-arm gated on IFF_RUNNING) - identical rig: the tick no longer parks (INPROG never latches) and the real et_stop RETURNED with the retry-tick chain live.