# DF-3806 — msk(4) msk_stop()/msk_tick() AB-BA callout deadlock (DF-3715 family)

## Verdict

**Mechanism PROVEN on the guest kernel by a deterministic replica of
if_msk.c's exact tick/stop callout protocol; msk-specific runtime
hardware-gated (inconclusive per family convention — the guest has no
Marvell Yukon hardware).** Severity Medium, confidence certain.
This is the DF-3715 (if_em.c) / DF-3738 (if_emx.c) / DF-3760 (if_alc.c) /
DF-3776 (if_jme.c) family — a fifth, independent instance, first filed
for this file (pass 1 marked the file clean; there were no pass-1
findings to exclude).

## Root cause (path:line)

- `msk_attach()` arms a **non-MPSAFE** callout: `callout_init(&sc_if->msk_tick_ch)`
  (sys/dev/netif/msk/if_msk.c:1375) — unlike the alc(4)/em(4) twins this
  callback also runs under the MP lock.
- `msk_tick()` (if_msk.c:2930-2946) **blocks** in
  `lwkt_serialize_enter(ifp->if_serializer)` (:2936), does the MII work,
  then **self re-arms at hz** (:2943) and exits the serializer (:2945).
- `msk_stop()` (if_msk.c:3675-3823) runs **under that same serializer**
  (`ASSERT_SERIALIZED`, :3687) and tears the tick down with
  `callout_stop(&sc_if->msk_tick_ch)` (:3689).
- DragonFly's `callout_stop()` is the **synchronous** variant
  (sys/kern/kern_timeout.c:1091 → `_callout_cancel_or_stop(..., CALLOUT_STOP,
  sync=1)`); when the callback is INPROG it `ssleep()`s on `"costp"`
  (kern_timeout.c:923) until the callback completes.

If `msk_tick` is INPROG and parked inside `lwkt_serialize_enter` on the
serializer held by the `msk_stop()` caller, neither thread can progress:
**permanent AB-BA deadlock.** Extras vs. the family: the parked callback is
non-MPSAFE, so the wedged softclock thread also holds the **MP lock**, and
its CPU stops delivering *all* callouts (observed live: an unrelated
`sleep 6` process starved because its nanosleep timeout lived on the wedged
softclock's wheel).

## Callers that reach msk_stop() under the serializer

- `msk_init()` :3352 — from `msk_ioctl` SIOCSIFFLAGS up/down (:880-881, :895),
  and `msk_watchdog()` (:2663, :2686) — TX-timeout reinit
- `msk_detach()` :1762-1764 — detach / kldunload if_msk
- `mskc_shutdown()` :2699-2701, `mskc_suspend()` :2719-2722,
  `mskc_resume()` → `msk_init` :2756

Trigger on real hardware: any `ifconfig mskN down`, re-up, detach, suspend
or TX-timeout reinit that races the 1 Hz tick — a privileged local user
flaps the interface and can retry indefinitely; watchdog-triggered reinits
can occur from unprivileged TX activity on a flapping link. Race window
~0.1-1% per stop (family-measured); consequences are permanent.

## Guest proof (deterministic replica, no msk hardware needed)

`msk_callout_replica.ko` replicates the protocol exactly (non-MPSAFE
`callout_init`, blocking enter, hz self re-arm, sync `callout_stop` under
the serializer) and exposes two shapes:

- **fix-shape control** (`lwkt_serialize_try` + 1-tick retry = this pack's
  fix.diff): `mskrep[try]: callout_stop returned, pending=0` — validated
  live, twice.
- **shipped-shape** (blocking enter): park caught after 99 ticks
  (`fired=4 ran=3`), then:
  - `1045 D0 costp sysctl kern.mskrep.stop_blk=1` — stopper permanently
    inside `callout_stop`'s wait (kern_timeout.c:923)
  - `softclock 1 B1 slize` — the msk_tick-shaped callback permanently
    parked in `lwkt_serialize_enter` (if_msk.c:2936 shape)
  - bystander `sleep 6` (`I1 nanslp`) starved — softclock blackout cascade

An earlier orchestration variant (tsleep-based park detection) wedged one
step earlier — the stopper's own sleep timeout was delivered by the very
softclock thread the parked callback took out — an additional cascade
unique to the non-MPSAFE msk shape (see run.log history / VERDICT notes).

Files: `run.log` (session transcript), `wedge.log` (dmesg + ps capture of
the D0/costp + B1/slize pair), `build.log`, `msk_callout_replica.c`,
`fix.diff`, `env.txt`.

## Fix

`fix.diff`: make `msk_tick()` non-blocking — `lwkt_serialize_try()` with a
1-tick `callout_reset` retry when contended (DF-3715 fix pattern). Mechanism
validated live on this kernel via the replica's fix-shape control. A
driver-level build/boot validation requires Yukon hardware (not available
on the audit guest), hence fix_status=not_testable.

## Kernel references

sys/dev/netif/msk/if_msk.c:1375, :2930, :2936, :2940-2943, :2945, :3352,
:880-881, :1762-1764, :2663, :2686, :2699-2701, :2719-2722, :2756,
:3687, :3689; sys/kern/kern_timeout.c:923, :1091.

## Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest `make -j6 nativekernel KERNCONF=X86_64_GENERIC` completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.