# VERDICT — DF-4260

**Status: reproduced (deterministic). Impact: dos (permanent kernel deadlock).**
Guest: DragonFly 6.5-DEVELOPMENT #0 Thu Jul 2 2026, x86_64, hw.ncpu=6
(X86_64_GENERIC), fully updated audit snapshot.

## What was verified

The vr(4) deadlock pattern was reproduced **deterministically** on the
shipping guest kernel with a kld module (`poc_deadlock.c`) that replicates the
exact code shape of if_vr.c — a callout callback that (1) re-arms itself and
(2) `lwkt_serialize_enter()`s a serializer, plus an administrative path that
holds that same serializer and calls the (synchronous) `callout_stop()`:

1. `kldload poc_deadlock.ko`
2. `sysctl kern.poc_deadlock=2` — **fixed pattern** (`callout_stop_async()`
   under the serializer + `callout_drain()` outside, exactly the remedy in
   fix.diff): returns cleanly, dmesg `fixed pattern completed, state=2`.
3. `sysctl kern.poc_deadlock=1` — **bugged pattern** (synchronous
   `callout_stop()` under the serializer, i.e. what `vr_stop()` at
   sys/dev/netif/vr/if_vr.c:1630 does on every `ifconfig vrX down` /
   `kldunload` / interrupt / watchdog path): **hangs forever**.
   - The `sysctl` process is stuck in state `D3`, wait channel **`costp`** —
     the `ssleep(c, &c->spin, 0, "costp", 0)` in `_callout_cancel_or_stop()`
     (sys/kern/kern_timeout.c:920). Unkillable (sleep has no PCATCH);
     `timeout(1)` itself blocks forever in `wait()` (pid 1704 `pause`).
   - The softclock thread of cpu4 (where the tick was INPROG) is wedged
     acquiring the serializer held by the sleeping stopper. Witness: a
     `usched_set`-pinned `sleep 3` returns in 3s on cpu0,1,2,3,5 but **never
     wakes on cpu4** (rc=124 only via timeout's SIGTERM, run.2.log) — cpu4's
     timer wheel is dead for the life of the boot.
   - Collateral: any later write to the same sysctl OID blocks forever on
     `oidlk` (pid 1735, `D5`) because the stuck writer holds the OID lock.

## Why it happens (code trace)

- `vr_tick()` starts with `lwkt_serialize_enter(ifp->if_serializer)`
  (sys/dev/netif/vr/if_vr.c:1165) and re-arms itself (if_vr.c:1178).
- softclock marks the callout `CALLOUT_INPROG` *before* invoking the callback
  (sys/kern/kern_timeout.c:566-568), so a callback blocked on the serializer
  keeps INPROG set.
- `vr_stop()` calls `callout_stop(&sc->vr_stat_timer)` (if_vr.c:1630) and is
  *always* called with `ifp->if_serializer` held: `vr_detach` (if_vr.c:822-826),
  `vr_ioctl(SIOCSIFFLAGS)` — ifioctl dispatches `if_ioctl` under
  `ifnet_serialize_all()` (sys/net/if.c:2168-2170) —, `vr_intr` (if_vr.c:1195),
  `vr_watchdog` (if_vr.c:1609), `vr_init` (if_vr.c:1408).
- On this kernel `callout_stop()` is **synchronous**
  (`_callout_cancel_or_stop(cc, CALLOUT_STOP, 1)`, sys/kern/kern_timeout.c:1091-1094):
  with INPROG set and the caller not being the softclock thread it
  `ssleep("costp")`s (kern_timeout.c:910-920) **while the caller still holds
  the lwkt serializer the callback needs**. lwkt serializers have no owner
  recursion/wakeup path for a sleeping holder (sys/kern/lwkt_serialize.c:105-111,
  adaptive spin then sleep) → neither side can ever progress.
- The documentation block directly above `callout_stop()` still claims it
  "does not block / does not interlock against a callout that is in-progress"
  (kern_timeout.c:1086-1093) — the code contradicts it; drivers were written
  against the documented (async) semantics. `callout_stop_async()`
  (kern_timeout.c:1060-1063) preserves the documented behavior.

SMP is required (the callback must be dequeued by another CPU's softclock
while the stopper holds the serializer) — satisfied on any multi-core box.

## vr(4)-specific reachability

- `ifconfig vrX down` (root) → vr_ioctl → vr_stop under serializer — 1 Hz
  tick race window of ~µs per attempt (statistical, ~1e-6..1e-4/s while
  hammering; the vr hardware itself could not be exercised — QEMU 7.2 has no
  via-rhine PCI device).
- `kldunload if_vr` (root) → vr_detach — same.
- `vr_intr` → `vr_stop` when IFF_UP was cleared (if_vr.c:1194-1197): the
  synchronous stop is then executed from the **interrupt thread**, sleeping
  in interrupt context while holding the serializer — the nastiest variant.
- `vr_watchdog` (TX stall) → vr_stop under serializer (if_vr.c:1609).
- The identical `tick-takes-serializer` / `stop-under-serializer` shape exists
  in other DFly netif drivers (e.g. sys/dev/netif/rl/if_rl.c:1163 vs 1587),
  so this is a systemic hazard; fixing kern_timeout.c semantics (or auditing
  all drivers) is the upstream-scale remedy. This finding is filed for
  if_vr.c per audit scope.

## Fix validation

- `fix.diff` (vr_stop → `callout_stop_async()`; vr_detach → `callout_drain()`
  moved outside the serializer, before children/resources are torn down):
  applied to the guest's /usr/src copy, `if_vr.ko` builds cleanly
  (`PATCHED_BUILD_OK`); guest tree reverted afterwards.
- The remedy's runtime semantics were validated live on the same kernel by
  module **mode 2** (identical call sequence): completes cleanly, callback
  drained, no wedge, state=2.
- The real vr(4) hardware path cannot be run (no via-rhine emulation in
  QEMU 7.2) → fix_status `not_testable` at the driver level, with
  compile + pattern validation recorded here.

## Honest limits

- The vr(4) device itself was not attached (no QEMU model); the reproduction
  is the exact code pattern on the exact kernel primitives, not the NIC irq
  path. All code-level claims carry path:line citations.
- The deadlock is a hang, not memory corruption: no escalation attempted;
  impact ceiling is permanent local DoS (privileged trigger).

## Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest `make -j6 nativekernel KERNCONF=X86_64_GENERIC` completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.