# DF-2813 — VERDICT

**Status: reproduced (deterministic, 3/3 runs).  Impact: local DoS
(unkillable thread / permanent M_WAITOK block).  Confidence: certain.**
**Fix: validated on a rebuilt guest kernel (#1) — bad behavior gone.**

## Root cause (line-accurate)

* `objcache_get()` exhaustion + `M_WAITOK` path:
  sys/kern/kern_objcache.c:547-555 —
  `ssleep(depot, &depot->spin, 0, "objcache_get", 0)`.
  flags=0 ⇒ no `PCATCH` (tsleep is not signal-interruptible, the victim
  process cannot be killed); timo=0 ⇒ no timeout.
* Hot-path wakeup gating: sys/kern/kern_objcache.c:647-648 and :660-661 —
  `if (cpucache->waiting) wakeup_mycpu(&oc->depot[...])` where `cpucache`
  is **the putter's** per-cpu structure (objcache_put:622).  A remote
  sleeper set `cache_percpu[A]->waiting` (objcache_get:548), which the
  putter on CPU B never checks; `wakeup_mycpu` (sys/kern/kern_synch.c:1189-1193)
  only scans the current CPU's sleep queue anyway.
* Objects put on the hot path land in the putting CPU's magazine
  (objcache_put:646, :659) and are invisible to other CPUs.  They reach the
  depot — the only place a remote sleeper can find them
  (objcache_get:484-497) — only when a **full** magazine is cycled
  (objcache_put:682-703, `wakeup(depot)` at :700-701), i.e. after
  2 × mag_capacity objects accumulate on one CPU.
* The magazine rebalance callout that would eventually drain partial
  magazines is disabled: sys/kern/kern_objcache.c:1029-1034 (`#if 0`).
* There is no other wakeup source: `objcache_dtor` (:734), reclaim
  (:886-887, :896-897) and `objcache_set_cluster_limit` (:371) all require
  depot-level events that never happen while objects sit in a remote
  partial magazine.

Net: thread on CPU A sleeps forever although objects are free and cached on
CPU B.  Verified interlocked-sleep correctness elsewhere (ssleep's
spin-interlock at kern_synch.c:815-828 rules out same-visibility-window lost
wakeups; the hole is purely the per-cpu-magazine hysteresis).

## Reproduction narrative

Deterministic KLD harness (private cache: cluster_limit=8, mag_capacity=6,
`unallocated_objects = 6 cpus × 6 × 2 + 8 = 80`):

1. ARM: `objcache_get(oc, M_NOWAIT)` ×80 → exhausted (`nheld=80`, magcap
   derived = 6 — matches kern_objcache.c:296-301 accounting exactly).
2. SLEEPER: kernel thread pinned to cpu5 (`lwkt_create(..., cpu=5)`) calls
   `objcache_get(oc, M_WAITOK)`.  dmesg: `Warning: objcache(ochang)
   exhausted on cpu5!` (kern_objcache.c:540).  Thread state: `TDF_TSLEEPQ=1
   wmesg="objcache_get"` — inside the ssleep.
3. STRAND: 6 puts (= exactly one magazine) on cpu0 via IPI.  Observation
   over 10 s: **STUCK for 1102 ticks (11 s)**.
4. Behavioral proof of stranding: `PROBE cpu0` runs
   `objcache_get(oc, M_NOWAIT)` on cpu0 → **OBJECT AVAILABLE** (hot path
   pops from cpu0's loaded magazine before any exhaustion check,
   kern_objcache.c:445-450); `PROBE cpu5` → **NULL (exhausted)**.  Same
   cache, same instant, opposite outcomes on different CPUs.
5. RESCUE (positive control): 12 more puts on cpu0 → both magazines fill →
   depot cycle → `wakeup(depot)` → **sleeper acquired 0.000 s after
   RESCUE** (latency=1102 ticks total, i.e. it woke the instant the
   magazine was delivered).  This proves the wakeup machinery itself works
   and the defect is precisely the delivery path.

Runs: run.log, run.2.log, run.3.log — identical results, 3/3.

## Impact assessment

* Local DoS.  Unprivileged reachability: `sosend_generic()` →
  `m_getl(..., M_WAITOK, ...)` (sys/kern/uipc_socket.c:1024; also :866,
  :1164) — a blocking `write(2)`/`send(2)` on any socket blocks inside
  `objcache_get(M_WAITOK)` when the mbuf caches are exhausted
  (sys/kern/uipc_mbuf.c:797-851 limits; guest nmbclusters=33296 ≈ 66 MB of
  clusters / ~16 k+maxfiles mbufs — exhaustible by an unprivileged user
  through socket buffers).  `m_get`'s reclaim-retry
  (sys/kern/uipc_mbuf.c:1020-1032) never executes because pure M_WAITOK
  never returns NULL — it sleeps inside objcache.
* The sleeping thread is not signal-interruptible: the owning process
  becomes unkillable; repeated attacks strand threads/fds/memory.
* Not a memory-safety bug; no escalation path identified (the sleep holds
  no spinlocks at the ssleep — ssleep releases the depot spin — and kernel
  callers do not hold critical tokens across M_WAITOK mbuf allocation
  paths in the audited call sites).

## Fix validation

* `fix.diff`: in `objcache_put()`, before the hot-path push, if the depot
  has waiters (`depot->waiting`) and there is no *local* waiter (the
  existing `wakeup_mycpu` covers that case) and the loaded magazine is
  non-empty, flush the loaded magazine to the depot full list (swapping in
  an empty magazine) and `wakeup(depot)`.  Decision re-checked under the
  depot spinlock; unlocked `waiting` probe is benign (skip-or-flush).
* Baseline kernel #0 (stock, `Thu Jul 2 06:02:54 UTC 2026`): sleeper stuck
  11 s with objects available on cpu0 (run.log/run.2.log/run.3.log).
* Patched kernel #1 (`Tue Sep  1 16:06:47 UTC 2026`, built in-guest with
  `make -j6 nativekernel KERNCONF=X86_64_GENERIC` + installkernel):
  **sleeper woke 0.25 s after STRAND** (first driver poll tick after the
  flush; `latency=100 ticks` = woke at strand time), no hang; both probes
  OBJECT AVAILABLE; DRAIN/DESTROY/kldunload clean; the whole SSH-driven
  run exercised the mbuf objcache paths on the patched kernel without
  incident (run.patched.log).
* Guest reset to the clean `with-src` snapshot afterwards (`vm.sh reset
  with-src`), stock kernel #0 back, guest up.

## Why the other suspected issues in this file are NOT this finding

* Same-CPU putter vs sleeper: covered by `cpucache->waiting` +
  `wakeup_mycpu` — correct.
* Depot-level deliveries (full mag cycle, no-empty-mag free, dtor,
  reclaim, set_cluster_limit): all `wakeup(depot)` — correct, and proven
  by the RESCUE control.
* DF-0064 (stats race on total_objects) — unrelated, stats-only, known.

## Artifacts

ochang.c, ochang_drv.c, build.sh, run.sh, build.log, run.log, run.2.log,
run.3.log, run.patched.log, env.txt, fix.diff, manifest.json, verdict.json