# DF-2813 — objcache_get() M_WAITOK lost wakeup: objects stranded in remote per-cpu magazines cause a permanent, unkillable sleep

## What

`objcache_get()` (sys/kern/kern_objcache.c:547-555) parks an exhausted
`M_WAITOK` caller in `ssleep(depot, &depot->spin, 0, "objcache_get", 0)` —
flags `0` (no `PCATCH`, not signal-interruptible) and timeout `0` (infinite).

An `objcache_put()` on the hot path (kern_objcache.c:644-651, 656-664) only
wakes sleepers **on the putting CPU** (`wakeup_mycpu`, gated on the putter's
own `cpucache->waiting`).  Objects pushed into a per-cpu magazine are
invisible to every other CPU.  They only become globally visible when a full
magazine is cycled into the depot (kern_objcache.c:682-703, which does
`wakeup(depot)`), i.e. only after **2 × mag_capacity** objects accumulate on
one CPU.  The magazine rebalance callout that would eventually drain partial
magazines is compiled out (`#if 0`, kern_objcache.c:1029-1034).

Consequence: a thread on cpu A sleeps forever inside `objcache_get()` while
freed objects sit in cpu B's partially-filled magazines.  Nothing in the
kernel ever delivers them.  The sleep has no timeout and no signal
interruption — a user process stuck here is **unkillable** (`kill -9` sets
the flag; the thread never returns to userret).

## Reproduce

On the audit QEMU guest (6-cpu DragonFly 6.5-DEVELOPMENT, stock INVARIANTS
kernel #0):

```
./build.sh          # on guest: builds ochang.ko + ochang_drv (as maxx, in ~/ochang)
sudo sh run.sh      # kldload ./ochang.ko ; ./ochang_drv ; dmesg tail ; kldunload
```

The KLD creates a private objcache (cluster_limit=8, mag_capacity=6,
unallocated = 6·6·2+8 = 80), then:

1. **ARM** — drain with `M_NOWAIT` gets: exactly 80 objects held.
2. **SLEEPER** — kernel thread pinned to cpu5 calls
   `objcache_get(oc, M_WAITOK)` → `Warning: objcache(ochang) exhausted on
   cpu5!` → sleeps on the depot.  `wmesg="objcache_get"`, `TDF_TSLEEPQ=1`.
3. **STRAND** — free exactly 6 objects (one magazine) on cpu0 via IPI.
4. Observe: sleeper still **STUCK for 1102 ticks (11 s)** while
   **PROBE cpu0 → OBJECT AVAILABLE** (an `M_NOWAIT` get on cpu0 succeeds
   from its loaded magazine, ahead of any exhaustion check) and
   **PROBE cpu5 → NULL**.  Same cache, same instant: the objects exist and
   are reachable from cpu0 but the sleeper cannot get them.
5. **RESCUE** (positive control) — free 12 more on cpu0: both magazines
   fill, the depot cycle deposits a full magazine, `wakeup(depot)` fires →
   **sleeper acquired 0.000 s after RESCUE**.

Reproduced 3/3 runs (run.log, run.2.log, run.3.log), deterministically.

## Expected output (bug present)

```
sleeper: started=1 got=0 STUCK for 1102 ticks (11 s), TDF_TSLEEPQ=1 wmesg="objcache_get"
ochang: PROBE cpu0 -> OBJECT AVAILABLE
ochang: PROBE cpu5 -> NULL (exhausted)
=== RESCUE ... ===
sleeper acquired 0.000 s after RESCUE
```

## Unprivileged reachability (no KLD needed in principle)

Every cluster-limited cache is reachable from an unprivileged user with
blocking syscalls: `sosend_generic()` allocates with `m_getl(..., M_WAITOK,
...)` (sys/kern/uipc_socket.c:1024, also :866, :1164), i.e. a blocking
`write(2)`/`send(2)` on any socket bottoms out in `objcache_get(...,
M_WAITOK)`.  The mbuf caches are limited by `nmbufs`/`nmbclusters`
(sys/kern/uipc_mbuf.c:797-851); on this guest `nmbclusters=33296` (~66 MB of
2 KB clusters, or ~16 k mbufs) — exhaustible by an unprivileged user through
socket buffers across many sockets.  Freeing fewer than 2×magcap objects on
a CPU different from the blocked one (CPU placement is not under user
control but is statistically spread across 6 CPUs and repeatable at will)
strands them; each attempt that lands cross-CPU wedges one thread of a
victim process permanently.

## Fix

`fix.diff` (validated — see VERDICT.md): when the depot has remote waiters,
`objcache_put()` flushes its partially-filled loaded magazine to the depot
(making the objects globally visible and waking the sleeper) before caching
the new object.  Zero behavior change when nobody waits.

## Files

* `ochang.c`      — KLD driver (the PoC)
* `ochang_drv.c`  — userspace orchestrator (pins to cpu2)
* `build.sh` / `run.sh`
* `build.log`, `run.log`, `run.2.log`, `run.3.log`, `run.patched.log`
* `env.txt`, `fix.diff`, `manifest.json`, `verdict.json`