# DF-3017 weaponization forensics (QEMU gdbstub / DDB, stock kernel)

All observations from the stock INVARIANTS kernel (same `/boot/kernel/kernel`
as the standard guest), captured via a QEMU gdbstub (custom RSP client
`rsp.py`) and DDB over a bidirectional serial console.

## F1 — the victim dereferences a *reclaimed* cdev chunk (race reaches the forge sink)

Panic (weaponization rig v4, unpriv `maxx`):

```
Fatal user address access from kernel mode from dx at ffffffff8091cb98
Fatal trap 12: page fault while in kernel mode
fault virtual address = 0x6ad27326
instruction pointer   = 0xffffffff8091cb98   == devfs_allocv+0x1b8
Stopped at devfs_allocv+0x1b8: testb $0x80,0xe(%rax)
```

`devfs_allocv+0x1b8` is `sys/vfs/devfs/devfs_core.c:379`
`node->d_dev->si_ops->head.flags & D_KVABIO` — i.e. the racing victim
read `node->d_dev` (stale pointer into the freed cdev), passed
`dev_is_good()` and `sysref_get()` on the *reclaimed* chunk (the flags
word at +236 had SRF_PUTAWAY clear, so no assert), and then dereferenced
the reclaimed chunk's `si_ops`.  This proves the victim reaches the
forged-cdev sink with attacker-replacement content; with our weapon image
in the chunk (`si_ops` -> userspace fake dev_ops, no SMAP) this same path
completes silently and caches the poisoned vnode.

## F2 — kernel-resident weapon image, same slab zone as the racing cdev

Panic (rig v5, 6-cpu): `sysref_get` SRF_PUTAWAY assert through
`reference_dev <- v_associate_rdev <- devfs_allocv <- devfs_vop_nresolve`.
Frame `0xfffff80118573528`.  Stack-walking the frame and testing every
kernel-heap candidate for cdev shape (`si_name@+88 == "pts/*"`) found:

```
[!] CDEV  fffff801175536c0 name=pts/1000   si_ops=ffffffff810e5ea0(pts98_ops)
                                     sysref.flags@236 = 0x6 = SRF_ALLOCATED|SRF_PUTAWAY
[!] CHUNK fffff80117553800 si_ops=0000000040000000  sysref.flags@236 = 0
```

Interpretation:

* `0xfffff801175536c0` is the event cdev: still fully *stale* (live pts
  content, PUTAWAY set) -> the victim asserted.  This is the panic.
* `0xfffff80117553800` is the **immediately adjacent 320-byte chunk**
  (`0x5536c0 + 320 = 0x553800`) holding **our complete weapon image**:
  `si_ops = 0x40000000` (the userspace fake `struct dev_ops`), sysref
  flags PUTAWAY-clear.  The writev(20-iovec) reclaim vehicle demonstrably
  plants fully-formed fake cdevs into the same slab zone the racing cdevs
  live in.  The single missing step was filling *that particular chunk*
  before the victim's read (see F3).

## F3 — why the freed event chunk is not refilled in time: slab zone
        ownership is per-CPU and the devfs core frees on CPU 0

Zone header of both chunks (`struct SLZone`, `sys/sys/slaballoc.h`):

```
zone fffff80117400000 (4MB-aligned):
  z_Magic  = 0x736c6162 ('slab')
  z_Cpu    = 5                  <- zone owned by CPU 5, not CPU 0
  z_CpuGd  = fffff8008968d000
  z_NFree  = 1254  (z_NMax 1257)  <- >1k-chunk free backlog
```

Mechanics (`sys/kern/kern_slaballoc.c`):

* `kfree()` on a cpu that does not own the chunk's zone pushes the chunk
  onto `z_RChunks` (remote list) and, only under a narrow `RSignal`
  interlock, sends a passive IPI.  `kfree_remote` eventually runs
  `clean_zone_rchunks()` on the *owner* cpu, appending the remote chunks
  to the tail of `z_LChunks`.
* `kmalloc()` pops the head of `z_LChunks` of the *TAIL* zone in the
  allocating cpu's `ZoneAry`; locally-freed chunks are pushed to the head
  (LIFO) — a ping-pong allocator recycles the same chunk forever and
  never reaches appended remote chunks behind a 1000+-chunk backlog.

The devfs core thread (`devfs_msg_core`, verified running on CPU 0 via
`ps -ax -o cpu`) performs the cdev `objcache_put -> kfree` for every pty
destroy.  Event cdev chunks are carved by whichever cpu the *opening*
thread ran on; on the 6-cpu guest the event chunk's zone is owned by a
cpu != 0 with overwhelming probability, so the free is remote, invisible
behind the backlog, and the victim reads stale PUTAWAY content
(KKASSERT panic) ~9 times out of 9 observed events (0 wins in 12+
6-cpu driver attempts).

Local (cpu0-owned-zone) frees would be head-of-LIFO and intercepted by
any cpu0-running hammer within microseconds — the full chain is staged
for exactly that case — but unprivileged CPU pinning is unavailable:
`usched_set(USCHED_SET_CPU)` requires SYSCAP_NOSCHED_CPUSET
(sys/kern/kern_usched.c:230-234), and no cpuset(2) affinity API exists
for unprivileged users on this system.

## F4 — supporting measurements

* `sizeof(struct cdev) == 320` (guest-compiled offsetof program,
  `offsets.c`): shares the 320-byte slab zone with
  `writev(iovcnt=20)` iovec arrays (`iovec_copyin` ->
  `kmalloc(320, M_IOV)`, sys/kern/kern_subr.c:458).
* writev to a reader-less pipe returns EPIPE from `pipe_write()`'s
  `PIPE_WEOF` fast path (sys/kern/sys_pipe.c:749-755) *before* touching
  the uio, so the copied-in image survives in the chunk after kfree.
* CPU feature flags (dmesg): no SMAP, no SMEP (Features2 has only
  SSE3,CX16,x2APIC,VMM) -> kernel dereferences/calls of userspace fake
  `dev_ops`/shellcode pages are permitted on this guest.
* `/dev/ptmx` is 0666; every `open()+close()` creates and destroys a
  pts+ptm cdev/node pair (unpriv race driver).
* User processes *do* get scheduled on CPU 0 (`ps -axl` shows R0
  spinners), so cpu0-owned 320-zones exist once cpu0-running threads
  kmalloc that class; the event chunks simply do not land there without
  affinity control.

## F5 — SMP-count sensitivity experiments (same stock kernel)

* 6 vCPU (standard guest): events fire (~1 per run before the fatal
  stale-PUTAWAY panic); 0 local wins in 12+ driver attempts.
* 1 vCPU: all frees are local (no RChunks problem at all) but the race
  window collapses — victim and devfs core cannot run concurrently, the
  event rate falls to ~0; 420s runs produced neither panic nor win.
* 2 vCPU: events fire within seconds (first run panicked <60s in),
  locality odds ~1/2 — the sweet spot for composing the chain; see
  `run2cpu.log` for the retry-driver results.