# PoC DF-4528 — nvme(4) admin submission-queue lock-protocol violation

## What this is

`nvme_submit_request()` documents its contract as "Caller must hold the
queue lock" (nvme.c:509-513) and mutates shared submission-ring state
without any internal lock:

```c
cmd = &queue->ksubq[queue->subq_tail];       /* nvme.c:520 */
--queue->unsubmitted;                        /* nvme.c:521 */
if (++queue->subq_tail == queue->nqe) ...    /* nvme.c:522 */
```

The I/O path honors the contract (`nvme_strategy_core()` takes
`subq->lk` around the submit, nvme_disk.c:336-338), but **every
admin-queue submitter violates it** — `nvme_getlog_ioctl()`
(nvme_ioctl.c:56), `nvme_create_subqueue()` (nvme.c:800),
`nvme_create_comqueue()` (nvme.c:843), `nvme_delete_subqueue()`
(nvme.c:877), `nvme_delete_comqueue()` (nvme.c:902) and all the admin
thread's identify flows (nvme_admin.c:233, 319, 534, 598, 629) call
`nvme_submit_request()` with no lock held. `queue->unsubmitted` is
incremented under `queue->lk` in `nvme_get_request()` (nvme.c:480) but
decremented unlocked here, so concurrent submitters corrupt the ring
accounting, can double-write `ksubq[subq_tail]` (lost commands) and can
drive `nvme_get_request()` to its NULL path (KKASSERT is a no-op on
production, non-INVARIANTS kernels — sys/systm.h:116-118) or into the
ring-full check with a corrupted counter.

Two concurrent submitters exist whenever the admin thread is issuing its
attach/detach command sequence (identify controller, SET FEATURES number
of queues, identify namespace × N, create/delete queue × N) while a user
holds an open namespace fd and issues NVMEIOCGETLOG — e.g. hot-plug /
rescan of a second... on the SAME controller: attach storm right after
the namespace device appears, or the delete-queue storm during removal.

## What was verified on the guest

* The admin ioctl path itself is live and was stress-tested clean:
  1 × 60 s solo: 783,730 ioctls, 0 errors, max latency 6 ms; 6 × 90 s
  concurrent: ~1.35 M ioctls, 0 errors, max latency 0.82 s
  (ioctl_lk serializes the getlog submitters among themselves — see
  run.log). The race needs the *admin thread* as the second submitter.
* The deterministic trigger environment was built (nvmebang test module:
  on-demand driver attach/detach method invocation → repeated admin
  command storms); during these runs the teardown-race panic of DF-4529
  (same files, same rig) fires first — the detach-side defect masks the
  submit-side race before attach-storm data could be collected.
* The identical locking violation also underlies the DF-4529 teardown
  race, which *was* reproduced to a panic on this rig.

**Verdict: not_reproduced (dynamically), code-certain (statically)** —
the contract violation and its state mutations are unambiguous in the
source; a clean dynamic demonstration requires the DF-4529 fix (or
instrumented kernels) so the teardown panic does not preempt it.

## Build / run

Same environment as DF-4529 (see ../DF-4529/README.md). The load tools:

```sh
cc -O2 -o /root/nvspam nvspam.c     # glob /dev/nvme*, loop NVMEIOCGETLOG
cc -O2 -o /root/nvslow nvslow.c     # per-call latency + errno stats
cc -O2 -o /root/nvgetlog nvgetlog.c # single-call sanity check
```

```sh
/root/nvgetlog once /dev/nvme0                    # sanity
/root/nvslow /dev/nvme0 60 0.1                    # solo load
for i in 1 2 3 4 5 6; do /root/nvslow /dev/nvme0 90 0.2 >/root/slow$i.log 2>&1 & done
# attach-storm window (needs DF-4529 fix in place to observe cleanly):
sysctl hw.nvmebang.action=2   # re-runs identify/make-queues admin storm
```

## Fix

see fix.diff — admin submissions take the queue lock (new
`nvme_submit_admin_request()` wrapper; all qid-0 submitters converted).