# DF-4926 VERDICT — REPRODUCED (unload-race kernel crash; fix validated: FIXED)

## Finding

`sys/dev/virtual/nvmm/nvmm_dragonfly.c` tears the NVMM hypervisor down in
the wrong order on module unload:

* `nvmm_detach()` (:366-374): an **unsynchronized** `nmachines == 0` check
  at :369, then `nvmm_fini()` at :372 — which `lockuninit()`s **all
  128×128 machine/vcpu locks** (nvmm.c:998-1003), runs `nvmm_impl->fini()`
  (VMXON regions / MSR bitmaps freed) and sets **`nvmm_impl = NULL`**
  (nvmm.c:1005-1006).
* `nvmm_modevent()` MOD_UNLOAD (:399-405) calls `destroy_dev(dev)` only
  **after** `nvmm_detach()` has done all of that — the device is open and
  dispatching ioctls the entire time. `destroy_dev()` is precisely the
  operation that revectors ops to `dead_dev_ops`, drains in-flight calls
  and blocks new ones (kern/kern_conf.c:334-349, devfs destroy path); it
  must come FIRST.

Three detonation paths for an unprivileged user in group `nvmm` (device is
`crw-r----- root nvmm`; DFly ioctls do not require FWRITE, so an O_RDONLY
open drives every `NVMM_IOC_*`):

1. in-flight ioctl during `nvmm_fini()` → `nvmm_impl` deref (NULL or freed
   impl) — e.g. `nvmm_capability()` nvmm.c:245,251;
2. `NVMM_IOC_MACHINE_CREATE` landing between the :369 check and `nvmm_fini()`
   → machine alive with `lockuninit`ed locks and freed impl state (UAF);
3. fd kept open across a successful unload → `dfbsd_nvmm_dtor()`
   (:325-335, module text, stored as the cdevpriv dtor at :315) runs
   `nvmm_kill_machines()` through unmapped text at close.

## Reproduction (2026-09-25, guest DragonFly 6.5-DEVELOPMENT #0 X86_64_GENERIC, backend x86-vmx, QEMU -cpu host)

* Racer (unprivileged `testnv`, uid 1002, group nvmm, O_RDONLY open):
  ~1.1M `NVMM_IOC_CAPABILITY`/s + ~20k machine create/destroy cycles/s
  (standalone calibration in run.log header; fixed-run totals: 36M caps,
  684k cycles).
* Admin side: `kldunload nvmm` (root) while the racer runs.

**Baseline (stock /boot/kernel/nvmm.ko): TWO runs, both panicked the guest
with the identical crash site:**

    Fa[tal trap ...] Stopped at nvmm_ioctl+0x444:  movq 0x48(%rax),%rax

Run 2 panicked on the **first** `kldunload`. `%rax == nvmm_impl == NULL`
(just set by `nvmm_fini()` at nvmm.c:1006 during the in-flight unload);
`movq 0x48(%rax)` is the `nvmm_impl->capability` function-pointer load in
the dispatch (offset 0x48 of `struct nvmm_impl`, nvmm_internal.h:106-133).
Guest down both times; `vm.sh status` ⇒ down. (The "Fatal trap 12" banner
was truncated by the crashing serial console — the `Fa` fragment — but the
faulting RIP/instruction and the dead guest are on the serial log; see
panic.txt.)

**Patched (fix.diff applied to guest /usr/src, module rebuilt, same race):**

* all 40 `kldunload` attempts during the storm returned **EBUSY**
  (`rc=1` × 40, `rc=0` × 0) — the new `nvmm_opens` gate held;
* no panic; guest survived the full storm (36M ioctls, 684k machine cycles);
* after the racer died (fd closed → dtor ran against the LIVE module),
  the final `kldunload` returned 0 and the module unloaded cleanly.

## Impact ceiling

Race-condition kernel memory-safety bug at the hypervisor boundary: the
demonstrated result is a NULL-deref panic from unprivileged-steerable
ioctls racing an admin unload; paths (2)/(3) give UAF on freed
implementation state (VMXON regions, MSR bitmaps, machine/vcpu locks) and
execution through unmapped module text — plausible corruption/escalation
material for the group-nvmm user, not weaponized here (weaponization needs
the admin to run kldunload concurrently, hence Medium rather than High).
CVSS: CVSS:3.1/AV:L/AC:H/PR:L/UI:R/S:U/C:N/I:L/A:H (UI:R = the required
concurrent `kldunload` by root).

## Fix

`fix.diff` (git-apply clean against sys/):

1. an `nvmm_opens` counter (inc at successful open, dec at dtor — after
   `nvmm_kill_machines()`, so a concurrent detach observes `nmachines==0`
   before `opens==0`);
2. `nvmm_detach()` refuses while any fd is open or any machine exists,
   then `destroy_dev()` **first**, then (stable, drained) re-check with
   device restore on bail, and only then `nvmm_fini()`;
3. MOD_UNLOAD no longer destroys the dev after the teardown.

Validated end-to-end on the guest (baseline panic vs patched EBUSY/clean
unload under the identical storm; race_driver_fixed.log,
run_fixed_racer.log, fix_build.log).

## PoC changes vs any seed

No executable seed (fresh racer + driver authored). Practical fixes made
while bringing it up: `su -m testnv -c '... >/tmp/race.log 2>&1 &'` runs
via root's csh → "Ambiguous output redirect"; replaced with a `#!/bin/sh`
wrapper script executed via `su -m testnv -c /home/testnv/goracer.sh &`.
The racer opens `/dev/nvmm` O_RDONLY (0640 gives group read only; DFly
ioctls need no FWRITE — first attempt with O_RDWR got EACCES).

## Guest hygiene

`vm.sh reset with-src` performed at the end of the run (guest left clean;
the testnv user, /root packs and /usr/src patch are on the reverted disk
state only).

## Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest `make -j6 nativekernel KERNCONF=X86_64_GENERIC` completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.