# DF-2832 — VERDICT

**Bottom line:** the defect is real and its exact interleaving was
demonstrated on a running kernel; a stock-kernel crash was **not** observed
(and is not expected to be observable without KASAN, because the primitive
is a bounded atomic write (±2 / bit-0) onto freed heap, usually silently
absorbed). The fix closes the window and was validated on-guest.

## 1. Code-level proof (no run needed)

* `trigger_syncer*()/speedup_syncer()` read `mp->mnt_syncer_ctx` and RMW the
  ctx with **no lock, no refcount, no mount hold** — vfs_sync.c:570-571,
  584-586, 595-596, 608-611.
* `vn_syncer_thr_stop()` NULLs the pointer under `sc_token` (which the
  trigger family never takes) and then `hashdestroy`+`kfree`s the ctx —
  vfs_sync.c:353-357. `hashdestroy`'s only protection is a KASSERT on the
  *lists*, nothing protects the `syncer_trigger` field readers.
* The caller's `struct mount *` is also unreferenced; `dounmount` can free
  it (vfs_syscalls.c:1107-1117) while a hammer2 frontend thread is inside
  the trigger functions.
* The other side of the same API defect: after the NULL publish, any
  lock-free `vn_syncer_add`/`vsetisdirty` (`reassignbuf` — every dirty
  buffer) does `lwkt_gettoken(&ctx->sc_token)` with `ctx == NULL` →
  guaranteed page fault. In the *stock* ordering this facet is unreachable
  (VFS_UNMOUNT's vflush drains all vnodes before thr_stop runs — verified:
  `vgone_vxlocked` force-dequeues at vfs_subr.c:1573-1580, and dounmount
  panics on dangling vnodes at vfs_syscalls.c:1085-1086); it becomes
  reachable the moment any code path stops the syncer on a live mount
  (demonstrated below; the failed-mount thr_stop paths at
  vfs_syscalls.c:458 / vfs_conf.c:369,508 are the speculative stock
  candidates).

## 2. On-guest demonstration (witness kernels, guest-only instrumentation)

The witness kernel inserts a **park** at the exact stock instruction
boundary — after `ctx = mp->mnt_syncer_ctx` and before the atomic RMW —
which is precisely where a preemption lands on the stock kernel. A park is
a faithful preemption emulation for the stock code because the stock code
holds **no lock** there. The teardown is the **real**
`vn_syncer_thr_stop()` (invoked deterministically via a debug sysctl once
parks are observed, because the full `umount -f` path cannot complete
while the unpriv storm saturates hammer2's dirty-chain backpressure — see
§4). Load side is genuinely unprivileged: `nobody` running `churn_open`
(stall reached via `open(O_CREAT)` → `ncp_writechk` → `VFS_MODIFYING` →
`hammer2_pfs_memory_wait`).

**Decisive run (panic.txt):**

```
DF2832: stop: parked=6, tearing down syncer of mp=0xfffff8008fa6b000
DF2832: thr_stop FREE ctx=0xfffff801174ee9a0 mp=0xfffff8008fa6b000
pan[D]iF…2:8 …HIT stale ctx=0xn f…   <- kprintf streams interleaved
Stopped at  lwkt_gettoken+0x64:  movq (%r12),%rax      <- db>
```

* `parked=6`: six unprivileged trigger callers were inside the window.
* `FREE`: the real `vn_syncer_thr_stop` destroyed the ctx under them.
* `HIT stale ctx=0x…`: a parked caller observed the teardown — on the
  stock code it would have executed `atomic_fetchadd_int(&ctx->syncer_trigger, 2)`
  **on freed memory**.
* The `lwkt_gettoken` panic is the NULL-ctx facet (a churner's
  `vn_syncer_add` after teardown of the still-live mount) — same API
  defect, second facet, artifact of the deterministic teardown bypassing
  VFS_UNMOUNT ordering (not stock-reachable; see §1).

Also observed across cycles: the freed ctx address is **reused by the next
mount** (0xfffff80117b51ee0/…ee0/…f40 repeatedly), so a stale RMW that
lands late corrupts the *next* mount's `syncer_trigger` — cross-mount
corruption, silent.

## 3. Fix validation (fix.diff)

`fix.diff` (against the repo `sys/`, never applied there):
1. `trigger_syncer`/`trigger_syncer_start`/`trigger_syncer_stop`/
   `speedup_syncer` hold `mnt_token` **shared** across load+RMW;
2. `vn_syncer_thr_stop` NULLs+frees under `mnt_token` **exclusive**;
3. NULL guards in `vn_syncer_add`/`vn_syncer_remove`/`vsetisdirty`/
   `vsetobjdirty` (defense-in-depth; also what stops the §2 NULL-panic
   facet).

Validation build = witness + fix, with the park switched to a **busy-spin**
(a `tsleep` would drop the lwkt token and no longer emulate preemption;
real preemption holds tokens). Identical procedure:

```
DF2832: stop: parked=3, tearing down syncer of mp=0xfffff8008fa6b000
DF2832: thr_stop FREE ctx=0xfffff8011750e9a0 mp=0xfffff8008fa6b000
panic: hashdestroy: hash not empty        <- artifact, see below
```

* **No HIT**: with the interlock, the teardown cannot slip between the
  load and the RMW — the exclusive acquire blocks until every in-window
  caller completes. The window demonstrated in §2 is closed.
* No `lwkt_gettoken` NULL-deref panic (guards work; execution proceeds
  until the teardown-of-a-live-mount hits `hashdestroy`'s "hash not
  empty" KASSERT — an artifact of bypassing VFS_UNMOUNT's vflush that is
  orthogonal to the fix and would fire identically on the unfixed kernel
  if the NULL-deref hadn't fired first).

## 4. Honesty notes

* **Stock full-umount race:** 40 iterations early (defective harness —
  `/tmp` denies exec for nobody, so the unpriv binaries never ran; only
  the teardown side was exercised) + ~8 iterations after the fix
  (`run.stock2.log`, working unpriv storm: 3×dirty_writer + 6×churn_open,
  `umount -f` cycles) — **no crash**. Expected: the window is a few ns
  vs. an umount taking seconds; without KASAN the ±2 write is invisible.
* **Why the deterministic sysctl teardown:** with the storm saturating
  `vfs.hammer2.limit_dirty_chains`, `umount -f` cannot get through
  `VFS_SYNC(mp, MNT_WAIT)` (vfs_syscalls.c:1002) — hammer2's backpressure
  oscillates (churners stall → chains drain → churners resume) forever.
  That interaction (unpriv dirty-pressure wedging forced unmount in
  VFS_SYNC) may deserve its own look; it is a livelock, not corruption.
* Witness parks were scoped to never fire on the root filesystem (an
  unscoped first attempt wedged the guest; instrumentation artifact).
* `stat_rush_requests` non-atomic increment (vfs_sync.c:569) observed
  while reading — cosmetic, not filed.