DF-4563 — vn(4): teardown-from-close-path race → cdev sysref double-put panic + unkillable stalls

• File: sys/dev/disk/vn/vn.c
• Class: refcount/lifetime race → kernel panic (assertion-guarded double-put), DoS
• Trigger: local, requires SYSCAP_RESTRICTEDROOT (vn ioctls are root-gated)
• Demonstrated: kernel panic 2/2 fresh-boot runs of a 2-thread racing-close program (no other load)

TL;DR

When a cloned vn unit is detached (VNIOCDETACH sets VNF_DESTROY) and two file descriptors on the unit are closed concurrently, the winning (last) close runs the entire disk teardown from inside the rawdev close path:

sys_close → diskclose (generic, holds ds_token, d_opencount 2→1→0)
  └─ dev_dclose(rawdev) → vnclose()                    [vn.c:181-187]
       └─ disk_destroy(&vn->sc_disk)                    synchronous disk-core msg
            └─ devfs_destroy_related(d_cdev)            synchronous devfs-core msg
                 └─ devfs_destroy_dev_worker: release_dev() × 3   ← races the
                     loser close()'s own cdev ref drops               sysref

The losing close (d_opencount 2→1) simultaneously drops its cdev sysref from its own diskclose/fdrop path. The destroy worker's release_dev() triple-put then double-puts the same sysref → panic: assertion "(sr->flags & SRF_PUTAWAY) == 0" failed in sysref_put. Before the panic lands, the same interleave wedges the losing close (and any new devfs operations system-wide) in uninterruptible sleep for minutes.

A second consequence of the same teardown-in-close design: vnclose() also SLIST_REMOVEs and kfree()s the vn_softc (with the embedded struct disk) while the losing diskclose() still dereferences dp->d_slice / dp->d_rawdev after acquiring ds_token (sys/kern/subr_disk.c diskclose) — a post-free deref that requires slab-block recycling to fault.

Files

• stall.c — minimal trigger: clone vn unit, attach (swap-backed), detach, open 2nd fd, race two close()s, repeat. Prints per-iteration close latencies; the panic lands within the first iterations.
• vn_race.c — heavier harness (4 racing pairs + p_args 'A' spray aimed at the post-free dp->d_slice read); produced the same wedge/panic.
• build.sh / run.sh
• run.log, run.2.log — console/dmesg excerpts of both fresh-boot runs
• panic.txt — the panic + backtrace (run 1), panic.2.txt (run 2)
• env.txt — guest uname / state
• fix.diff — deferred-teardown fix (callout), validated: patched kernel survives the same trigger, unpatched panics 2/2.
• VERDICT.md — full narrative

Reproduce

cc -O2 -pthread -o stall stall.c
./stall 800          # panics within the first ~dozen iterations
                     # (may first wedge close() for minutes, then panic)

Expected (unpatched):

panic: assertion "(sr->flags & SRF_PUTAWAY) == 0" failed in sysref_put at sysref2.h:85
release_dev() at release_dev+0x6e
release_dev() at release_dev+0x6e
devfs_destroy_dev_worker() at devfs_destroy_dev_worker+0x18b
devfs_destroy_related_worker() at devfs_destroy_related_worker+0x80

Expected (patched with fix.diff): program runs to DONE n=800 worst_close=<ms> with no panic and no multi-second close stalls.

DF-4563 VERDICT — reproduced (kernel panic), root-gated trigger

Status

• status: reproduced / impact: panic / confidence: certain
• 2/2 fresh-boot reproductions (vm.sh reset with-src between runs), minimal 2-thread trigger, no other load needed. Guest: DragonFly 6.5-DEVELOPMENT X86_64_GENERIC (see env.txt).

Root cause (path:line)

• sys/dev/disk/vn/vn.c:455-477 — VNIOCDETACH: allows the last-close teardown when disk_getopencount() <= 1 and arms VNF_DESTROY for cloned units (dkunit(dev) >= VN_PREALLOCATED_UNITS, vn.c:473-475).
• sys/dev/disk/vn/vn.c:169-189 — vnclose() runs, from inside the rawdev close path: disk_destroy() + devfs_clone_bitmap_put() + SLIST_REMOVE()
• kfree(vn).
• sys/kern/subr_disk.c:1126-1161 — generic diskclose(): decrements d_opencount, then under ds_token calls dsclose(dev, type, dp->d_slice) and, on the last close, dev_dclose(dp->d_rawdev) → the vnclose teardown. The losing close blocks on ds_token while the winner frees; after the token it dereferences dp->d_slice/dp->d_rawdev from the (potentially freed) softc — the post-free deref window.
• sys/kern/subr_disk.c:527-555 — DISK_DISK_DESTROY handler: devfs_destroy_related(d_cdev) + destroy_dev(d_cdev) + destroy_only_dev(d_rawdev).
• sys/vfs/devfs/devfs_core.c:1440-1460 — devfs_destroy_dev_worker() calls release_dev(dev) three times; racing the loser close's own release_dev() of the same cdev sysref double-puts it.

Demonstrated manifestation

panic: assertion "(sr->flags & SRF_PUTAWAY) == 0" failed in sysref_put at /usr/src/sys/sys/sysref2.h:85
cpuid = 1
release_dev() at release_dev+0x6e
release_dev() at release_dev+0x6e
devfs_destroy_dev_worker() at devfs_destroy_dev_worker+0x18b
devfs_destroy_related_worker() at devfs_destroy_related_worker+0x80
devfs_msg_exec() at devfs_msg_exec+0x6c
devfs_msg_core() at devfs_msg_core+0xb5
Debugger("panic")

• Run 1: heavy harness (vn_race.c, 2 racing pairs + p_args spray) — panic after ~10 minutes of racing (several minutes-long unkillable D-state stalls observed first, ps -H showed 4 threads in D state; kill -9 not processed while stalled).
• Run 2: minimal harness (stall.c, 2 threads, no spray) from a fresh reset with-src snapshot — identical panic; first racing iteration wedged the guest's devfs (new ssh sessions hung at banner) until the panic.

Exploitability assessment

• The sysref assertion catches the double-put before free-in-use in this kernel config (KKASSERT-enabled). Without the assertion this is a use-after-free / double-free of a cdev (kernel heap).
• The post-free dp->d_slice deref by the losing diskclose() is a read of a freed vn_softc (2816-byte slab class, measured via vmstat -m) followed by dsclose() bit-clear writes through the garbage pointer — exploitable in principle via same-class slab recycling (execve p_args of ~2700 bytes, content user-controlled, needs kern.ps_arg_cache_limit raised or a different same-class allocation stream).
• Trigger requires SYSCAP_RESTRICTEDROOT (vnioctl, vn.c:434) for the detach that arms VNF_DESTROY. So the demonstrated panic is a privileged-user DoS/crash, not an unprivileged escalation. Severity Medium (same bar as the known vn race findings, e.g. DF-1699).

Fix validation

• fix.diff: defer the VNF_DESTROY teardown out of vnclose via callout_reset(..., hz, vn_destroy_deferred, ...); the loser's close has long completed (and released its cdev refs) by the time the destroy worker runs; callout_drain() added to MOD_UNLOAD.
• Baseline: unpatched kernel → panic 2/2.
• Patched: make buildkernel KERNCONF=X86_64_GENERIC && make installkernel, reboot, same stall 800 trigger → no panic, program completes (DONE n=800 ...), no multi-second close stalls. See run.patched.log.

Negative results recorded during this audit (pass 2)

• attach-path dev_dopen() imbalance: none — the internal open targets the raw dev whose vnopen doesn't touch d_opencount (subr_disk.c layering).
• devfs_clone_bitmap_get is fully serialized by devfs_bitmap_lock (devfs_helper.c:221-233) — no clone-unit race.
• swap-backed vn_size overflow at vn.c:632-633 (PAGE_SIZE * vn_size as off_t) wraps for vn_size ≥ 2^51, but swap_pager_strategy / swp_pager_meta_build never bound-check object->size, so the result is a sparse swap-disk of bogus advertised geometry, not memory corruption.
• concurrent double-VNIOCATTACH (no INITED serialization) leaks one backing vnode + file, no UAF — adjacent to DF-2528's thesis, folded there.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.