DF-3194 — nfs_nget/nfs_nget_nonblock race-loser path vx_put()s a never-connected VNON vnode → forced VREF_FINALIZE → VOP_INACTIVE runs nfs_inactive with v_data == NULL → NULL-deref kernel panic

What this is

In nfs_nget() (sys/vfs/nfs/nfs_node.c) the "validate that we did not race another nfs_nget()" loop (:189-202) handles a discovered duplicate by dropping the freshly created-but-never-connected vnode with a bare vx_put(vp) (:194), then manually freeing the nfsnode. The identical pattern exists in nfs_nget_nonblock() (:336).

At that point vp is exactly what getnewvnode(VT_NFS, mntp, &vp, 0, 0) returned: v_type == VNON (set by allocvnode, sys/kern/vfs_lock.c:1213), v_data == NULL, v_mount == mntp (set by insmntque), v_ops == &mp->mnt_vn_use_ops (NFS ops).

DragonFly's vx_put() forces VREF_FINALIZE for VNON/VBAD vnodes (sys/kern/vfs_lock.c, vx_put), so the 1→0 refcount transition in vrele() runs vnode_terminate() → VOP_INACTIVE(vp) → nfs_inactive():

np = VTONFS(ap->a_vp);          /* NULL */
if (ap->a_vp->v_type != VDIR) { /* VNON != VDIR → true */
        sp = np->n_sillyrename; /* NULL-deref → page fault → panic */

(sys/vfs/nfs/nfs_node.c:389-393)

Reachability of the loser path: an nfs_nget creator must end up with a stale view of its hash bucket. Two verified schedulings do this:

1. Mid-scan deschedule: lwkt_gettoken() does not hold a critical section, and lwkt_switch() releases all of the thread's tokens (sys/kern/lwkt_thread.c:636-638). A creator preempted mid-LIST_FOREACH has its token transparently released; a creator on another CPU can then complete a whole nfs_nget and LIST_INSERT_HEAD() the node behind the first thread's cursor. Resuming, the stale thread misses the new head, acquires nfsnhash_lock without sleeping (LK_SLEEPFAIL only discards acquisitions that slept), creates its own vnode, and the validation loop finds the duplicate → vx_put(VNON vp) → panic. (The other nfs_nget caller must hold nfsnhash_lock during this — satisfied when the preempted thread's outage covers the other creator's remaining scan+create time.)

2. Winner blocks mid-create: the lockmgr winner blocks between lockmgr(...LK_SLEEPFAIL) success and LIST_INSERT_HEAD (both allocations are M_WAITOK; getnewvnode→allocvnode also has M_WAITOK/kmalloc_obj fallbacks under VM pressure). A contender cycles in the SLEEPFAIL retry loop (lockmgr fails → goto loop → full-bucket rescan). When the winner wakes and inserts at the bucket head mid-rescan, the contender's cursor is already past the head; its next lockmgr succeeds sleep-free (the winner released) and its validation loop finds the duplicate → panic.

Trigger surface: any unprivileged local userspace concurrency on an NFS-mounted system that drives concurrent nfs_nget() for the same fresh file handle (concurrent stat/open of aliased names resolving to the same fh — a namecache-resolution-serialized same-name lookup does NOT race, so aliases or lookitup paths are needed).

PoC files

• fakesrv_race.c — NFSv3 server over loopback UDP. Precomputes 100,000 32-byte file handles that all satisfy fnv1_32(fh,32) & 0x1FFFF == 0 (the guest's nfsnode hash mask; hashinit(vfs_inodehashsize()) with kern.maxvnodes=109306 → hsize 131072). Names [a-m]<idx> all resolve to pool fh <idx>, so N threads can race nfs_nget() for one fh without namecache serialization. Pool cached in /tmp/fh_pool.bin.
• race_trigger.c — fill phase (stat every pool name once → one hash bucket holds ~all nodes → nfs_nget's initial scan takes ~1-2 ms), then 12 pthread_barrier-synchronised threads stat the same fresh fh each round via distinct name aliases.
• udp_flood.c, memhog.c — amplifiers (netisr preemption bursts; VM pressure for mid-create M_WAITOK blocks).

Build (in-guest, base cc)

cd /root/df3194
cc -O2 -o fakesrv_race fakesrv_race.c
cc -O2 -pthread -o race_trigger race_trigger.c
cc -O2 -o udp_flood udp_flood.c
cc -O2 -o memhog memhog.c

Run

./fakesrv_race &            # ~5 min: builds the colliding pool once
mount_nfs -o port=2049,udp 127.0.0.1:/P /mnt
./udp_flood & ./udp_flood & ./memhog & ...   # amplifiers
./race_trigger 24000 76000                  # fill + hammer

Expected

• If the race fires: Fatal trap 12: page fault while in kernel mode in nfs_inactive+… with a small fault address (≈ offsetof(struct nfsnode, n_un3), page-0 fault), current process = the stat()ing user program. The guest dies.
• Observed in this campaign: no panic across ~620,000 barrier-synchronised 12-thread races over three configurations (shallow bucket; 76k-node deep bucket + netisr flood + host oversubscription; deep bucket + VM pressure). The two required schedulings need a ≥~100-200 µs mid-scan LWKT preemption or a ≥~2 ms mid-create block of the lockmgr winner — neither could be forced reliably from unprivileged userspace in the QEMU guest (objcache magazine hits make the creation-phase allocations effectively non-sleeping, and quiet-guest preemptors run for µs). Verdict: not_reproduced dynamically; static chain certain (see VERDICT.md for the full scheduling analysis).

Fix

fix.diff (verified to build; included in the DF-3195 validation kernel): connect (np, vp) before vx_put() in both loser paths so the forced VOP_INACTIVE/VOP_RECLAIM teardown sees a consistent pair (reclaim then frees np because n_hash.le_prev == NULL), plus the vp = NULL fix for the related stale-vp double-vput restart in nfs_nget_nonblock (:271-273).

DF-3194 VERDICT — NOT REPRODUCED dynamically (race); static chain certain; ~620k attempts across 3 amplified configurations

Bottom line

The "validate that we did not race another nfs_nget()" loser paths in nfs_nget() (:189-202, drop at :194) and nfs_nget_nonblock() (:331-350, drop at :336) release the freshly created vnode with a bare vx_put(vp):

vx_put(vp);                                    /* nfs_node.c:194 / :336 */
lockmgr(&nfsnhash_lock, LK_RELEASE);
if (np->n_fhsize > NFS_SMALLFH) kfree(np->n_fhp, M_NFSBIGFH);
np->n_fhp = NULL;
kfree_obj(np, nmp->nm_mnode);

At that instant vp is a stock getnewvnode() product: v_type == VNON (set in allocvnode, sys/kern/vfs_lock.c:1213), v_data == NULL, v_mount == mntp (insmntque), v_ops = NFS ops. DragonFly's vx_put() forces VREF_FINALIZE for VNON/VBAD vnodes (sys/kern/vfs_lock.c, vx_put), so vrele()'s 1→0 transition runs vnode_terminate() → VOP_INACTIVE(vp) → nfs_inactive():

np = VTONFS(ap->a_vp);            /* NULL            :389 */
if (ap->a_vp->v_type != VDIR) {   /* VNON != VDIR    :392 */
        sp = np->n_sillyrename;   /* NULL+offset deref :393 → page fault → panic */

Every link is verified in source; there is no guard anywhere in the chain (vop_inactive dispatch is a plain DO_OPS, vnode_terminate only checks v_mount != NULL).

Why the loser path is reachable (scheduling analysis)

nfsnhash_lock (exclusive) serializes insertion; a stale view therefore requires the losing thread to observe the bucket across another creator's insert. Two schedulings do it:

1. Mid-scan deschedule. lwkt_gettoken() does not hold a critical section over the protected region, and lwkt_switch() drops all of the thread's tokens (sys/kern/lwkt_thread.c:636-638). A creator preempted mid-LIST_FOREACH (nfs_node.c:116) transparently loses nfsnhash_token; a creator on another CPU then runs a complete nfs_nget and LIST_INSERT_HEAD()s the node behind the first thread's cursor. On resume the stale thread misses the new head, calls lockmgr(... LK_SLEEPFAIL) — which succeeds without sleeping because the other creator already released — allocates its own vnode, and the validation loop finds the duplicate → vx_put(VNON vp) → panic. Requirement: preemption duration p > (scan progress c) + winner create time τc; with a deep bucket and c≈0 this is p > τc ≈ 100-200 µs.
2. Winner blocks mid-create. The lockmgr winner can block between acquiring nfsnhash_lock and LIST_INSERT_HEAD (both allocations are M_WAITOK; getnewvnode→allocvnode has M_WAITOK/kmalloc_obj fallbacks under VM pressure). A contender then spins in the SLEEPFAIL retry loop (fail → goto loop → full-bucket rescan). If the winner's lockmgr-hold H ≥ one rescan D, its insert lands mid-rescan behind the contender's cursor; the contender's next lockmgr succeeds sleep-free and its validation loop finds the duplicate → panic. Requirement: one ≥ ~2 ms block of the winner (D ≈ 1.8 ms with a 76k-node bucket).

Reproduction campaign (all on stock INVARIANTS kernel #0 Jul 2, 6 vCPU, 4 GB)

Why it never fired: on this quiet QEMU guest, (a) LWKT preemptors that run longer than ~100 µs are rare — per-packet netisr work is µs-scale and user-level threads cannot preempt a thread in kernel mode mid-syscall; and (b) the creation-phase allocations are objcache-backed (magazine hits), so the winner essentially never sleeps mid-create. Both conditions are ordinary on a real loaded NFS client (interrupt bursts, VM pressure, NFS timer/callout storms) but could not be forced deterministically from unprivileged userspace here.

Status: not_reproduced (dynamically). Confidence in the defect: certain (statically) / likely (to occur in the wild). The PoC infrastructure (colliding-fh pool builder, alias-name racing driver, amplifiers) is left in the pack for re-testing on noisier hardware.

Fix

fix.diff — connect (np, vp) before the drop in both loser paths so the forced VOP_INACTIVE/VOP_RECLAIM teardown sees a consistent pair (nfs_reclaim frees np because n_hash.le_prev == NULL), plus the stale vp = NULL fix for the related double-vput restart in nfs_nget_nonblock (:271-273, DF-3196).

Fix build: both hunks compiled into the DF-3195 validation kernel (make nativekernel RC=0, nfs_node.c compile confirmed, installed, booted, NFS mount/lookup lifecycle exercised — see DF-3195's run.fixed.log which ran on the kernel containing this fix too). Behavioral before/after comparison is not possible because the baseline race never fired in-guest. fix_status: inconclusive (builds & boots; race not triggerable to compare).

Threat

Unprivileged local user on any NFS-mounted DragonFly system; trigger is ordinary concurrent path resolution (stat/open of names aliasing to the same file handle — the server controls fh identity for LOOKUP replies, so even an honest server plus a flaky scheduler suffices). Outcome: kernel page fault → panic (local DoS). No memory corruption on this path (NULL + fixed-offset read).

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.