DF-2865 — pbuf reservation counters: unsynchronized check-then-decrement race

sys/vm/vm_pager.c — getpbuf() / getpbuf_kva() / getpbuf_mem() / trypbuf() / trypbuf_kva().

The bug

The per-subsystem pbuf reservation counters (pfreecnt: nsw_rcount, nsw_wcount_sync/async, vp->v_pbuf_count, mp->mnt_pbuf_count, nfs/smbfs/vinum counters) are meant to cap how many pbufs one subsystem holds concurrently. The allocator gates on the counter without any lock (vm_pager.c:418/463/512):

while (pfreecnt && *pfreecnt <= 0) {      /* unlocked read */
        tsleep_interlock(pfreecnt, 0); ...
}
... bucket scan ...
spin_lock(&bswspin_kva[iter]);            /* per-BUCKET spinlock */
TAILQ_REMOVE(...);
if (pfreecnt)
        atomic_add_int(pfreecnt, -1);     /* :489-490 */

Because the gate and the decrement are not atomic together — and the spinlock is per-bucket (iter = mycpuid & BSWHMASK), so two CPUs are typically under different locks — N CPUs can each observe *pfreecnt == 1, each pass the gate, and each decrement, driving a counter that should never go below 0 to 1-N and putting N pbufs in flight for a reservation of 1.

Consequences: 1. Subsystem pbuf caps are violable (over-consumption of the shared kva/raw pools by up to ncpu-1 concurrent allocators). 2. Negative excursions make getpbuf* sleepers block on *pfreecnt <= 0 while pbufs are actually free — an additional recovery delay stacked on the (separately filed, DF-0953) wake threshold defect.

Unprivileged reachability: trypbuf_kva(&mnt_pbuf_count) / getpbuf_kva(&mnt_pbuf_count) run for every clustered file read/write (sys/kern/vfs_cluster.c:924-927, 1525-1527); nsw_rcount runs for swap-in under memory pressure (sys/vm/swap_pager.c:1447).

Distinct from DF-0953 (known): DF-0953 is the release-side wake threshold in relpbuf(); this is the allocator-side check/decrement atomicity. Different function, different lines, different fix.

PoC

pbufres.c — KLD harness. Six kernel threads (one per CPU) hammer getpbuf_kva()/relpbuf() against a reservation counter with cap 1, tracking the minimum observed counter value and the maximum number of simultaneous pbuf holders.

./build.sh          # build pbufres.ko on the guest
./run.sh            # load, spin 10 s, read stats, rescue, unload

Success criterion (stock kernel): vm.pbres_neg > 0, vm.pbres_min < 0, vm.pbres_maxinflight > 1. On a kernel with the cmpxchg reservation fix: all three clean (neg=0, min >= 1, maxinflight <= 1).

vm.pbres_bump=100 is harness plumbing only: it wakes sleepers left gated by the known DF-0953 wake-threshold defect so the module can be unloaded cleanly (the stall it relieves is DF-0953's, not this finding's).

Artifacts

• run.2.log — decisive stock-kernel run: pbres_neg: 2, pbres_min: -1, pbres_maxinflight: 2, console PBUFRES: counter NEGATIVE (-1), cap=1.
• run.log — first stock run (same numbers; the unload step timed out because cap=1 also exercises DF-0953's missed wakeup — 5 threads asleep while vm.pbuf_kva_count: 512, i.e. the pool 100 % idle).
• fix.diff — cmpxchg reservation (all five allocators) and the DF-2866 pagertab[] OBJT_MARKER slot (combined kernel build).
• run.patched.log — same harness on the fixed kernel.

DF-2865 — VERDICT

Status: reproduced (stock kernel) / fixed (patched kernel). Impact: dos (reservation-counter integrity + transient I/O throttling corruption). Not memory corruption.

What was run

KLD harness pbufres.c (6 kernel threads, one per CPU, reservation counter cap = 1, getpbuf_kva()/relpbuf() hammered for 10 s) on the single-tenant QEMU guest, stock INVARIANTS kernel DragonFly 6.5-DEVELOPMENT #0: Thu Jul 2 06:02:54 UTC 2026 (env.txt).

Baseline (stock kernel) — run.2.log (and run.log, independent boot)

vm.pbres_neg: 2            # counter observed < 0 twice in 10 s
vm.pbres_min: -1           # reservation counter driven below zero
vm.pbres_maxinflight: 2    # TWO pbufs held simultaneously, cap was 1
PBUFRES: counter NEGATIVE (-1), cap=1, inflight=1     (console)

maxinflight = 2 with cap = 1 is the direct, race-proven statement of the finding: two CPUs passed the *pfreecnt <= 0 gate (vm_pager.c:463) with the counter at 1 and both decremented (vm_pager.c:489-490), exactly as read from the source. The negative counter is the observable residue of the same race.

Incidentally, run.log (first run, before the rescue sysctl existed) also captured the known DF-0953 behaviour end-to-end: 5 threads asleep on the reservation gate while vm.pbuf_kva_count: 512 — pool 100 % idle — because relpbuf() only wakes pfreecnt sleepers on the 1→2 transition. That stall is DF-0953's defect (do-not-re-report); this finding is the allocator-side atomicity hole that additionally drives the counter negative and past its cap.

Patched kernel (fix.diff: cmpxchg reservation in all five allocators)

Combined kernel build make -j6 nativekernel KERNCONF=X86_64_GENERIC (+ the DF-2866 pagertab hunk), see run.patched.log / fix_build.log:

vm.pbres_neg: 0
vm.pbres_min: 1            # never below the floor
vm.pbres_maxinflight: 1    # cap respected

The race window is closed: the reservation is taken with atomic_cmpset_int(pfreecnt, c, c-1) re-validating c > 0; on failure the buffer is put back and the outer gate re-entered.

Why not higher severity

The counter is self-correcting (each over-decrement is matched by its release), so the effect is transient over-consumption and extra blocking latency on subsystems whose throttles exist to keep the shared pools fair (swap-in, clustered VFS I/O). No memory-safety consequence; impact ceiling is local I/O-latency DoS — the same family DF-0953 occupies, with a distinct root cause and fix.

Kernel references

• sys/vm/vm_pager.c:417-422 (unlocked gate, getpbuf), :442-447 (raw decrement under per-bucket lock)
• sys/vm/vm_pager.c:462-467, :487-491 (getpbuf_kva)
• sys/vm/vm_pager.c:511-516, :536-540 (getpbuf_mem)
• sys/vm/vm_pager.c:564-578 (trypbuf), :596-610 (trypbuf_kva)
• Real-world counters: sys/vm/swap_pager.c:1447,163-166; sys/kern/ vfs_cluster.c:924-927,1525-1527; sys/sys/mount.h:221 (u_int mnt_pbuf_count — also type-punned through the int * API); sys/sys/vnode.h:153.