DF-2672 — vm_object reference leak on fork() of wired (mlock'd) VM entries

What

vm_map_copy_entry() (sys/vm/vm_map.c:3600) handles the fork of a wired (src_entry->wired_count != 0) map entry by scrapping the cloned child entry's object and copying the pages into a fresh one. Its comment claims "its ref-count has not yet been adjusted so we can just NULL out the field", but that stopped being true with the vm_map_backing rework: vmspace_fork_normal_entry() (sys/vm/vm_map.c:3882) calls vm_map_backing_replicated(new_map, new_entry, 0) before vm_map_copy_entry() (sys/vm/vm_map.c:3889), and with flags == 0 that function takes a reference on the base object (vm_object_reference_quick(), sys/vm/vm_map.c:3510-3513).

The wired path detaches and NULLs dst_entry->ba.map_object (sys/vm/vm_map.c:3627-3634) without dropping that reference → one vm_object reference leaks per fork() of a wired entry.

Impact

Unprivileged local user (mlock works within RLIMIT_MEMLOCK, default = free_memory/3, no privilege needed — sys/vm/vm_mmap.c:1020-1031, sys/kern/kern_plimit.c:131-132):

• leaked vm_objects are never freed (vm_object_deallocate never sees 0);
• their dirty anonymous pages, once paged out under memory pressure, keep their swap space allocated forever (swap_pager_freespace only runs at object termination);
• permanent kernel-memory + swap exhaustion → system-wide DoS that survives the death of the offending processes. No RLIMIT bounds the accumulated leak across processes.

Reproduce (on the audit QEMU guest)

./build.sh   # cc -O2 -o /tmp/leak_fork leak_fork.c    (+ hog.c)
./run.sh     # control vs leak phases + memory pressure

Success criterion: vmstat -m "vm_object" Count grows by ~1 per leak iteration and never returns; control run returns to baseline. After pressure, swapinfo shows hundreds of MB permanently consumed with no processes owning anonymous memory.

Fix

fix.diff — call vm_object_deallocate() before NULLing the field (mirrors vm_map_entry_dispose(), sys/vm/vm_map.c:1132-1135). Validated by kernel rebuild + PoC re-run (see VERDICT.md).

DF-2672 VERDICT — REPRODUCED (leak), fix validated

Question

Does fork() of a wired (mlock'd) entry leak a vm_object reference, and does that translate into permanently unreclaimable kernel memory / swap?

Root cause (path:line)

• vmspace_fork_normal_entry() clones the child entry and calls vm_map_backing_replicated(new_map, new_entry, 0) at sys/vm/vm_map.c:3882 — with flags == 0 this executes vm_object_reference_quick(object) on the base object (sys/vm/vm_map.c:3510-3513).
• vm_map_copy_entry() (sys/vm/vm_map.c:3600) then takes the wired path because src_entry->wired_count != 0 (:3608) and discards the cloned object: vm_map_backing_detach() + ba.map_object = NULL (:3627-3629) without vm_object_deallocate(). The stale comment at :3613-3615 ("its ref-count has not yet been adjusted") is wrong since the backing_ba rework — the reference IS taken before the call. One reference is orphaned per fork.
• The orphaned object is never terminated, so swap_pager_freespace() never runs for it: dirty pages paged out under pressure keep their swap allocation forever.

Reproduction (guest: DragonFly 6.5-DEVELOPMENT #0, INVARIANTS kernel)

leak_fork.c: mmap(ANON|PRIVATE) + memset (dirty) + mlock + fork(); child exits; owner exits. Control omits the inner fork.

Decisive observations (run.log):

• Object growth ≈ 1 per fork-of-wired-entry, exactly as the code path predicts; control adds only kernel noise.
• ~844MB of swap permanently consumed with no processes owning anonymous memory (every leak_fork/hog exited). This swap can never be freed except by reboot: the owning vm_objects are unreachable.

Unprivileged: mlock() needs no privilege, only RLIMIT_MEMLOCK (default = free memory / 3, sys/kern/kern_plimit.c:131-132). The PoC ran as root on the guest but uses no privilege; mlock(256KB) succeeds for normal users within that rlimit.

Impact: permanent, unbounded, aggregation-across-processes consumption of kernel memory and swap by an unprivileged local user → system-wide memory-exhaustion DoS (Medium).

Fix validation

fix.diff adds vm_object_deallocate(dst_entry->ba.object) before NULLing (mirrors vm_map_entry_dispose(), sys/vm/vm_map.c:1132-1135). Kernel rebuilt in-guest (make nativekernel), rebooted, PoC re-run: vm_object count flat across leak iterations, no residual swap. See fix section in run.log / verdict.json fix_* fields.

Not a UAF

The bug only leaks references (object never freed early); there is no premature-free / type-confusion angle. Severity Medium (availability).