β¬’ DragonFlyBSD Kernel Audit
← triage Β· dashboard
DF-2834

Unbounded vmspace allocation in sys_vmspace_create: no per-process cap, each create pins a wired PML4 page + 8KB KVA + vmspace struct β†’ unprivileged kernel-memory exhaustion DoS (requires vm.vkernel_enable=1)

Field Value
ID DF-2834
Status new
Severity Medium
CVSS 3.1 CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H
CWE CWE-770 Allocation of Resources Without Limits
File sys/vm/vm_vmspace.c
Lines 146-201 (chain vm_map.c:309-340 β†’ pmap.c:2368-2387)
Area vm
Confidence certain
Discovered 2026-08-31
Pass 2 (GLM 5.3 second pass)
Bucket base:vm
Reported pending
Known CVE none
CVE match novel

Summary

sys_vmspace_create has no limit on how many vmspaces a process may create. Each successful create allocates a vmspace_entry, a full vmspace, and via pmap_pinit a 2-page kmem_alloc_pageable KVA allocation plus a WIRED PML4 vm_page, and pmap_pinit2. An unprivileged loop with distinct ids grows unswappable kernel memory and wired pages without any rlimit or sysctl cap β€” unlike mmap entries (capped by vm.max_proc_mmap). Exhaustion is system-wide (wired-page/KVA starvation wedging or panicking the machine). Gated by vm.vkernel_enable=1 (root tunable, default 0 β€” but exactly the configuration on any vkernel host). Same reachability class as DF-0952: no privilege check beyond p_vkernel != NULL. Allocation chain fully source-proven; live OOM run deferred (would wedge the single-tenant guest).

Per-vkernel_proc cap on live vmspaces checked under vkp->token, mirroring the vm.max_proc_mmap pattern (vm.max_vkernel_vmspaces sysctl, default 64) β€” sketch diff in the row above / pack.

Timeline

  • 2026-08-31 Discovered during pass-2 audit of vm_vmspace.c (GLM 5.3).

Discussion (0)

No comments yet.

PoC verification

Evidence pack

findings/poc/DF-2834 Β· 13 files
FileTypeDescriptionSize
README.md β€” 3.4 KB ↓ raw
VERDICT.md β€” 7.0 KB ↓ raw
vmsp_grow.c β€” 3.5 KB view raw
vmsp_cap.c β€” 1.1 KB view raw
build.sh β€” 284 B view raw
run.sh β€” 143 B view raw
build.log β€” 202 B view raw
run.log β€” 2.5 KB view raw
run.fixed.log β€” 2.6 KB view raw
env.txt β€” 1.4 KB view raw
fix.diff β€” 2.0 KB view raw
manifest.json β€” 1.4 KB view raw
verdict.json β€” 4.7 KB view raw

DF-2834 evidence pack β€” unbounded vmspace_create kernel-memory exhaustion

File: sys/vm/vm_vmspace.c:146-201 (sys_vmspace_create) Severity: Medium (CWE-770) β€” unprivileged local DoS, gated on vm.vkernel_enable=1

What was verified

  1. No cap exists on the path (static + dynamic): - sys_vmspace_create (sys/vm/vm_vmspace.c:146-201) consults no rlimit, no priv_check, no counter. The only gate is vkernel_enable (sys/vm/vm_vmspace.c:155). Contrast: mmap entries ARE capped by vm.max_proc_mmap (1000000 on this guest). - Empirical: 120,000 live vmspaces created by ONE unprivileged (uid 65534) process in 6.9 seconds, zero failures.

  2. Per-iteration cost (line-accurate): - kmalloc(sizeof(struct vmspace_entry)) β€” sys/vm/vm_vmspace.c:182 - vmspace_alloc() β€” sys/vm/vm_vmspace.c:183 β†’ sys/vm/vm_map.c:309-340, objcache vmspace_cache (sys/vm/vm_map.c:233) - pmap_pinit() β€” sys/platform/pc64/x86_64/pmap.c:2347

    • 8192 bytes pageable kernel KVA: kmem_alloc_pageable(kernel_map, PAGE_SIZE * 2, VM_SUBSYS_PML4) (pmap.c:2368-2370)
    • 1 WIRED PML4 vm_page: pmap_allocpte(pmap_pml4_pindex()) (pmap.c:2376-2380) β†’ vm_page_wire(m) (pmap.c:2757) β€” the page is wired and therefore unswappable/unreclaimable
    • pmap_pinit2() (sys/vm/vm_vmspace.c:187) is a NO-OP on x86_64 (pmap.c:2509-2512) β€” refinement of the finding text, which listed it as a cost contributor
    • Measured totals: 2.374-2.376 wired pages per create (9.7 KB) and 9752 bytes KVA per create (cold), perfectly linear.
  3. Growth is unswappable while the vmspaces are live β€” wired pages are not reclaimable by the page daemon; the only releases are vmspace_destroy or process exit (vkernel_exit β†’ rb_vmspace_delete).

  4. Exhaustion extrapolation (NOT executed β€” single-tenant guest): - guest physmem 4,257,873,920 B = 1,039,519 pages - measured rate 17,350 creates/s (120k in 6.915 s) - quiescent wirable pool (wire+free) β‰ˆ 992,529 pages β†’ β‰ˆ 418,000 creates β†’ β‰ˆ 24 seconds for one unprivileged process to wire all 4 GB - failure mode at exhaustion: vm_page_alloc(VM_ALLOC_NORMAL) inside pmap_allocpte (pmap.c:2751-2756) fails β†’ vm_wait(0) loop; wired pages can never satisfy the wait β†’ every M_WAITOK allocator system-wide stalls β†’ hard wedge. No clean error return exists on this path.

Files

  • vmsp_grow.c β€” main PoC: unpriv create loop with wire/KVA/free sampling
  • vmsp_cap.c β€” fix-validation helper: cap hit / destroy-frees-slot check
  • run.log β€” stock-kernel runs (20k cold, 20k warm-reuse, 120k capped)
  • run.fixed.log β€” patched-kernel A/B (cap, per-proc, raised-cap legit use)
  • fix.diff β€” the verified fix (cap + ABI guards), applied only in the guest /usr/src scratch copy, never to the audit tree
  • build.log β€” full compiler output (warning-free)
  • env.txt β€” guest/kernel/compiler state
  • VERDICT.md β€” full narrative and math
  • manifest.json, verdict.json β€” machine-readable results

Reproduce

build.sh                 # cc -O2 -Wall -o /tmp/vmsp_grow vmsp_grow.c
run.sh                   # enable sysctl, capped 120k unpriv run, restore

Success criterion (stock kernel): # DONE created=120000 ... grow_exit=0 with linear wire growth ~2.37 pages/iteration and zero failures.

VERDICT.md
↓ download raw

VERDICT β€” DF-2834 (unbounded vmspace_create kernel-memory exhaustion)

status: reproduced / impact: dos / confidence: certain

Guest: DragonFly 6.5-DEVELOPMENT #0 (stock INVARIANTS kernel, X86_64_GENERIC), QEMU/KVM 4GB / 6 vcpu. All runs 2026-10-20 UTC.

1. Reachability and the no-cap claim

sys_vmspace_create (sys/vm/vm_vmspace.c:146-201) is a plain syscall (SYS #486) with no privilege check of any kind. The ONLY gate is vkernel_enable at sys/vm/vm_vmspace.c:155 (vm.vkernel_enable, root sysctl, default 0). With it set to 1 β€” the documented operating state of any machine that hosts virtual kernels β€” any local user can create vmspaces without limit:

  • statically: no rlimit (lim_*), no priv_check, no counter, no sysctl cap anywhere in the function; nothing else in the kernel counts p_vkernel vmspaces;
  • empirically: 120,000 live vmspaces in one unprivileged process (uid 65534), zero failures, 6.9 seconds (run.log, PRE3..120000 block). For contrast, mmap entries per proc are capped (vm.max_proc_mmap = 1000000); no analogous knob existed for vmspaces.

2. Per-iteration cost, line-accurate

Each successful create, held until vmspace_destroy or process exit:

cost where
kmalloc(sizeof(struct vmspace_entry)) sys/vm/vm_vmspace.c:182
vmspace_alloc() objcache object (sizeof(struct vmspace)) sys/vm/vm_vmspace.c:183 β†’ sys/vm/vm_map.c:309-340; cache at vm_map.c:233
8192 B pageable kernel KVA (kmem_alloc_pageable(kernel_map, PAGE_SIZE*2, VM_SUBSYS_PML4)) sys/platform/pc64/x86_64/pmap.c:2368-2370
1 wired PML4 vm_page (pmap_allocpte(pmap_pml4_pindex()) β†’ vm_page_wire(m)) pmap.c:2376-2380 β†’ pmap.c:2757
pmap_pinit2() β€” no-op on x86_64 pmap.c:2509-2512

The finding text listed pmap_pinit2 among the cost contributors; on x86_64 it is an empty function (refinement, not a contradiction β€” the substance of the cost chain is confirmed).

3. Measured growth (stock kernel)

Three runs, ids 0x10000000+i, sampling vm.stats.vm.v_wire_count, vm.kvm_free, vm.stats.vm.v_free_count:

run creates Ξ”wire pages/create Ξ”kvm_free B/create time
1 (cold) 20,000 +47,526 2.376 195,035,136 9,752 1.543 s
2 (warm) 20,000 +46,066 2.303* 12,618,048* β€” 1.214 s
3 120,000 +284,878 2.374 970,981,376 ~9,999 6.915 s

() warm run reused objcache-retained objects (see Β§5); wire slope stays ~2.37 pages/create. Growth is linear (run 1 vs run 3 slope delta 0.1%). Rate: 17,350 creates/s* sustained.

120,000 live vmspaces = 431,414 wired pages observed mid-run (~1.68 GB wired held by one unprivileged process). Guest stayed healthy (load 0.18 after).

4. Exhaustion extrapolation β€” NOT executed (single-tenant guest)

  • total physical pages: 4,257,873,920 / 4096 = 1,039,519
  • quiescent wirable pool (wire 143,045 + free 849,484) = 992,529 pages
  • creates needed: 992,529 / 2.374 β‰ˆ 418,000
  • time: 418,000 / 17,350 β‰ˆ 24 seconds for one unprivileged process to wire the entire 4 GB of this guest (pageout cannot help: wired pages are unswappable).

Failure mode at the wall: vm_page_alloc(VM_ALLOC_NORMAL) inside pmap_allocpte (pmap.c:2751-2756) fails β†’ vm_wait(0) loop; because the exhaustion is in WIRED memory the wait can never be satisfied, and every M_WAITOK allocation system-wide blocks β†’ hard hang, no clean error. The path offers no error return before that point (kmalloc(M_WAITOK) at vm_vmspace.c:165/:182, vmspace_alloc(M_WAITOK)). This is the panic-free-but-wedge class: RUN timeout + unresponsive guest would be the observed artifact if executed; deliberately not driven to that state on the shared audit guest per policy β€” the linear rate + totals above are the honest proof of the exhaustion primitive.

5. Post-exit reclamation (hygiene + not-a-permanent-leak)

Process exit (vkernel_exit sys/vm/vm_vmspace.c:773-805 β†’ rb_vmspace_delete β†’ vmspace_entry_delete) reclaims live vmspace memory: run 3 dropped wire from 431,414 β†’ 154,535 within 3 s of exit. Some memory stays CACHED (objcache magazines retain vmspace objects with their pmap β€” see vm_map.c:244 comment; vmstat -m showed 390 MB M_VKERNEL retained right after exit) β€” run 2 proved this cache is REUSED, not leaked (Ξ”kvm_free β‰ˆ 0 for the first 15,000 warm creates). The DoS primitive is the LIVE pinned set, which the attacker controls the duration of.

6. Fix validation (A/B on patched kernel)

Patch: fix.diff β€” adds nvmspaces to struct vkernel_proc (sys/sys/vkernel.h), vm.max_vkernel_vmspaces sysctl (default 64), cap checked+incremented under vkp->token in the same critical section as the RB insert (mirroring the EEXIST unwind), decremented in vmspace_entry_delete under the token. Also the DF-2835 EINVAL guards.

Built with make nativekernel KERNCONF=X86_64_GENERIC in the guest scratch /usr/src (6.1 min, warning-free), installed, rebooted into #1: Tue Oct 20 11:55:22 UTC 2026. sysctl vm.max_vkernel_vmspaces: 64 present.

leg result
[A] unpriv vmsp_grow 100000 -u FAILED iter=64 rc=-1 errno=59 (ETOOMANYREFS) β€” clean error, wire flat at baseline
[B] cap arithmetic (vmsp_cap, uid 1001) create #65 β†’ ETOOMANYREFS; destroy β†’ 0; create again β†’ 0 (slot freed)
[C] legitimate use unchanged root raises cap to 100000 β†’ unpriv creates 5000 in 313 ms, all succeed, reclaimed on exit
[D] per-process cap two concurrent filler processes EACH hit #65 independently

Legitimate vkernel workloads (vkernel userland creates one vmspace per emulated process; typical counts are tens) sit far below 64; operators running huge vkernels raise one sysctl. Not boot-tested end-to-end (vkernel-in-vkernel out of scope for this guest) β€” reasoned from vmspace_create call sites.

Baseline vs patched: stock = unbounded linear growth, no error; patched = clean ETOOMANYREFS at the cap, legitimate paths intact.

7. Team directive: privesc?

NOT privesc. There is no uid=0 route β€” the primitive is resource exhaustion (impact dos). It is gated on vm.vkernel_enable=1, a root tunable that defaults to 0, so default-config machines return EOPNOTSUPP to unprivileged callers. On vkernel-hosting machines (tunable = 1, the supported configuration) any local unprivileged user can wedge the box in seconds. Medium severity is the correct rating; the finding stands as filed.

8. Guest hygiene

vm.vkernel_enable restored to 0 immediately after each stock-kernel run and after the fixed-kernel legs; guest vm.sh reset with-src performed at the end (patched kernel replaced by the clean snapshot kernel); guest_dirty = 0.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.

Fix verification

not_testable
↓ fix.diffper-fix-DF-2834

Confirmed kernel references

Detail

Exploit chain

1) operator sets vm.vkernel_enable=1 (vkernel host); 2) unprivileged local user runs vmsp_grow looping syscall 486 with distinct ids; 3) kernel pins 2.37 wired pages + 9.75KB KVA per create with no limit (17.3k creates/s measured); 4) about 418000 creates (about 24s measured rate) wire all 1039519 pages of the 4GB guest; 5) vm_page_alloc in pmap_allocpte fails and vm_wait cannot be satisfied because the exhaustion is in wired memory, stalling every M_WAITOK allocator system-wide - hard wedge of the machine (deliberately not executed to the wedge point; rate and totals measured and extrapolated).

Evidence (decisive lines)

run.log (stock: 20k cold +2.376 pages/create, 20k warm reuse proving retention is cache not leak, 120k capped run +2.374 pages/create linear, exit-reclaim samples), run.fixed.log (cap A/B legs A-D), VERDICT.md section 3-6 for the math and fix validation, build.log warning-free, env.txt for guest state

PoC changes

no seed pack existed (findings/poc/DF-2834 was empty); PoC written fresh in verify phase: syscall 486 via libc syscall(), self-drop to uid/gid 65534, per-sample sysctl accounting of vm.stats.vm.v_wire_count / vm.kvm_free / vm.stats.vm.v_free_count; run deliberately capped at 120k creates to keep the single-tenant guest healthy; vmsp_cap.c added for fix-validation cap arithmetic

Verified recommended fix

per-vkernel_proc nvmspaces counter capped by vm.max_vkernel_vmspaces (default 64), checked and incremented under vkp->token with the RB insert, decremented in vmspace_entry_delete; excess create returns ETOOMANYREFS

Verdict

Verified on the stock INVARIANTS kernel: with vm.vkernel_enable=1 an unprivileged process (uid 65534) created 120000 live vmspaces in 6.9 seconds at a steady 17350 creates/s with zero failures and no cap of any kind on the path (no rlimit, no priv_check, no sysctl; only the vkernel_enable gate at sys/vm/vm_vmspace.c:155). Each create linearly pins 2.374-2.376 wired pages (9.7KB unswappable: 1 wired PML4 page via pmap_allocpte/vm_page_wire at pmap.c:2757 plus vmspace slab) and 9752B of kernel KVA (8KB kmem_alloc_pageable at pmap.c:2368) until destroy or exit; pmap_pinit2 is a no-op on x86_64 (pmap.c:2509), refining the finding text. Extrapolated honestly per single-tenant guest policy: 992529 wirable pages / 2.374 per create = about 418000 creates = about 24 seconds for one unprivileged process to wire all 4GB; the exhaustion point is a vm_wait loop on unwireable memory (pmap.c:2751-2756) with no clean error return, i.e. a hard system wedge. NOT privesc (no uid=0 route; pure resource exhaustion) and gated on the root tunable that defaults to 0, so Medium severity stands for vkernel-hosting configurations. Fix validated A/B on a patched kernel (nativekernel build in the guest): default cap 64 stops the unpriv loop at create 65 with clean ETOOMANYREFS and flat wire count, destroy frees slots (counter decrement verified), two concurrent processes are capped independently, and with the operator-raised sysctl (vm.max_vkernel_vmspaces=100000) 5000 unpriv creates still succeed, so legitimate usage is unchanged.