DF-4576 — amdgpu GEM_VA alloc_pts-before-validate: page-table memory exhaustion

What it is

amdgpu_gem_va_ioctl() (MAP and REPLACE) allocates VM page tables for the whole attacker-supplied [va_address, va_address+map_size) range via amdgpu_vm_alloc_pts() before amdgpu_vm_bo_map() validates offset_in_bo/map_size against the BO. When the validation rejects the mapping (which the attacker deliberately provokes), the freshly-allocated page-table BOs are never freed — they stay owned by the per-fd VM until the DRM fd is closed. One 4 KiB BO + one failed ioctl pins ~(span >> 18) bytes of GTT/VRAM (128 TiB span → ~512 MiB). Repeat with fresh render-node fds → GTT/kernel-memory exhaustion → GPU unusable for every process (local, cross-process DoS by an unprivileged render-node client).

Files

• gem_va_pt_exhaust.c — self-contained PoC (raw ioctls, no libdrm needed)
• build.sh / run.sh — exact commands
• build.log / run.log / env.txt — guest evidence
• VERDICT.md — full analysis narrative + line-level trace
• fix.diff — validation-before-allocation patch (NOT applied to sys/)
• manifest.json / verdict.json

Build

cc -O2 -Wall -o gem_va_pt_exhaust gem_va_pt_exhaust.c

(verified to build clean on the DragonFly 6.5-DEVELOPMENT audit guest)

Run

./gem_va_pt_exhaust [iterations] [span_bytes_hex]     # defaults: 64, 0x800000000000 (128 TiB)

Expected output

On real AMD hardware (amdgpu render node present): - every iteration prints MAP rejected (Invalid argument) — page tables for 128.0 TiB of VA now pinned by fd N — the EINVAL is the rejection of the mapping; the leak is the already-committed page tables; - GTT usage grows ~512 MiB per iteration (cat /sys/kernel/debug/dri/<n>/amdgpu_mm_info); - after enough iterations: amdgpu allocation failures in dmesg, GPU clients fail for all users.

On a machine without an AMD GPU (this audit's QEMU guest: virtio VGA only, /dev/dri absent, amdgpu not even in the guest kernel build):

iter 0: cannot open any DRM node (No such file or directory)
  -> no AMD GPU on this machine; PoC needs real amdgpu hardware.

exit code 2 — which is what run.log records.

DF-4576 VERDICT — amdgpu GEM_VA alloc_pts-before-validate page-table exhaustion

Status: untested (attempted on the audit guest; target hardware absent — see "Verification attempts"). Impact if hardware present: dos (local, cross-process GPU/GTT memory exhaustion by an unprivileged render-node client). Confidence in the code path: high — pure code trace, every hop cited.

Root cause (line-level trace)

1. amdgpu_gem.c:637-646 (AMDGPU_VA_OP_MAP) and amdgpu_gem.c:657-666 (AMDGPU_VA_OP_REPLACE): the ioctl calls amdgpu_vm_alloc_pts(adev, bo_va->base.vm, args->va_address, args->map_size) with the raw, still-unvalidated user arguments, and only afterwards calls amdgpu_vm_bo_map(... args->offset_in_bo, args->map_size ...).

2. amdgpu_vm.c:945-971 (amdgpu_vm_alloc_pts): the only validation is GPU-page alignment of saddr/size and eaddr >= max_pfn. For anything inside the VA limit it then walks the whole range (for_each_amdgpu_vm_pt_leaf, amdgpu_vm.c:973) and allocates a BO per missing page table (amdgpu_vm.c:995) plus kvmalloc_array entry arrays (amdgpu_vm.c:981). Nothing in the ioctl frees any of this.

3. amdgpu_vm.c:2503-2512 (amdgpu_vm_bo_map): the actual mapping validation — size == 0, page alignment, saddr >= eaddr wrap, and offset + size > amdgpu_bo_size(bo) — runs after the page tables are committed. An attacker forces this check to fail by passing a huge, page-aligned offset_in_bo (e.g. 0x1000000000000000), so the mapping is rejected with -EINVAL.

4. amdgpu_gem.c:675-680 (error path): error_backoff: merely releases the reservations; the page tables allocated in step 2 remain in the VM's page-table tree (vm->root) until VM teardown at fd close. Page tables are not freed on unmap in this driver generation, and here there is not even a mapping to unmap — the allocation is pure waste.

Damage arithmetic

With 4 KiB page tables of 512 PTEs mapping 2 MiB pages, one PTB covers 1 GiB of VA. A failed MAP spanning S bytes of VA pins ≈ S / 2^30 × 4 KiB = S / 2^18 bytes: - 128 TiB span → ~512 MiB per ioctl - 256 TiB (full 48-bit VM) → ~1 GiB per ioctl

Each iteration needs only a 4 KiB GTT BO and one doomed ioctl. Opening a new render-node fd creates a fresh VM (amdgpu_kms.c: amdgpu_driver_open), so the loop open → GEM_CREATE(4K) → GEM_VA(MAP, doomed) × N pins N × ~1 GiB until GTT is full. Consequences: amdgpu TTM allocations start failing for every process using the GPU (cross-process DoS), kernel kvmalloc memory is consumed by PD entry arrays, and the per-ioctl cost includes amdgpu_vm_clear_bo of every new PT (long stalls). GTT is not freed until the attacker closes the fds; the wedge persists while they are open. No MAP_SIZE-vs-GTT accounting exists anywhere on this path.

Notably this is an amplification primitive: the legitimate way to force page-table allocation (a real mapping) is bounded by offset+size <= amdgpu_bo_size(bo), i.e. by how much GTT the attacker already paid for. The bug removes that bound — 4 KiB of BO buys ~1 GiB of page tables.

Threat model / reachability

• Ioctls are registered DRM_AUTH|DRM_RENDER_ALLOW (amdgpu_kms.c:1197-1213) → reachable by any unprivileged user with render-node access (/dev/dri/renderD128, conventionally 0666 or group video).
• No GPU command submission, no special privileges needed — just the two ioctls.

Distinction from known findings

• DF-1719 is the wrap of offset+size inside amdgpu_vm_bo_map (GPU page table corruption). This finding is the ordering bug in the ioctl — the leak happens even with non-wrapping, honest-to-bounds-check failure, and nothing is corrupted; the resource is simply never reclaimed.
• DF-4522 (vm fini leak of mappings) is a different lifetime bug at teardown; the leak here persists during the fd's whole lifetime regardless of fini.

Verification attempts (why "untested")

1. PoC authored self-contained (raw Linux-encoding ioctls, no libdrm), fixed a comment-termination compile error, built clean on the DragonFly 6.5-DEVELOPMENT guest (build.log: BUILD_RC=0).
2. Executed as the unprivileged user (run.log): iter 0: cannot open any DRM node (No such file or directory), exit 2.
3. Guest environment (env.txt): QEMU virtio VGA only (chip 0x1111:1234), /dev/dri does not exist, amdgpu does not appear in the guest's sys/conf/files (driver not in this kernel build), no drm kld, no drm/ amdgpu lines in dmesg. The audit guest cannot attach an AMD GPU, so the finding cannot be executed to reproduction here — it requires real AMD hardware (or an AMD-GPU CI runner). Status recorded as untested (device absent), impact none on this guest; expected impact on target hardware: dos.

The path itself is live in the shipped tree: the code under audit is unconditional in amdgpu_gem.c and its call graph (amdgpu_gem_va_ioctl → amdgpu_vm_alloc_pts → amdgpu_bo_create) has no hardware-dependent guards beyond device attach.

Fix validation

fix.diff adds the bo_map-equivalent validation (alignment, size, wrap of both va_address+map_size and offset_in_bo+map_size, and the offset+size <= amdgpu_bo_size(bo) bound) to the ioctl before amdgpu_vm_alloc_pts(), so a mapping that would be rejected can no longer allocate page tables. The wrap guards additionally pre-fix the DF-1719 wrap for the MAP/REPLACE ioctl path. Fix validation (fix_status) is not_testable for the same reason as above: the guest cannot attach amdgpu hardware, so baseline-vs-patched comparison cannot be executed. The patch is a pure argument pre-check inserted before the operation switch; the cited identifiers (AMDGPU_GPU_PAGE_MASK via amdgpu_gart.h, amdgpu_bo_size() via amdgpu_object.h) are both already visible in amdgpu_gem.c's include set.

Bottom line

Real, unbounded, attacker-repeatable resource-exhaustion primitive on the amdgpu render node, inherited from the Linux 4.x-era ioctl ordering, present verbatim in DragonFly's tree. Severity Medium (local DoS, requires GPU render-node access — no privilege, no memory corruption). Fix is a five-line argument check before the allocation.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.