β¬’ DragonFlyBSD Kernel Audit
← triage Β· dashboard
DF-4791

ttm_shrink ↔ ttm_bo_swapout unbounded recursion: the TTM memory-accounting shrinker re-enters itself through the lazy page-populate of the very BO it is swapping out β€” attacker-controlled nesting depth exhausts the 16KiB kernel stack

Summary

alloc_zone calls ttm_shrink on limit failure with only a 4-iteration retry counter and no recursion guard; ttm_bo_swapout moves a VRAM BO to SYSTEM first (no_wait_gpu false), whose lazy populate goes back through alloc_page -> alloc_zone -> ttm_shrink while the zone is still over limit. One BO per level off swap_lru, ~12 frames/level vs the 16KiB kernel stack (no guard page) - an unpriv /dev/dri client parks 64 unpopulated VRAM BOs and pushes the zone over limit: stack smash/corruption or panic. Chain proven hop-by-hop at path:line (no GPU on guest); upstream corroborates by deleting the whole accounting shrinker (b12dacf5fa64, 2021). Nesting-cap fix.

Discussion (0)

No comments yet.

PoC verification

Evidence pack

findings/poc/DF-4791 Β· 11 files
FileTypeDescriptionSize
README.md β€” 2.2 KB ↓ raw
VERDICT.md β€” 5.4 KB ↓ raw
ttm_shrink_recursion.c β€” 5.1 KB view raw
build.sh β€” 114 B view raw
run.sh β€” 116 B view raw
build.log β€” 9 B view raw
run.log β€” 123 B view raw
env.txt β€” 340 B view raw
fix.diff β€” 1.8 KB view raw
manifest.json β€” 1.3 KB view raw
verdict.json β€” 3.5 KB view raw

DF-4791 β€” TTM memory-accounting shrinker unbounded recursion β†’ kernel stack exhaustion

What to run

./build.sh          # cc -O2 -Wall -o ttm_shrink_recursion ttm_shrink_recursion.c
./run.sh            # ./ttm_shrink_recursion 64 268435456

Expected output

On a machine with a radeon- or amdgpu-driven GPU and /dev/dri/card0 readable by the invoking user (the trigger uses the radeon GEM uapi; amdgpu has the identical move/populate path):

  1. 64 VRAM BOs parked on swap_lru (ttm unpopulated) β€” recursion fuel (fresh VRAM BOs whose ttm_tt exists but was never populated).
  2. N x 4MiB GTT BOs β€” each populated page pushes the TTM kernel zone's used_mem up by PAGE_SIZE until it exceeds max_mem (ttm_mem_global_reserve, ttm_memory.c:477-481).
  3. The next accounting-checked allocation enters ttm_mem_global_alloc_zone β†’ ttm_shrink β†’ ttm_bo_swapout, which picks a phase-1 VRAM BO and moves it to SYSTEM β€” the move must populate the tt, the populate's first ttm_mem_global_alloc_page re-fails the limit check and calls ttm_shrink again, nesting once per parked BO (~12 frames / ~1-2 KiB per level vs a 16 KiB kernel stack, UPAGES=4, sys/cpu/x86_64/include/param.h:126).
  4. Success criterion: kernel panic / stack-smash splat with a backtrace repeating ttm_shrink / ttm_bo_swapout / ttm_bo_handle_move_mem / ttm_pool_populate frames, or non-deterministic corruption of kernel memory adjacent to the overflowing thread stack (thread stacks are kmem allocations on DragonFly β€” no guard page).

On a machine without a DRM device (this audit guest: QEMU std-VGA): the run exits 2 at open() β€” recorded in run.log; the defect stands on the source-level proof in VERDICT.md.

Files

  • ttm_shrink_recursion.c β€” trigger source (self-contained uapi defs)
  • build.sh, run.sh β€” exact commands
  • build.log, run.log β€” guest compiler + runtime output (untrimmed)
  • env.txt β€” guest state (uname, /dev/dri, vm.dma_reserved)
  • VERDICT.md β€” full narrative with the path:line recursion chain
  • fix.diff β€” bounded-nesting guard in ttm_memory.c (advisory)
  • manifest.json, verdict.json β€” machine-readable records
VERDICT.md
↓ download raw

DF-4791 β€” VERDICT

Finding: unbounded recursion between ttm_shrink() and ttm_bo_swapout() in TTM memory accounting exhausts the 16 KiB kernel stack; depth is attacker-controlled via the number of swappable VRAM-resident buffer objects on ttm_bo_global::swap_lru.

Status: not_reproduced on the audit guest β€” feature absent. The audit guest is QEMU with std-VGA: no radeon/amdgpu/i915 hardware, no /dev/dri (env.txt, run.log). The trigger builds cleanly on the guest (build.log: BUILD_OK) and exits 2 at open("/dev/dri/card0", ENOENT). The bug is proven structurally below; runtime proof requires a GPU-driven DRM device, which this guest cannot provide (kernel DRM drivers in this tree bind only real hardware).

The recursion chain (every hop verified in this tree)

  1. Any accounting-checked allocation under memory pressure: ttm_mem_global_alloc_zone (sys/dev/drm/ttm/ttm_memory.c:502-521) fails ttm_mem_global_reserve (used_mem > max_mem/emer_mem, ttm_memory.c:477-481) and, because ctx->no_wait_gpu == false, calls ttm_shrink (ttm_memory.c:517).
  2. ttm_shrink (ttm_memory.c:213-229) drops glob->lock and calls ttm_bo_swapout (ttm_memory.c:222 β†’ sys/dev/drm/ttm/ttm_bo.c:1768).
  3. ttm_bo_swapout picks the next BO from swap_lru (ttm_bo.c:1776-1785). Which BOs sit there? ttm_bo_add_to_lru (ttm_bo.c:182-187) adds every BO with a ttm that is not SG and not already swapped β€” including freshly-created, never-touched VRAM BOs whose ttm_tt exists in tt_unpopulated state (ttm pages are populated lazily). An unprivileged /dev/dri client creates those at will via GEM create.
  4. For such a BO (bo->mem.mem_type != TTM_PL_SYSTEM), swapout must move it to SYSTEM|CACHED first (ttm_bo.c:1807-1819) using its local ctx = { .interruptible=false, .no_wait_gpu=false } (ttm_bo.c:1809).
  5. The move (radeon_bo_move VRAM→SYSTEM → radeon_move_vram_ram's staging TT bind, or directly ttm_bo_handle_move_mem for a non-FIXED destination, ttm_bo.c:331-347) reaches ttm_tt_bind → ttm_tt_populate (sys/dev/drm/ttm/ttm_tt.c:341) → driver populate → ttm_pool_populate (sys/dev/drm/ttm/ttm_page_alloc.c:894-917).
  6. ttm_pool_populate accounts each page: ttm_mem_global_alloc_page (ttm_page_alloc.c:910) β†’ ttm_mem_global_alloc_zone β€” and we are still above the zone limit (that is why step 1 is swapping), no_wait_gpu is still false (step 4's ctx) β†’ ttm_shrink again (ttm_memory.c:517).

β†’ step 2 with the next swap_lru entry (the current BO was removed from the LRU at ttm_bo.c:1800 before the move, so each level descends to a fresh BO). The count in ttm_mem_global_alloc_zone bounds iterations per level, never nesting depth; the only depth bound is the swap_lru run length β€” attacker-controlled.

Why it overflows

Per nesting level the kernel stack gains β‰₯10 frames: ttm_shrink β†’ ttm_bo_swapout β†’ ttm_bo_handle_move_mem β†’ radeon_bo_move β†’ radeon_move_vram_ram β†’ ttm_bo_move_ttm β†’ ttm_tt_bind β†’ ttm_tt_populate β†’ radeon_ttm_tt_populate β†’ ttm_pool_populate β†’ ttm_mem_global_alloc_page β†’ ttm_mem_global_alloc_zone β†’ (next ttm_shrink). ~1-2 KiB per level. DragonFly x86_64 kernel stacks are UPAGES * PAGE_SIZE = 4 Γ— 4 KiB = 16 KiB (sys/cpu/x86_64/include/param.h:126); lwkt thread stacks are kmem allocations with no guard page, so overflow smashes adjacent kernel memory before/instead of faulting. ~10-16 levels suffice; the trigger parks 64.

Attack prerequisites

  • Machine with a GPU driven by this tree's radeon or amdgpu driver (both share the TTM core; amdgpu has the same VRAMβ†’system staging path via amdgpu_move_vram_ram).
  • Unprivileged local user able to open /dev/dri/card0 (video/operator class membership on a desktop system).
  • ~zone-limit worth of populated GTT BOs to push used_mem over max_mem (zone sized from the vm.dma_reserved pool at driver load, ttm_memory.c:334 β€” typically tens of MiB) plus Nβ‰₯32 untouched VRAM BOs as recursion fuel. All GEM ioctls, unprivileged.

Impact

Kernel stack overflow from unprivileged context = kernel memory corruption (adjacent-kmemory smearing) / guaranteed panic. Primary classification: local DoS with memory-corruption ceiling; controlled exploitation beyond panic is not claimed.

Not a false positive / not already-fixed

  • The whole chain exists in this tree today β€” every hop cited above.
  • The DFly kobject/workqueue compat introduces no guard that would cap nesting (linux_kobject.c only stubs sysfs; workqueue is real).
  • Upstream Linux removed this entire mechanism in 2021 (commit b12dacf5fa64 "drm/ttm: nuke memory accounting shrinker"), citing the shrinker's re-entrancy/deadlock problems β€” strong evidence the mechanism, not just our trigger, is unsound.

Verification honesty

  • Guest run attempted: build OK, run exits 2 (no DRM device) β€” run.log.
  • No kernel was patched (fix.diff is authored, never applied to sys/), so no fix validation run was possible either β€” see verdict.json fix_status: not_testable.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.

Fix verification

not_testable
baseline no→ patch + rebuild →patched clean

No DRM device on the audit guest, so neither baseline nor patched kernel can execute the trigger; fix.diff is validated by inspection only (guard increments outside glob->lock, caps depth at 4 x ~2KiB well within the 16KiB stack).

fix.diff in this directory; guest hardware inventory in env.txt (Bochs VGA, no /dev/dri).
↓ fix.diffper-fix-DF-4791

Confirmed kernel references

Detail

Evidence (decisive lines)

VERDICT.md (full path:line recursion chain); run.log (build OK on guest, open ENOENT - feature absent); env.txt (no /dev/dri, no drm modules, Bochs VGA); fix.diff (bounded-nesting guard, advisory)

PoC changes

PoC authored fresh (no seed): had to inline Linux-style drm ioctl encodings because the DFly guest ships no drm uapi headers and its _IOWR is the 3-arg BSD form; second build attempt succeeded.

Verified recommended fix

Cap ttm_shrink nesting depth in ttm_memory.c (bounded guard, see fix.diff); long-term follow upstream b12dacf5fa64 and remove the accounting shrinker.

Verdict

Structurally proven but not runtime-reproducible on the audit guest: ttm_mem_global_alloc_zone (ttm_memory.c:502-521) re-enters ttm_shrink (ttm_memory.c:517) with no recursion guard whenever ttm_bo_swapout's move-to-system step (ttm_bo.c:1807-1819, ctx no_wait_gpu=false at :1809) must lazily populate the victim BO's ttm pages (ttm_bo.c:344 -> ttm_tt.c:341 -> ttm_page_alloc.c:910 -> ttm_mem_global_alloc_page). Nesting depth equals the run of unpopulated-tt VRAM BOs an unprivileged /dev/dri client parks on swap_lru (ttm_bo.c:182-187), ~1-2KiB of kernel stack per level against the 16KiB x86_64 kernel stack (UPAGES=4) -> kernel stack overflow / adjacent-kernel-memory corruption. The audit guest has no DRM device (QEMU Bochs VGA chip 0x1234:1111, no /dev/dri, no drm modules), so the trigger exits 2 at open(); the PoC builds cleanly on the guest. Upstream removed this entire accounting shrinker in 2021 (b12dacf5fa64 'drm/ttm: nuke memory accounting shrinker'), corroborating that the mechanism is unsound. impact recorded as 'none' strictly for the guest runtime observation; the source-level classification is local kernel-memory-corruption/DoS (High) on GPU-equipped machines.