DF-4507 — TTM page pools have no memory-pressure reclamation on DragonFly

File: sys/dev/drm/ttm/ttm_page_alloc.c (live, unguarded — compiled into drm.ko) Severity: Medium (local memory-pressure DoS amplification on drm systems) Confidence: certain (statically proven dead reclamation path) Verification status: untested at runtime — the QEMU audit guest has no GPU (see VERDICT.md); the defect itself is proven by static_proof.sh, which is fully reproducible offline.

What is broken

ttm_pool_mm_shrink_init() (ttm_page_alloc.c:442-449) tries to give the TTM page pools a memory-pressure hook two ways, and both are dead:

1. It fills in manager->mm_shrink.{count_objects,scan_objects,seeks} — but DragonFly has no register_shrinker() definition anywhere in sys/ (the Linux shim sys/dev/drm/include/linux/shrinker.h defines the struct types only). The callbacks are never called. (For comparison: i915's register_shrinker() call is #if 0'd out, and ttm_page_alloc_dma.c is entirely preprocessed out.)
2. It registers ttm_pool_shrink_scan via EVENTHANDLER_REGISTER(vm_lowmem, ...) — but no code in the entire kernel executes EVENTHANDLER_INVOKE(vm_lowmem, ...). The event list is created by the registration itself (slow-handler lists are created lazily) and then never fired. FreeBSD fires vm_lowmem from its VM subsystem; this DragonFly port never acquired the invoke site.

Net effect: the WC/UC page pools are never drained under VM pressure. The only live trimming is the put-path over-limit trim (ttm_page_alloc.c:709-721) at options.max_size, which defaults to half the kernel zone in pages (sys/dev/drm/ttm/ttm_memory.c:353). On a drm desktop (any system where radeon/amdgpu/i915 attach), an unprivileged client churning GPU buffers can leave up to max_size pages per pool permanently pinned; when the system later hits real memory pressure, the page daemon cannot reclaim them, converting reclaimable memory into avoidable OOM kills.

Secondary hazard: ttm_pool_shrink_scan was registered with the wrong signature. vm_lowmem handlers are invoked as func(void *arg, int flags); the scan callback dereferences its second argument as struct shrink_control * (sc->gfp_mask = M_WAITOK at line 398, sc->nr_to_scan at line 418). Anyone who naively wires the event up in the future turns this into a near-NULL pointer write (kernel panic). The proposed fix adds a typed adapter. (This deadness is also why DF-1663's fini-vs-shrinker race has no practical trigger on this tree.)

How to reproduce (static proof — no guest required)

./static_proof.sh > static_proof.log

Decisive lines in the output: - [1] vm_lowmem is referenced only by ttm_page_alloc.c:447,453. - [2] the complete list of EVENTHANDLER_INVOKE sites in sys/ — vm_lowmem is not among them (COUNT=0). - [3] register_shrinker has no definition; the only calls sit in #if 0 (i915_gem_shrinker.c:523-531) or in preprocessed-out code (ttm_page_alloc_dma.c:36). - [4] ttm_page_alloc.c has no preprocessing guard — it is live code. - [5] the only live reclamation: put-path trim vs options.max_size (ttm_memory.c:353). - [6] the signature landmine at lines 398/418.

Why no runtime PoC

The audit QEMU guest (DragonFly 6.5-DEVELOPMENT X86_64_GENERIC) has a QEMU std-VGA device (vendor 0x1234 device 0x1111), no /dev/dri, and no radeon/amdgpu/i915 attachment — the TTM pools are never instantiated there, so pool-growth-under-pressure cannot be demonstrated on it. See env.txt.

Fix

See fix.diff (never applied to sys/): declare the vm_lowmem event type in sys/sys/eventhandler.h, invoke it from the page daemon's catastrophic shortage block (sys/vm/vm_pageout.c), and register a correctly-typed adapter instead of casting the scan callback. Fix validation is not_testable on this guest for the same hardware reason.

DF-4507 VERDICT — untested at runtime; defect certain by static proof

Bottom line

The finding is certain: the memory-pressure reclamation path of the TTM page pool (sys/dev/drm/ttm/ttm_page_alloc.c) is dead code on DragonFly, by two independent and independently-verifiable facts:

1. ttm_pool_mm_shrink_init() assigns manager->mm_shrink.count_objects / scan_objects / seeks (ttm_page_alloc.c:444-446) but nothing can ever call them: register_shrinker() has no definition anywhere in sys/ (rg -n "register_shrinker" sys/ returns only the two call sites in dead code — i915_gem_shrinker.c:525 inside #if 0, and ttm_page_alloc_dma.c:1149 inside the whole-file #if defined(CONFIG_SWIOTLB) || defined(CONFIG_INTEL_IOMMU) guard at ttm_page_alloc_dma.c:36, which is never defined on DragonFly). The shim header sys/dev/drm/include/linux/shrinker.h defines the struct types only — there is no registration API at all.
2. The fallback registration EVENTHANDLER_REGISTER(vm_lowmem, ttm_pool_shrink_scan, manager, EVENTHANDLER_PRI_ANY) (ttm_page_alloc.c:447-448) creates a slow-handler list named "vm_lowmem" (eventhandler lists are created lazily at registration — sys/sys/eventhandler.h:128-129), but no code in the kernel ever executes EVENTHANDLER_INVOKE(vm_lowmem, ...) (rg -n "EVENTHANDLER_INVOKE" sys/ enumerates every invoke site in the tree: usb, acpi, shutdown, power, network — vm_lowmem is not among them). The event therefore never fires, ever.

Everything in the file was read end-to-end; both claims above are reproduced mechanically by static_proof.sh (output in static_proof.log).

Consequence chain (assessed)

• The four live pools (wc, uc, wc dma32, uc dma32) are trimmed only by the put path when pool->npages > _manager->options.max_size (ttm_page_alloc.c:709-721). max_size comes from ttm_page_alloc_init(glob, glob->zone_kernel->max_mem/(2*PAGE_SIZE)) (sys/dev/drm/ttm/ttm_memory.c:353) — i.e. half the kernel memory zone, in pages, per pool.
• ttm_page_alloc.c itself is live (no preprocessing guard; sys/dev/drm/ builds drm.ko from it; radeon/amdgpu sit in the same SUBDIR tree).
• On any system where a drm GPU driver attaches, unprivileged GPU clients (open /dev/dri/*) drive ttm_pool_populate → ttm_get_pages → pool fills, and ttm_pool_unpopulate → ttm_put_pages → pool growth. Freed WC/UC pages accumulate up to max_size per pool and are unreclaimable under VM pressure because both pressure hooks are dead.
• Under a genuine memory squeeze the page daemon cannot drain these pools, so the system takes OOM kills (or stalls) while potentially gigabytes are sitting in the pools. On Linux this exact situation is handled by the mm shrinker this port dropped.

Severity was set to Medium rather than High: the impact is availability-only, requires a drm-attached system plus a sustained unprivileged GPU workload and coincident memory pressure, and the pinned amount is bounded by 4 × max_size pages.

Interaction with the known pass-1 finding DF-1663

DF-1663 (fini races vm_lowmem shrinker on static_buf and _manager lifetime) is not re-reported here. Note however that its racing counterpart is precisely this dead shrinker: because the event never fires, DF-1663's race has no practical trigger on this tree either. Both resolve together by either wiring the event correctly (see fix.diff) or deleting the dead registration outright.

Why the status is untested (not not_reproduced)

The PoC would require instantiating the TTM pools, i.e. a GPU whose drm driver attaches. The audit QEMU guest has:

DragonFly dfbsd 6.5-DEVELOPMENT #0 X86_64_GENERIC
vgapci0@pci0:0:2.0: chip=0x11111234   (QEMU std VGA — no drm driver)
/dev/dri: absent
drm.ko: present at /boot/kernel/drm.ko, not loaded at boot

so the finding is neither reproducible nor refutable there; the static proof is the decisive evidence (and it is certain). A runtime validation would need bare-metal or passthrough GPU hardware.

Side observation from the guest session (out of scope for this file, recorded for the maintainers): kldload drm on the 6.5-DEVELOPMENT guest wedged the kernel linker lock (kldstat afterwards blocks forever in state D on kldlk; three such processes captured). drm_core_init() itself is #if 0-gutted, and MODULE_DEPEND(drm, agp, ...) (sys/dev/drm/drm_drv.c:1143) forces the agp.ko dependency load — the wedge lives in that module-load path, not in ttm_page_alloc.c (whose init runs later, per ttm device global init). The guest was reset clean afterwards (vm.sh reset with-src, verified up).

Exploit chain

None (availability class). Sketch of the DoS chain that a GPU-equipped system exposes: unprivileged user churns WC/UC GEM buffers → pools saturate at max_size pages each → user (or any other load) drives the system into the page daemon's shortage path → pools are not drained (dead hooks) → OOM kills of unrelated processes. Chain not executable on this guest (no GPU).

Fix validation

not_testable on this guest (baseline cannot be reproduced without a GPU). fix.diff proposes the minimal correct wiring:

1. sys/sys/eventhandler.h: declare the event with a real type (typedef void (*vm_lowmem_fn)(void *, int); EVENTHANDLER_DECLARE(...)), which is also what makes future EVENTHANDLER_INVOKE sites type-safe.
2. sys/vm/vm_pageout.c: fire EVENTHANDLER_INVOKE(vm_lowmem, 0) in the catastrophic-shortage block of vm_pageout_scan (before the OOM-kill path), where FreeBSD-equivalent semantics belong.
3. sys/dev/drm/ttm/ttm_page_alloc.c: register a typed adapter (ttm_pool_lowmem(void *, int)) that builds a local struct shrink_control and calls ttm_pool_shrink_scan; drop the dead mm_shrink assignments. Registering the raw ttm_pool_shrink_scan (current code) would, once the event is wired, cast the integer flags argument to struct shrink_control * and write through it (ttm_page_alloc.c:398) — a near-NULL write and instant panic.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.