β¬’ DragonFlyBSD Kernel Audit
← triage Β· dashboard
DF-2232

Signed integer overflow in idr_grow() growth loop causes kernel hang or panic-sized kmalloc

Summary

idr_grow() doubles nf with nf=2*nf+1 in signed int until nf>want. For want>=2^29 loop exits with nf=2^30-1 or 2^31-1 driving kmalloc to request 16-32GiB panicking/hanging under M_WAITOK. For want==INT_MAX predicate INT_MAX<=INT_MAX true so loop runs once more 2*INT_MAX+1 is signed overflow wraps to -1 and -1<=want keeps loop spinning forever holding lwkt token. Starting from 255=2^8-1 after k iterations nf=2^(8+k)-1. At k=23 nf=2^31-1=INT_MAX. want flows from caller-supplied start/sid: idr_alloc passes start directly idr_get_new_above stores idr_maxwant=max(maxwant sid) before returning -EAGAIN retry pattern calls idr_pre_get which grows to idr_maxwant. No upper-bound check on want sid or nf anywhere in file. In-tree DRM callers currently bounded (drm_gem.c start=1 amdgpu PASID 16-bit) but API provides no guardrail. Impact: system-wide DoS livelock on token or kernel panic. Any caller of idr KPI reachable once DRM device open. Latent DoS primitive fires moment any caller forwards user-influenced integer into start/sid. [VERIFIED pass2: reproduced 3 ways - idr_alloc start 2^29 drives live-captured kmalloc 17179869168 bytes M_WAITOK that never returns; sid INT_MAX poisons idr_maxwant and idr_pre_get spins forever holding the token while a second thread livelocks; proven on both verbatim copy and shipped binary via hard-IRQ watchdog; fix-validated - IDR_MAX_IDS bound plus ENOMEM propagation, largest post-fix request 4080 bytes; NOT privesc - wrapped nf can never under-allocate]

Discussion (0)

No comments yet.

PoC verification

Evidence pack

findings/poc/DF-2232 Β· 24 files
FileTypeDescriptionSize
df2232_rig.c β€” 18.7 KB view raw
Makefile β€” 271 B ↓ download
build.sh β€” 92 B view raw
run.sh β€” 135 B view raw
build.log β€” 5.5 KB view raw
env.txt β€” 599 B view raw
run.a.log β€” 2.3 KB view raw
run.b.log β€” 3.3 KB view raw
panic.b.txt β€” 3.3 KB view raw
run.c1.log β€” 2.8 KB view raw
panic.c1.txt β€” 2.9 KB view raw
run.c1b.log β€” 2.8 KB view raw
panic.c1b.txt β€” 2.7 KB view raw
panic.c2.txt β€” 2.7 KB view raw
fix.diff β€” 2.1 KB view raw
fix_build.log β€” 9.4 KB view raw
fix_run.log β€” 3.9 KB view raw
fix/run-all.log β€” 257 B view raw
fix/console-fixed.log β€” 3.9 KB view raw
fix/control-fixed.log β€” 4.2 KB view raw
fix/kbuild-tail.log β€” 9.0 KB view raw
fix/build-fixed.log β€” 5.5 KB view raw
VERDICT.md β€” 7.7 KB ↓ raw
verdict.json β€” 5.4 KB view raw
VERDICT.md
↓ download raw

VERDICT β€” DF-2232 (sys/libkern/linux_idr.c idr_grow signed-overflow growth loop)

Status: REPRODUCED β€” impact dos (hang / panic-sized kmalloc), confidence certain. Fix authored and validated on a rebuilt kernel. No memory-unsafety path exists (analysis below); no uid=0 chain is possible from this bug.

Environment

  • Guest: DragonFly 6.5-DEVELOPMENT, kernel X86_64_GENERIC (options INVARIANTS), 6 vCPU, 4 GiB RAM, QEMU/KVM. Baseline kernel: #0 Thu Jul 2 06:02:54 UTC 2026. Fixed kernel: #1 Sat Oct 17 10:36:44 UTC 2026.
  • Compiler: cc 8.3 (DragonFly base); module built with kernel flags -O2 -fno-strict-overflow (signed wrap deterministic, same as kernel).
  • Verbatim gate: guest /usr/src/sys/libkern/linux_idr.c sha256 22d421c35bfd7ebc7ae2df137b52b7ece0a9cf349a8f9369b440389e033a95de β€” equal to the audited tree; the rig includes that file byte-identical.

What was run (evidence trail)

  1. Control (run.a.log): phases 1+2 β€” normal wants 1..1000 round-trip on the verbatim KLD copy and on the kernel's own exported KPI. Both PASSED with identical results (grow 255β†’1023, pre_get/get_new_above protocol settles in 2 iterations, remove/realloc reuses slot 1000). This also demonstrates the idr_maxwant poisoning protocol the attack rides on: first idr_get_new_above(sid) with a fresh higher sid returns -EAGAIN after raising idr_maxwant (linux_idr.c:327-331).

  2. Giant kmalloc (run.b.log, panic.b.txt): idr_alloc(start=1<<29, end=0) through the verbatim copy. The allocator-boundary hook (outside the verbatim file) captured the request live:

[DF2232-B] ALLOCATOR BOUNDARY: kmalloc(17179869168 bytes [0x3fffffff0], M_IDR, flags 0x102) -> real allocator

17179869168 = (2^30-1) * sizeof(struct idr_node=16) β€” exactly the predicted ~16 GiB M_ZERO|M_WAITOK request. On the 4 GiB guest the allocation never returned (page-by-page vm_page_alloc + vm_wait() in kmem_slab_alloc, sys/kern/kern_slaballoc.c:1706-1800; the per-type limit check in _kmalloc only trips on cumulative use, not on a single first request). The hard-IRQ watchdog panicked at 15 s with the captured request size in the marker; backtrace shows it fired from systimer_intr β†’ pcpu_timer_process_oncpu β†’ splz_timer (interrupt context, proving the design requirement: immune to the wedge).

  1. Infinite spin, verbatim copy (panic.c1.txt; repeat in panic.c1b.txt): thread A: idr_get_new_above(sid=INT_MAX) β†’ -EAGAIN, idr_maxwant=2147483647 poisoned; then idr_pre_get β†’ never returns (a_done=0, a_stage=2): the growth loop computed 2*INT_MAX+1 β†’ -1 and spins forever holding the lwkt idr token. Thread B called idr_remove on the same idr and never returned (b_started=1, b_acquired=0 in run 1; in the repeat run B hung already inside its pre-spin token check) β€” the livelock the finding predicted. Watchdog panic at 15 s with the state flags.

  2. Infinite spin, shipped binary (panic.c2.txt): identical outcome with the kernel's OWN exported idr_get_new_above / idr_pre_get (last_kmalloc_req=0: the rig's kmalloc hook is not in this path β€” the spinning code is the stock kernel text at 0xffffffff809d5590). This closes the "your recompile behaves differently" objection.

Root cause (line-accurate)

  • sys/libkern/linux_idr.c:427-430 β€” nf = 2*nf+1; while (nf <= want) in signed int: want β‰₯ 2^29-1 drives the kmalloc size toward (2^30-1)*16 = 16 GiB; want == INT_MAX makes 2*INT_MAX+1 wrap to -1 and the predicate -1 <= INT_MAX true forever.
  • sys/libkern/linux_idr.c:437-438 β€” the wrapped/oversized nf flows into kmalloc(nf * sizeof(struct idr_node), ..., M_ZERO|M_WAITOK).
  • Reachability of want: idr_alloc start (linux_idr.c:358-375, only start < 0 rejected), idr_get_new_above sid β†’ idr_maxwant (linux_idr.c:327-331) β†’ idr_pre_get grow (linux_idr.c:274-275). No upper bound anywhere. In-tree callers (drm_gem start=1; amdgpu PASID 16-bit) are bounded β€” latent KPI hazard, hence the rig drives the exported KPI.

Why DoS and not memory corruption / privesc (team directive)

  • The wrapped nf = -1 cannot escape the loop into the size computation: the do-while only exits when nf > want; exiting with negative nf requires want < -1, and every reachable want is non-negative (idr_alloc rejects negative start at :365-366; idr_maxwant only rises via max() at :328; the forced-growth want = idr_count at :303 is > 0).
  • Even in the impossible case, nf * sizeof(struct idr_node) promotes nf to size_t: (size_t)(-1) * 16 = 0xFFFFFFFFFFFFFFF0 β€” a gigantic OVER-allocation request (allocator refuses), never an under-allocation. The subsequent bcopy at :451-452 uses the OLD idr_count, and the race guard at :441 frees the new buffer if it is not larger. There is no path to a smaller-than-needed allocation, so no heap corruption, no forged-object primitive, no uid=0 chain. Impact ceiling: kernel-mode livelock holding an lwkt token, unbounded M_WAITOK draw (memory exhaustion of every page into wired zero pages), watchdog-observable hang β€” DoS.

Fix validation (fix.diff, fix_run.log, fix_build.log)

fix.diff: idr_grow() becomes static int, rejecting want >= IDR_MAX_IDS ((1<<24)-1) with ENOMEM; growth loop otherwise unchanged (with the cap, max computed nf = 2^25-1 β€” no signed overflow is reachable); idr_pre_get maps failure to result 0 (its documented failure contract β€” drm-style callers then give up instead of looping), idr_alloc maps it to -ENOMEM (both grow sites, token correctly released via the existing done: path).

Cycle: guest reset to clean source β†’ patch applied in guest /usr/src (patch -p1, 6/6 hunks) β†’ make -j6 nativekernel KERNCONF=X86_64_GENERIC (320 s, -Werror clean) β†’ installkernel β†’ reboot into #1 Sat Oct 17 10:36:44. Rig rebuilt against the FIXED file (sha 1c04da32…, recorded in build.log) and all phases re-run:

  • control phases: PASSED, output identical to baseline (fix/run-all.log, fix/control-fixed.log);
  • phase 3: idr_alloc RETURNED -12 β€” and the hook proves the largest request that reached the allocator was the 4080-byte idr_init allocation: no giant kmalloc was ever issued;
  • phases 4/5: pre_get RETURNED 0, a_done=1 b_acquired=1, "no spin, no livelock" β€” on both the fixed verbatim copy and the fixed kernel functions;
  • grep -c panic fix_run.log β†’ 0; guest stayed up throughout.

Note on rig history: the first fixed-kernel validation run completed all phases successfully but crashed AFTER success during kldunload β€” a defect in my first rig (systimer teardown race + a leaked idr freed the module text too early), not in the fix; corrected by keeping the module resident (EBUSY) after crash-phases and destroying the leftover idr; the second run (fix_run.log) is clean end-to-end. The crash and its analysis are part of the serial console history and were re-verified to not involve linux_idr.c code paths beyond malloc_uninit bookkeeping.

Bottom line

The finding is fully confirmed on the stock kernel, at exact byte sizes, with livelock proof, on both a verbatim recompile and the shipped binary; the one-line-bound fix eliminates both manifestations with bit-identical normal-path behavior. Class: latent-KPI kernel DoS (Medium) β€” no privilege-escalation potential exists for this bug.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.

Fix verification

fixed
baseline reproduced→ patch + rebuild →patched clean

Single-fix kernel (patch -p1, make nativekernel, 320s, -Werror clean, installkernel, reboot): baseline manifestations gone - phase 3 returns -ENOMEM with the largest allocator request observed at 4080 bytes (idr_init), phases 4/5 show pre_get RETURNED 0 and a_done=1 b_acquired=1 (no spin, no livelock) on both the fixed verbatim copy and the fixed kernel functions; control phases bit-identical to baseline; zero panics; guest stayed up.

fix_run.log (fixed-kernel serial console), fix/control-fixed.log (control PASSED), fix_build.log (kernel build), build.log (rig rebuilt against the fixed file, sha 1c04da32d604f7b5164832be1da227b24980c0804665facdf20fe0fa263275e3)
↓ fix.diffDragonFly dfbsd 6.5-DEVELOPMENT #1: Sat Oct 17 10:36:44 UTC 2026 root@dfbsd:/usr/obj/usr/src/sys/X86_64_GENERIC x86_64

Confirmed kernel references

Detail

Evidence (decisive lines)

run.b.log + panic.b.txt line kmalloc(17179869168 bytes [0x3fffffff0]) at the allocator boundary followed by watchdog panic DF2232-WD; panic.c1.txt / panic.c1b.txt state a_stage=2 a_done=0 b_started=1 b_acquired=0 (infinite spin, token livelock); panic.c2.txt same on the shipped binary with last_kmalloc_req=0; fix_run.log shows idr_alloc RETURNED -12, pre_get RETURNED 0, a_done=1 b_acquired=1, zero panics on the fixed kernel; fix/control-fixed.log shows control PASSED unchanged

PoC changes

Built the rig from scratch (no seed existed): verbatim #include of linux_idr.c with exported-symbol rename guard and sha256 gate; kmalloc observation wrapper outside the verbatim file; systimer watchdog armed on a remote CPU firing from hard-IRQ context so every phase terminates with decisive output; second kernel thread as token-livelock probe; fixed rig defects found during runs (expectation arithmetic 2047 vs 1023, EPERM on nonzero sysctl return, unload-time systimer race -> module stays resident after crash-phases)

Verified recommended fix

Bound growth in idr_grow: return ENOMEM when want >= IDR_MAX_IDS ((1<<24)-1), make idr_grow static int, propagate to idr_pre_get (result 0 failure) and idr_alloc (-ENOMEM) - see fix.diff

Verdict

Reproduced on the stock DragonFly 6.5 X86_64_GENERIC kernel through the real exported KPI. (b) idr_alloc(start=1<<29) drives idr_grow to nf=2^30-1 and the live allocator-boundary hook captured kmalloc(17179869168 bytes, M_ZERO|M_WAITOK) which never returned on the 4 GiB guest (vm_wait wedge in kmem_slab_alloc); the hard-IRQ watchdog panicked at 15s with the captured size. (c) idr_get_new_above(sid=INT_MAX) poisons idr_maxwant, then idr_pre_get spins forever in the growth loop (2*INT_MAX+1 wraps to -1, -1 <= INT_MAX) holding the lwkt idr token while a second thread livelocks in idr_remove on the same idr; reproduced both on the verbatim-compiled KLD copy and on the kernel shipped binary. No memory-unsafety path exists (wrapped nf cannot escape the loop to any size computation, and any escape would over- not under-allocate), so the impact ceiling is kernel hang / OOM draw / livelock - DoS, no privesc. fix.diff (bound want at IDR_MAX_IDS, propagate ENOMEM) validated on a rebuilt kernel: control output bit-identical, boundary wants fail cleanly with -ENOMEM, no giant kmalloc reaches the allocator, no spin, no livelock, zero panics.