Signed integer overflow in idr_grow() growth loop causes kernel hang or panic-sized kmalloc
Summary
idr_grow() doubles nf with nf=2*nf+1 in signed int until nf>want. For want>=2^29 loop exits with nf=2^30-1 or 2^31-1 driving kmalloc to request 16-32GiB panicking/hanging under M_WAITOK. For want==INT_MAX predicate INT_MAX<=INT_MAX true so loop runs once more 2*INT_MAX+1 is signed overflow wraps to -1 and -1<=want keeps loop spinning forever holding lwkt token. Starting from 255=2^8-1 after k iterations nf=2^(8+k)-1. At k=23 nf=2^31-1=INT_MAX. want flows from caller-supplied start/sid: idr_alloc passes start directly idr_get_new_above stores idr_maxwant=max(maxwant sid) before returning -EAGAIN retry pattern calls idr_pre_get which grows to idr_maxwant. No upper-bound check on want sid or nf anywhere in file. In-tree DRM callers currently bounded (drm_gem.c start=1 amdgpu PASID 16-bit) but API provides no guardrail. Impact: system-wide DoS livelock on token or kernel panic. Any caller of idr KPI reachable once DRM device open. Latent DoS primitive fires moment any caller forwards user-influenced integer into start/sid. [VERIFIED pass2: reproduced 3 ways - idr_alloc start 2^29 drives live-captured kmalloc 17179869168 bytes M_WAITOK that never returns; sid INT_MAX poisons idr_maxwant and idr_pre_get spins forever holding the token while a second thread livelocks; proven on both verbatim copy and shipped binary via hard-IRQ watchdog; fix-validated - IDR_MAX_IDS bound plus ENOMEM propagation, largest post-fix request 4080 bytes; NOT privesc - wrapped nf can never under-allocate]
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-2232 Β· 24 files| File | Type | Description | Size | |
|---|---|---|---|---|
| df2232_rig.c | β | 18.7 KB | view raw | |
| Makefile | β | 271 B | β download | |
| build.sh | β | 92 B | view raw | |
| run.sh | β | 135 B | view raw | |
| build.log | β | 5.5 KB | view raw | |
| env.txt | β | 599 B | view raw | |
| run.a.log | β | 2.3 KB | view raw | |
| run.b.log | β | 3.3 KB | view raw | |
| panic.b.txt | β | 3.3 KB | view raw | |
| run.c1.log | β | 2.8 KB | view raw | |
| panic.c1.txt | β | 2.9 KB | view raw | |
| run.c1b.log | β | 2.8 KB | view raw | |
| panic.c1b.txt | β | 2.7 KB | view raw | |
| panic.c2.txt | β | 2.7 KB | view raw | |
| fix.diff | β | 2.1 KB | view raw | |
| fix_build.log | β | 9.4 KB | view raw | |
| fix_run.log | β | 3.9 KB | view raw | |
| fix/run-all.log | β | 257 B | view raw | |
| fix/console-fixed.log | β | 3.9 KB | view raw | |
| fix/control-fixed.log | β | 4.2 KB | view raw | |
| fix/kbuild-tail.log | β | 9.0 KB | view raw | |
| fix/build-fixed.log | β | 5.5 KB | view raw | |
| VERDICT.md | β | 7.7 KB | β raw | |
| verdict.json | β | 5.4 KB | view raw |
VERDICT β DF-2232 (sys/libkern/linux_idr.c idr_grow signed-overflow growth loop)
Status: REPRODUCED β impact dos (hang / panic-sized kmalloc), confidence certain. Fix authored and validated on a rebuilt kernel. No memory-unsafety path exists (analysis below); no uid=0 chain is possible from this bug.
Environment
- Guest: DragonFly 6.5-DEVELOPMENT, kernel X86_64_GENERIC (
options INVARIANTS), 6 vCPU, 4 GiB RAM, QEMU/KVM. Baseline kernel:#0 Thu Jul 2 06:02:54 UTC 2026. Fixed kernel:#1 Sat Oct 17 10:36:44 UTC 2026. - Compiler: cc 8.3 (DragonFly base); module built with kernel flags
-O2 -fno-strict-overflow(signed wrap deterministic, same as kernel). - Verbatim gate: guest
/usr/src/sys/libkern/linux_idr.csha25622d421c35bfd7ebc7ae2df137b52b7ece0a9cf349a8f9369b440389e033a95deβ equal to the audited tree; the rig includes that file byte-identical.
What was run (evidence trail)
-
Control (run.a.log): phases 1+2 β normal wants 1..1000 round-trip on the verbatim KLD copy and on the kernel's own exported KPI. Both PASSED with identical results (grow 255β1023, pre_get/get_new_above protocol settles in 2 iterations, remove/realloc reuses slot 1000). This also demonstrates the
idr_maxwantpoisoning protocol the attack rides on: firstidr_get_new_above(sid)with a fresh higher sid returns -EAGAIN after raisingidr_maxwant(linux_idr.c:327-331). -
Giant kmalloc (run.b.log, panic.b.txt):
idr_alloc(start=1<<29, end=0)through the verbatim copy. The allocator-boundary hook (outside the verbatim file) captured the request live:
[DF2232-B] ALLOCATOR BOUNDARY: kmalloc(17179869168 bytes [0x3fffffff0], M_IDR, flags 0x102) -> real allocator
17179869168 = (2^30-1) * sizeof(struct idr_node=16) β exactly the
predicted ~16 GiB M_ZERO|M_WAITOK request. On the 4 GiB guest the
allocation never returned (page-by-page vm_page_alloc + vm_wait()
in kmem_slab_alloc, sys/kern/kern_slaballoc.c:1706-1800; the per-type
limit check in _kmalloc only trips on cumulative use, not on a single
first request). The hard-IRQ watchdog panicked at 15 s with the captured
request size in the marker; backtrace shows it fired from
systimer_intr β pcpu_timer_process_oncpu β splz_timer (interrupt context,
proving the design requirement: immune to the wedge).
-
Infinite spin, verbatim copy (panic.c1.txt; repeat in panic.c1b.txt): thread A:
idr_get_new_above(sid=INT_MAX)β -EAGAIN,idr_maxwant=2147483647poisoned; thenidr_pre_getβ never returns (a_done=0,a_stage=2): the growth loop computed2*INT_MAX+1 β -1and spins forever holding the lwkt idr token. Thread B calledidr_removeon the same idr and never returned (b_started=1,b_acquired=0in run 1; in the repeat run B hung already inside its pre-spin token check) β the livelock the finding predicted. Watchdog panic at 15 s with the state flags. -
Infinite spin, shipped binary (panic.c2.txt): identical outcome with the kernel's OWN exported
idr_get_new_above/idr_pre_get(last_kmalloc_req=0: the rig's kmalloc hook is not in this path β the spinning code is the stock kernel text at 0xffffffff809d5590). This closes the "your recompile behaves differently" objection.
Root cause (line-accurate)
- sys/libkern/linux_idr.c:427-430 β
nf = 2*nf+1; while (nf <= want)in signed int: want β₯ 2^29-1 drives the kmalloc size toward (2^30-1)*16 = 16 GiB; want == INT_MAX makes2*INT_MAX+1wrap to -1 and the predicate-1 <= INT_MAXtrue forever. - sys/libkern/linux_idr.c:437-438 β the wrapped/oversized
nfflows intokmalloc(nf * sizeof(struct idr_node), ..., M_ZERO|M_WAITOK). - Reachability of
want: idr_alloc start (linux_idr.c:358-375, onlystart < 0rejected), idr_get_new_above sid β idr_maxwant (linux_idr.c:327-331) β idr_pre_get grow (linux_idr.c:274-275). No upper bound anywhere. In-tree callers (drm_gem start=1; amdgpu PASID 16-bit) are bounded β latent KPI hazard, hence the rig drives the exported KPI.
Why DoS and not memory corruption / privesc (team directive)
- The wrapped
nf = -1cannot escape the loop into the size computation: the do-while only exits whennf > want; exiting with negativenfrequireswant < -1, and every reachablewantis non-negative (idr_alloc rejects negative start at :365-366; idr_maxwant only rises viamax()at :328; the forced-growthwant = idr_countat :303 is > 0). - Even in the impossible case,
nf * sizeof(struct idr_node)promotes nf to size_t: (size_t)(-1) * 16 = 0xFFFFFFFFFFFFFFF0 β a gigantic OVER-allocation request (allocator refuses), never an under-allocation. The subsequentbcopyat :451-452 uses the OLDidr_count, and the race guard at :441 frees the new buffer if it is not larger. There is no path to a smaller-than-needed allocation, so no heap corruption, no forged-object primitive, no uid=0 chain. Impact ceiling: kernel-mode livelock holding an lwkt token, unbounded M_WAITOK draw (memory exhaustion of every page into wired zero pages), watchdog-observable hang β DoS.
Fix validation (fix.diff, fix_run.log, fix_build.log)
fix.diff: idr_grow() becomes static int, rejecting
want >= IDR_MAX_IDS ((1<<24)-1) with ENOMEM; growth loop otherwise
unchanged (with the cap, max computed nf = 2^25-1 β no signed overflow is
reachable); idr_pre_get maps failure to result 0 (its documented failure
contract β drm-style callers then give up instead of looping), idr_alloc
maps it to -ENOMEM (both grow sites, token correctly released via the
existing done: path).
Cycle: guest reset to clean source β patch applied in guest /usr/src
(patch -p1, 6/6 hunks) β make -j6 nativekernel KERNCONF=X86_64_GENERIC
(320 s, -Werror clean) β installkernel β reboot into #1 Sat Oct 17 10:36:44.
Rig rebuilt against the FIXED file (sha 1c04da32β¦, recorded in build.log)
and all phases re-run:
- control phases: PASSED, output identical to baseline (fix/run-all.log, fix/control-fixed.log);
- phase 3:
idr_alloc RETURNED -12β and the hook proves the largest request that reached the allocator was the 4080-byte idr_init allocation: no giant kmalloc was ever issued; - phases 4/5:
pre_get RETURNED 0,a_done=1 b_acquired=1, "no spin, no livelock" β on both the fixed verbatim copy and the fixed kernel functions; grep -c panic fix_run.logβ 0; guest stayed up throughout.
Note on rig history: the first fixed-kernel validation run completed all
phases successfully but crashed AFTER success during kldunload β a defect
in my first rig (systimer teardown race + a leaked idr freed the module text
too early), not in the fix; corrected by keeping the module resident (EBUSY)
after crash-phases and destroying the leftover idr; the second run
(fix_run.log) is clean end-to-end. The crash and its analysis are part of
the serial console history and were re-verified to not involve linux_idr.c
code paths beyond malloc_uninit bookkeeping.
Bottom line
The finding is fully confirmed on the stock kernel, at exact byte sizes, with livelock proof, on both a verbatim recompile and the shipped binary; the one-line-bound fix eliminates both manifestations with bit-identical normal-path behavior. Class: latent-KPI kernel DoS (Medium) β no privilege-escalation potential exists for this bug.
Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.
Fix verification
fixedSingle-fix kernel (patch -p1, make nativekernel, 320s, -Werror clean, installkernel, reboot): baseline manifestations gone - phase 3 returns -ENOMEM with the largest allocator request observed at 4080 bytes (idr_init), phases 4/5 show pre_get RETURNED 0 and a_done=1 b_acquired=1 (no spin, no livelock) on both the fixed verbatim copy and the fixed kernel functions; control phases bit-identical to baseline; zero panics; guest stayed up.
fix_run.log (fixed-kernel serial console), fix/control-fixed.log (control PASSED), fix_build.log (kernel build), build.log (rig rebuilt against the fixed file, sha 1c04da32d604f7b5164832be1da227b24980c0804665facdf20fe0fa263275e3)
Confirmed kernel references
Detail
Evidence (decisive lines)
run.b.log + panic.b.txt line kmalloc(17179869168 bytes [0x3fffffff0]) at the allocator boundary followed by watchdog panic DF2232-WD; panic.c1.txt / panic.c1b.txt state a_stage=2 a_done=0 b_started=1 b_acquired=0 (infinite spin, token livelock); panic.c2.txt same on the shipped binary with last_kmalloc_req=0; fix_run.log shows idr_alloc RETURNED -12, pre_get RETURNED 0, a_done=1 b_acquired=1, zero panics on the fixed kernel; fix/control-fixed.log shows control PASSED unchanged
PoC changes
Built the rig from scratch (no seed existed): verbatim #include of linux_idr.c with exported-symbol rename guard and sha256 gate; kmalloc observation wrapper outside the verbatim file; systimer watchdog armed on a remote CPU firing from hard-IRQ context so every phase terminates with decisive output; second kernel thread as token-livelock probe; fixed rig defects found during runs (expectation arithmetic 2047 vs 1023, EPERM on nonzero sysctl return, unload-time systimer race -> module stays resident after crash-phases)
Verified recommended fix
Bound growth in idr_grow: return ENOMEM when want >= IDR_MAX_IDS ((1<<24)-1), make idr_grow static int, propagate to idr_pre_get (result 0 failure) and idr_alloc (-ENOMEM) - see fix.diff
Verdict
Reproduced on the stock DragonFly 6.5 X86_64_GENERIC kernel through the real exported KPI. (b) idr_alloc(start=1<<29) drives idr_grow to nf=2^30-1 and the live allocator-boundary hook captured kmalloc(17179869168 bytes, M_ZERO|M_WAITOK) which never returned on the 4 GiB guest (vm_wait wedge in kmem_slab_alloc); the hard-IRQ watchdog panicked at 15s with the captured size. (c) idr_get_new_above(sid=INT_MAX) poisons idr_maxwant, then idr_pre_get spins forever in the growth loop (2*INT_MAX+1 wraps to -1, -1 <= INT_MAX) holding the lwkt idr token while a second thread livelocks in idr_remove on the same idr; reproduced both on the verbatim-compiled KLD copy and on the kernel shipped binary. No memory-unsafety path exists (wrapped nf cannot escape the loop to any size computation, and any escape would over- not under-allocate), so the impact ceiling is kernel hang / OOM draw / livelock - DoS, no privesc. fix.diff (bound want at IDR_MAX_IDS, propagate ENOMEM) validated on a rebuilt kernel: control output bit-identical, boundary wants fail cleanly with -ENOMEM, no giant kmalloc reaches the allocator, no spin, no livelock, zero panics.
No comments yet.