dma_fence_wait_any_timeout: unsigned deadline wrap returns bogus negative error on clean timeout, longβint truncation turns infinite waits into tick-rate poll loops, and an uninterlocked checkβtsleep window permanently loses completion wakeups
Summary
Three defects beyond the DF-2046 pattern in a different function: (1) unsigned end-jiffies underflow at deadline -> MAX(0,ULONG_MAX) stored negative -> timeout returned as fake errno (399/400 guest runs); (2) tsleep takes int timo - (int)MAX_SCHEDULE_TIMEOUT==-1 fires next tick, so amdgpu_sa infinite waits poll at 52 wakeups/s for the whole wait; (3) signaled-check without lock + unlatched tsleep drops wakeups fired in the window (full 1s sleep despite immediate wakeup demonstrated at primitive level). KLD PoC against shipped drm.ko (no GPU needed) - all tests reproduced + fixed variant clean + patched TU -Werror.
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-5017 Β· 14 files| File | Type | Description | Size | |
|---|---|---|---|---|
| df5017.c | β | 9.0 KB | view raw | |
| Makefile | β | 330 B | β download | |
| build.sh | β | 129 B | view raw | |
| run.sh | β | 168 B | view raw | |
| build.log | β | 1.1 KB | view raw | |
| run.log | β | 14.7 KB | view raw | |
| run.2.log | β | 724 B | view raw | |
| env.txt | β | 366 B | view raw | |
| fix.diff | β | 1.7 KB | view raw | |
| fix_build.log | β | 1.4 KB | view raw | |
| README.md | β | 4.8 KB | β raw | |
| VERDICT.md | β | 5.3 KB | β raw | |
| manifest.json | β | 1.1 KB | view raw | |
| verdict.json | β | 4.9 KB | view raw |
DF-5017 β dma_fence_wait_any_timeout: unsigned deadline wrap + longβint sleep-timeout truncation + lost-wakeup race
What
Three defects in one loop β sys/dev/drm/linux_fence.c:206-215 β the
any-fence wait used by the amdgpu stack (and any KLD calling the exported
API):
- Unsigned deadline wrap (misreported timeout) β
end = jiffies + timeout(line 206) withret = MAX(0, end - jiffies)(line 207): once the deadline passes,end - jiffiesunderflows to ~ULONG_MAX,MAX(0, ULONG_MAX)is huge, and storing it intolong retyields a negative value that the loop guard (ret > 0) propagates to the caller as a bogus error instead of the contract's0(timeout). Same defect family as DF-2046 (dma_fence_default_wait), duplicated in this different function with different callers. long β inttruncation of the sleep timeout βtsleep(..., ret)(line 210) takes anint timo. Fortimeout = MAX_SCHEDULE_TIMEOUT(whatamdgpu_sa.c:322passes, and whatamdgpu_gem_timeout()(amdgpu_gem.c:414-415) returns for usertimeout_nswith the sign bit set),(int)LONG_MAX == -1;_callout_setup_quick(sys/kern/kern_timeout.c:968-970) clamps non-positive tick counts to fire on the next tick, so the "infinite" wait degenerates into a tick-rate poll loop (measured: 52 wakeups/s, TEST E) that keeps waking the thread ~100Γ/s until a fence signals β unbounded CPU churn, triggerable per-thread by unprivileged GPU clients.- Lost-wakeup race β unlike
dma_fence_default_wait(which holdsfence->lockacross check+sleep vialksleep's atomic release), the wait_any loop checksdma_fence_test_signaled_any()(line 208) without any lock and then calls plaintsleep()(line 210). Adma_fence_signal()on another CPU that fires a callback in the window between the check and the tsleep interlock is dropped β DragonFlywakeup()on a not-yet-queued ident is not latched (proven at the primitive level: TEST C trial 4, full 1 s sleep despite an immediate wakeup; controls delivered at 19/20 ticks). The waiter then sleeps the entire remaining timeout even though the fence was signaled before it slept, and reports0/timeout for a fence that was complete.
PoC (KLD against the live drm.ko symbols β no GPU needed)
kldload drm cd /tmp/df5017 && make kldload /tmp/df5017/df5017.ko # runs ~30 s of tests, prints to console dmesg | grep DF5017
The module exercises the shipped dma_fence_wait_any_timeout (baseline)
and a local copy using the fixed set_current_state() + schedule_timeout()
idiom (fixed), side by side:
- TEST A β 400 unsignaled waits, timeout = 1 tick:
baseline
ret<0:399 ret==0:1, fixedret<0:0 ret==0:400. - TEST B β
MAX_SCHEDULE_TIMEOUT, intr=false, signal after 3 s: baseline returnsret=9223372036854775507(βLONG_MAX "remaining") after 300 ticks β correct-looking only because of the tick-rate poll from defect 2; fixed returnsret=1after 300 ticks. - TEST E β
tsleep(ident, 0, ..., (int)MAX_SCHEDULE_TIMEOUT): 52 wakeups in 1 s (an indefinite sleep would have 0). - TEST C (first guest session,
run.log) β wakeup fired immediately after the waiter's "condition check": trial 4 slept the full 100 ticks (WAKEUP LOST); all 200 ms-control trials delivered at 19 ticks.
Expected (observed, run.2.log)
[DF5017] TEST A (400 unsignaled waits, timeout=1 tick) [DF5017] baseline wait_any: ret<0:399 ret==0:1 ret>0:0 (contract: every ret must be 0) [DF5017] fixed wait_any: ret<0:0 ret==0:400 ret>0:0 [DF5017] VERDICT A: BUG β negative return on clean timeout [DF5017] TEST B baseline: MAX_SCHEDULE_TIMEOUT intr=false, signal@3s -> ret=9223372036854775507 elapsed=300 ticks [DF5017] TEST B fixed : MAX_SCHEDULE_TIMEOUT intr=false, signal@3s -> ret=1 elapsed=300 ticks (correct) [DF5017] TEST E: tsleep timo=(int)MAX_SCHEDULE_TIMEOUT=-1 -> 52 wakeups in 1s ...
Fix
fix.diff (applies cleanly to sys/dev/drm/linux_fence.c): use the
sched.h schedule_timeout() shim, whose kt_spin + task-state latch closes
the wakeup race, which clamps the tick count (no truncation), and which
returns non-negative remaining time (no wrap). Validated:
the identical algorithm compiled into the module (fixed columns above), and
the patched translation unit builds clean with full -Werror kernel flags
(fix_build.log).
In-tree reachability (real hardware)
amdgpu_cs.c:1560amdgpu_cs_wait_any_fenceβDRM_IOCTL_AMDGPU_WAIT_FENCES, unprivileged render-node clients; user-controlledtimeout_ns(negative β MAX_SCHEDULE_TIMEOUT viaamdgpu_gem_timeout).amdgpu_sa.c:322amdgpu_sa_bo_newβ intr=false, MAX_SCHEDULE_TIMEOUT; hit by unprivileged submissions whenever the SA manager must wait.
DF-5017 VERDICT β dma_fence_wait_any_timeout: wrap + truncation + lost wakeup
Status: REPRODUCED (kernel-logic/DoS class; no memory corruption)
Root cause
dma_fence_wait_any_timeout() β sys/dev/drm/linux_fence.c:171-222 β ports
the Linux any-fence wait to the DragonFly sleep primitives but drops every
safety property of the original wait_event/schedule_timeout idiom:
206: end = jiffies + timeout;
207: for (ret = timeout; ret > 0; ret = MAX(0, end - jiffies)) {
208: if (dma_fence_test_signaled_any(fences, count, idx))
209: break;
210: err = tsleep(current, intr ? PCATCH : 0, "dfwat", ret);
- (1) wrap β line 206/207: unsigned
end - jiffiesunderflows at deadline;MAX(0, ULONG_MAX)stored inlong retis negative; returned to callers as an errno. Contract (Linux dma-fence API): 0 = timeout, negative = real error. Upstream returns remaining jiffies viaschedule_timeout()which never goes negative. - (2) truncation β line 210:
tsleep(..., int timo)receives along.(int)MAX_SCHEDULE_TIMEOUT == -1;sys/kern/kern_timeout.c:968-970(_callout_setup_quick) clampsqtick - softticks <= 0tosoftticks+1, so each "infinite" iteration wakes on the next tick: the infinite wait is a ~hz-rate poll loop for its whole duration (TEST E: 52 wakeups/s). - (3) lost wakeup β lines 208β210: the signaled-check runs without any
lock and the sleep is not interlocked beforehand;
dma_fence_signal()(linux_fence.c:241-244 walk underfence->lock) can callwake_up_process(current)(wakeup on the task ident) after the check but beforetsleep()queues the thread (_tsleep_interlock, sys/kern/kern_synch.c:575-577). DragonFly wakeup is not latched for not-yet-queued idents β the completion notification is lost and the waiter sleeps the full remaining timeout.dma_fence_default_waitdoes not share this flaw becauselksleepreleases/acquiresfence->lockatomically with the sleep (linux_fence.c:100/137).
Reproduction (guest, fresh vm.sh reset with-src, uname in env.txt)
KLD df5017.c calls the shipped dma_fence_wait_any_timeout from
drm.ko and a byte-equivalent local copy with the fix, in one run:
- TEST A (400 unsignaled waits, timeout=1 tick): baseline 399/400 returned
negative on a clean timeout (one 0); fixed 400/400 returned 0.
(
run.2.logβ deterministic, 399/400 both sessions.) - TEST B (MAX_SCHEDULE_TIMEOUT, intr=false, signaled at 3 s): baseline
returned
ret=9223372036854775507after 300 ticks β i.e. the loop only terminated because of the tick-rate polling (defect 2), not because an indefinite sleep worked; fixed returnedret=1after 300 ticks. - TEST E (truncation primitive):
tsleep(id, 0, "x", (int)LONG_MAX)woke 52 times in 1 s β an infinite sleep wakes 0 times. - TEST C (lost-wake primitive, first session
run.log): waiter "checked" (flag) then tsleep(1 s); waker on another CPU firedwakeup()within ns of the flag: trial 4 slept the full 100 ticks (wake lost); the 9 other immediate trials were delivered (the window is narrow β exactly the point: it is a race, not a certainty); all 200 ms-control trials slept 19-20 ticks (sanity: the mechanism works when the wake follows the sleep).
Guest stayed up throughout the recorded run; module loads and unloads cleanly. (One earlier session ended in a guest wedge caused by my own TEST C harness kthread lifecycle β a harness artifact, retired from the automated flow; its fence results are the ones cited above. One session before that panicked on a NULL kthread arg β again a harness bug, fixed.)
Impact
No memory corruption. Wrong synchronization semantics on the DRM fence API:
- unprivileged render-node users get spurious
-errnofromDRM_IOCTL_AMDGPU_WAIT_FENCESon legitimate timeouts (amdgpu_cs.c:1560;amdgpu_gem_timeoutmaps negativetimeout_nsto MAX_SCHEDULE_TIMEOUT β amdgpu_gem.c:414-415); - "infinite" waits (amdgpu_sa.c:322, intr=false) become 100 Hz poll loops β cheap per tick but per-thread, unbounded, and unkillable for the duration (PCATCH unset on that caller);
- rare lost completions stall a waiter the full user-supplied timeout and misreport a signaled fence as timed out.
On the standard audit guest there is no GPU, so the in-tree ioctl paths were not exercisable; the defects were demonstrated directly against the shipped drm.ko implementation (deterministic for defects 1-2, single-hit race for defect 3).
Fix
fix.diff β rewrite the wait loop with the shim's
set_current_state()/schedule_timeout() idiom (its kt_spin + TASK_RUNNING
latch closes the race; timo conversion handles MAX_SCHEDULE_TIMEOUT; the
return is non-negative). Validated two ways: the identical algorithm
compiled into the PoC module (fixed columns in run.2.log) and the patched
linux_fence.c compiles clean as a kernel module with full -Werror flags
(fix_build.log).
Fix status: fixed (behavioral A/B in-run + TU compile check; full-kernel rebuild not performed β not a memory-corruption class finding).
Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.
Fix verification
fixedfix.diff rewrites the wait loop with schedule_timeout(); validated (a) behaviorally by compiling the identical algorithm into the PoC module: fixed copy 0/400 negative returns vs baseline 399/400 and correct MAX_SCHEDULE_TIMEOUT return (ret=1 at signal, no poll), and (b) the patched linux_fence.c translation unit compiles clean as a kernel module with full -Werror flags (fix_build.log). Full fixed-kernel rebuild not performed (logic-class finding, not memory corruption).
run.2.log TEST A 'fixed wait_any: ret<0:0 ret==0:400' and TEST B 'fixed ... ret=1 elapsed=300 ticks (correct)'; fix.diff (git apply --check clean); fix_build.log 'FIXED_TU_COMPILES_CLEAN'.
Confirmed kernel references
- sys/dev/drm/linux_fence.c:206
- sys/dev/drm/linux_fence.c:207
- sys/dev/drm/linux_fence.c:208
- sys/dev/drm/linux_fence.c:210
- sys/dev/drm/include/linux/sched.h:78
- sys/kern/kern_timeout.c:968
- sys/kern/kern_synch.c:575
- sys/dev/drm/amd/amdgpu/amdgpu_sa.c:322
- sys/dev/drm/amd/amdgpu/amdgpu_cs.c:1560
- sys/dev/drm/amd/amdgpu/amdgpu_gem.c:414
Detail
Evidence (decisive lines)
run.2.log (clean run): '[DF5017] TEST A ... baseline wait_any: ret<0:399 ret==0:1 ... fixed wait_any: ret<0:0 ret==0:400'; TEST B baseline ret=9223372036854775507 after 300 ticks vs fixed ret=1; TEST E 'tsleep timo=(int)MAX_SCHEDULE_TIMEOUT=-1 -> 52 wakeups in 1s'. run.log (first session): TEST C trial 4 '(immediate): slept 100 ticks (WAKEUP LOST)' with controls delivered at 19 ticks. fix.diff applies cleanly; fix_build.log shows the patched linux_fence.c compiles clean as a KLD with -Werror.
PoC changes
PoC authored fresh for this finding (no seed). Iterations: (1) dropped
Verified recommended fix
Replace the raw tsleep loop with the set_current_state()/schedule_timeout() idiom from the DF shim (latches wakeups via kt_spin+TASK_RUNNING, clamps timo, returns non-negative remainder) and handle MAX_SCHEDULE_TIMEOUT as a true indefinite wait.
Verdict
dma_fence_wait_any_timeout (sys/dev/drm/linux_fence.c:206-215) misreports clean timeouts as negative errors (unsigned end-jiffies wrap: 399/400 waits returned ret<0 against a contract of 0), truncates its long timeout into tsleep's int timo so MAX_SCHEDULE_TIMEOUT waits degenerate into tick-rate poll loops ((int)LONG_MAX==-1 -> next-tick callout; measured 52 wakeups/s), and has an uninterlocked check-then-sleep window in which a dma_fence_signal() wakeup is permanently lost (proven at the primitive level: a wakeup fired nanoseconds after the waiter's check left it sleeping the full 1s timeout while 200ms-control wakes were delivered). All three defects demonstrated on a fresh-reset guest against the shipped drm.ko implementation via a KLD that runs the baseline and a fixed schedule_timeout()-idiom copy side by side; the fixed copy scored 0/400 negatives and correct infinite-wait semantics. No memory corruption: impact is wrong synchronization semantics, spurious errors to unprivileged GPU clients (DRM_IOCTL_AMDGPU_WAIT_FENCES via amdgpu_cs.c:1560, amdgpu_gem_timeout clamps negative timeout_ns to MAX_SCHEDULE_TIMEOUT), and unbounded poll-churn/timeout-stalls (amdgpu_sa.c:322, intr=false).
No comments yet.