β¬’ DragonFlyBSD Kernel Audit
← triage Β· dashboard
DF-5017

dma_fence_wait_any_timeout: unsigned deadline wrap returns bogus negative error on clean timeout, long→int truncation turns infinite waits into tick-rate poll loops, and an uninterlocked check→tsleep window permanently loses completion wakeups

Summary

Three defects beyond the DF-2046 pattern in a different function: (1) unsigned end-jiffies underflow at deadline -> MAX(0,ULONG_MAX) stored negative -> timeout returned as fake errno (399/400 guest runs); (2) tsleep takes int timo - (int)MAX_SCHEDULE_TIMEOUT==-1 fires next tick, so amdgpu_sa infinite waits poll at 52 wakeups/s for the whole wait; (3) signaled-check without lock + unlatched tsleep drops wakeups fired in the window (full 1s sleep despite immediate wakeup demonstrated at primitive level). KLD PoC against shipped drm.ko (no GPU needed) - all tests reproduced + fixed variant clean + patched TU -Werror.

Discussion (0)

No comments yet.

PoC verification

Evidence pack

findings/poc/DF-5017 Β· 14 files
FileTypeDescriptionSize
df5017.c β€” 9.0 KB view raw
Makefile β€” 330 B ↓ download
build.sh β€” 129 B view raw
run.sh β€” 168 B view raw
build.log β€” 1.1 KB view raw
run.log β€” 14.7 KB view raw
run.2.log β€” 724 B view raw
env.txt β€” 366 B view raw
fix.diff β€” 1.7 KB view raw
fix_build.log β€” 1.4 KB view raw
README.md β€” 4.8 KB ↓ raw
VERDICT.md β€” 5.3 KB ↓ raw
manifest.json β€” 1.1 KB view raw
verdict.json β€” 4.9 KB view raw

DF-5017 — dma_fence_wait_any_timeout: unsigned deadline wrap + long→int sleep-timeout truncation + lost-wakeup race

What

Three defects in one loop β€” sys/dev/drm/linux_fence.c:206-215 β€” the any-fence wait used by the amdgpu stack (and any KLD calling the exported API):

  1. Unsigned deadline wrap (misreported timeout) β€” end = jiffies + timeout (line 206) with ret = MAX(0, end - jiffies) (line 207): once the deadline passes, end - jiffies underflows to ~ULONG_MAX, MAX(0, ULONG_MAX) is huge, and storing it into long ret yields a negative value that the loop guard (ret > 0) propagates to the caller as a bogus error instead of the contract's 0 (timeout). Same defect family as DF-2046 (dma_fence_default_wait), duplicated in this different function with different callers.
  2. long β†’ int truncation of the sleep timeout β€” tsleep(..., ret) (line 210) takes an int timo. For timeout = MAX_SCHEDULE_TIMEOUT (what amdgpu_sa.c:322 passes, and what amdgpu_gem_timeout() (amdgpu_gem.c:414-415) returns for user timeout_ns with the sign bit set), (int)LONG_MAX == -1; _callout_setup_quick (sys/kern/kern_timeout.c:968-970) clamps non-positive tick counts to fire on the next tick, so the "infinite" wait degenerates into a tick-rate poll loop (measured: 52 wakeups/s, TEST E) that keeps waking the thread ~100Γ—/s until a fence signals β€” unbounded CPU churn, triggerable per-thread by unprivileged GPU clients.
  3. Lost-wakeup race β€” unlike dma_fence_default_wait (which holds fence->lock across check+sleep via lksleep's atomic release), the wait_any loop checks dma_fence_test_signaled_any() (line 208) without any lock and then calls plain tsleep() (line 210). A dma_fence_signal() on another CPU that fires a callback in the window between the check and the tsleep interlock is dropped β€” DragonFly wakeup() on a not-yet-queued ident is not latched (proven at the primitive level: TEST C trial 4, full 1 s sleep despite an immediate wakeup; controls delivered at 19/20 ticks). The waiter then sleeps the entire remaining timeout even though the fence was signaled before it slept, and reports 0/timeout for a fence that was complete.

PoC (KLD against the live drm.ko symbols β€” no GPU needed)

kldload drm
cd /tmp/df5017 && make
kldload /tmp/df5017/df5017.ko     # runs ~30 s of tests, prints to console
dmesg | grep DF5017

The module exercises the shipped dma_fence_wait_any_timeout (baseline) and a local copy using the fixed set_current_state() + schedule_timeout() idiom (fixed), side by side:

  • TEST A β€” 400 unsignaled waits, timeout = 1 tick: baseline ret<0:399 ret==0:1, fixed ret<0:0 ret==0:400.
  • TEST B β€” MAX_SCHEDULE_TIMEOUT, intr=false, signal after 3 s: baseline returns ret=9223372036854775507 (β‰ˆLONG_MAX "remaining") after 300 ticks β€” correct-looking only because of the tick-rate poll from defect 2; fixed returns ret=1 after 300 ticks.
  • TEST E β€” tsleep(ident, 0, ..., (int)MAX_SCHEDULE_TIMEOUT): 52 wakeups in 1 s (an indefinite sleep would have 0).
  • TEST C (first guest session, run.log) β€” wakeup fired immediately after the waiter's "condition check": trial 4 slept the full 100 ticks (WAKEUP LOST); all 200 ms-control trials delivered at 19 ticks.

Expected (observed, run.2.log)

[DF5017] TEST A (400 unsignaled waits, timeout=1 tick)
[DF5017]   baseline wait_any: ret<0:399  ret==0:1  ret>0:0  (contract: every ret must be 0)
[DF5017]   fixed     wait_any: ret<0:0  ret==0:400  ret>0:0
[DF5017]   VERDICT A: BUG β€” negative return on clean timeout
[DF5017] TEST B baseline: MAX_SCHEDULE_TIMEOUT intr=false, signal@3s -> ret=9223372036854775507 elapsed=300 ticks
[DF5017] TEST B fixed    : MAX_SCHEDULE_TIMEOUT intr=false, signal@3s -> ret=1 elapsed=300 ticks (correct)
[DF5017] TEST E: tsleep timo=(int)MAX_SCHEDULE_TIMEOUT=-1 -> 52 wakeups in 1s ...

Fix

fix.diff (applies cleanly to sys/dev/drm/linux_fence.c): use the sched.h schedule_timeout() shim, whose kt_spin + task-state latch closes the wakeup race, which clamps the tick count (no truncation), and which returns non-negative remaining time (no wrap). Validated: the identical algorithm compiled into the module (fixed columns above), and the patched translation unit builds clean with full -Werror kernel flags (fix_build.log).

In-tree reachability (real hardware)

  • amdgpu_cs.c:1560 amdgpu_cs_wait_any_fence β€” DRM_IOCTL_AMDGPU_WAIT_FENCES, unprivileged render-node clients; user-controlled timeout_ns (negative β†’ MAX_SCHEDULE_TIMEOUT via amdgpu_gem_timeout).
  • amdgpu_sa.c:322 amdgpu_sa_bo_new β€” intr=false, MAX_SCHEDULE_TIMEOUT; hit by unprivileged submissions whenever the SA manager must wait.
VERDICT.md
↓ download raw

DF-5017 VERDICT β€” dma_fence_wait_any_timeout: wrap + truncation + lost wakeup

Status: REPRODUCED (kernel-logic/DoS class; no memory corruption)

Root cause

dma_fence_wait_any_timeout() β€” sys/dev/drm/linux_fence.c:171-222 β€” ports the Linux any-fence wait to the DragonFly sleep primitives but drops every safety property of the original wait_event/schedule_timeout idiom:

206:    end = jiffies + timeout;
207:    for (ret = timeout; ret > 0; ret = MAX(0, end - jiffies)) {
208:        if (dma_fence_test_signaled_any(fences, count, idx))
209:            break;
210:        err = tsleep(current, intr ? PCATCH : 0, "dfwat", ret);
  • (1) wrap β€” line 206/207: unsigned end - jiffies underflows at deadline; MAX(0, ULONG_MAX) stored in long ret is negative; returned to callers as an errno. Contract (Linux dma-fence API): 0 = timeout, negative = real error. Upstream returns remaining jiffies via schedule_timeout() which never goes negative.
  • (2) truncation β€” line 210: tsleep(..., int timo) receives a long. (int)MAX_SCHEDULE_TIMEOUT == -1; sys/kern/kern_timeout.c:968-970 (_callout_setup_quick) clamps qtick - softticks <= 0 to softticks+1, so each "infinite" iteration wakes on the next tick: the infinite wait is a ~hz-rate poll loop for its whole duration (TEST E: 52 wakeups/s).
  • (3) lost wakeup β€” lines 208β†’210: the signaled-check runs without any lock and the sleep is not interlocked beforehand; dma_fence_signal() (linux_fence.c:241-244 walk under fence->lock) can call wake_up_process(current) (wakeup on the task ident) after the check but before tsleep() queues the thread (_tsleep_interlock, sys/kern/kern_synch.c:575-577). DragonFly wakeup is not latched for not-yet-queued idents β†’ the completion notification is lost and the waiter sleeps the full remaining timeout. dma_fence_default_wait does not share this flaw because lksleep releases/acquires fence->lock atomically with the sleep (linux_fence.c:100/137).

Reproduction (guest, fresh vm.sh reset with-src, uname in env.txt)

KLD df5017.c calls the shipped dma_fence_wait_any_timeout from drm.ko and a byte-equivalent local copy with the fix, in one run:

  • TEST A (400 unsignaled waits, timeout=1 tick): baseline 399/400 returned negative on a clean timeout (one 0); fixed 400/400 returned 0. (run.2.log β€” deterministic, 399/400 both sessions.)
  • TEST B (MAX_SCHEDULE_TIMEOUT, intr=false, signaled at 3 s): baseline returned ret=9223372036854775507 after 300 ticks β€” i.e. the loop only terminated because of the tick-rate polling (defect 2), not because an indefinite sleep worked; fixed returned ret=1 after 300 ticks.
  • TEST E (truncation primitive): tsleep(id, 0, "x", (int)LONG_MAX) woke 52 times in 1 s β€” an infinite sleep wakes 0 times.
  • TEST C (lost-wake primitive, first session run.log): waiter "checked" (flag) then tsleep(1 s); waker on another CPU fired wakeup() within ns of the flag: trial 4 slept the full 100 ticks (wake lost); the 9 other immediate trials were delivered (the window is narrow β€” exactly the point: it is a race, not a certainty); all 200 ms-control trials slept 19-20 ticks (sanity: the mechanism works when the wake follows the sleep).

Guest stayed up throughout the recorded run; module loads and unloads cleanly. (One earlier session ended in a guest wedge caused by my own TEST C harness kthread lifecycle β€” a harness artifact, retired from the automated flow; its fence results are the ones cited above. One session before that panicked on a NULL kthread arg β€” again a harness bug, fixed.)

Impact

No memory corruption. Wrong synchronization semantics on the DRM fence API:

  • unprivileged render-node users get spurious -errno from DRM_IOCTL_AMDGPU_WAIT_FENCES on legitimate timeouts (amdgpu_cs.c:1560; amdgpu_gem_timeout maps negative timeout_ns to MAX_SCHEDULE_TIMEOUT β€” amdgpu_gem.c:414-415);
  • "infinite" waits (amdgpu_sa.c:322, intr=false) become 100 Hz poll loops β€” cheap per tick but per-thread, unbounded, and unkillable for the duration (PCATCH unset on that caller);
  • rare lost completions stall a waiter the full user-supplied timeout and misreport a signaled fence as timed out.

On the standard audit guest there is no GPU, so the in-tree ioctl paths were not exercisable; the defects were demonstrated directly against the shipped drm.ko implementation (deterministic for defects 1-2, single-hit race for defect 3).

Fix

fix.diff β€” rewrite the wait loop with the shim's set_current_state()/schedule_timeout() idiom (its kt_spin + TASK_RUNNING latch closes the race; timo conversion handles MAX_SCHEDULE_TIMEOUT; the return is non-negative). Validated two ways: the identical algorithm compiled into the PoC module (fixed columns in run.2.log) and the patched linux_fence.c compiles clean as a kernel module with full -Werror flags (fix_build.log).

Fix status: fixed (behavioral A/B in-run + TU compile check; full-kernel rebuild not performed β€” not a memory-corruption class finding).

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.

Fix verification

fixed
baseline reproduced→ patch + rebuild →patched clean

fix.diff rewrites the wait loop with schedule_timeout(); validated (a) behaviorally by compiling the identical algorithm into the PoC module: fixed copy 0/400 negative returns vs baseline 399/400 and correct MAX_SCHEDULE_TIMEOUT return (ret=1 at signal, no poll), and (b) the patched linux_fence.c translation unit compiles clean as a kernel module with full -Werror flags (fix_build.log). Full fixed-kernel rebuild not performed (logic-class finding, not memory corruption).

run.2.log TEST A 'fixed wait_any: ret<0:0 ret==0:400' and TEST B 'fixed ... ret=1 elapsed=300 ticks (correct)'; fix.diff (git apply --check clean); fix_build.log 'FIXED_TU_COMPILES_CLEAN'.
↓ fix.diffper-fix-DF-5017

Confirmed kernel references

Detail

Evidence (decisive lines)

run.2.log (clean run): '[DF5017] TEST A ... baseline wait_any: ret<0:399 ret==0:1 ... fixed wait_any: ret<0:0 ret==0:400'; TEST B baseline ret=9223372036854775507 after 300 ticks vs fixed ret=1; TEST E 'tsleep timo=(int)MAX_SCHEDULE_TIMEOUT=-1 -> 52 wakeups in 1s'. run.log (first session): TEST C trial 4 '(immediate): slept 100 ticks (WAKEUP LOST)' with controls delivered at 19 ticks. fix.diff applies cleanly; fix_build.log shows the patched linux_fence.c compiles clean as a KLD with -Werror.

PoC changes

PoC authored fresh for this finding (no seed). Iterations: (1) dropped (no such header), (2) fence arrays passed as struct dma_fence **, (3) first run panicked on a NULL kthread arg in my own signaler thread (fixed), (4) TEST C harness kthread lifecycle wedged the guest at the end of its run (harness artifact; TEST C retired from the automated flow, its trial-4 result kept in run.log), (5) final clean run: module loads, runs ~30s, unloads. File transport: direct ssh stream (guest b64decode/openssl were unreliable for large payloads).

Verified recommended fix

Replace the raw tsleep loop with the set_current_state()/schedule_timeout() idiom from the DF shim (latches wakeups via kt_spin+TASK_RUNNING, clamps timo, returns non-negative remainder) and handle MAX_SCHEDULE_TIMEOUT as a true indefinite wait.

Verdict

dma_fence_wait_any_timeout (sys/dev/drm/linux_fence.c:206-215) misreports clean timeouts as negative errors (unsigned end-jiffies wrap: 399/400 waits returned ret<0 against a contract of 0), truncates its long timeout into tsleep's int timo so MAX_SCHEDULE_TIMEOUT waits degenerate into tick-rate poll loops ((int)LONG_MAX==-1 -> next-tick callout; measured 52 wakeups/s), and has an uninterlocked check-then-sleep window in which a dma_fence_signal() wakeup is permanently lost (proven at the primitive level: a wakeup fired nanoseconds after the waiter's check left it sleeping the full 1s timeout while 200ms-control wakes were delivered). All three defects demonstrated on a fresh-reset guest against the shipped drm.ko implementation via a KLD that runs the baseline and a fixed schedule_timeout()-idiom copy side by side; the fixed copy scored 0/400 negatives and correct infinite-wait semantics. No memory corruption: impact is wrong synchronization semantics, spurious errors to unprivileged GPU clients (DRM_IOCTL_AMDGPU_WAIT_FENCES via amdgpu_cs.c:1560, amdgpu_gem_timeout clamps negative timeout_ns to MAX_SCHEDULE_TIMEOUT), and unbounded poll-churn/timeout-stalls (amdgpu_sa.c:322, intr=false).