β¬’ DragonFlyBSD Kernel Audit
← triage Β· dashboard
DF-4741

drm_sched_fini returns without cancelling the armed work_tdr or draining ring_mirror_list β†’ delayed-work/callout executes on the freed scheduler after driver teardown (softclock panic)

Summary

fini only stops the thread; with any job pending, work_tdr is armed and fini neither cancels it nor drains the list - callers free the scheduler container immediately, leaving the callout to fire on freed memory. REPRODUCED on guest: fini + kfree exactly as amdgpu frees amdgpu_ring -> ~2s later panic in softclock_handler. Privileged trigger (unload/rebind/reset-failure teardown); freed-memory execution unconditional once fini runs with a pending TDR. cancel_delayed_work_sync fix validated on rebuilt amdgpu.ko.

Discussion (0)

No comments yet.

PoC verification

Evidence pack

findings/poc/DF-4741 Β· 11 files
FileTypeDescriptionSize
README.md β€” 1.2 KB ↓ raw
VERDICT.md β€” 2.7 KB ↓ raw
build.sh β€” 103 B view raw
run.sh β€” 135 B view raw
build.log β€” 75 B view raw
run_fixed.log β€” 2.1 KB view raw
panic.txt β€” 2.7 KB view raw
env.txt β€” 290 B view raw
fix.diff β€” 1.6 KB view raw
verdict.json β€” 3.6 KB view raw
../DF-4740/df4740.c β€” 14.9 KB view raw

DF-4741 β€” drm_sched_fini returns with an armed work_tdr / undrained job list β†’ post-teardown UAF

Build

Same harness as DF-4740 (df4740.c, Makefile) β€” see ../DF-4740/README.md.

Run

kldload amdgpu
kldload /tmp/df4740/df4740.ko
sysctl hw.df4740_test=2

Expected (baseline, unpatched scheduler β€” observed, panic.txt)

[DF4741] job ... in flight, work_tdr armed (timeout=200 ticks); calling drm_sched_fini()
[DF4741] drm_sched_fini() returned WITHOUT cancelling work_tdr ...
[DF4741] reclaim block 0 at 0xfffff8008f844440 (zeroed)   <- exact freed sched address
...
panic: _callout 0xfffff8008f968b80 verifier 0xfffff8008f844630 toc 0 (expected ...) func ...
softclock_handler() at softclock_handler+0x502
Debugger("panic")

The armed work_tdr delayed-work callout fires on the freed (and reclaimed) scheduler ~1-2 s after drm_sched_fini() returned and the caller freed the scheduler, exactly as amdgpu_fence_driver_fini() (amdgpu_fence.c:495-519) frees struct amdgpu_ring (which embeds ring->sched) right after drm_sched_fini(&ring->sched).

Post-fix (fix.diff adds cancel_delayed_work_sync to drm_sched_fini)

run_fixed.log: same test on the patched amdgpu.ko β€” no callout fire, no panic, guest stays up ("STILL-UP-4s").

VERDICT.md
↓ download raw

DF-4741 VERDICT β€” drm_sched_fini leaves armed TDR / pending jobs on teardown

Status: REPRODUCED (panic via softclock on freed scheduler; fix validated)

Root cause

drm_sched_fini() β€” sys/dev/drm/scheduler/sched_main.c:652-656:

void drm_sched_fini(struct drm_gpu_scheduler *sched)
{
    if (sched->thread)
        kthread_stop(sched->thread);
}

It neither cancels sched->work_tdr (armed by drm_sched_start_timeout whenever jobs sit on ring_mirror_list, sched_main.c:194-199) nor drains the job list, and takes no reference-participation in pending finish_work items. Every in-tree caller frees the scheduler's container right after: amdgpu_fence_driver_fini() calls drm_sched_fini(&ring->sched) (amdgpu_fence.c:509) and the enclosing struct amdgpu_ring is released with the device during teardown. The pending delayed work / callout then executes against freed memory.

Reproduction (guest, harness test 2 β€” df4740.c)

From a fresh reset (panic.txt):

  1. Scheduler initialized with a 2 s timeout; a hung job sits on ring_mirror_list, so job_begin armed work_tdr (sched_main.c:246).
  2. drm_sched_fini() called (thread stopped; work_tdr STILL armed).
  3. Scheduler kfree()d and the block reclaimed with zeroed allocations (reclaim block 0 landed exactly on the freed scheduler address) β€” the stand-in for amdgpu freeing struct amdgpu_ring.
  4. ~2 s later the callout fired on the freed object: panic: _callout ... verifier ... (expected ...) ... softclock_handler β€” guest down at db>.

Even when the callout verifier does not trip first, the queued drm_sched_job_timedout work item itself runs on freed memory (list walk, lockmgr(&sched->job_list_lock) inside the freed block, ops->timedout_job indirect call) β€” the same class with RIP-control-grade consequences on this guest.

Severity context: teardown is reached via driver unload/rebind (privileged) or device removal/reset-failure paths, hence Medium rather than High; the panic and the freed-memory execution are unconditional once fini runs with a pending TDR.

Fix

fix.diff hunk 3 (shared with DF-4740): cancel_delayed_work_sync(&sched->work_tdr) after kthread_stop in drm_sched_fini. Validated: patched amdgpu.ko, test 2 re-run β†’ no callout fire, no panic, guest stays up (run_fixed.log). A full upstream-grade fix should additionally drain ring_mirror_list (signal + free leftover jobs) as Linux 5.x does.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.

Fix verification

fixed
baseline reproduced→ patch + rebuild →patched clean

fixed: patched amdgpu.ko rebuild; test 2 re-run shows no callout fire on the freed scheduler, no panic, guest stays up (run_fixed.log) vs baseline softclock verifier panic (panic.txt).

fix.diff; run_fixed.log; ../DF-4740/fix_build.log
↓ fix.diffper-fix-DF-4741

Confirmed kernel references

Detail

Exploit chain

driver teardown (module unload / rebind / GPU removal) with any job pending on ring_mirror_list -> drm_sched_fini returns without cancelling work_tdr -> caller frees scheduler container (amdgpu_ring) -> armed delayed work fires on freed+reclaimed memory -> drm_sched_job_timedout executes from freed struct: freed-memory lockmgr, ring_mirror_list walk on dangling head, ops->timedout_job call through the freed sched->ops pointer.

Evidence (decisive lines)

panic.txt (test2 sequence + 'panic: _callout ... verifier' + softclock_handler trace, db> wedge); run_fixed.log (patched module: no fire, STILL-UP); ../DF-4740/df4740.c (harness); VERDICT.md

PoC changes

Shares the DF-4740 harness; test-2 timing tuned after the first run showed the TDR firing before fini (2*hz timeout + 2-tick pre-fini wait) and the reclaim switched from 0xde fill to zeroed blocks because queue_work skips entries whose on_queue/running flags are truthy.

Verified recommended fix

cancel_delayed_work_sync(&sched->work_tdr) in drm_sched_fini after kthread_stop (fix.diff hunk 3; full upstream-grade fix should also drain ring_mirror_list).

Verdict

Reproduced on a fresh-reset guest: drm_sched_fini (sched_main.c:652-656) only calls kthread_stop β€” it does not cancel the armed work_tdr delayed work and does not drain ring_mirror_list. The harness (test 2 of df4740.c) left a hung job on the list (work_tdr armed at sched_main.c:246), called drm_sched_fini, then kfree()d the scheduler exactly as amdgpu_fence_driver_fini frees struct amdgpu_ring after drm_sched_fini(&ring->sched) (amdgpu_fence.c:495-519); the block was reclaimed by a zeroed allocation at the identical address. ~2s later the armed callout fired on the freed object and softclock's verifier tripped: 'panic: _callout ... verifier ... (expected ...) ... softclock_handler', guest down at db> (panic.txt). Absent the verifier trip, the queued drm_sched_job_timedout work itself executes on freed memory (lockmgr on freed job_list_lock, ops->timedout_job indirect call). Trigger is teardown-time (driver unload/rebind/device removal β€” privileged), hence Medium; the freed-memory execution is unconditional once fini runs with a pending TDR. Fix (cancel_delayed_work_sync after kthread_stop, fix.diff hunk 3) validated: patched amdgpu.ko, same test -> no callout fire, no panic, guest stays up (run_fixed.log).