DF-3862 — amdgpu_device_gpu_recover() dereferences the timed-out job that the fence-completion path can already have freed (UAF, concurrent-reinit family)

File: sys/dev/drm/amd/amdgpu/amdgpu_device.c — amdgpu_device_gpu_recover() (3388-3464)

Root cause

Lifetime chain (all in this tree):

1. drm_sched_job_timedout() (sys/dev/drm/scheduler/sched_main.c:250-289) picks job = list_first_entry_or_null(&sched->ring_mirror_list, ...) under job_list_lock, drops the lock, then calls sched->ops->timedout_job(job) (line 271). No reference of any kind is taken on the job.
2. amdgpu_job_timedout() (amdgpu_job.c:31-48) forwards to amdgpu_device_gpu_recover(adev, job) (amdgpu_job.c:47).
3. Meanwhile, if the job's hardware fence was signaled just before/while the TDR fires (GPU hung but this job completed; or amdgpu_fence_process/force_completion raced), the normal completion chain runs asynchronously: drm_sched_process_job → signals s_fence->finished → finish_cb → drm_sched_job_finish (sched_main.c:203-227) → list_del_init(&s_job->node) + sched->ops->free_job(s_job) = amdgpu_job_free_cb → kfree(job) (amdgpu_job.c:110-120).
4. amdgpu_device_gpu_recover() keeps dereferencing job: - loop 1, amdgpu_device.c:3414 if (job && job->base.sched != &ring->sched) — executed for EVERY ring i (up to 17 iterations) across the parked scheduler set-up; - amdgpu_device.c:3417 drm_sched_hw_job_reset(..., job ? &job->base : NULL); - loop 2 after the multi-second amdgpu_device_reset() (MMIO waits, PSP handshake, VRAM recovery): amdgpu_device.c:3438 if ((!job || job->base.sched == &ring->sched) && !r).

Between (3) and each of these reads there is no lock that protects the job object (job_list_lock only guards list membership, and the job is already off the list once finish ran). ⇒ use-after-free read; a groomed reuse of the freed struct amdgpu_job slab additionally steers which rings get drm_sched_job_recovery() and feeds a forged job->base.sched pointer comparison.

Note amdgpu_fence_driver_force_completion() (amdgpu_fence.c:582-586) is called at amdgpu_device.c:3420 inside this window — it force-signals every outstanding fence on the ring, actively pushing in-flight jobs toward the free path above (for jobs whose parent callback detached in drm_sched_hw_job_reset the chain is severed; for a job that completed a moment earlier the chain is already running).

Preconditions

• amdgpu_gpu_recovery=1 module parameter (default -1 = recovery disabled on bare metal; enabled for SR-IOV VFs) — amdgpu_device.c:3363-3377.
• A GPU hang (lockup_timeout, default 10 s) plus completion of the timed-out job racing the TDR — the classic hang-just-as-job-finished window.

Code is inherited verbatim from Linux ~4.20; upstream later reworked scheduler/timeout lifetime handling (5.x).

Trigger

tdr_race.c — renders the race: submits an endless IB to the compute ring (hang), times a concurrently-completing job, and loops. Needs the module parameter amdgpu.gpu_recovery=1 on hardware. Deterministic wins are not guaranteed (timing race); a DEBUG_MALLOC/invariant kernel would flag the kfree'd-slab read.

Fix

fix.diff snapshots job->base.sched once at entry (before any fence can be force-completed) and stops dereferencing job in both loops. The fully complete fix additionally needs scheduler-side job refcounting (as upstream did later); this patch removes the reachable post-reset dereferences in this file.

DF-3862 VERDICT — not_reproduced at runtime (hardware-gated), speculative-likely race, source-traced

Reproduction attempt

Guest has no AMD GPU (see env.txt): no render node, amdgpu attaches to nothing, jobs can never be submitted, no TDR can fire. The race is therefore untestable here: status = not_reproduced, reproduced = 0.

Source trace establishing the lifetime gap (path:line)

1. TDR picks the victim without holding any job reference: sys/dev/drm/scheduler/sched_main.c:258-271 — job = list_first_entry_or_null(&sched->ring_mirror_list, ...) under job_list_lock; lockmgr(&sched->job_list_lock, LK_RELEASE); at :268; sched->ops->timedout_job(job); at :271.
2. amdgpu_job.c:31-48 — amdgpu_job_timedout() → amdgpu_device_gpu_recover(ring->adev, job) at :47.
3. Free chain: dma_fence_signal on the hw fence → drm_sched_process_job (sched_main.c, s_fence->cb) → signals s_fence->finished → drm_sched_job_finish_cb (sched_main.c:229-235) → schedule_work(&job->finish_work) → drm_sched_job_finish (sched_main.c:203-227): list_del_init(&s_job->node) at :220 then sched->ops->free_job(s_job); at :226 → amdgpu_job_free_cb (amdgpu_job.c:110-120) → kfree(job); at :119.
4. Dangling dereferences in amdgpu_device.c:3414 (job->base.sched read for every ring), 3417 (&job->base), 3438 (job->base.sched again after the multi-second amdgpu_device_reset()).
5. Ammunition inside the window: amdgpu_fence_driver_force_completion(ring) at amdgpu_device.c:3420 → amdgpu_fence.c:582-586 force-processes every outstanding fence — for any job whose parent callback was NOT detached in drm_sched_hw_job_reset (i.e. it completed a moment earlier, callback already firing) the free chain (3) runs to completion during the reset.

Why speculative and not "certain": the deref requires the specific interleaving where the timed-out job itself completes concurrently. The window is real and the code holds no protection, but I could not execute it (no GPU) — filing confidence "speculative" for exploitation, with the structural UAF demonstrated from source.

Preconditions (honest gating)

• amdgpu_gpu_recovery=1 (root module param; default off on bare metal; on for SR-IOV VF) — amdgpu_device.c:3363-3377 (should_recover_gpu).
• GPU hang + racing completion of the same job. Impact if hit: UAF read on a freed heap object; with slab grooming, drm_sched_job_recovery() can be steered onto additional rings via a forged job->base.sched pointer — Medium.

Fix validation

fix.diff (job_sched snapshot) is git apply --check clean. Building the patched kernel here cannot exercise the path (no AMD GPU): fix_status = not_testable.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.