β¬’ DragonFlyBSD Kernel Audit
← triage Β· dashboard
DF-4685

amdgpu_fence_slab_fini: rcu_barrier() is a no-op β€” kldunload with pending call_rcu entries unmaps module text before the >=1s delay line fires amdgpu_fence_free: fatal trap 12 (reproduced; fix validated on rebuilt kernel)

Summary

Every fence free is call_rcu into amdgpu.ko text executing >=1s later; slab_fini calls rcu_barrier() to drain - but the port's rcu_barrier is an empty inline. kldunload within ~1s of fence activity unmaps the text while armed func pointers target it: REPRODUCED deterministically (Fatal trap 12 at the armed callback address, guest down). Root-triggered (module unload) DoS; the broken contract affects every module using the port's call_rcu/kfree_rcu. Real drain-the-delay-lines rcu_barrier fix, VALIDATED on a rebuilt guest kernel (barrier blocks, unload clean). Side-discovery: linker_file_add_dependancy never increments dep->refs (latent linker bug, noted).

Discussion (0)

No comments yet.

PoC verification

Evidence pack

findings/poc/DF-4685 Β· 8 files
FileTypeDescriptionSize
README.md β€” 1.4 KB ↓ raw
VERDICT.md β€” 4.7 KB ↓ raw
run.sh β€” 172 B view raw
panic.txt β€” 1.6 KB view raw
run.fixed.log β€” 448 B view raw
fix.diff β€” 1.7 KB view raw
manifest.json β€” 1.0 KB view raw
verdict.json β€” 5.2 KB view raw

DF-4685 β€” no-op rcu_barrier() lets kldunload unmap module text with armed call_rcu callbacks (Low, panic)

Reproduce (shared module with DF-4684)

cd /root/poc/DF-4684 && make -m /usr/share/mk KMODDIR=$PWD   # df4684.ko (STOCK headers)
kldload drm            # manual: module deliberately has NO MODULE_DEPEND (see df4684.c note)
kldload ./df4684.ko
sysctl kern.df4684_run=2       # arm: call_rcu + no-op rcu_barrier
kldunload df4684               # completes
sleep 4                        # ~1s later: Fatal trap 12 (panic.txt)

Expected (baseline): Fatal trap 12: page fault while in kernel mode, faulting IP == the callback address printed at arm time; db> prompt. Expected (patched kernel + patched-header module): rcu_barrier() blocks until the callback runs; unload completes; guest survives (run.fixed.log).

See VERDICT.md (full narrative incl. the DF linker dependency-refcount side-discovery that initially wedged the demo) and fix.diff.

VERDICT.md
↓ download raw

DF-4685 β€” rcu_barrier() is a no-op: amdgpu_fence_slab_fini() permits module unload while call_rcu delay-line callbacks into module text are pending β†’ fatal page fault (reproduced)

Root cause (line-accurate)

amdgpu_fence_slab_fini (sys/dev/drm/amd/amdgpu/amdgpu_fence.c:69-73) is the amdgpu module-exit drain:

void amdgpu_fence_slab_fini(void)
{
        rcu_barrier();                        /* amdgpu_fence.c:71 */
        kmem_cache_destroy(amdgpu_fence_slab);
}

Every amdgpu fence free goes through amdgpu_fence_release β†’ call_rcu(&f->rcu, amdgpu_fence_free) (amdgpu_fence.c:646-649), and amdgpu_fence_free β€” code that lives in amdgpu.ko's text β€” is executed by the port's per-CPU delay line (sys/dev/drm/linux_rcu.c rcu_timer, callbacks fire once an entry is >= hz ticks old). The rcu_barrier() call exists precisely so that no amdgpu_fence_free callback is still queued when the module text is unmapped.

On this port it does nothing:

/* sys/dev/drm/include/linux/rcutree.h:30 */
static inline void rcu_barrier(void) {}

So kldunload amdgpu (or module teardown of any consumer of this port's call_rcu) within ~1 s of fence activity leaves armed elm->func pointers into soon-to-be-unmapped module text; when rcu_timer sweeps the expired entry it calls straight into freed text.

Reproduction (guest, deterministic)

/root/poc/DF-4684/df4684.ko mode 2 replicates amdgpu_fence_slab_fini's contract faithfully with the port's own primitives:

  1. call_rcu(&dummy_rcu, df4684_dummy_cb) β€” callback in module text, fires in β‰₯1 s;
  2. rcu_barrier() β€” returns instantly (no-op);
  3. operator runs kldunload df4684 β€” unload completes, text unmapped;
  4. ~1 s later the delay line executes df4684_dummy_cb at its old address.

panic.txt (serial console, verbatim):

df4684: call_rcu enqueued (callback 0xffffffff831dd020 fires in >=1s)
df4684: rcu_barrier() returned (no-op on this port) β€” kldunload NOW; ...
df4684: unloading

Fatal trap 12: page fault while in kernel mode
fault virtual address   = 0xffffffff831dd020      <- the unmapped callback
fault code              = supervisor read instruction, page not present
instruction pointer     = 0x8:0xffffffff831dd020
current process         = Idle
...
Stopped at      iicbus_devclass+0x1e348:         <- ddb nearest-symbol noise
db>

Guest dropped to db> (down). That is exactly what kldunload amdgpu after GPU work does on real hardware: amdgpu_device_fini β†’ amdgpu_fence_driver_fini puts all slot fences (each amdgpu_fence_release β†’ call_rcu), then amdgpu_fence_slab_fini's no-op barrier "drains" nothing, module unmaps, β‰₯1 s later rcu_timer calls amdgpu_fence_free at a dead address β†’ fatal trap. (Root-triggered, kldunload; severity Low/DoS. The same broken contract affects any module using the port's call_rcu/kfree_rcu, but this file's amdgpu_fence_slab_fini is the instance audited.)

Note: the first two attempts of this demo wedged instead of panicking β€” the culprit turned out to be DF's linker not refcounting module dependencies (kern_linker.c:574 linker_file_add_dependancy() never does dep->refs++), so an earlier MODULE_DEPEND(df4684, drm, ...) made kldunload df4684 recursively tear down drm.ko, whose headless exit blocks forever in a DRM wait_event (wchan "lwe", linux/wait.h:133). Removing the MODULE_DEPEND (manual kldload drm) made the unload complete and the panic reproduce deterministically. That linker refcount gap is a separate latent issue, noted for the record.

Fix (validated A/B on the guest)

fix.diff implements a real rcu_barrier() in linux_rcu.c (wait until every per-CPU delay line is drained) and turns the rcutree.h stub into the extern declaration. Validation on the rebuilt kernel (make -j6 nativekernel KERNCONF=X86_64_GENERIC, RC=0 under -Werror, installed + rebooted):

  • baseline (stock kernel): mode 2 + kldunload β†’ Fatal trap 12 at the unmapped callback, guest down (panic.txt);
  • patched kernel + module rebuilt against the patched rcutree.h: mode 2's rcu_barrier() actually blocks (~1 s) until the dummy callback has run ("dummy RCU callback executed ... still mapped" printed before unload), kldunload completes, guest survives indefinitely (run.fixed.log) β€” the amdgpu unload contract restored.

See ../DF-4684/ for the shared module source and the guest environment.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.

Fix verification

fixed
baseline reproduced→ patch + rebuild →patched clean

A/B on the guest: baseline kernel -> Fatal trap 12 at the unmapped callback (panic.txt); after applying fix.diff to the guest /usr/src, rebuilding (nativekernel RC=0 under -Werror), installing and rebooting, the same trigger drains: the real rcu_barrier blocks inside the sysctl until 'dummy RCU callback executed' prints, then kldunload completes and the guest survives indefinitely (run.fixed.log, uptime confirmed). amdgpu's slab_fini unload contract restored.

panic.txt vs run.fixed.log; fix.diff (git-apply clean against sys/; touches rcutree.h + linux_rcu.c); build_kernel.log (nativekernel RC=0); post-validation md5 restore of rcutree.h/linux_rcu.c to pristine
↓ fix.diffper-fix-DF-4685

Confirmed kernel references

Detail

Exploit chain

kldunload amdgpu (root) within ~1s of fence activity -> fence_driver_fini's puts enqueue amdgpu_fence_free callbacks -> no-op rcu_barrier drains nothing -> module text unmapped -> rcu_timer executes elm->func into unmapped text -> fatal trap 12 (page-not-present instruction fetch at the old callback address) -> kernel panic / DoS

Evidence (decisive lines)

panic.txt (verbatim serial console: callback armed at 0xffffffff831dd020, Fatal trap 12 at exactly that address after 'df4684: unloading'); run.fixed.log (patched kernel: real rcu_barrier blocks, 'dummy RCU callback executed ... still mapped' prints BEFORE 'rcu_barrier() returned', kldunload completes, guest survives); VERDICT.md (full narrative + the linker-dependency side-discovery)

PoC changes

mode 2 of the DF-4684 KLD (shared module): call_rcu with an in-module callback + the port's rcu_barrier + operator kldunload; first two runs used MODULE_DEPEND(drm) which triggered the DF linker dep-teardown wedge β€” removed the dependency (manual kldload drm) and the panic reproduced deterministically

Verified recommended fix

implement rcu_barrier() in linux_rcu.c (drain all per-CPU delay lines) and make rcutree.h declare it instead of the no-op inline

Verdict

Reproduced as a deterministic fatal trap on the guest: amdgpu_fence_slab_fini (amdgpu_fence.c:69-73) relies on rcu_barrier() to drain the port's call_rcu delay line (every fence free is call_rcu(&f->rcu, amdgpu_fence_free), amdgpu_fence.c:646-649, executed >=1s later by linux_rcu.c rcu_timer) before module text may be unmapped β€” but rcu_barrier() is a static-inline no-op (rcutree.h:30). The KLD replica (same module as DF-4684) enqueued a call_rcu whose callback lives in module text, called the port's rcu_barrier() (returned instantly), and kldunload(8) then completed and unmapped the text; ~1s later rcu_timer called the callback at its old address: 'Fatal trap 12: page fault while in kernel mode, fault virtual address = 0xffffffff831dd020, supervisor read instruction, page not present' β€” guest down at db> (panic.txt). This is exactly what kldunload amdgpu after GPU fence activity does on real hardware. Root-triggered (module unload), so Low/DoS. Two earlier attempts wedged instead of panicking β€” root-caused to DF's linker not refcounting module dependencies (kern_linker.c:574 linker_file_add_dependancy never bumps dep->refs), which made kldunload recursively tear down drm.ko (auto-loaded via MODULE_DEPEND) whose headless exit blocks forever in a DRM wait_event (wchan 'lwe', linux/wait.h:133) β€” removing the MODULE_DEPEND made the panic reproduce deterministically; that linker refcount gap is a separate latent issue worth its own look.