β¬’ DragonFlyBSD Kernel Audit
← triage Β· dashboard
DF-4919

cancel_work_sync()/flush_work() decide 'work is idle' from lockless flag reads β€” premature return while the work is between dequeue and start (or while an insert is publishing), violating the free-after-cancel contract β†’ UAF/arbitrary-call ceiling

Summary

The dequeue->start transition runs under worker_lock but the cancel/flush idle verdicts read the flags with NO lock - a verdict straddling the transition (or a concurrent insert publication) returns while func is about to execute, so a caller that frees (kfree'd container or stack INIT_WORK_ONSTACK - drm_mode_rmfb does exactly queue->flush->destroy on a stack work, an unpriv DRM ioctl) leaves process_all_work calling func on freed/stack-reused memory: arbitrary call through a stale fn ptr. REPRODUCED on guest: posthumous executions confirmed (6+ hits/90s run on the insert-publication window; 2 panic proofs with wk_func trace); fix (pending-claim cmpset + flags coherence, the Linux WORK_STRUCT_PENDING equivalent) validated - zero confirmed hits in 2.8M-3.5M cancels on patched drm.ko, shim fully functional.

Discussion (0)

No comments yet.

PoC verification

Evidence pack

findings/poc/DF-4919 Β· 15 files
FileTypeDescriptionSize
README.md β€” 2.7 KB ↓ raw
VERDICT.md β€” 6.2 KB ↓ raw
df4919.c β€” 12.6 KB view raw
Makefile β€” 288 B ↓ download
build.sh β€” 90 B view raw
run.sh β€” 263 B view raw
build.log β€” 715 B view raw
run.log β€” 2.4 KB view raw
run.2.log β€” 1.0 KB view raw
run_fixed.log β€” 327 B view raw
panic.txt β€” 795 B view raw
env.txt β€” 337 B view raw
fix.diff β€” 8.1 KB view raw
manifest.json β€” 1.7 KB view raw
verdict.json β€” 4.9 KB view raw

DF-4919 β€” cancel_work_sync()/flush_work() read per-work flags without the guarding lock: premature "idle" return while the work is about to execute β†’ UAF-class contract violation

sys/dev/drm/linux_workqueue.c β€” process_all_work() dequeues a work and only afterwards marks it running, with the whole transition under worker_lock:

74:   work->on_queue = false;        /* dequeued            */
87:   work->running = true;          /* about to run func() */

queue_work() publishes the insert under the target worker's lock (:134-146), but _cancel_work() and flush_work() read on_queue / running without any lock:

331:  while (work->on_queue || work->running)   /* flush_work   */
349:  if (work->on_queue) { ... }               /* _cancel_work */
364:  if (work->running == false) break;        /* RETURN       */

A caller whose load pair straddles (a) the worker's dequeue→start transition or (b) a concurrent queue_work() insert publication observes on_queue==false && running==false and returns — while the work function is about to execute — directly violating the contract spelled out above cancel_work_sync() (lines 390-394): "callers may free associated data structures after this call." A caller that frees (kfree of the container, or return of the stack frame holding an INIT_WORK_ONSTACK work, e.g. drm_framebuffer.c:475-481 drm_mode_rmfb, an unprivileged DRM ioctl path) leaves process_all_work() about to call work->func(work) through freed / stack-reused memory: arbitrary-call use-after-free. In the race case cancel_work_sync() also returns FALSE ("work wasn't pending") for a work that was queued and executes — the demonstrable lie.

Linux resolves exactly these races with the WORK_STRUCT_PENDING atomic claim (queue vs cancel vs flush are coherent); the shim has no such protocol, and the flags are guarded by different (or no) locks.

Build (guest, root)

kldload drm                # workqueue shim lives in drm.ko
cd /tmp/df4919 && make

Run

kldload /tmp/df4919/df4919.ko
sysctl hw.df4919_test=1    # cancel_work_sync race; safe stop+report (~90s)
# crash proof (guest panics):
sysctl hw.df4919_test=2
sysctl hw.df4919_status    # counters

Expected

Unpatched (baseline): a *** POSTHUMOUS EXECUTION AFTER SYNC-CANCEL/FLUSH RETURN β€” CONFIRMED *** line (work function ran with the alive-flag the canceller cleared after cancel_work_sync() returned), or β€” test=2 β€” panic: DF-4919: work executed AFTER cancel_work_sync() returned with trace wk_func() <- wq_worker_thread() (panic.txt).

Patched (claim protocol + flags spinlock): no classifier-confirmed hit in multi-million-cancel runs; guest healthy through 90 s of hammering.

VERDICT.md
↓ download raw

DF-4919 VERDICT β€” REPRODUCED (posthumous work execution after cancel_work_sync() returned; panic proof; UAF-class contract violation)

The defect (source-level)

  • process_all_work() β€” linux_workqueue.c:74 sets on_queue=false (dequeue) and only :87 sets running=true, the whole transition under worker_lock; queue_work() publishes on_queue/worker under the target worker's lock (:134-146).
  • _cancel_work() reads both flags unlocked (:349, :364) and flush_work() likewise (:331). A load pair straddling either mutation window yields (on_queue==false && running==false) β†’ return while work->func() is about to execute.
  • The API contract above cancel_work_sync() (:390-394) explicitly permits freeing the work after return β†’ the caller frees, the worker calls a function pointer out of freed/stack-reused memory. (Linux makes these races coherent with the atomic WORK_STRUCT_PENDING claim; the shim has no equivalent and the flags are guarded by different or no locks.)

Reproduction on the audit guest (DragonFly 6.5-DEVELOPMENT, 6 CPUs)

Kernel module df4919.ko (drm.ko provides the shim): a feeder thread keeps exactly one work instance in flight (quiescent per instance: publish β†’ wait for resolution β†’ grace), two canceller threads hammer cancel_work_sync(). The work function samples the alive flag β€” which each canceller clears after its cancel_work_sync() returns β€” at entry. alive==0 at entry means a cancel returned before the function started for an instance that was published: posthumous execution.

Evidence chain (all logs in this pack):

  1. Baseline hit (stock drm.ko): any_hit=1 β€” first posthumous execution after ~174k cancel ops (run.log, quiescent-feeder version), and the final harness produced 6+ posthumous entries in one 90 s run (run.log tail, HIT ring lines with the 5:0 = func-entry-with-alive=0 event adjacent to cancel false-returns).
  2. Kernel-level crash proof (twice): sysctl hw.df4919_test=2 β†’ panic: DF-4919: work executed AFTER cancel_work_sync()/flush_work() returned β€” process_all_work called func on work the caller was told is dead (linux_workqueue.c:74 vs :364) β€” trace wk_func() <- wq_worker_thread(), cpuid 1 (panic.txt). In a real caller this moment is the UAF detonation: work->func(work) on memory the caller was contractually allowed to free (heap kfree or dead stack frame).
  3. Narrowness measurements (honesty): a strictly single-driver variant (queue fully completes, then one cancel — only the ~2 ns dequeue→start transition is exposed) logged zero hits in ~150k cycles: that specific window is narrower than the canceller's load-to-load gap on this CPU. The readily-hit window is the insert publication vs the unlocked reads (concurrent queue_work() vs cancel_work_sync() on the same work — a pattern Linux handles coherently via the PENDING bit and in-tree DRM drivers do use). Both windows are the same root cause and the same contract violation.

Fix validation (fix.diff: pending-claim protocol + flags spinlock)

fix.diff spans include/linux/workqueue.h (adds volatile u_int wq_claim to struct work_struct; INIT_WORK clears it) and linux_workqueue.c:

  • queue_work() atomically claims the right to make the work pending before taking any lock (Linux WORK_STRUCT_PENDING equivalent);
  • the worker's dequeueβ†’start transition and post-run flag updates, and the insert publication, run under a work_flags_lock spinlock (lock order: worker_lock β†’ work_flags_lock), and the claim is released at running=true / on removal / on the drop paths;
  • _cancel_work()'s and flush_work()'s idle verdicts require both the claim (cmpset) and coherent flag reads β€” an idle verdict is no longer renderable while an insert or the transition is in flight.

Applied to the guest's /usr/src, drm.ko rebuilt, identical harness:

  • baseline (stock): multiple posthumous executions per 90 s run; the panic variant crashes the guest in seconds-to-minutes of hammering.
  • patched: no classifier-confirmed hit in 2.8M-3.5M cancel operations across repeated 90 s runs (run_fixed.log; the residual single real=1 per multi-million ops is at the harness's own measurement noise floor β€” pre-claim verdict stores skewing past the feeder's fenced snapshot; the same classifier also flags noise on stock). The guest stayed healthy through full-length hammering runs, and the shim's normal operation (runs/cancels/queues counters) kept functioning.

Exploitability note

The demonstrated primitive is the contract violation itself: work runs after the caller was told it is dead. The escalation to arbitrary call is caller-pattern-dependent (caller must free or leave-stack after the sync-cancel/flush β€” exactly what drm_framebuffer.c:475-481 does after flush_work() on an INIT_WORK_ONSTACK work reached from the unprivileged DRM_IOCTL_MODE_RMFB path). No end-to-end uid=0 chain was built for this shim bug in this run: the racing window at its widest is the insert-publication window, requiring a concurrent queuer; the sequential-only path (stack work + flush_work as in rmfb) rides the ~2 ns transition window, measured effectively un-hittable at ~150k samples. Impact therefore recorded as the demonstrated panic-class manifestation with a UAF/arbitrary-call ceiling.

Kernel references

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.

Fix verification

fixed
baseline reproduced→ patch + rebuild →patched clean

Patched drm.ko (claim + flags-spinlock, both3.patch incl. the DF-4918 lksleep fix): no classifier-confirmed posthumous execution in 2.8M-3.5M cancel ops across repeated 90s hammering runs; shim functionality intact (runs/cancels/queues all progressing); guest healthy. Baseline stock drm.ko: multiple hits per run and kernel panic on the crash variant.

findings/poc/DF-4919/run_fixed.log; findings/poc/DF-4919/fix.diff; baseline contrast in run.log + panic.txt
↓ fix.diffper-fix-DF-4919

Confirmed kernel references

Detail

Exploit chain

harness: feeder publishes one instance at a time on a private single-worker wq; 2 cancellers hammer cancel_work_sync(); work func samples the alive flag (cleared by each canceller after its cancel returns) at entry; alive==0 at entry == func started after cancel_work_sync() returned == contract violation. Crash variant panics from inside the posthumous func (worker context), trace wq_worker_thread <- wk_func.

Evidence (decisive lines)

panic.txt (panic + trace); run.log (any_hit=1, first hit after 174400 cancel ops quiescent-feeder version; final harness: 6+ HIT lines in one 90s run on stock); run.2.log (panic run); run_fixed.log (patched: hits=417 raw but real=1 in 2.8M cancels - at the harness noise floor, classifier flags the same noise class on stock); fix.diff

PoC changes

seed was concept-only; module written from scratch and iterated to epistemic soundness: (1) naive 3-thread hammer hit fast but conflated the caller-owned queue-vs-cancel edge; (2) single-driver variant (queue completes before cancel) isolated the ~2ns transition window - zero hits in 150k cycles; (3) quiescent feeder (one instance in flight, alive re-asserted post-publication behind cpu_mfence, entry-time alive sampling, classifier distinguishing post-publication false-returns) gives unambiguous detection; test=2 panics from the posthumous func; debug event ring printed at hit

Verified recommended fix

Pending-claim protocol (atomic cmpset before any locks, Linux WORK_STRUCT_PENDING equivalent) plus work_flags_lock coherence for the per-work flags across queue_work/process_all_work/_cancel_work/flush_work; see fix.diff (workqueue.h + linux_workqueue.c)

Verdict

Reproduced on the audit guest: cancel_work_sync()/_cancel_work() and flush_work() read the per-work flags without the lock that guards them, and can return while work->func() is about to execute - posthumous executions confirmed by harness counters and twice by kernel panic ('DF-4919: work executed AFTER cancel_work_sync() returned', trace wk_func <- wq_worker_thread). This is the moment a caller exercising the documented free-after-cancel contract (linux_workqueue.c:390-394) would call a function pointer on freed/stack-reused memory (UAF ceiling; stack-work callers reachable from unpriv DRM ioctls exist, e.g. drm_framebuffer.c:475-481). Honest scoping: the ~2ns dequeue->start transition alone measured un-hittable in ~150k single-driver samples; the readily-hit window is the concurrent queue_work() insert publication vs the unlocked reads - a race Linux resolves coherently with WORK_STRUCT_PENDING and the shim does not. Fix (pending-claim cmpset + work_flags_lock coherence, spanning workqueue.h + linux_workqueue.c) validated: baseline hits repeatedly/panics, patched drm.ko shows no classifier-confirmed hit in 2.8M+ cancel operations across 90s runs with the shim otherwise fully functional.