DF-4482 — vinum revive waitlist drain use-after-free (kernel panic)

What this proves

revive_block() drains sd->waitlist by launching each request while it is still linked into the waitlist, then reading sd->waitlist->next after the launch (sys/dev/raid/vinum/vinumrevive.c:216-217). A launched request is freed by complete_rqe() -> freerq() as soon as its I/O completes, and nothing ever unlinks a request from sd->waitlist at completion time. With an md-backed (malloc disk) vinum drive the completion happens synchronously inside dev_dstrategy() (see sys/dev/disk/md/md.c mdstrategy_malloc() -> biodone(bio)), so the request is already freed when the drain loop reads rq->next and hands the (freed, possibly reallocated) chain to the next launch_requests() — which then fires request/buf structures that have already been completed and unlocked:

panic: bpdone: bp 0xfffff80116ae1eb0 not busy
bpdone() at bpdone+0x326
bpdone() at bpdone+0x326
mdstrategy() at mdstrategy+0x3b6
dev_dstrategy() at dev_dstrategy+0xb7
diskstrategy() at diskstrategy+0x3b      <- vinum DRIVE[].dev = /dev/md0s0
dev_dstrategy() at dev_dstrategy+0xb7    <- vinum launch_requests()
Debugger("panic")

Reproduced 2/2 runs. Controls: revive alone (clean) and 150,000 plain volume writes without revive (clean) — the panic requires the revive-conflict waitlist machinery.

Setup (root, one time)

sh ./setup_vinum.sh — loads vinum, GPT-partitions two md memory disks with vinum-typed partitions, creates a mirrored 3 MB volume vol (concat plex vol.p0 on md0, concat plex vol.p1 on md1), marks vol.p1.s0 stale.

Trigger

sh ./run_race.sh — starts 6 writer.sh sweepers writing 4 KB blocks all over /dev/vinum/vol/vol and runs vinum start vol.p1.s0 (revive). Writes that straddle the moving revive cursor are waitlisted (checksdstate() sets XFR_REVIVECONFLICT), and every completed revive block drains the waitlist → UAF.

Expected

Kernel panic bpdone: bp ... not busy within seconds (~5 s run 1, <30 s run 2); guest drops to db>, ssh dies.

Fix validation

fix.diff unlinks each request before launching it (2-line semantic change). Applied to the guest's /usr/src, kernel rebuilt (make nativekernel), rebooted, and the exact same PoC re-run: no panic (writers complete, revive completes). See VERDICT.md.

DF-4482 verdict — REPRODUCED (panic 4/4), memcorrupt bucket; two minimal fixes attempted, both insufficient

• finding: vinum revive-conflict machinery launches/frees requests across unsynchronized, still-referenced structures (waitlist drain UAF, waitlist append UAF, and at least one further lifetime defect) → reliable kernel panic under concurrent conflicting I/O during a subdisk revive
• guest: DragonFly dfbsd 6.5-DEVELOPMENT #0 (X86_64_GENERIC), x86_64, 6 CPUs, vinum.ko + in-kernel md, vinum(8) from base

Proven defects (path:line)

1. Drain UAF — sys/dev/raid/vinum/vinumrevive.c:216-217 (orig): launch_requests(sd->waitlist, 1) launches the head while it is still linked; with md-backed drives the I/O completes synchronously inside dev_dstrategy() (sys/dev/disk/md/md.c mdstrategy_malloc() ends in an inline biodone(bio)), so complete_rqe() -> freerq() (sys/dev/raid/vinum/vinuminterrupt.c:211-262) frees the request before the drain's sd->waitlist->next read. Nothing ever unlinks a request from sd->waitlist at completion time.
2. Append UAF — sys/dev/raid/vinum/vinumrequest.c:327-339 (orig): a new conflicting write walks sd->waitlist to append at the tail; the walk dereferences chain nodes that the drain frees concurrently (cross-CPU, no lock).
3. Additional arm(s) — after fixing (1)+(2) (see fix.diff) the panic still reproduces (run.4.patched.log), so at least one more lifetime defect participates; candidates include the launch loop's post-free walk of all-bad rqgroups after an inline completion frees the request (sys/dev/raid/vinum/vinumrequest.c:395-439: the final active group can complete and freerq() inside its own dev_dstrategy, after which the outer loop continues into the freed remaining groups of the chain — benign only while the freed chunks stay intact).

Run matrix (all on identical md-backed mirrored 3 MB volume vol)

Panic signature (identical all four times):

panic: bpdone: bp 0xfffff80116ae1eb0 not busy
bpdone() at bpdone+0x326
bpdone() at bpdone+0x326
mdstrategy() at mdstrategy+0x3b6
dev_dstrategy() at dev_dstrategy+0xb7
diskstrategy() at diskstrategy+0x3b      <- vinum DRIVE[].dev = /dev/mdNs0
dev_dstrategy() at dev_dstrategy+0xb7    <- vinum launch_requests()
Debugger("panic")

bpdone() KASSERTs BUF_LOCKINUSE(bp) (sys/kern/vfs_bio.c bpdone) — the fired buffer had already been completed/unlocked/freed by a previous completion: a double-launch of freed request memory.

What the matrix says

• The panic needs a fast-moving revive (default 64 KB blocks on a memory disk: ~48 drains/second) and writes that straddle/transition around the revive cursor — i.e. exactly the revive-conflict waitlist machinery under churn (runs 5/6 isolate the individual behaviors and are clean).
• Both individually-provable UAF arms (drain walk, append walk) were fixed and the panic persisted (runs 3/4) — the defect is a cluster of lifetime bugs around relaunching requests whose completion can free them synchronously; point fixes are whack-a-mole.

Exploitability

Demonstrated: kernel panic (proof of the double-launch-of-freed-request primitive). The primitive itself is stronger than a panic: the chain walk and launch_requests() operate on attacker-groomable freed struct request / struct rqgroup memory (vinum's own per-I/O Mallocs from the same M_DEVBUF zones are the spray), reaching dev_dstrategy() with attacker-influenced bio_driver_info/b_data/bio_offset/b_bcount. Driven to uid=0: not attempted (setup-gated by the required root-configured revive; the natural unprivileged entry is a filesystem mounted on the volume).

Preconditions: root configures a mirrored vinum volume and starts a subdisk revive; an unprivileged user issues concurrent conflicting writes through a mounted filesystem on the volume.

Fix validation (mandatory for memcorrupt)

• baseline: panic reproduced (runs 1, 2) — fix_baseline_reproduced = 1
• attempt 1 (unlink-before-launch in the drain): kernel rebuilt (make -j8 nativekernel KERNCONF=X86_64_GENERIC, ~25 min), installed, rebooted; same workload → panic again (run.3.patched.log)
• attempt 2 (attempt 1 + atomic head-steal in drain + cmpset head-push in append): rebuilt, installed, rebooted; same workload → panic again (run.4.patched.log)
• fix_status: fix_failed — the two provable arms are fixed but the machinery still crashes; a redesign is required:
• freerq()/complete_rqe() must detach the request from any waitlist, and
• the drain must not synchronously launch requests whose inline completion frees them while the walker still holds references — e.g. relaunch via a deferred/queued context, or make waitlisted completion-aware (refcount the request until both the launch and the drain drop it). fix.diff in this pack is attempt 2, kept for the record (marked INSUFFICIENT).

Artifacts

• panic.txt / run.2.log — baseline panics (runs 1, 2)
• run.3.patched.log, run.4.patched.log — panics on fix-attempt kernels
• setup_vinum.sh, writer.sh, race.sh, run.sh — exact reproduction scripts
• env.txt — guest/kernel info

FIX v2 ADDENDUM (2026-09-24, pass-2 re-audit of vinumrevive.c) — VALIDATED

The residual panic after attempts 1/2 was the launch-loop arm now filed as DF-4721 (launch_requests() walks the freed group chain after an inline completion frees the whole request) — the waitlist arms were already correctly fixed by attempt 2. fix v2 (fix2.diff, this pack) closes the whole cluster with a completion-aware lifetime:

1. launch_requests() takes a launcher reference on rq->active before firing, so complete_rqe() can never biodone()+freerq() while the launcher still walks the group chain; the launcher drops the reference under the mplock afterwards and runs the factored finish_request() itself when everything completed inline;
2. the drain atomically steals the waitlist and unlinks each request before launching it (attempt 2's drain, kept);
3. the append pushes at the head with a cmpset loop and never walks the chain (attempt 2's append, kept).

Validation on the guest (module rebuild, same workload):

• fresh stock baseline panic this session (5th independent reproduction): panic.fresh-20260923.log;
• fix2_run.log — 3 rounds of the exact stock trigger + pattern roundtrips: ALL CLEAN;
• fix2_soak.log — 10 more rounds (~2600 writer iterations each): ALL CLEAN, dmesg | grep -c panic = 0;
• fix2_4721_run.log — DF-4721's degraded RAID-5 write trigger on the same module: 5 writes + readback clean (stock trapped 2/2).

fix v2 verdict: fixed (baseline_reproduced=1, patched_reproduced=0). See VERDICT2.md for the machine-readable summary. Related pass-2 findings extending this work: DF-4800 (concurrent-revive cursor race), DF-4801 (orphaned waitlist hang — its fix stacks on v2).

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.