vcdstrategy() drains sc->bio_queue with no serialization β latent SMP TAILQ-corruption race, shielded today only by the missing D_MPSAFE flag (mplock)
Summary
bioq ops + blocking host I/O inside the drain loop with no lock - the exact shape vdisk's cothread rework fixed; shielded today by the non-MPSAFE mplock (4x20k mixed I/O survived clean on a 4-vCPU vkernel). Any D_MPSAFE conversion without serialization makes the race live with a full-host-I/O window. Adopt-vdisk-cothread fix before MPSAFE.
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-5172 Β· 12 files| File | Type | Description | Size | |
|---|---|---|---|---|
| README.md | β | 3.1 KB | β raw | |
| VERDICT.md | β | 2.6 KB | β raw | |
| vcd_race_poc.c | β | 2.2 KB | view raw | |
| odcat.c | β | 843 B | view raw | |
| guest_prep.sh | β | 1.3 KB | view raw | |
| run.log | β | 2.0 KB | view raw | |
| run.2.log | β | 1.9 KB | view raw | |
| run.s2-failed.log | β | 1.7 KB | view raw | |
| run.s3-failed.log | β | 1.6 KB | view raw | |
| env.txt | β | 462 B | view raw | |
| manifest.json | β | 1013 B | view raw | |
| verdict.json | β | 3.1 KB | view raw |
DF-5172 β vcdstrategy() drains sc->bio_queue with no serialization (latent race)
vkernel virtual CD-ROM driver β sys/dev/virtual/vkernel/cdrom/vcd.c
What
vcdstrategy() (vcd.c:148-149) manipulates sc->bio_queue with
bioqdisksort() (sorted TAILQ insertion, sys/kern/subr_disk.c:1353) and
bioq_takefirst() (sys/sys/buf2.h:167) with no lock, crit section, or
cothread β and then performs a blocking pread/pwrite inside the
drain loop, which holds the queue open for the full duration of the host
I/O. Two concurrent strategy calls on different vCPUs would corrupt the
TAILQ (lost/double-removed bios β double biodone β buffer-cache state
corruption inside the vkernel process).
The vdisk sibling was reworked precisely to fix this shape: vkd moved all
queue manipulation under cothread_lock() (vdisk.c:228-231, 250-255,
279-298). vcd never got the rework.
Why it is only latent (the shield, verified)
vcd_ops is declared { "vcd", 0, D_DISK } β without D_MPSAFE
(vcd.c:66-73). For any device whose ops lack D_MPSAFE,
dev_dstrategy() takes the global kernel lock around the strategy call:
dev_needmplock() => (dev->si_ops->head.flags & D_MPSAFE) == 0 /* kern_device.c:119-121 */
...
if (needmplock) get_mplock(); /* kern_device.c:384 */
(void)dev->si_ops->d_strategy(&ap);
if (needmplock) rel_mplock(); /* kern_device.c:388 */
So today every vcdstrategy() call on the system executes under the
mplock β concurrent entries are impossible and the unlocked bio_queue
cannot be raced. Exactly like DF-5010's dscheck shield, this is an
external, incidental guard: the day someone marks vcd D_MPSAFE (the
usual modernization step for vkernel drivers β note the mplock is a
known scalability bottleneck), the race goes live with a window that
includes a full host-side pread.
Verification (guest, 2026-09-30)
Static + dynamic:
- Dynamic attempt: 4-vCPU vkernel (
-n 4; the default boots 2), PID1 forked 4 children hammering/dev/vcd0with 20,000 mixed 64 KB reads/writes each (offsets 2048-aligned; the first attempt with 64-byte alignment was bounced bydscheckwith ~2000 "not on a sector boundary" messages β those bios never reached the driver, so the test was fixed). Result: no crash, clean exit (RACE SURVIVED, run.log / run.2.log) β consistent with the mplock shield, not with a racy driver. - Shield trace: kern_device.c:119-121 + 384-388 (above); vcd.c:67 has
no D_MPSAFE. The vkernel is fully SMP (
-n N, MAXCPU; cpu0..cpu3 observed in run logs).
Not reproduced because it cannot currently happen; filed as a latent hardening defect in the same spirit as DF-5010 (Info).
Recommended fix
When touching vcd (e.g. for DF-5171), either adopt vdisk's cothread
structure or, minimally, wrap the bioq manipulation + drain in a
crit_enter()/crit_exit() (single-CPU-per-thread) or a spinlock β and
only then consider D_MPSAFE. Not validated in this run (no live bug to
observe); the validated diff in this batch is DF-5171's.
DF-5172 VERDICT β vcd unlocked bio_queue: not reproduced β provably shielded by the mplock today; latent
Reproduced? NO β and the verification explains why that is the expected
result, not a failure of the PoC.
-
The defect is real in the driver: vcdstrategy() (vcd.c:148-149) does
bioqdisksort()/bioq_takefirst()onsc->bio_queuewith no lock of any kind, then runs a blockingpread/pwriteinside the drain loop (vcd.c:156-163) β the queue stays "open" for the duration of host I/O. vdisk's 2016 cothread rework exists exactly because this shape is unsafe (all vkd queue ops are undercothread_lock, vdisk.c:228-231/250-255/279-298). -
The shield:
vcd_opslacksD_MPSAFE(vcd.c:67), sodev_dstrategy()wraps every call in the global mplock (kern_device.c:119-121, 384-388). Concurrent entries into vcdstrategy are impossible on any SMP configuration. Same class of incidental external guard as DF-5010's dscheck shield. -
Dynamic attempt (honest): 4-vCPU vkernel (
-n 4), 4 children Γ 20,000 Γ 64 KB mixed reads/writes on/dev/vcd0, 2048-aligned (first attempt was 64-aligned and was bounced wholesale by dscheck β 2000Γ "not on a sector boundary (ssize 2048)", bios never reached the driver; fixed and rerun). Result:RACE SURVIVED - no crash this run, clean init exit (run.log, run.2.log). Zero crashes β consistent with the mplock serialization. -
Side observation: the first race attempt used
-s 2believing-sset ncpus;-sis actually RB_SINGLE (init.c:308-310) and "2" became a positional arg, breaking disk attach (mountroot panic). The ncpus flag is-n; the default vkernel already boots 2 vCPUs.
Verdict
not_reproduced / impact none (as observed) / the finding stands as
a latent hazard: Info β if vcd is ever marked D_MPSAFE without
adding driver-internal serialization, concurrent strategy calls corrupt
the bio TAILQ (window includes a full host pread/pwrite). Recommended
fix: adopt vkd's cothread (or minimally crit/lock the bioq) before any
D_MPSAFE conversion. No fix.diff in this pack (nothing observable to
validate against); DF-5171's validated diff touches the same function.
Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.
(No fix.diff for this item: documented recommendation only - see Fix above.)
Fix verification
not_testableConfirmed kernel references
Detail
Evidence (decisive lines)
run.2.log:37-39 ('DF-5172 vcd bioq race attempt', 'mixed r/w: reopening O_RDWR', 'RACE SURVIVED - no crash this run', clean init exit on 4 CPUs); run.log (read-only variant); run.s2-failed.log (documents that -s is RB_SINGLE, not ncpus - ncpus is -n, default boots 2 vCPUs); shield trace: vcd.c:67 + kern_device.c:119-121,384-388; sibling divergence: vdisk.c:228-231,250-255,279-298 (cothread_lock around all queue ops).
PoC changes
Wrote a static PID1 binary (vcd_race_poc.c) that opens /dev/vcd0 and forks 4 children doing 20000 lseek+read (second variant: 50/50 read/write via O_RDWR) each, logging to /dev/console (init stdio is dead in the vkernel). Two lessons baked in: offsets must be 2048-aligned (vcd media sector size) or dscheck silently rejects every bio before the driver sees them; -s is RB_SINGLE not ncpus (ncpus flag is -n; default vkernel boots 2 vCPUs).
Verified recommended fix
Adopt vdisk's cothread structure for vcd (or minimally guard bioqdisksort/bioq_takefirst and the drain loop with a lock/crit section) before ever marking vcd_ops D_MPSAFE.
Verdict
vcdstrategy() manipulates sc->bio_queue (bioqdisksort/bioq_takefirst, vcd.c:148-149) with zero serialization and holds the queue open across a blocking pread/pwrite - the same shape vdisk's 2016 cothread rework fixed for vkd. NOT reproducible today because vcd_ops lacks D_MPSAFE (vcd.c:67), so dev_dstrategy() executes every strategy call under the global mplock (kern_device.c:119-121, 384-388): concurrent entries are impossible. Attempted on a 4-vCPU vkernel with 4x20000 mixed aligned reads/writes on /dev/vcd0 - clean survival, as the shield predicts. Filed as a latent hazard (Info): the day vcd is marked D_MPSAFE the race goes live with a window spanning a full host-side I/O syscall.
No comments yet.