cd9660 ihash publish-before-validate: vget_internal leaves the half-initialized iso_node in the hash on every post-publish error path; a concurrent cd9660_ihashget blocked in vget wakes with the orphaned VNON vnode (i_mnt == NULL, i_devvp == NULL) and returns it as SUCCESS β NULL-deref panic in cd9660_getattr/read on first use
Summary
cd9660_vget_internal (vfsops.c:763) inserts the freshly allocated iso_node into the global ihash and sets vp->v_data BEFORE reading/validating the on-media record or initializing i_mnt/i_devvp (:825-827). All five error paths (:775-779 lbn>=vss, :781-787 off+34>lbs, :789-797 bread error, :800-809 record-crosses, :837-840 relocated devblkatoff error) vx_put() without removing the node from the hash - removal only happens in cd9660_reclaim (node.c:228) at refcount 0. A concurrent cd9660_ihashget for the same (dev,ino) has already taken a reference inside vget and blocks on the creator's VX lock; the creator's failure releases the lock with refcount 2->1 (no terminate, no reclaim), the waiter's post-block re-walk (:124-130) finds the orphan still hashed and returns it as success - a half-initialized VNON vnode. Lookup hands it to namei; the first stat() dereferences ip->i_mnt->im_fmask (vnops.c:158) -> page fault -> panic. DF-3194 (NFS) twin: there the loser's own vx_put panicked; cd9660's NULL guards defuse that variant while the orphan-handoff variant survives. PoC (crafted ISO with RRIP CL lba=0x1000 beyond device, two parents defeating the vnode lock): failing path executes ~45,000/s and in-device CL proves the relocated branch ran, but ~80M raced attempts across 6 configurations did not open the window on the stock guest - LWKT tokens drop only on voluntary block, never on preemption; not_reproduced with full why-not analysis; near-deterministic whenever an in-window block exists. Unpriv + crafted ISO (mount precondition); panic-only (NULL-deref read), no uid0 route. Fix: ihashpurge on all five post-publish error paths + ihashget vput-on-vanished.
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-3208 Β· 12 files| File | Type | Description | Size | |
|---|---|---|---|---|
| README.md | β | 5.6 KB | β raw | |
| VERDICT.md | β | 7.3 KB | β raw | |
| craft_df3208.py | β | 5.6 KB | view raw | |
| trigger.c | β | 3.1 KB | view raw | |
| build.sh | β | 103 B | view raw | |
| run.sh | β | 212 B | view raw | |
| build.log | β | 63 B | view raw | |
| run.log | β | 2.0 KB | view raw | |
| run.2.log | β | 268 B | view raw | |
| env.txt | β | 951 B | view raw | |
| fix.diff | β | 3.1 KB | view raw | |
| verdict.json | β | 4.9 KB | view raw |
DF-3208 β cd9660 ihash publish-before-validate: race loser receives an
orphaned half-initialized VNON vnode (i_mnt == NULL) as SUCCESS β NULL-deref
panic in any subsequent VOP
What (root cause)
cd9660_vget_internal() (sys/vfs/isofs/cd9660/cd9660_vfsops.c:732-909)
publishes the freshly allocated iso_node into the global inode hash
(cd9660_ihashins(), cd9660_node.c:143-160) at vfsops.c:763 β before
the on-media record is read and validated, and before i_mnt/i_devvp
are set (vfsops.c:825-827). Every post-publish error path
- :775-779 lbn >= volume_space_size β ESTALE
- :781-787 off+34 > logical_block_size β ESTALE
- :789-797 bread() error β error
- :800-809 record crosses block boundary β ESTALE
- :837-840 relocated-path
cd9660_devblkatoff()error β error
does vx_put(vp) without removing the node from the hash. Hash removal
only happens in cd9660_reclaim() (cd9660_node.c:228), which runs at
refcount 0 β but a concurrent cd9660_ihashget() for the same (dev, ino)
already took a reference inside vget() (vfs_lock.c:589, ref before lock)
and is blocked on the VX lock the creator holds. When the creator fails and
releases, the waiter wakes, re-walks the hash (cd9660_node.c:124-130),
finds the orphaned node still there, and returns it as a SUCCESS
(cd9660_node.c:131-132). cd9660_vget_internal() then returns 0 with
*vpp = a half-initialized vnode: v_type == VNON, i_mnt == NULL,
i_devvp == NULL, iso_extent/i_size/inode.* == 0 (kmalloc M_ZERO,
init at :825+ never ran).
The first VOP on that vnode dereferences NULL: cd9660_getattr() reads
ip->i_mnt->im_fmask at cd9660_vnops.c:158 β kernel page fault β panic;
read() likewise via blksize(imp=ip->i_mnt,...) in cd9660_read /
cd9660_devblkatoff.
This is the cd9660 twin of the DF-3194 (NFS) "race-loser gets an
unconnected VNON" class β but the failure mode is different and worse in
shape: in DF-3194 the loser's own error path panicked on v_data == NULL;
here cd9660_inactive()/cd9660_reclaim() explicitly tolerate ip == NULL
(cd9660_node.c:199/205/227) and defuse that twin, while the handoff of
the orphan returns SUCCESS and pushes the detonation into any later VOP.
Threat model
Crafted ISO9660/RRIP image, mounted (root setup or vfs.usermount=1
self-mount β same precondition family as DF-0878/DF-3146/DF-3156).
After the mount, any unprivileged user with traverse permission races two
lookups that resolve to the same crafted ino through two different parent
directories (defeats the per-parent vnode lock taken by
vop_compat_nresolve, kern/vfs_default.c:200) β e.g. directory entries
"X/PWN" and "Y/PWN" both carrying an RRIP CL entry pointing beyond the
device: rrip_pclink (cd9660_rrip.c:310-315) sets dp->i_ino = lba << 11,
dp->i_ino != ino makes lookup pass relocated=1 (cd9660_lookup.c:366-367)
and cd9660_devblkatoff(vp,0) at vfsops.c:837 fails (dscheck EINVAL/EIO on
the beyond-device offset) β the :838 error path β poison window.
Panic (local DoS) when it fires; no memory corruption (all derefs are NULL reads at small offsets).
Dynamic verdict (see VERDICT.md)
not_reproduced β the static chain is proven link-by-link, and the
failing path demonstrably executes on every attempt (~45k/s live:
stat /mnt/x/X/PWN β EINVAL from vfsops.c:837-838 while the same CL
pointing in-device makes stat succeed, proving the vget ran the
relocated branch), but the race window cannot be opened on a stock
system: cd9660 is not MPSAFE (VFS_SET(..., VFCF_READONLY) β no
VFCF_MPSAFE), so every entry into vget_internal is serialized by the
global mp_token (an LWKT token, sys/sys/mplock2.h). The token is only
dropped when the holder blocks; lwkt_preempt() never releases a running
thread's tokens (kern/lwkt_thread.c β preempt into/out of token holders is
refused), and none of the reachable failure windows contains a voluntary
block β the beyond-device bread fails synchronously via dscheck. ~80M
attempted races across 6 configurations (up to 6 CPUs/6 threads, network
interrupt pressure, 8-40 concurrent buffer-churn readers, and boot-tuned
kern.nbuf of 1024 and 256 buffers to force geteblk allocator waits)
produced no hit.
The window would open (and the chain become near-deterministic) whenever the in-window code blocks: genuinely failing-but-slow device I/O (real media errors, NFS-backed stores), an MPSAFE conversion of cd9660, or any future sleep added between ihashins and the error paths.
Reproduce (baseline attempt; guest must be up)
Host:
python3 craft_df3208.py df3208.iso scp -F dfbsd-qemu/config df3208.iso trigger.c dfbsd:/root/ scp -F dfbsd-qemu/config craft_df3208.py dfbsd:/root/
Guest (root setup; trigger as nobody):
cc -O2 -pthread -o /root/trigger /root/trigger.c chmod 755 /root/trigger vnconfig -c /dev/vn0 /root/df3208.iso # NOTE: no -S (see VERDICT.md) mkdir -p /mnt/df3208 && mount_cd9660 /dev/vn0 /mnt/df3208 su -m nobody -c "/root/trigger /mnt/df3208 5000000 6"
Expected on a vulnerable+open window: kernel panic (Fatal trap 12, page fault in cd9660_getattr β NULL i_mnt). Observed on stock guest #0: EINVAL per attempt, no panic (window closed β see VERDICT.md).
Fix
fix.diff (authored post-verification, never applied to sys/): a new
cd9660_ihashpurge(vp, ip) helper (unlink from hash, drop the devvp ref if
taken, disconnect v_data, free the node) called on all five post-publish
error paths, plus the now-mandatory vput(vp) before the goto loop retry
in cd9660_ihashget() (with the purge, a successful vget can legitimately
observe the node gone from the hash and must drop the lock+ref it holds).
DF-3208 β VERDICT
status: not_reproduced (dynamic) / static chain certain end-to-end by inspection and live partial demonstration.
The static chain (every link cited)
- Publish before validate.
cd9660_vget_internal(): - vfsops.c:745ihashgetmiss β :749getnewvnode(returns VX-locked, ref=1,v_type=VNON,v_data=NULLβ kern/vfs_mount.c:194-221) β :754kmalloc(M_ZERO)β :763cd9660_ihashins()β node now globally visible β :769vp->v_data = ip. - :771-823 (isodir==NULL: NFS fhtovp via VFS_VGET) validations, and :829-842 (relocated:cd9660_devblkatoff(vp,0)) can all fail after the publish; each error path doesvx_put(vp)(vfs_lock.c:761-768) without touching the hash. - Hash removal only at reclaim.
cd9660_ihashrem()is called only fromcd9660_reclaim()(cd9660_node.c:228), which requires the vnode refcount to hit 0 (vnode_terminate β VOP_INACTIVE β vrecycle β vclean β VOP_RECLAIM, kern/vfs_subr.c:1284-1421). - The waiter's ref blocks the reclaim. A concurrent
cd9660_ihashget(dev, ino)(cd9660_node.c:107-136) finds the node under the ihash token (:114-116), andvget()references before locking (kern/vfs_lock.c:589) then blocks on the creator's VX lockmgr. While blocked it holds a ref β creator'svx_putdrops refcount 2β1 β no terminate, no reclaim, node stays in the hash. - The waiter receives the orphan as success. Creator's unlock wakes
the waiter;
vn_locksees VRECLAIMED clear,vgetreturns 0; the post-block re-walk (:124-130) still finds the node (nobody removed it) βihashgetreturns the half-initialized vnode βvget_internalreturns 0 at :745-746 with*vpp= VNON /i_mnt==NULL/i_devvp==NULL/ all-zero inode fields. - Detonation. Lookup hands it to namei as the resolved component
(cd9660_lookup.c:360/375, vop_compat_nresolve kern/vfs_default.c:231-235);
the first
stat()βcd9660_getattr()readsip->i_mnt->im_fmask(cd9660_vnops.c:158) β NULL-deref page fault β panic. Any other VOP (read, lookup-as-dir) is equally fatal.
Live partial demonstration (what was proven on the guest)
Stock INVARIANTS kernel #0, 6 vCPU, QEMU/KVM; crafted df3208.iso
(sectors: PVD@16 with volume_space_size=0x200000, root@32, X@33, Y@34;
X and Y each contain "PWN" β a directory record (flags=2) carrying
RRIP PX + CL lba=0x1000, beyond the 40-sector device):
CL β in-device(earlier 2GB-sparse image):stat /mnt/x/X/PWNsucceeds with ino = 0x800000 (= 0x1000<<11) β proving therrip_pclink β dp->i_ino β vget_internal(relocated=1) β devblkatoffpath executes (it read the CL target and built the node from it).CL β beyond-device(80KB device):stat /mnt/x/X/PWNfails with EINVAL from vfsops.c:837-838 (devblkatoff bread of 0x800000 β dscheck) β i.e. the creator publishes the node and then fails on the intended path, ~45,000 times/second under the two-directory hammer.
Why it did not reproduce (the honest core)
cd9660 is not MPSAFE (VFS_SET(cd9660_vfsops, cd9660, VFCF_READONLY),
cd9660_vfsops.c:97 β no VFCF_MPSAFE). Every entry into vget_internal
(lookup via vop_old_lookup, NFS VFS_VGET, VFS_ROOT) is wrapped by
VFS_MPLOCK() (kern/vfs_vopops.c:185-187) = lwkt_gettoken(&mp_token)
(sys/sys/mplock2.h). Two facts close the window:
- Tokens are only released when the holder blocks (
lwkt_switchβlwkt_relalltokens, kern/lwkt_thread.c:lwkt_switch). Interrupt preemption does not release them (lwkt_preempt()refuses to preempt into/out of token-holding threads β kern/lwkt_thread.c). - None of the reachable failure windows contains a voluntary block.
The failing bread at vfsops.c:837/:789 fails synchronously in the vn
driver's dscheck (out-of-range I/O is rejected before any strategy I/O β
sys/dev/disk/vn/vn.c:292-296); an in-range read never fails (so the
relocated/validation paths either sleep-and-succeed or
fail-without-sleeping). The only theoretical in-window sleep is
getblk's buffer-allocation wait under free-list starvation.
Attempted window-openers, all negative:
| config | attempts | result |
|---|---|---|
| 4 threads, idle guest | 200k | no hit |
| 6 threads Γ 10M + ping flood + 3 churn readers | ~60M | no hit |
| 2 threads timed (rate measurement) | 400k (45k/s) | no hit |
kern.nbuf=1024 (boot-tuned), 4 threads |
1.2M | no hit |
kern.nbuf=1024 + 8Γ1MB dd churn readers, 6 threads |
~6M | no hit |
kern.nbuf=256 + 40Γ1MB dd churn readers, 6 threads Γ 2M |
12M | no hit |
Total β 80M races. The buffer-churn configurations were meant to force
geteblk waits inside the window; either the free list never empties at
the failing getblk (virtio I/O is too fast; error buffers may also short-
circuit subsequent breads) or the resulting block is too short for the
handshake. Guest never panicked; every attempt returned EINVAL cleanly
(and the transient node was reclaimed synchronously each time β no leak:
vmstat-stable vnode counts across runs).
Conclusion: the defect is structural and provable by inspection; on a stock system the non-MPSAFE mp_token serializes all creators so the half-initialized node can never be observed by a second thread. The window opens whenever the in-window code blocks: genuinely failing-but-slow device I/O (real media errors, network-backed stores), an MPSAFE conversion of cd9660 (VFCF_MPSAFE), or any future sleep added between ihashins and the error paths. DF-3194 (NFS twin, not_reproduced for the same class of reason) is the precedent.
PoC notes (what had to be fixed vs the obvious sketch)
-S 2048in the DF-3146-era vnconfig recipe means 2048 MB device size, which silently grew the backing file to 2GB sparse and made every "beyond-device" offset in-device (reads succeed, zero-filled). Attached without-S(device = file size = 80KB).- RRIP lookup derives a non-directory entry's ino from
bp->b_bio1.bio_offset + entryoffsetinblock(cd9660_lookup.c:255-259), notisodirinoβ so the trigger entry must be a directory (flags=2) with a CL entry to steerdp->i_inoat an attacker-chosen LBA. - Directory records need even-padded length β₯ 34+namelen (readdir rejects otherwise at cd9660_vnops.c:524) and RRIP-mount wants PX on entries to avoid "RRIP without PX" noise.
- Two different parent directories (X, Y) are required: same-directory
lookups serialize on the parent vnode lock in
vop_compat_nresolve(kern/vfs_default.c:200) before the mp_token is even taken.
Fix
fix.diff: cd9660_ihashpurge(vp, ip) on all five post-publish error
paths (unlink from hash, vrele i_devvp if taken, v_data=NULL, kfree);
vput(vp) added to cd9660_ihashget()'s ripped-out branch, which becomes
reachable once purges exist (a successful vget can observe the node gone
and must drop the lock+ref before retrying). Not applied to sys/; not
kernel-built (baseline never reproduced, so fix validation is vacuous β
fix_status: not_testable).
Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.
Fix verification
not_testableBaseline never reproduced on the stock guest (window closed by the non-MPSAFE mp_token), so a fix-build A/B validation is vacuous: fix_status=not_testable. The patch is a direct structural closure of the proven handoff (remove-from-hash before vx_put) and compiles by inspection against the cited lines; it was NOT applied to sys/.
[]
Confirmed kernel references
Detail
Evidence (decisive lines)
['findings/poc/DF-3208/VERDICT.md (full chain with path:line + why-not analysis + run matrix)', 'findings/poc/DF-3208/run.log (six race runs, ~80M attempts, EINVAL each attempt, no panic)', 'findings/poc/DF-3208/craft_df3208.py (RRIP CL-based trigger image; in-device vs beyond-device CL demonstrated both outcomes)', 'findings/poc/DF-3208/trigger.c (two-directory hammer defeating the per-parent vnode lock)', 'findings/poc/DF-3208/fix.diff (cd9660_ihashpurge on all 5 error paths + vput in ihashget retry)']
PoC changes
Crafted ISO per finding sketch was wrong in three ways, all fixed: (1) RRIP lookup derives a non-dir entry's ino from b_bio1.bio_offset+entryoffset (lookup.c:255-259), not isodirino -- trigger entry must be a DIRECTORY (flags=2) with an RRIP CL entry to steer dp->i_ino to an attacker-chosen LBA (relocated=1 -> vfsops.c:837-838 failure); (2) the DF-3146-era 'vnconfig -S 2048' recipe creates a 2048MB device and silently grows the backing file, making every beyond-device offset in-device -- attach without -S; (3) directory records need even-padded reclen >= 34+namelen (vnops.c:524) and PX fields. Two different parent directories are required to defeat the per-parent vnode lock in vop_compat_nresolve (vfs_default.c:200).
Verified recommended fix
Purge the iso_node from the ihash (new cd9660_ihashpurge: v_data=NULL, ihashrem, vrele i_devvp if set, kfree) on all five post-ihashins error paths in cd9660_vget_internal, and add the now-required vput(vp) before the goto-loop retry in cd9660_ihashget.
Verdict
Static chain certain end-to-end: cd9660_vget_internal publishes the iso_node into the global ihash (vfsops.c:763, node.c:143-160) before on-media validation; all five post-publish error paths (:775/:782/:793/:802/:838) vx_put() without hash removal, so a concurrent cd9660_ihashget() blocked in vget (ref-before-lock, vfs_lock.c:589) wakes with the orphaned half-initialized VNON vnode (i_mnt==NULL) and RETURNS IT AS SUCCESS; the first VOP (cd9660_getattr vnops.c:158) NULL-derefs -> panic. Live on the guest: the failing path executes ~45k/s (stat -> EINVAL from :837-838 with a beyond-device RRIP CL entry; in-device CL -> stat succeeds, proving the vget ran). Dynamically NOT reproduced across ~80M races in 6 configurations: cd9660 is not MPSAFE (vfsops.c:97), every entry is serialized by the global mp_token (LWKT token), tokens are released only on a voluntary block (lwkt_switch), and no reachable failure window contains one -- the beyond-device bread fails synchronously in vn dscheck; interrupt preemption does not release tokens (lwkt_preempt). Boot-tuned buffer starvation (kern.nbuf=1024/256) + up to 40 churn readers did not open a geteblk wait inside the window. The window opens (near-deterministic panic) whenever the in-window code blocks: failing-but-slow device I/O, an MPSAFE conversion of cd9660, or any future sleep added between ihashins and the error paths. DF-3194 (NFS twin, not_reproduced for the same structural reason) is the precedent.
No comments yet.