deget() publishes the denode in the global hash before taking its reference (refcnt store outside the token): SMP TOCTOU permits duplicate denodes for one directory entry
Summary
msdosfs_hashins() (called at deget:308) inserts the new denode and releases dehash_token; ldep->de_refcnt = 1 is stored afterwards at :323, outside the token. Both hash users treat de_refcnt <= 0 entries as absent (hashget:131, hashins:178), so a concurrent deget() on another CPU scanning the bucket inside the window can insert a second denode for the same (dev, dirclust, diroffset) - two vnodes for one dirent, divergent de_FileSize/de_StartCluster caches, two deupdat writers for one slot, writes through a stale denode into freed/reallocated clusters (cross-linked files on multi-user volumes; INVARIANTS usemap_alloc KASSERT later). Empirically NOT reproduced: an honest detector (ftruncate-publish-on-success + persistent-divergence confirmation) saw 0 confirmed duplicates in ~2-4M synchronized deget pairs on 6 vCPUs; an earlier naive detector's 'hits' were root-caused as its own artifact (detector lesson in pack). Code-level window real but a handful of instructions wide - speculative hardening with the one-line ordering fix built and boot-validated (refcnt=1 before hashins; included in DF-3157's fix.diff).
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-3158 Β· 8 files| File | Type | Description | Size | |
|---|---|---|---|---|
| README.md | β | 2.0 KB | β raw | |
| VERDICT.md | β | 2.3 KB | β raw | |
| hammer.c | β | 3.5 KB | view raw | |
| build.sh | β | 94 B | view raw | |
| run.sh | β | 110 B | view raw | |
| run.log | β | 1.3 KB | view raw | |
| manifest.json | β | 671 B | view raw | |
| verdict.json | β | 2.3 KB | view raw |
DF-3158 β deget() publishes the denode in the hash before taking its reference (duplicate-denode TOCTOU)
deget() (sys/vfs/msdosfs/msdosfs_denode.c:308) inserts the new denode
into the global hash β releasing dehash_token inside msdosfs_hashins β
and only sets ldep->de_refcnt = 1 at :323, outside the token. Both
msdosfs_hashget() (:131) and msdosfs_hashins() (:178) treat
de_refcnt <= 0 entries as absent. A concurrent deget() for the same
(dev, dirclust, diroffset) on another CPU that scans the bucket inside
that window sees de_refcnt == 0, concludes no denode exists, and appends
its own β two vnodes/denodes for one directory entry. Consequences if
won: divergent de_FileSize/de_StartCluster caches, two deupdat
writers for one dirent slot (stale-denode clobber), writes through a stale
denode into freed/reallocated clusters (cross-linked files on multi-user
volumes), INVARIANTS usemap_alloc KASSERT on later allocation.
The hammer races parallel open()s of one file and detects a duplicate
pair by persistent size divergence: thread 0 ftruncates (publishing the
round only on success), checkers stat their own fd and confirm any
divergence by truncating via a fresh fd and re-reading β a true duplicate
never converges, a transient does.
Outcome
NOT REPRODUCED in bounded runs: 2 x ~2-4M synchronized deget pairs on 6 vCPUs, 0 confirmed persistent duplicates on both the stock and the patched kernel. An earlier detector version reported thousands of "hits"; hardening the detector (publish-only-on-success + convergence confirmation) showed every divergence was transient and self-correcting β the initial "reproduction" was a measurement artifact and is documented here as a lesson. The code-level window analysis stands (the refcnt store is genuinely outside the token-protected insertion); the window appears too narrow to win even under saturated parallel lookup load. Reported as speculative; the one-line ordering fix is included in DF-3157's fix.diff and is pure hardening.
DF-3158 VERDICT β not reproduced (window too narrow; earlier positive was a detector artifact)
What was tried
Two detector generations, both on the 6-vCPU INVARIANTS guest, file on a live msdosfs mount:
- Naive (ftruncate return ignored, no persistence check): reported 2180 and 1795 "hits" on stock and patched kernels respectively.
- Hardened (truncator publishes the round only after ftruncate() == 0;
every candidate divergence is confirmed by truncating through a second
fd and re-statting the first β a duplicate denode never converges):
0 confirmed duplicates in 1.6M+ rounds on the stock kernel; the
naive detector's "hits" all classify as
[transient].
Per the honest-status table this is
Expected marker absent, run exits 0, guest stays up -> not_reproduced.
Why it is still reported
The race exists as a matter of code: the ldep->de_refcnt = 1 store sits
at msdosfs_denode.c:323, after msdosfs_hashins() (:308) has already
published the entry and released dehash_token; both hash users skip
de_refcnt <= 0 entries (hashget :131, hashins :178). Nothing in the
LWKT token protocol orders a remote reader against that post-insertion
store. The window is only a handful of instructions, and empirically it
did not fire in ~4M synchronized deget pairs on this guest β but a
preemption or interrupt landing in the window keeps it theoretically
winnable, and the failure mode (duplicate denodes for one dirent) is
severe (divergent size/cluster caches, stale-slot dirent clobber,
cross-linked allocations). FreeBSD's msdosfs deget sets the reference
state before making the denode reachable for the same class of reason.
Fix (included and validated in DF-3157's fix.diff)
Move ldep->de_refcnt = 1; before msdosfs_hashins(ldep) so the denode
becomes visible with its reference already held. Build+boot validated
(kernel #1/#3); no regressions across the DF-3157 matrix and the hammer.
Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.
(No fix.diff for this item: documented recommendation only - see Fix above.)
Fix verification
not_testableConfirmed kernel references
Detail
Evidence (decisive lines)
['findings/poc/DF-3158/run.log -- naive vs hardened detector runs; hardened shows only [transient] divergences, 0 confirmed', 'sys/vfs/msdosfs/msdosfs_denode.c:308-323 -- the insertion/refcnt window']
PoC changes
Detector hardened after the initial version produced false positives: truncator publishes rounds only on ftruncate()==0 and every candidate divergence is confirmed persistent by truncating via a second fd and re-statting (duplicates never converge).
Verified recommended fix
Set ldep->de_refcnt = 1 before msdosfs_hashins(ldep) in deget() (included in DF-3157 fix.diff; validated)
Verdict
deget() sets de_refcnt=1 at msdosfs_denode.c:323, after msdosfs_hashins() (:308) has published the denode and released dehash_token; hashget (:131) and hashins (:178) both treat refcnt<=0 entries as absent, so a concurrent deget on another CPU could insert a duplicate denode for the same dirent. Empirically NOT reproduced: with an honest detector (ftruncate success required + persistent-divergence confirmation) 0 confirmed duplicates in ~2-4M synchronized deget pairs on 6 vCPUs, stock and patched kernels; an earlier naive detector produced thousands of false hits (transient divergences) which were root-caused to the detector, not the kernel. Reported as speculative hardening; the one-line ordering fix was built and boots clean.
No comments yet.