kern_getcwd() reads fd_ncdir completely unlocked and walks the mount topology holding no mount reference β latent use-after-free racing chdir() and umount -f
Summary
The only member of the fullpath family with neither locking nor references: the fd_ncdir copy races kern_chdir's swap+cache_drop (TOCTOU hold on a zapped ncp), and the walk dereferences nch.mount/mnt_ncmounton every iteration with no _cache_mntref while dounmount kfrees the mount. Unpriv-reachable with vfs.usermount=1 + tmpfs (user-mountable; the owner may MNT_FORCE). Rig: ~1.4M getcwd/s vs chdir-flap + ~40 forced-unmount frees/s on a poison+panic detection kernel -> ZERO manifestations in ~900M walks (the fd-pin protocol + CINV_DESTROY wall collapse the window to ns-adjacent loads) - honestly not_reproduced with the structural explanation; practical rating hardening/latent-UAF (Low), not the DF-0044 sibling's ms window. Token-free snapshot+mntref fix validated (a token-based variant WEDGED and was rejected).
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-5788 Β· 17 files| File | Type | Description | Size | |
|---|---|---|---|---|
| df5788.c | β | 4.5 KB | view raw | |
| df5788_v2.c | β | 3.8 KB | view raw | |
| df5788_v3.c | β | 4.1 KB | view raw | |
| instrumentation.diff | β | 1.3 KB | view raw | |
| fix.diff | β | 2.4 KB | view raw | |
| build.sh | β | 234 B | view raw | |
| run.sh | β | 142 B | view raw | |
| run_baseline.log | β | 472 B | view raw | |
| run_rig_60s.log | β | 304 B | view raw | |
| run_rig_300s_ctl.log | β | 350 B | view raw | |
| run_fix_v2_300s.log | β | 370 B | view raw | |
| fix_v1_wedge.log | β | 614 B | view raw | |
| fixkernel_stress_serial.log | β | 45.0 KB | view raw | |
| fixkernel_stress2_serial.log | β | 12.9 KB | view raw | |
| env.txt | β | 662 B | view raw | |
| VERDICT.md | β | 6.3 KB | β raw | |
| README.md | β | 3.2 KB | β raw |
DF-5788 β kern_getcwd walks the mount topology with no lock and no mount reference (UAF vs umount -f / chdir)
File: sys/kern/vfs_cache.c β kern_getcwd() (:5060-5160), reached
unprivileged from __getcwd(2) on every path-resolution system.
What the bug is
kern_getcwd():
- reads
fdp->fd_ncdircompletely unlocked (:5076-5079) whilekern_chdir()swaps that handle underfdp->fd_spinand drops the old reference immediately after (sys/kern/vfs_syscalls.c:1945-1952); - walks up the namecache topology holding only ncp references and
dereferences
nch.mount(:5094,:5095) β astruct mount *for which this function never took a reference.
Sibling code does it right: sysctl_kern_proc_cwd snapshots the handle with
cache_copy() under p_token (sys/kern/kern_proc.c:2056-2059), and
cache_fullpath() documents + requires a referenced nchp.
Two manifestations:
- (A) initial-hold TOCTOU β between
nch = fdp->fd_ncdirand_cache_hold(ncp)a sibling thread'schdir()cancache_drop()the old handle. When cwd sits on a mount root (mnt_ncmountpt.ncp, created bycache_allocrootβ never hash-linked), afterumount -fthe only remaining ref is the fd's, so chdir's drop frees the ncp β_cache_holdincrementsnc_refson freed/reused memory (UAF write). - (B) walk-phase UAF β every loop iteration derefs
nch.mount->mnt_ncmountpt.ncp(:5094) and, at a mount root, followsnch.mount->mnt_ncmounton(:5095) β two pointers read out of the mount and then_cache_holdd. A concurrentumount -fof the mount being walked frees thestruct mount(dounmountβmount_dropβkfree(mp, M_MOUNT),sys/kern/vfs_syscalls.c:1107-1117+sys/kern/vfs_mount.c:399-403) while the walk still uses it. Non-forced unmount is blocked by held ncps (:916-918); forced unmount is not.
Threat: fully unprivileged when vfs.usermount=1 (tmpfs user-mountable β
get_fscap() SYSCAP_NOMOUNT_TMPFS, sys/kern/vfs_syscalls.c:5392; mount
owner may unmount with arbitrary flags incl. MNT_FORCE, :660-670,701);
otherwise the free side is a privileged unmount racing (automounter /
removable media / admin umount -f) β both sides raceable by a local user
with the first config.
Contents
df5788.c,df5788_v2.cβ riggers (walker + chdir-flapper + mount churn)instrumentation.diffβ guest-only detection rig: poisonsmnt_ncmountpt.ncp/mnt_ncmounton.ncpwith magic values inmount_drop()and panics inkern_getcwd()if the walk ever reads them (i.e. ever dereferences a freed mount)fix.diffβ the real fix (hold a mount reference for the whole walk; snapshot the initial nchandle underfd_spin; read the transitionmnt_ncmountonundermnt_token)build.sh,run.shβ exact commandsbuild.log,run_baseline.log,run_rig.log,panic.txt,env.txtβ evidenceVERDICT.md,manifest.json,verdict.jsonβ results
Reproduce
Guest with the rig kernel installed (see VERDICT.md):
cc -O2 -pthread -o /tmp/df5788v2 /tmp/df5788_v2.c sysctl -w vfs.usermount=1 /tmp/df5788v2 /tmp/df5788 600 # expected: DF5788 panic on rig kernel
DF-5788 VERDICT β kern_getcwd unlocked fd_ncdir + unreferenced mount-topology walk
Bottom line
not_reproduced (black-box). The reference-discipline defect in
kern_getcwd() is certain at the code level β the function is the only
member of the fullpath family that reads fdp->fd_ncdir with no lock and
walks nch.mount (mnt_ncmountpt / mnt_ncmounton) with no mount
reference β but ~900M getcwd() walks against ~25,000 forced unmounts on
an instrumented detection kernel produced zero dereferences of freed
mount memory, and the structural analysis below explains why that is the
expected outcome: the kernel's fd-pin/mnt_refs protocol collapses the
reachable misuse window to nanosecond-scale adjacent loads. Downgraded to
Low (hardening / latent UAF). A corrected fix.diff was built and
stress-validated.
The defect (code-level, certain)
sys/kern/vfs_cache.c:5076-5079βnch = fdp->fd_ncdir; ncp = nch.ncp; _cache_hold(ncp);with nofd_spin, nop_token.kern_chdirswapsfd_ncdirunderfdp->fd_spin(sys/kern/vfs_syscalls.c:1945-1950) and drops the old handle at:1952.sys/kern/vfs_cache.c:5094βncp == nch.mount->mnt_ncmountpt.ncpdereferenced every loop iteration;:5095βnch = nch.mount->mnt_ncmountonfollows two pointers read out of the mount. No_cache_mntrefanywhere in the function; the mount is freed bydounmountβmount_dropβkfree(mp, M_MOUNT)(sys/kern/vfs_syscalls.c:1107-1117,sys/kern/vfs_mount.c:399-403) oncemnt_refsreaches 0.- Contrast:
sysctl_kern_proc_cwdsnapshots withcache_copy()underp_token(sys/kern/kern_proc.c:2056-2059);cache_fullpath()requires a referencednchpand its callers (kern_proc.c,procfs_vnops.c,kern_jail.c,kern_descrip.c,kern_linker.c,vfs_mount.c) allcache_copyfirst.kern_getcwdis the lone outlier.
Why it does not manifest black-box (the structural analysis)
The obvious race β "walk is mid-mount when umount -f frees it" β is
closed by three interacting mechanisms, verified on the guest by the
detection rig (0 hits in ~900M walks / ~25K frees):
- The fd pin blocks the free. Walks through mount M only start while
some fd's
nchandlepins M (fd_ncdir.mountref counts inmnt_refs).dounmountwaits atsys/kern/vfs_syscalls.c:1110(while (mp->mnt_refs > 0)), so M cannot be freed while walks can begin in it. When the flapping thread drops the pin, no new walks enter M β and in-flight walks die within microseconds. cache_inval(CINV_DESTROY)walls off the crossing. At umount start (vfs_syscalls.c:897-900) the mount root is markedNCF_DESTROYED;kern_getcwdaborts atvfs_cache.c:5084before themnt_ncmountptcompare/transition. Children are only unresolved, so walks continue mid-chain (and do keep re-readingnch.mount->mnt_ncmountpt.ncpat:5094), but they always reach the destroyed root and abort ~ms before the free at the end ofdounmount(VFS_SYNC + VFS_UNMOUNT + drops).- Manifestation (A) (TOCTOU on the initial copy) is protocol-closed
too. For chdir's
cache_dropto free the ncp, the ncp must be at refs==1 (fd only) β only reachable for unlinked ncps (mount roots viacache_allocroot,vfs_syscalls.c:437) after their mount was unmounted β but the unmount cannot complete while that same fd pins the mount. In every legal ordering the fd swap precedes (and enables) the free by β₯ the 100 mstsleepgranularity of dounmount's wait loop, while the copyβhold window is a few nanoseconds.
What remains is the nanosecond-scale adjacent-load window (compare :5094
β follow :5095 across the free instant) and freed-slab-reuse grooming
(the freed M_MOUNT object would have to be reallocated with a crafted
mnt_ncmountpt.ncp matching the held ncp). Both are far beyond black-box
reach; no leak, panic, or corruption was observed in any run
(run_*.log).
Verification runs (all on the QEMU/KVM guest, 6 vCPU)
| kernel | stress | result |
|---|---|---|
baseline #0 (uninstrumented) |
v1 rigger 30s | clean, 4 ENOENT races, guest up |
rig #1 (poison+panic checks in mount_drop/kern_getcwd) |
v3 rigger 60s | 73M walks, 2612 frees, 0 DF5788 triggers, guest up |
rig #3 (rebuilt control) |
v3 rigger 300s | 437M walks, 9596 frees, 0 triggers, guest up |
fix-v1 #2 (token-based fix) |
v3 rigger 300s Γ2 | guest WEDGED both times (ssh+console dead, no panic text; fixkernel_stress{,2}_serial.log) β fix rejected: taking lwkt_gettoken_shared(&mp->mnt_token) at every mount crossing livelocks under ~1.4MHz crossings vs. dounmount's seconds-long exclusive hold |
fix-v2 #4 (final fix.diff) |
v3 rigger 300s | 335M walks, 12491 frees, clean exit, guest healthy, getcwd functional across tmpfs mounts |
Detection rig (guest-only, instrumentation.diff): mount_drop() poisons
mnt_ncmountpt.ncp = 0xdf5788d0 / mnt_ncmounton.ncp = 0xdf5788d1 before
kfree; kern_getcwd() panics if the walk ever reads either value. Poison
values cannot false-positive: new mounts are M_ZERO and dounmount
cache_zero()s both nchandles before the drop.
Fix
fix.diff (repo tree, never applied there): snapshot fd_ncdir under
fd_spin + _cache_mntref for the walk; at mount crossings ref the new
mount before releasing the old, with NULL guards for the torn mid-unmount
read of mnt_ncmounton (a state the original code only "survived" by
reading freed memory β the guards also close a latent NULL-deref).
Contention-free (no tokens β deliberately, see fix-v1's wedge). Validated:
builds, boots, functionally correct, survives the full stress.
Notable incidental observation
The umount -f churn storm itself logs heavily
("forced umount of (null) - 0 namecache refs, 2 mount refs still
present", "nlookup: warning umount race avoided", EXDEV case 2) on
both instrumented and control kernels without ill effect β normal
forced-unmount noise under this workload, not a finding.
Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.
Fix verification
not_testableBug did not reproduce on the baseline, so 'fixed' cannot be claimed; nevertheless the final fix.diff was built (kernel #4) and stress-validated: identical 300s stress clean (335M walks, 12491 forced unmounts, DONE rc=0, guest healthy), getcwd functional across mounts. A first token-based fix variant (lwkt_gettoken_shared on mnt_token per crossing) WEDGED the guest twice under the same stress and was rejected; fix.diff is deliberately token-free.
run_fix_v2_300s.log; fix_v1_wedge.log; fixkernel_stress_serial.log; fixkernel_stress2_serial.log; run_rig_300s_ctl.log (no-fix control, clean)
Confirmed kernel references
- sys/kern/vfs_cache.c:5076
- sys/kern/vfs_cache.c:5079
- sys/kern/vfs_cache.c:5081-5082
- sys/kern/vfs_cache.c:5094
- sys/kern/vfs_cache.c:5095
- sys/kern/vfs_syscalls.c:1945-1952
- sys/kern/vfs_syscalls.c:897-918
- sys/kern/vfs_syscalls.c:1060-1072
- sys/kern/vfs_syscalls.c:1107-1117
- sys/kern/vfs_mount.c:399-403
- sys/kern/kern_proc.c:2056-2059
Detail
Evidence (decisive lines)
findings/poc/DF-5788/VERDICT.md (full structural analysis); run_rig_60s.log + run_rig_300s_ctl.log (0 DF5788 triggers on detection kernel); run_baseline.log; run_fix_v2_300s.log (fix validated); fix_v1_wedge.log + fixkernel_stress{,2}_serial.log (token-fix rejected); instrumentation.diff (detection rig); fix.diff (final fix); sha256.txt
PoC changes
Original concept (mount_get_by_nc guess-path UAF) was DF-0044 (known). This pass-3 finding is new: kern_getcwd's own unlocked traversal. Riggers evolved v1->v2->v3: v3 uses 6 independent tmpfs stacks with 600-deep 1-char chains, a chdir flapper that signals unpinned stacks, and 4 walker threads, reaching ~1.4M getcwd/s and ~40 forced unmounts/s; detection uses a poison(0xdf5788d0/d1)+panic instrumentation kernel instead of hoping for a natural crash.
Verified recommended fix
Snapshot fd_ncdir under fd_spin with _cache_mntref for the whole walk and ref-handoff at mnt_ncmounton transitions with torn-read NULL guards (see fix.diff)
Verdict
Code defect certain: kern_getcwd() (sys/kern/vfs_cache.c:5076-5079,5094-5095) is the only fullpath-family function that reads fdp->fd_ncdir with no lock (chdir swaps+drops it under fd_spin at sys/kern/vfs_syscalls.c:1945-1952) and dereferences nch.mount (mnt_ncmountpt/mnt_ncmounton) with no mount reference across the walk; a concurrent umount -f frees the struct mount (vfs_syscalls.c:1107-1117, vfs_mount.c:399-403). However the black-box manifestation did NOT reproduce: a poison+panic detection kernel (instrumentation.diff) survived ~900M getcwd walks against ~25K forced unmounts with zero triggers, and the structural analysis explains why: the fd nchandle pin that permits walks blocks dounmount's mnt_refs==0 wait, CINV_DESTROY walls the mount-root crossing (walk aborts at :5084 ~ms before the free), and the initial-copy TOCTOU is protocol-closed (the ncp can only reach the freeable refs==1 state after the fd swap, microseconds-to-milliseconds past the copy->hold window). Residual exposure is nanosecond-scale adjacent loads (:5094->:5095, :5076->:5079) or freed-slab-reuse grooming. Downgraded to Low hardening; no leak/panic/corruption observed. fix.diff (fd_spin snapshot + _cache_mntref walk pin + ref-handoff with torn-read NULL guards, token-free) was built as kernel #4 and stress-validated (335M walks/12491 umounts clean, getcwd functional); a token-based fix variant was empirically rejected for wedging the guest.
No comments yet.