DragonFlyBSD Kernel Audit
DF-2630 / verdict.json
← back to finding ↓ download raw
{
  "finding_id": "DF-2630",
  "status": "reproduced",
  "reproduced": 1,
  "impact": "dos",
  "confidence": "certain",
  "verdict": "VERIFIED on stock INVARIANTS kernel: a single forged byte (PFS inode meta.pfs_nmasters 0x00->0x02, minimal realistic value, CHECK_NONE brefs + volhdr CRC recompute) on a single-device hammer2 image makes every mount-crossing operation wedge forever in hammer2_vfs_root (wchan h2root, 1 Hz PCATCH retry loop with no timeout/failure path at vfsops.c:1966-2011 because nquorum=2>nchains=1 can never be met); the first wedged lookups are signal-killable, but once umount -f takes the mount exclusively busy and itself wedges in teardown (h2twait), every further lookup piles up UNKILLABLE D-state in vfs_busy (observed 40+ minutes, kill -9 immune, pids 877/884), plain umount-by-path wedges in sys_unmount's nlookup, even a bare `mount` listing wedges via mountctl->nlookup, and shutdown is degraded ('init: some processes would not die', vn0 unmount fails BUSY, forced unmounts) - reboot is the only recovery. Causality proven by the one-byte-different control image (nmasters=1): mount/ls/umount all RC=0. Fix validated on rebuilt kernel #1: bounded (sysctl vfs.hammer2.root_timeout, default 60 s, 0=legacy) + sticky HAMMER2_PMPF_ROOTFAILED failure makes ls fail with EIO after the bound, retries fail instantly, zero h2root/vfs_busy waiters remain, umount -f cleanly REMOVES the bogus mount, further mounts work, control and stock images behave identically, and 280 s post-run soaks were clean. Residual documented: plain umount-by-path of the bogus mount fails EIO (path lookup fails once the root is marked failed) - umount -f removes it. vfs.usermount=0 on this guest, so mounts were root-position (same caveat as DF-2620).",
  "exploit_chain": "",
  "evidence": "run.log (steps 1-10 incl. MOUNT_RC=0 same-second, ls child wchan h2root, kill -9 survival, umount/umount -f wedges); dstate_census.txt (7 D-state processes wchan vfs_busy/h2twait 40+ min after kill -9); shutdown_serial.log (init: some processes would not die; unmount of /dev/vn0@testvol failed (BUSY); forced unmounts); fix_run.log (WEDGE_LS1 EIO after exactly 5 s, sticky instant fail, WEDGE_UMOUNTF_RC=0, dmesg 'root quorum cannot be reached (nmasters=2 nchains=1)'); control run CONTROL_*_RC=0",
  "kernel_refs": [
    "sys/vfs/hammer2/hammer2_vfsops.c:527",
    "sys/vfs/hammer2/hammer2_vfsops.c:1966",
    "sys/vfs/hammer2/hammer2_vfsops.c:2007",
    "sys/vfs/hammer2/hammer2_cluster.c:348",
    "sys/kern/vfs_nlookup.c:1053",
    "sys/kern/vfs_cache.c:4504",
    "sys/kern/vfs_syscalls.c:635"
  ],
  "poc_changes": "Clean standalone forger derived from DF-2620's forge (same CHECK_NONE/CRC technique) with a minimal realistic nmasters=2 variant plus a one-byte-different nmasters=1 control; trigger script hardened after the first run (bare `mount` replaced by `mount -p` because prmount->mountctl->nlookup itself wedges on the mountpoint - accidentally proven, mount(8) pid 940 wchan h2root).",
  "attempts": 3,
  "guest_uname": "DragonFly dfbsd 6.5-DEVELOPMENT #0: Thu Jul  2 06:02:54 UTC 2026 (baseline) / #1: Sat Aug 29 05:10:55 UTC 2026 (fix), x86_64",
  "runtime_sec": 9600,
  "guest_dirty": 0,
  "build_cmd": "python3 forge_df2630.py base.img wedge wedge.img; python3 forge_df2630.py base.img control control.img; (fix kernel) cd /usr/src && make nativekernel && make installkernel",
  "run_cmd": "sh trigger_df2630.sh   # stock wedge; sh control_df2630.sh  # causality; sh fix_validate.sh  # fix kernel",
  "code_hash": "8c4c328401248c1cefb70d27914e2f3b787e631fda13effd49c237595e0817d6",
  "notes": "Anomaly recorded honestly: one early `stat /mnt/h2` returned the covered directory's attributes without wedging (hashed root-fs inum 2501377919); all operations that must cross into the mount wedge permanently. Full untrimmed nativekernel build logs were lost to QEMU disk reverts between fix iterations (BUILD_RC=0 recorded per build). Guest left clean (vm.sh reset with-src).",
  "recommended_fix": "Bound hammer2_vfs_root's quorum wait with sysctl vfs.hammer2.root_timeout (default 60 s, 0=legacy) and fail VFS_ROOT with EIO, remembering the failure in HAMMER2_PMPF_ROOTFAILED so subsequent lookups fail immediately; see fix.diff.",
  "fix_status": "fixed",
  "fix_kernel_uname": "DragonFly dfbsd 6.5-DEVELOPMENT #1: Sat Aug 29 05:10:55 UTC 2026     root@dfbsd:/usr/obj/usr/src/sys/X86_64_GENERIC  x86_64",
  "fix_baseline_reproduced": 1,
  "fix_patched_reproduced": 0,
  "fix_verdict": "On the rebuilt kernel the previously-observed bad behavior is GONE: wedge-image ls fails EIO after the (5 s demo) bound instead of wedging forever, the sticky flag makes retries instant, no h2root/vfs_busy/ncplk D-state pile forms, `umount -f` returns 0 and removes the bogus mount, and stock/control images are unaffected (mount/ls/umount all RC=0). 280 s post-run soaks clean, zero leftover threads, no panics. The 2630 hunks are byte-identical across all validated fix builds.",
  "fix_evidence": "fix_run.log (SANITY/CONTROL RC=0, WEDGE_LS1_RC=1 after exactly 5 s with 'Input/output error', WEDGE_LS2_RC=1 same-second sticky, CENSUS_DONE zero waiters, dmesg quorum message, WEDGE_UMOUNTF_RC=0, HEALTH_MOUNT_RC=0); fix.diff"
}