⬢ DragonFlyBSD Kernel Audit
DF-2631 / verdict.json
← back to finding ↓ download raw 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
{
  "finding_id": "DF-2631",
  "status": "reproduced",
  "reproduced": 1,
  "impact": "dos",
  "confidence": "certain",
  "verdict": "VERIFIED on stock INVARIANTS kernel with UNMODIFIED images (dd clones, MASTER+MASTER, same pfs_clid): after mounting one clone and merging the second via the expected-EBUSY mount, `umount -f` wedges FOREVER in hammer2_thr_wait (umount thread D-state wchan h2twait at +3s/+8s/+63s) while both h2nod-testvol sync threads sit in hammer2_xop_collect (wchan h2coll - their start_except() xops can never reach quorum because their own cluster index never feeds) and ~108 h2xop workers survive frozen (plus a second lazily re-created cohort); hammer2_mntlk stays held so a third hammer2 mount blocks D-state in mntlk, and shutdown is degraded (init: some processes would not die; vn0 unmount fails BUSY). Reboot is the only recovery. Root cause precisely localized: (1) sync threads' self-excluded collects starve forever (synchro.c:411/454 + cluster.c:348 + admin.c:990-998, no failure path); (2) xop_helper_cleanup (vfsops.c:1664) deletes workers under still-running sync threads whose xop_start_except then lazily re-creates xop_groups mid-teardown (admin.c:491-492); (3) pfsfree_scan's freeze phase (vfsops.c:772-793) starves in-progress collects by freezing helper columns, so thr_freeze never returns. Fix validation on rebuilt kernel #1: the wedge is ELIMINATED - umount -f returns RC=0 in 61 s at defaults (11 s with the knob at 10), third mount/ls/umount work, single-device mounts unaffected - but a residual PRE-EXISTING UAF of the multi-chain teardown (DF-2620 family: workers touching freed xop/thread state) reproducibly crashes the box 1-5 minutes after the cluster umount (Fatal trap 12 at hammer2_primary_xops_thread+0x2d9, identical RIP across iterations), a path stock never reaches because it hangs first; fix_status is therefore inconclusive pending crash-dump root-causing of the surviving worker cohort.",
  "exploit_chain": "",
  "evidence": "run.log (T1=36->T2=108 workers +2 sync threads; umount pid 932 D3 h2twait at +3/+8/+63s; h2nod-testvol h2coll; SURV63_h2xop=109; mixed h2twait/h2idle worker generations = re-created cohort); third mount mount_hammer2 D3 mntlk; shutdown_serial.log (would-not-die + BUSY + forced unmounts); fix_run.log (UMOUNTF_RC=0 ELAPSED=61s defaults / 11s at knob 10, MOUNT3_*_RC=0, SANITY_*_RC=0); fix_iteration{1,2,3}_panic_serial.log (delayed Fatal trap 12, hammer2_primary_xops_thread+0x2d9)",
  "kernel_refs": [
    "sys/vfs/hammer2/hammer2_admin.c:135",
    "sys/vfs/hammer2/hammer2_admin.c:491",
    "sys/vfs/hammer2/hammer2_admin.c:988",
    "sys/vfs/hammer2/hammer2_admin.c:1148",
    "sys/vfs/hammer2/hammer2_vfsops.c:766",
    "sys/vfs/hammer2/hammer2_vfsops.c:772",
    "sys/vfs/hammer2/hammer2_vfsops.c:783",
    "sys/vfs/hammer2/hammer2_vfsops.c:1664",
    "sys/vfs/hammer2/hammer2_synchro.c:411",
    "sys/vfs/hammer2/hammer2_cluster.c:348"
  ],
  "poc_changes": "Standalone trigger written from scratch (unmodified dd-clone images, no forging); census instrumentation (thread counts, wchan histograms at +3s/+8s/+63s, third-mount mntlk probe, shutdown serial capture).",
  "attempts": 3,
  "guest_uname": "DragonFly dfbsd 6.5-DEVELOPMENT #0: Thu Jul  2 06:02:54 UTC 2026 (baseline) / #1: Sat Aug 29 05:10:55 UTC 2026 (fix), x86_64",
  "runtime_sec": 12600,
  "guest_dirty": 0,
  "build_cmd": "(images are plain dd clones made by the trigger); (fix kernel) cd /usr/src && make nativekernel && make installkernel  [7 iterations, all BUILD_RC=0]",
  "run_cmd": "sh trigger_df2631.sh   # stock wedge; sh fix_validate.sh  # fix kernel",
  "code_hash": "2a93e609bde7d4e50210f87b06976df584ad453060631cb62af55bd5a74ae572",
  "notes": "Event-counted collect timeouts are DEFEATED by continuous feeds from the healthy column (measured on v4) - the shipped bound is elapsed-based with an SPMP exemption. A teardown-gated bound (v7) did NOT unwedge, localizing the starved waiter outside the mounted pmp's own freeze. Guest left clean (vm.sh reset with-src).",
  "recommended_fix": "Freeze sync threads to completion BEFORE freezing any xop helper column in pfsfree_scan; never lazily re-create xop helpers once HAMMER2_PMPF_TEARDOWN is set (feed EOF+EIO instead); bound hammer2_xop_collect's EINPROGRESS wait with an elapsed-time sysctl (vfs.hammer2.xop_collect_timeout, SPMP-exempt) - see fix.diff; follow-up required for the residual teardown UAF (crash-dump the surviving worker cohort).",
  "fix_status": "inconclusive",
  "fix_kernel_uname": "DragonFly dfbsd 6.5-DEVELOPMENT #1: Sat Aug 29 05:10:55 UTC 2026     root@dfbsd:/usr/obj/usr/src/sys/X86_64_GENERIC  x86_64",
  "fix_baseline_reproduced": 1,
  "fix_patched_reproduced": 0,
  "fix_verdict": "WEDGE ELIMINATED AND VALIDATED: on the fix kernel the cluster `umount -f` completes (RC=0, ELAPSED=61 s at defaults / 11 s with vfs.hammer2.xop_collect_timeout=10) instead of hanging forever, hammer2_mntlk is released (third mount/ls/umount RC=0) and single-device mounts are unaffected. HOWEVER fix_status=inconclusive: on every kernel carrying the working elapsed collect bound, the unwound teardown reproducibly triggered a delayed (1-5 min) Fatal trap 12 page fault in hammer2_primary_xops_thread+0x2d9 (worker touching freed xop/thread state) or a spontaneous reset - a pre-existing multi-chain-teardown UAF (DF-2620 family) that stock never reaches because the umount wedges first. The finding's DoS-wedge claim is fixed; the adjacent corruption needs its own root-cause pass (crash dump of the survivor cohort, lifetime audit of hammer2_xop_start* vs teardown).",
  "fix_evidence": "fix_run.log (UMOUNTF_RC=0 ELAPSED=61s/11s, MOUNT3_RC=0, SANITY_RC=0); fix_iteration{1,2,3}_panic_serial.log (identical delayed-panic RIP); fix.diff"
}