DF-2666 — HAMMER2IOC_RECLUSTER leaks the holdfp() file reference on error returns

Finding

hammer2_ioctl_recluster() (sys/vfs/hammer2/hammer2_ioctl.c:212-236) takes a file reference with holdfp(curthread, recl->fd, -1) and transfers it into the kernel iocom only on the two success branches:

fp = holdfp(curthread, recl->fd, -1);
if (fp) {
        error = VFS_ROOT(ip->pmp->mp, &vproot);
        if (error == 0) {
                cluster = &ip->pmp->iroot->cluster;
                if (cluster->focus != NULL) {
                        hammer2_cluster_reconnect(cluster->focus->hmp, fp);  /* ref transferred */
                        error = 0;
                } else if (cluster->nchains == 1 &&
                           cluster->array[0].chain != NULL) {
                        hammer2_cluster_reconnect(...array[0]..., fp);       /* ref transferred */
                        error = 0;
                } else {
                        kprintf(...); error = EINVAL;                         /* LEAK: no fdrop(fp) */
                }
                vput(vproot);
        }                                                                       /* LEAK: no fdrop(fp) on VFS_ROOT error */
} else {
        error = EINVAL;
}
return error;

hammer2_cluster_reconnect() (hammer2_iocom.c:73-85) and its backend kdmsg_iocom_reconnect() (kern_dmsg.c:128-131) implement a one-way ownership contract — "The caller must ref the fp for us ... We own that ref now" — with no failure path back to the caller. Both error returns in the ioctl therefore leak one struct file reference per call (each pinned ref keeps the file structure and everything it references — vnode/socket and buffers — alive forever).

Privilege: root-only (caps_priv_check(cred, SYSCAP_NOVFS_IOCTL) at hammer2_ioctl.c:83 gates HAMMER2IOC_RECLUSTER at :89-91). Severity: Low.

What the guest run established (stock INVARIANTS kernel #0)

1. Success-path ref accounting is balanced — 200 consecutive HAMMER2IOC_RECLUSTER calls on a healthy mount: 200× OK, 0× EINVAL, kern.openfiles delta 0 (each call transfers the new ref; the next kdmsg_iocom_reconnect drops the previous iocom->msg_fp).
2. The leak branch was not enterable on a healthy guest mount — iroot->cluster.focus is non-NULL for a successfully mounted local PFS, and VFS_ROOT() does not fail there; we found no way to construct the degraded-cluster precondition (focus == NULL && nchains != 1) from userland. The leak itself is certain by code inspection.
3. Bonus pass-2 stress (shutdown/retract ordering) — ~1250 rapid reconnect cycles (this run + the aborted sizing run): each cycle kills and reaps both iocom threads, tears down the auto-LNK_CONN state through the synthesized-failure path (kern_dmsg.c:1391-1407) — which re-enters hammer2_autodmsg's LNK_CONN|CREATE|REPLY block (tcmd is derived from the state's icmd, kern_dmsg.c:1050-1058; observed as per-cycle "VOLDATA DUMP / INITIATE SPANs / CONN WAS TERMINATED" console lines) — and re-creates the threads. No wedge, no panic, unmount clean.

Files

• df2666_trigger.c trigger (mount, 200× RECLUSTER, kern.openfiles deltas)
• run_df2666.sh guest-side orchestration (vn image, newfs, build, run)
• build.sh/run.sh host-side wrappers
• run.log decisive guest run (trigger stdout + console deltas)
• build.log cc output
• env.txt guest uname/env
• fix.diff the fix (fdrop on failure returns)
• VERDICT.md full narrative
• manifest.json, verdict.json

Build & run (from the repo root)

scp -F dfbsd-qemu/config findings/poc/DF-2666/df2666_trigger.c findings/poc/DF-2666/run_df2666.sh dfbsd:/root/poc/df2666/
dfbsd-qemu/vm.sh run_root 'sh /root/poc/df2666/run_df2666.sh'

Stock kernel: RECLUSTER_OK=200 RECLUSTER_EINVAL=0 DELTA=0 (no leak on the healthy path; leak branch not enterable). A kernel with the leak branch reachable would show RECLUSTER_EINVAL>0 and DELTA == RECLUSTER_EINVAL.

DF-2666 VERDICT — HAMMER2IOC_RECLUSTER holdfp() reference leak on error returns

Status: not_reproduced (leak branch not enterable on a healthy guest mount; the missing-fdrop defect itself is certain by code inspection) Impact: none demonstrated (ceiling: one permanently pinned struct file per failing call — privileged caller only) Guest: DragonFly dfbsd 6.5-DEVELOPMENT #0 (stock INVARIANTS kernel, X86_64_GENERIC, Jul 2 2026)

1. THE DEFECT (certain, static)

hammer2_ioctl_recluster() (hammer2_ioctl.c:212) takes a file reference via holdfp() and transfers it to the kernel iocom only on the two success branches (:217-225). hammer2_cluster_reconnect() (hammer2_iocom.c:77-85 -> kern_dmsg.c:132-167) implements a one-way ownership transfer ("We own that ref now", kern_dmsg.c:130) — it has no failure return and never drops the ref itself. Both error paths in the ioctl therefore leak the reference:

• VFS_ROOT() failure — fp never dropped (hammer2_ioctl.c:214,232-233)
• EINVAL branch — fp never dropped (hammer2_ioctl.c:226-231)

Each failing call permanently pins one struct file plus everything it references (vnode/inode or socket + buffers). Privilege-gated: caps_priv_check(SYSCAP_NOVFS_IOCTL) at hammer2_ioctl.c:83,89-91.

2. VERIFICATION RUN (what was tried and observed)

Two runs on the stock guest kernel (attempt 1 sized at 2000 iterations was aborted for runtime — each reconnect costs ~1s in hz/2 teardown sleeps — after ~1050 clean cycles; attempt 2 at 200 iterations completed):

OPENFILES_BEFORE=-4294967199 iters=200 (kern.openfiles is int-wrapped OPENFILES_AFTER=-4294967199 DELTA=0 on this long-lived guest; the RECLUSTER_OK=200 RECLUSTER_EINVAL=0 deltas are still live) RECLUSTER_OTHER=0 LAST_ERRNO=0 BADFD_RC=22 (Invalid argument) [expected EINVAL, no leak] UNMOUNT_OK OPENFILES_POST_UNMOUNT=-4294967201 (FINAL_DELTA=-2)

• 200/200 calls took the SUCCESS path: iroot->cluster.focus != NULL on a healthy local mount, so the ref-transfer path executed every time and the accounting balanced exactly (DELTA=0; next kdmsg_iocom_reconnect drops the previous iocom->msg_fp, :149-152).
• The leak branch requires focus==NULL && !(nchains==1 && array[0].chain) — a degraded iroot cluster — or a VFS_ROOT() failure. Neither could be constructed from userland on a healthy mount: every local PFS mount builds iroot's cluster from ≥1 live chain, and all-slaves-dead mounts fail before the ioctl is reachable. The lockless read of cluster->focus (:217) could in principle observe a transient NULL mid-rebuild, but that race was not observed (200/200 success).
• Control: recl->fd = 999999 -> holdfp() fails, EINVAL, no ref taken (not a leak — confirmed).

Conclusion: the missing fdrop is real and the fix is required, but a user-observable leak could not be demonstrated on a healthy system, hence not_reproduced / impact none / severity Low.

3. BONUS PASS-2 RESULT (shutdown/retract ordering, requested hunt)

The ~1250 reconnect cycles stress-tested exactly the ordering the orchestrator asked about: kdmsg_iocom_reconnect() kill+join of both iocom threads (kern_dmsg.c:139-144), wr-thread teardown drain + simulated failures (:543-570), auto-CONN re-initiation (hammer2_iocom.c:129 -> kern_dmsg.c:176-192), and teardown re-entry into hammer2_autodmsg's LNK_CONN|CREATE|REPLY block — empirically confirmed via the per-cycle "VOLDATA DUMP / INITIATE SPANs / CONN WAS TERMINATED" console lines (tcmd derives from the tracked state's icmd, kern_dmsg.c:1050-1058, not from the synthesized LNK_ERROR base command). No wedge, no panic, clean unmount, kern.openfiles back to baseline. This teardown-context dispatch runs WITHOUT msglk (kdmsg_state_abort releases it around the callback, kern_dmsg.c:1403-1405), so no voldata_lock/msglk lock-order inversion exists on this path either.

4. WHY NO FIX-KERNEL RUN

Fix validation is mandatory for reproduced memory-corruption findings; this finding did not reproduce (branch unenterable), and the patch is a two-line refcount fix with no behavior change on any executable path of this run (200/200 success before and after would be identical). fix_status: not_testable.