DF-2615 — kdmsg_iocom_uninit leaks the queued LNK_PING

What was verified

Baseline (instrumented kernel, fix NOT applied): exactly one leaked kdmsg_msg per mount/umount cycle, 20/20 cycles. Instrumentation (instrument.diff, kprintf counters inside kdmsg_iocom_uninit on an otherwise-stock kernel) shows per cycle:

DF2615_DBG: uninit entered rd_td=0 wr_td=0 flags=00000003
DF2615_DBG: after ping write, msgq.first=0xfffff801168ef4c0
DF2615_DBG: uninit exit msgq_nleak=1
DF2615_LEAK: 1 kdmsg_msg(s) queued at iocom_uninit exit

i.e. no iocom threads exist (rd_td=wr_td=0), EXITNOACC (0x8000) is clear in flags, the LNK_PING allocated at kern_dmsg.c:280-281 is TAILQ_INSERTed onto iocom->msgq by kdmsg_msg_write_locked (kern_dmsg.c:2018-2021), and nothing drains it before hmp (with the embedded iocom) is freed — kdmsg_iocom_uninit (kern_dmsg.c:264-317) only joins threads, drops freerd/freewr_state caches and msg_fp.

Precondition refined (important correction to the filed text): the leak requires the mount to have been made with cluster_fd < 0 (no iocom threads ever created). Stock mount_hammer2(8) normally auto-starts the hammer2 service daemon (system("/sbin/hammer2 -q service"), mount_hammer2.c:225) and connects to it (cluster_fd >= 0) — in that configuration the write thread exists, sets EXITNOACC on exit, and the PING self-drains at write time (verified live: 0 leak lines via plain mount -t hammer2). The leak path is taken whenever the mount proceeds without a cluster connection: daemon not running / connect failed (mount_hammer2 passes cluster_fd=-1 and still mounts, mount_hammer2.c:162-167), or any direct mount(2) caller — reproduced with df2615_trigger.c (mount(2) with cluster_fd=-1, 20 cycles). The finding's mechanism is fully confirmed; its "default local case" wording overstates how often stock mounts hit it.

Why the leak is invisible to vmstat on a stock kernel: the msg is allocated from the per-device malloc type hmp->mmsg (kmalloc_create(&hmp->mmsg,"HAMMER2-msg"), hammer2_vfsops.c:1176) which is kmalloc_destroy'ed at unmount (vfsops:1893) — the type vanishes from vmstat -m while the allocation lives. (Separately observed, out of scope: the global M_HAMMER2 "HAMMER2-mount" zone grows ~3.5KB/cycle even on daemon-connected mounts — a different, unfixed accounting leak.)

Fix validated: fix.diff sets EXITNOACC and calls kdmsg_drain_msgq(iocom) in uninit once the threads are gone (mirroring the write-thread exit path, kern_dmsg.c:545-554; uninit holds msglk, satisfying the drain's locking requirement). On the rebuilt kernel: same trigger, 20/20 cycles succeed, after ping write msgq.first=<ptr> (unchanged path) but uninit exit msgq_nleak=0 and 0 DF2615_LEAK lines. Mount/umount behavior unchanged.

Files

• df2615_trigger.c trigger: 20x mount(2){cluster_fd=-1}/unmount(2)
• run_df2615.sh stock-kernel vmstat variant (kept for reference)
• instrument.diff verification counters (NOT the fix)
• fix.diff the fix (against pristine sys/)
• build.log instrumented kernel build (nativekernel)
• build.attempt1_buildkernel_fail.log why nativekernel is used
• build_baseline_kernel4.log, fix_build.log baseline/fix kernels
• run_base.* decisive baseline run (20 leak lines, full dmesg)
• run_fix.* decisive fixed run (0 leak lines, full dmesg)

DF-2615 VERDICT

Reproduced: YES — one kdmsg_msg (LNK_PING) leaked per hammer2

mount/umount cycle when the mount had no cluster connection (cluster_fd < 0). Proven on an instrumented kernel (leak counter inside kdmsg_iocom_uninit); fix validated on a rebuilt kernel.

Chain (all path:line verified in source and at runtime)

• Every device mount runs hammer2_iocom_init (hammer2_vfsops.c:1341 -> hammer2_iocom.c:59-62); every unmount of the last PFS on the device runs hammer2_iocom_uninit (hammer2_vfsops.c:1778 -> hammer2_iocom.c:66-71) -> kdmsg_iocom_uninit (kern_dmsg.c:264-317).
• uninit allocates DMSG_LNK_PING on state0 and queues it (kern_dmsg.c:280-281 -> kdmsg_msg_write_locked :2018-2021) because EXITNOACC is clear whenever the iocom write thread never ran.
• The thread-join loop (:284-294) is a no-op when msgrd_td/msgwr_td are NULL; the function then never drains msgq — only freerd/freewr_state (:299-307) and msg_fp (:312-315) are released.
• The leaked msg (kmalloc from the per-device "HAMMER2-msg" type, vfsops:1176) stays linked via qentry into the embedded iocom that is kfree'd together with hmp — no later dereference: pure leak, no attacker-controlled data. Runtime proof: after ping write, msgq.first=0xfffff801168ef4c0 + uninit exit msgq_nleak=1 on every cycle, 20/20 (run_base.dmesg_after).

Precondition correction (honest scope narrowing)

The filed text calls the never-connected case "the default local case". Verified live: stock mount_hammer2(8) auto-starts the hammer2 service daemon and supplies cluster_fd >= 0 (mount_hammer2.c:225-247); with threads present the PING is drained (EXITNOACC set by the write-thread exit path, kern_dmsg.c:545) and plain mount -t hammer2 cycles leak 0 msgs. The leak manifests when the daemon is unavailable (connect fails -> mount proceeds with -1, mount_hammer2.c:162-167) or for any direct mount(2) caller passing cluster_fd=-1 — reproduced with df2615_trigger. Impact ceiling unchanged (slow root-driven kernel-heap growth; Info).

Also explained: why vmstat -m cannot see it on a stock kernel — the msg's malloc type is destroyed at unmount (vfsops:1893) with the allocation live.

Fix validation

fix.diff: set EXITNOACC + kdmsg_drain_msgq(iocom) in uninit after the threads are gone (mirrors :545-554; msglk held). Rebuilt kernel, same 20-cycle trigger: 0 DF2615_LEAK lines, uninit exit msgq_nleak=0 x20, msgq.first still non-NULL after the ping write (same path taken), CYCLES_DONE=20, TRIG_RC=0 — leak eliminated, behavior unchanged. DF-2614's trigger (cluster-connected mount/unmount) also runs clean on the fixed kernel.

Bottom line

status=reproduced (memory leak; Info-class impact, root-gated cycling), fix validated (fixed).