# DF-3531 — VERDICT **Finding**: netgraph7 ng_tty — closing the tty while the `chand` drain callout is armed permanently leaks the timer `ng_item` and its node reference (the ng_tty twin of DF-3515/DF-3523's ng_uncallout defect). **Status: REPRODUCED** (leak, certain) on the stock INVARIANTS guest. **Fix: leak measurably eliminated by layers 1+2, but stress-testing the patched state surfaced a residual fire/close race that can free an in-flight timer item → heap double-free (delayed `chunk_mark_free` panic). fix_status = inconclusive — see "Fix validation" below.** ## Root cause (line-accurate) - `ngt_start()` arms the pty-drain timer: `sys/netgraph7/tty/ng_tty.c:453-454` `ng_callout(&sc->chand, sc->node, NULL, 1, ngt_timeout, NULL, 0)` - `ng_callout()` (`sys/netgraph7/netgraph/ng_base.c:3248-3268`) allocates a `struct ng_item` carrying one `NG_NODE_REF` (ng_base.c:3257) and stores it as the callout argument. - `ngt_close()` cancels it: `sys/netgraph7/tty/ng_tty.c:252-253` `if (callout_pending(&sc->chand)) ng_uncallout(&sc->chand, sc->node);` - `ng_uncallout()` (`ng_base.c:3273-3296`) frees the item only under `if ((rval > 0) && callout_func(c) == trampoline && NGI_NODE(item) == node)`. On the stock kernel `callout_stop()` of an ARMED not-yet-fired callout completes inline and returns **0** (DF-3515: `_callout_cancel_or_stop` early `return 0` before the PREVENTED-reporting path, sys/kern/kern_timeout.c) → the gate never passes → the never-sent item (refs==0) is orphaned forever: **one ng_item + one node ref leaked per close**. Even if the gate passed, `NG_FREE_ITEM()` is flag-only in ng7 (`sys/netgraph7/netgraph.h:787-793`) — `ng_free_item()` is never invoked. ## Reproduction (baseline: stock kernel #0 + stock modules) Harness: `ngtty_leak.c` (root; ngt_open requires SYSCAP_NONET_NETGRAPH, ng_tty.c:188). Per cycle: /dev/ptmx pty → TIOCSETD=NETGRAPHDISC(6) → NGIOCGINFO → ng7 NGM_CONNECT from a data-socket node to "ttyN:hook" → sendto 1500 B (master never read → t_outq pinned at NGT_HIWATER=400, remainder sits in sc->outq, chand re-arms every tick) → TIOCSETD back to termios (the tty.c:1137 l_close path calls ngt_close synchronously; same ngt_close code as fd-close) → sockets closed. vmstat -m | grep netgraph: baseline 12 allocs 1.69K 2000 stall-close cycles 1.96K allocs 283K <-- ~1/cycle leaked 2000 drain-close (control) 2.32K allocs 334K (mostly flat; the residual = cycles whose drain outlasted the sleep windows and hit the same leak) 2000 more stall-close cycles 3.98K allocs 573K (monotone) kldunload ng_tty "Device busy" (refs=0 — pinned by leaked refs) Corroboration: the very first 20-cycle smoke run (stock modules, before the known DF-3514 module-build rehash fix was pre-applied) wedged the guest in `panic: hashdestroy: hash not empty` — the leaked node refs had already pushed ng_nodes past the rehash threshold within ~20 cycles (DF-3514 feed). Note on measurement: the node-count part of the baseline delta is *contaminated* by a second, independent ng_tty leak discovered during this verification (DF-3534: ngt_shutdown never drops the constructor node ref — every ng_tty node leaks regardless of the callout). The *item* leak (+1.95K/2000, M_NETGRAPH `netgraph` type) is DF-3531's alone: with DF-3534's node fix but stock ng_uncallout behavior … items still climb; with the ng_uncallout fix items go flat while nodes still climb (fix_run.3.log) — the two leaks are cleanly separable and each fix is independently effective for its own allocation class. ## Fix validation Layers applied (fix.diff): 1. `sys/kern/kern_timeout.c` — `_callout_cancel_or_stop` reports PREVENTED for inline-completed stops (the DF-3515 contract fix, same diff DF-3523 validated). 2. `sys/netgraph7/netgraph/ng_base.c` — `ng_uncallout()` calls `ng_free_item(item)` after the flag-only `NG_FREE_ITEM` (layer 2). Result (kernel #1 + rebuilt modules, `fix_run.final2.log`): baseline: netgraph(items)=2 in-use, netgraph_node=2 nodd 500 (ldisc on/off only): nodes 2 -> 2 (flat; DF-3534 effect) stall-close 2000: items 9 in-use (2.45K allocs cycled), nodes 2 (flat) stall-close 2000 (2nd): items 9, nodes 2 (FLAT over 4000) hooks/paths/socks in-use 0 throughout; kldunload ng_tty succeeded once the ref pinning was gone (final battery, earlier run: KLDUNLOAD_OK). **Residual hazard (why fix_status=inconclusive)**: under aggressive close-vs-fire racing (drain-mode stress, and earlier an instrumented build) the patched state double-freed an in-flight timer item: * instrumented module: `Fatal trap 12` in `ng_snd_item` (`lock addl $0x1,0x80(%rdi)`, el_node==NULL) — an item already dispatched to the cpu0 netgraph port had been freed by `ng_uncallout`; * clean modules: delayed `panic: memory chunk 0x... is already free!` (`chunk_mark_free` via `slab_cleanup`/`slotimer_callback`) after the flat 4000-cycle battery, during drain-mode stress. Mechanism: when `ngt_close`'s `ng_uncallout` races the 1-tick fire — the trampoline has dequeued the item and handed it to `lwkt_sendmsg` (cpu0) — the layer-1 patched `callout_stop` can report PREVENTED (rval>0) for a callout whose callback already dispatched, so `ng_uncallout` frees an item that is queued/in-flight (refs==1 held by ng_snd_item). Stock code cannot do this (it never frees). CONSEQUENCE FOR UPSTREAM: the DF-3515/DF-3523 kern_timeout fix layer must guarantee `PREVENTED` is reported *only* when the callback never dispatched; alternatively ng_uncallout must distinguish "stopped before dispatch" from "already dispatched" (e.g. trampoline-side arg clearance). This also means the `ng_callout` oitem-free hunk shipped in DF-3514/DF-3523's fix.diff is dangerous for self-rearming 1-tick timers (ng_tty re-arms from inside ngt_timeout): it frees the in-flight fired item on every re-arm — do NOT include that hunk for ng_tty-class users (it was excluded from this pack's fix.diff after it reproduced the same double-free signature immediately). ## What was NOT the leak (negative results) - v1 `sys/netgraph/tty/ng_tty.c` uses plain `callout_reset`/`callout_stop` with no ng_item — no ng_uncallout twin exists in v1. - The generic ng7 socket/connect/data path leaks nothing (socksock control flat) — the tty ldisc is the sole leaker of its own items/nodes. ## Environment Baseline kernel: DragonFly 6.5-DEVELOPMENT #0 (stock INVARIANTS X86_64_GENERIC, Thu Jul 2 06:02:54 2026). Fix kernel: #1 Mon Sep 14 09:24:11 2026 (kern_timeout layer-1). Modules built in-guest from /usr/src/sys/netgraph7 with gcc80. ## Fix status (batch validation) Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest `make -j6 nativekernel KERNCONF=X86_64_GENERIC` completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.