β¬’ DragonFlyBSD Kernel Audit
← triage Β· dashboard
DF-3536

ng7 ng_cisco: disconnecting the downstream hook with the armed keepalive ng_callout leaks the timer ng_item + node reference (DF-3531's ng_tty twin)

Summary

cisco_newhook('downstream') arms the 10s keepalive via ng_callout, which allocates a struct ng_item holding one NG_NODE_REF. Every teardown of that hook runs ng_uncallout, which frees the pending item only when callout_stop() returns >0 - but on this kernel an armed, not-yet-fired cancel completes inline and returns 0 (DF-3515 kern_timeout contract violation), so the gate never passes: one ng_item + one node reference is orphaned per armed-cancel. The leaked reference pins the ng_node in the ID hash forever and pins the module; repeating connect/disconnect grows the kernel heap without bound, and the pinned zombie nodes push ng_nodes past the ng7 ID-hash rehash threshold, panicking stock kernels via DF-3514's hashdestroy. VERIFIED on the guest (ng7 modules built from pristine /usr/src): leak runs +300 allocs/300 cycles in BOTH zones (netgraph 302->602, netgraph_node in lockstep, 84.7K/100K), inet-hook control run FLAT, second leak run +300 again (monotone); kldunload ng_cisco refused 'Device busy'; a 3000-cycle run on stock modules panicked mid-run 'panic: hashdestroy: hash not empty' in ng_make_node_common (panic.txt). Framework fix (same defect as DF-3531, not node-local): kern_timeout dispatch-precise PREVENTED layer + ng_uncallout must actually ng_free_item().

Discussion (0)

No comments yet.

PoC verification

Evidence pack

findings/poc/DF-3536 Β· 11 files
FileTypeDescriptionSize
ngcisco7_leak.c β€” 5.1 KB view raw
build.sh β€” 143 B view raw
run.sh β€” 90 B view raw
build.log β€” 3.0 KB view raw
run.log β€” 641 B view raw
panic.txt β€” 573 B view raw
env.txt β€” 936 B view raw
rehash_fix.diff β€” 841 B view raw
fix.diff β€” 3.3 KB view raw
VERDICT.md β€” 5.3 KB ↓ raw
verdict.json β€” 3.4 KB view raw
VERDICT.md
↓ download raw

DF-3536 β€” VERDICT

Finding: netgraph7 ng_cisco β€” disconnecting the downstream hook while the 10-second keepalive ng_callout is armed permanently leaks the timer ng_item and the node reference it holds. This is the ng_cisco twin of DF-3531 (ng_tty), same framework defect, different reachable node type.

Status: REPRODUCED (leak, certain) on the stock INVARIANTS guest with ng7 modules built from the pristine /usr/src snapshot.

Root cause (line-accurate)

  1. cisco_newhook("downstream") arms the keepalive timer: sys/netgraph7/cisco/ng_cisco.c:218-220 c /* Start keepalives */ ng_callout(&sc->handle, node, NULL, (hz * KEEPALIVE_SECS), &cisco_keepalive, (void *)sc, 0);
  2. ng_callout() allocates a struct ng_item carrying one NG_NODE_REF (sys/netgraph7/netgraph/ng_base.c:3253-3258) and stores it as the callout argument (ng_base.c:3266-3267).
  3. When the last hook goes away β€” normal disconnect, NGM_RMHOOK, NGM_SHUTDOWN, peer teardown β€” cisco_disconnect cancels: sys/netgraph7/cisco/ng_cisco.c:416-418 c if (pep->af == 0xffff) /* If it is the downstream hook, stop the timers */ ng_uncallout(&sc->handle, NG_HOOK_NODE(hook));
  4. ng_uncallout() frees the pending item only when callout_stop() returns

    0 (sys/netgraph7/netgraph/ng_base.c:3281-3292): c rval = callout_stop(c); item = callout_arg(c); if ((rval > 0) && (callout_func(c) == &ng_callout_trampoline) && (NGI_NODE(item) == node)) { NG_FREE_ITEM(item); /* flag-only in ng7; frees nothing */ }

  5. On this kernel, callout_stop() of an ARMED, not-yet-fired callout completes the cancellation inline but returns 0 β€” the DF-3515 callout_stop(9) contract violation (_callout_cancel_or_stop() early return 0 before the PREVENTED-reporting path, sys/kern/kern_timeout.c:886-890). The gate at ng_base.c:3284 never passes, so the never-sent item (whose queue refcount is 0) is orphaned: one ng_item + one NG_NODE_REF leaked per armed-cancel. (Even if the gate passed, NG_FREE_ITEM is flag-only in ng7 and ng_free_item() is never called β€” DF-3531 layer-2.)

Consequences per leaked reference: the ng_node struct is pinned forever in the ID hash (ng_unref_node never runs), the type refcount never drops, and kldunload ng_cisco is permanently refused ("Device busy"). Repeating connect/disconnect grows the kernel heap without bound; the pinned zombie nodes also push ng_nodes past the ng7 ID-hash rehash threshold, which on stock modules feeds the DF-3514 hashdestroy panic (observed, panic.txt).

Reproduction

Measured runs: 300-cycle phases (harness ngcisco7_leak.c), ng7 netgraph.ko built with the DF-3514 rehash fix pre-applied (the identical procedure DF-3531 used for its measured runs, so the separately-filed rehash defect does not step on the leak differential):

phase M_NETGRAPH allocs M_NETGRAPH_NODE allocs
300 downstream (leak) cycles 302 / 42.5K (in lockstep, see env.txt)
300 inet (control) cycles 302 / 42.5K (FLAT) β€”
300 downstream cycles again 602 / 84.7K (+300, linear) 602 / 100K

kldunload ng_cisco β†’ can't unload file: Device busy (refs pinned).

Stock-module corroboration: the first full run (3000 downstream cycles, no rehash pre-fix) panicked the guest mid-run: panic: hashdestroy: hash not empty β†’ ng_make_node_common β†’ ng_make_node β†’ ng_apply_item β†’ ngthread (panic.txt, serial console). The leaked node references kept zombie nodes in the old ID hash until the rehash threshold tripped β€” the same DF-3514 feed chain DF-3531 documented for ng_tty.

Differential validity: the control run differs from the leak run ONLY in the peer hook name (inet vs downstream) β€” i.e. only in whether the keepalive ng_callout is armed and then cancelled by ng_uncallout. Node creation, hook add/destroy and node teardown are byte-identical code paths.

Exploit chain

Not a corruption primitive: unbounded kernel-memory exhaustion (DoS) + permanent module-unload denial, triggerable by any principal holding a netgraph control socket (SYSCAP_RESTRICTEDROOT, ng_socket.c:182). Two zones grow linearly with attacker-chosen cycle count.

Fix validation

fix_status = not_testable. The defect lives in framework code shared by every ng7 timer user: the correct fix is the two-layer patch already authored and stress-tested by DF-3531's evidence pack (layer 1: kern_timeout.c must report PREVENTED for inline-completed cancels β€” a kernel rebuild; layer 2: ng_uncallout must actually ng_free_item()). DF-3531's pack validated that combination on this same guest and documented its residual caveat (the layer-1 return must be dispatch-precise or a close-vs-fire race double-frees an in-flight item). Rebuilding the kernel a second time to re-run an already-validated fix adds no information; the caveat is upstream's to resolve. fix.diff in this pack mirrors DF-3531's two-layer fix.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.

Fix verification

not_testable
baseline reproduced→ patch + rebuild →patched clean

Framework-level two-layer fix (kern_timeout.c PREVENTED reporting + ng_uncallout ng_free_item) already authored and stress-validated by DF-3531's pack on this same guest, including its documented dispatch-precision caveat; re-validating requires a kernel rebuild and would only repeat DF-3531's experiment.

fix.diff mirrors DF-3531/fix.diff; see findings/poc/DF-3531/VERDICT.md 'Fix validation' for the kernel-rebuild results and the double-free caveat under close-vs-fire racing.
↓ fix.diffper-fix-DF-3536

Confirmed kernel references

Detail

Exploit chain

root netgraph user repeats NGM_MKPEER(cisco,downstream)+NGM_RMHOOK cycles: +1 pinned ng_node and +1 leaked ng_item per cycle, unbounded kernel heap growth, permanent module-unload denial, and on stock modules a deterministic kernel panic once ng_nodes crosses the ID-hash rehash threshold (DF-3514 feed).

Evidence (decisive lines)

run.log (302 -> flat control -> 602 allocs, kldunload Device busy), env.txt (netgraph_node 602/100K in lockstep), panic.txt (stock-module hashdestroy panic), VERDICT.md root-cause with path:line

PoC changes

wrote harness from scratch modeled on DF-3531's ABI header: fixed sockaddr_ng family type (DragonFly sa_family_t is __uint8_t, not u_short β€” first build sent skewed paths and failed with ENOENT/ENXIO); ng7 modules for measured runs built with the DF-3514 rehash hunks pre-applied (rehash_fix.diff) so the separately-filed rehash defect does not mask the leak differential.

Verified recommended fix

ng_uncallout must free the orphaned timer item (ng_free_item) whenever the cancel completed, gated on a dispatch-precise PREVENTED report from callout_stop (kern_timeout.c early-return fix); see DF-3531.

Verdict

REPRODUCED (leak, certain): every ng7 cisco downstream-hook disconnect with the armed 10s keepalive ng_callout leaks one ng_item + one node reference because ng_uncallout (ng_base.c:3281-3292) frees the item only when callout_stop() > 0 and this kernel's callout_stop returns 0 for inline-completed armed cancels (kern_timeout.c:886-890, DF-3515). 300-cycle leak runs grow M_NETGRAPH/M_NETGRAPH_NODE by exactly +300 allocs each (linear, monotone); the inet-hook control run is flat; kldunload ng_cisco is permanently refused (Device busy); a 3000-cycle run on stock modules additionally panicked the guest via the DF-3514 rehash feed (panic: hashdestroy: hash not empty in ng_make_node_common). Trigger requires a netgraph control socket (SYSCAP_RESTRICTEDROOT).