β¬’ DragonFlyBSD Kernel Audit
← triage Β· dashboard
DF-3528

ng_car SHAPE timer leaks ng_item + node ref + hook ref on every armed-cancel (DF-3523 twin) - latent: module does not build on DragonFly

Summary

ng_car arms its per-hook SHAPE callout with netgraph7's item-carrying ng_callout() (fresh ng_item + NG_NODE_REF + NG_HOOK_REF per arm) and cancels it only in ng_car_shutdown via ng_uncallout. Every armed-cancel leaks the item and both references via the same two stacked layers DF-3523 measured in-guest (DF-3515 kern_timeout contract violation makes the cleanup branch unreachable on stock kernels; netgraph7's NG_FREE_ITEM is flag-only so ng_free_item is never called). Since ng_car's disconnect (DF-0622) never cancels, an armed timer at node destruction is the normal case. LATENT: ng_car.c was never ported - struct bintime/getbinuptime/bintime_sub/mtx_* exist nowhere else in the DragonFly tree; in-guest KLD build fails with 68 -Werror errors (build.log in pack); sys/conf/files marks it optional, enabled by no kernel config. Rearm-path leak KILLED (ng_car_q_event is an NGQF_WRITER item serialized by ng_apply_item's reader/writer protocol, so ng_car_schedule only ever re-arms an idle callout).

Discussion (0)

No comments yet.

PoC verification

Evidence pack

findings/poc/DF-3528 Β· 8 files
FileTypeDescriptionSize
README.md readme finding + latency + would-be trigger 4.0 KB ↓ raw
VERDICT.md verdict full narrative 3.5 KB ↓ raw
build.sh script in-guest module build attempt 84 B view raw
run.sh script documentation-only trigger 132 B view raw
build.log build-log FULL 295-line make output, 68 errors 16.1 KB view raw
env.txt env guest uname + state 152 B view raw
fix.diff suggested-fix ng_base ng_callout/ng_uncallout item free (DF-3523 layer-2 + rearm oitem) 1.6 KB view raw
verdict.json verdict machine verdict 3.0 KB view raw
README.md readme finding + latency + would-be trigger
↓ download raw

DF-3528 β€” ng_car SHAPE callout leaks ng_item + node/hook refs (DF-3523 twin)

What

sys/netgraph7/ng_car.c arms its per-hook SHAPE timer with netgraph7's item-carrying callout API and cancels it with ng_uncallout():

  • arm: ng_car_schedule() β€” ng_car.c:656-660 β†’ ng_callout(&hinfo->q_callout, node, hook, delay, ng_car_q_event, NULL, 0)
  • cancel: ng_car_shutdown() β€” ng_car.c:539-540 β†’ ng_uncallout(&priv->upper/lower.q_callout, node)

ng_callout() (sys/netgraph7/netgraph/ng_base.c:3248-3269) allocates a fresh ng_item and takes NG_NODE_REF + NG_HOOK_REF on it every call. ng_uncallout() (ng_base.c:3273-3296) is supposed to free that item on a successful armed-cancel, but:

  1. layer 1 (DF-3515): kern_timeout _callout_cancel_or_stop returns 0 for inline-completed stops, so on a stock kernel callout_stop(c) returns 0 and the cleanup at ng_base.c:3284-3292 is never reached;
  2. layer 2 (DF-3474/DF-3523): even when reached, NG_FREE_ITEM(item) is a flag-only macro in netgraph7 (netgraph.h:787-792) β€” ng_free_item() is never invoked, so the item + its node reference + its hook reference leak on ANY kernel.

Net effect for ng_car: every ng_car node destroyed while a SHAPE timer is armed leaks 1 ng_item + 1 node ref + 1 hook ref per armed hook (up to 2 per node). The leaked node ref pins the dead node and its name registration forever (DF-3523 measured this exact mechanism on ng_pptpgre: 5.76K allocs / 830 KB per 30k cycles, kldunload β†’ "Device busy", and at scale the leaked node refs drove the DF-3514 idhash-rehash panic).

Rearm note (lead (a) of this audit batch): the arm-then-rearm path (q_event β†’ ng_car_schedule) only ever re-arms an idle callout, because q_event runs as an NGQF_WRITER item and is serialized against rcvdata by ng_apply_item's reader/writer protocol (ng_base.c:1993-1999). The orphaned oitem dead-store at ng_base.c:3266 is therefore NOT reachable through ng_car's flow; the leak is exactly the cancel path. (The disconnect path never cancels at all β€” that is known DF-0622 β€” but ng_car_shutdown's ng_uncallout still runs at rmnode, so the armed timer IS cancelled-and-leaked there.)

LATENT β€” the module does not build on DragonFly

ng_car.c was never ported: it uses FreeBSD-only struct bintime, getbinuptime(), bintime_sub(), struct mtx, mtx_init/mtx_destroy/ mtx_lock/mtx_unlock β€” none of which exist anywhere else in the DragonFly tree (verified: rg over sys/ finds getbinuptime only inside ng_car.c). sys/conf/files:1696 lists it as optional netgraph7_car and no kernel config in sys/config/ enables that option.

In-guest build attempt (this pack, build.log): 295 lines, 68 compile errors (-Werror), first fatal at struct bintime lastRefill (ng_car.c:62) / getbinuptime implicit (ng_car.c:592).

Trigger (once the module is ported / on FreeBSD-derived ng7)

root (control socket is privilege-gated, ng_socket.c:182):

ngctl mkpeer ether:lower car lower upper?   # any graph with a car node
ngctl msg car: setconf '{ downstream={ mode=3 cbs=8192 ebs=8192 cir=100 } ... }'  # SHAPE
# send data until queue non-empty and timer armed
ngctl rmhook <hook>                          # rmnode β†’ shutdown β†’ ng_uncallout β†’ leak
# repeat; watch: vmstat -m | grep netgraph   (monotonic growth, never returned)

Unprivileged users cannot drive this one (unlike DF-3523, where the unprivileged data stream armed/cancelled the timers): arm requires SHAPE traffic, cancel only happens at node destruction β€” both are control-plane (root) operations. Hence Low severity even before the build issue.

Fix

Two layers, both required (identical to the validated DF-3523 fix): 1. kern_timeout contract fix (DF-3521/DF-3515, kern_timeout.diff in findings/poc/DF-3523/) so armed-cancels report rval>0; 2. ng_base.c ng_uncallout actually calls ng_free_item() β€” see fix.diff here (plus the oitem rearm free for completeness). Plus a port of ng_car.c to DragonFly time/lock APIs (or removal of the dead file) so the module can build at all.

VERDICT.md verdict full narrative
↓ download raw

DF-3528 VERDICT

status: build_failed (module cannot be built on this kernel) β€” finding is source-certain and LATENT; leak mechanism proven generically in-guest by DF-3523 on the same ng_base.c code path.

Did it reproduce on the guest?

No β€” and it cannot, on any DragonFly kernel built from this tree. The PoC setup (compile sys/netgraph7/ng_car.c as a KLD) fails with 68 errors (build.log, -Werror), because the file was never ported from FreeBSD:

  • struct bintime unknown (ng_car.c:62, 588)
  • getbinuptime() implicit (ng_car.c:210, 225, 592)
  • bintime_sub() implicit (ng_car.c:596)
  • struct mtx / mtx_init / mtx_destroy / mtx_lock / mtx_unlock unknown (ng_car.c:70-71, 214, 229, 541-542, 722, 765)

rg getbinuptime sys/ (host, whole tree) matches ONLY ng_car.c; no kernel config enables netgraph7_car (sys/conf/files:1696 optional, nothing in sys/config/). This confirms and matches the pass-1 DF-0622 record ("Dead code... doesn't compile") and retroactively explains why DF-0623 was source-only.

Why the finding is still certain

  1. ng_car.c:656-660 arms the timer via ng_callout() β€” the item-carrying API (ng_base.c:3248-3269): fresh ng_item + NG_NODE_REF + NG_HOOK_REF per arm. Certain by inspection.
  2. ng_car.c:539-540 is the ONLY cancel (ng_uncallout, ng_base.c:3273-3296). Its cleanup branch (ng_base.c:3284-3292) is (a) unreachable on the stock kernel because of the DF-3515 callout_stop contract violation, and (b) a leak even when reached because netgraph7's NG_FREE_ITEM is flag-only (netgraph.h:787-792). Both layers were measured in-guest by DF-3523 (ng_pptpgre, same ng_base functions): 30k arm/cancel cycles β†’ 5.76K allocs/830 KB never returned; fixed kernel + fixed ng_base β†’ flat.
  3. ng_car's disconnect (ng_car.c:553-580) never cancels the timer at all β€” known DF-0622 β€” so an armed SHAPE timer at node destruction is the normal case, and shutdown's ng_uncallout then cancels-and-leaks it.

Rearm-vs-arm leak (lead (a)): not reachable via ng_car β€” q_event is a NGQF_WRITER item serialized against rcvdata readers (ng_base.c:1993-1999), so ng_car_schedule() only ever re-arms an idle callout; the orphaned oitem dead-store (ng_base.c:3266) stays dead for this caller.

Impact ceiling

Low: the trigger is entirely control-plane (root: mkpeer/setconf/rmhook); the unprivileged-data-driven variant is DF-3523's (ng_pptpgre), already filed. Latent until the module is ported. If ported unchanged, each node destruction with an armed SHAPE timer permanently leaks ≀2 ng_items + node refs + hook refs (objcache exhaustion β†’ DF-3474-style M_WAITOK wedge; name-registration pinning; DF-3514-style idhash growth).

Fix validation

Not performed here (nothing to validate against β€” module cannot build). The code fix is the already-validated DF-3523 pair (kern_timeout contract + ng_free_item in ng_uncallout); fix.diff in this pack carries the ng_base layer plus the rearm-oitem free for completeness.

Bottom line

Real refcount-lifetime defect in dead code: file it, fix ng_base (DF-3523 fix covers it), and either port ng_car to DragonFly primitives or delete it.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.

Fix verification

not_testable
↓ fix.diffper-fix-DF-3528

Confirmed kernel references

Detail

Evidence (decisive lines)

build.log (295 lines, 68 -Werror errors, first fatal: 'struct bintime lastRefill' ng_car.c:62, 'getbinuptime' ng_car.c:592); VERDICT.md (layer-by-layer reasoning); mechanism precedent: findings/poc/DF-3523 (measured 5.76K allocs/830KB per 30k arm/cancel cycles on ng_pptpgre, flat after fix)

PoC changes

n/a - the PoC is the build attempt itself (orchestrator seed pattern from DF-3523 adapted to ng_car: out-of-tree kmod.mk Makefile with SYSDIR=/usr/src/sys and -I/usr/src/sys/netgraph7)

Verified recommended fix

Fix ng_base ng_callout/ng_uncallout to actually free callout items (fix.diff; DF-3523 layer-2) AND port ng_car.c to DragonFly time/lock APIs or delete the dead file.

Verdict

LATENT, source-certain: ng_car.c arms its SHAPE timer with the item-carrying ng_callout() (ng_car.c:656-660) and the only cancel is ng_car_shutdown()'s ng_uncallout() (ng_car.c:539-540); on this kernel every armed-cancel leaks the ng_item plus one node and one hook reference (DF-3515 contract violation makes the cleanup unreachable, and netgraph7's flag-only NG_FREE_ITEM leaks it even when reached - both layers measured in-guest by DF-3523 on the same ng_base.c functions). HOWEVER ng_car.c was never ported to DragonFly (struct bintime / getbinuptime / bintime_sub / mtx_* do not exist in this tree): the in-guest KLD build fails with 68 errors (build.log), sys/conf/files:1696 marks it optional netgraph7_car which no kernel config enables, so the leak cannot manifest on any kernel built from this tree. Rearm-leak (lead a) is NOT reachable via ng_car: q_event is a writer item serialized against rcvdata readers (ng_base.c:1993-1999), so ng_car_schedule only re-arms an idle callout. Trigger is root-only control-plane (setconf SHAPE + rmnode), so even when ported the ceiling is Low.