DF-3515 — callout_stop() returns 0 for a synchronously-stopped armed callout; breaks netgraph7 ng_uncallout contract (HCI CC machine wedged; callout items+noderefs leak)

What

Two coupled defects:

1. sys/kern/kern_timeout.c _callout_cancel_or_stop() (lines 859-901): when the STOP request is completed inline by _callout_update_spinlocked() (callout removed from the wheel, SET cleared, CALLOUT_PREVENTED set), execution falls into the early exit if (sync == 0 || (c->flags & (CALLOUT_INPROG | CALLOUT_SET)) == 0) return 0; — returning 0 even though OUR stop succeeded and PREVENTED is set. The documented contract (kern_timeout.c:1029-1044: "Returns 1 if the cancel is responsible for stopping the callout") is violated for the synchronous-stop case. callout_stop/cancel/drain are all affected.

2. netgraph7's ng_uncallout() (sys/netgraph7/netgraph/ng_base.c:3273-3296) relies on that return value (rval = callout_stop(c); ... return rval), with consumers treating 0 as "not cancelled". Live-instrumented on the guest (kprintf probes): a callout armed for 500 ticks (default HCI command timeout), stopped 300ms in with toc valid and func=ng_callout_trampoline, returned rval=0. Consequence for ng_hci_command_untimeout() (ng_hci_misc.c:390-392): ETIMEDOUT for a successful cancellation -> complete_command() (ng_hci_cmds.c:368-369) silently drops every Command_Complete that arrives while the timeout is pending (the normal case) -> the HCI unit can NEVER initialize (Read_BD_ADDR CC dropped -> bdaddr stays 00:00:00:00:00:00 -> NGM_HCI_NODE_INIT returns ENXIO) -> the whole ng7 bluetooth stack is wedged on this kernel. Same contract is used by ng_l2cap RTX/ERTX, ng_car, ng_pptpgre, ng_l2tp timers.

Additionally, both callout paths leak in netgraph7: * ng_callout() (ng_base.c:3266-3268) reads oitem = callout_arg(c) and never frees it when overwriting a pending callout's item (item + its node/hook refs leak). * ng_uncallout() frees the cancelled item with flag-only NG_FREE_ITEM(item) (ng_base.c:3291) — the item and its node reference are never released on the rval>0 path; on the rval=0 path the item stays armed as the callout arg and callout_set_arg(c,NULL) orphans it. Both routes permanently pin ng_node references (feeds DF-3514's rehash panic threshold).

Evidence (guest, instrumented ng7 core + hci modules)

ARM: rv=0 ticks=500                <- ng_callout armed (default 5s timeout)
CC: state=0x5 q=1                  <- complete_command: PENDING set, cmd queued
CC: op=0x1009 qop=0x1009           <- opcode MATCHES
UNC: rval=0 toc=0xfffff800897a4f80 func=0xffffffff82603460   <- ARMED callout, callout_stop()==0
UT: rval=0                          <- command_untimeout => ETIMEDOUT, CC dropped
state = 0x5 forever; bdaddr stays 00:00:00:00:00:00

Reproduce

Same harness as DF-3513 (poc_leak handshake stage): the Read_BD_ADDR handshake never completes; GET_STATE shows COMMAND_PENDING (0x4) stuck indefinitely; GET_BDADDR returns zeros. (This is also why DF-3513's measurement needed a guest-local INITED bypass.)

Fixes

• kern_timeout.c (needs kernel rebuild — authored, not rebuilt here): in _callout_cancel_or_stop, replace the early return 0 for the sync case with return ((c->flags & CALLOUT_PREVENTED) != 0); (the same expression the later path already uses).
• ng7 side (module-level, VALIDATED as part of the 20-run survival test in DF-3514's pack): free oitem in ng_callout; use ng_free_item(item) (refs==0, never sent) instead of flag-only NG_FREE_ITEM in ng_uncallout.

DF-3515 — VERDICT

Status: reproduced (impact: dos — netgraph7 timer subsystem contract broken; HCI initialization wedged; callout items/node refs leak). ng7-side fix validated (module); kern_timeout.c fix authored, NOT kernel-rebuilt (fix_status: not_testable for that half, documented).

Live proof on guest with instrumented modules: armed 500-tick callout, valid toc, func=trampoline, callout_stop() returns 0 (UNC probe); command completion then always takes the silent ETIMEDOUT path (complete_command, ng_hci_cmds.c:368) — reproduced on every run; unit state stuck at 0x5, bdaddr never set, NODE_INIT ENXIO forever.

Root cause in kern_timeout.c verified by code-reading: inline-completed STOP hits the (flags & (INPROG|SET)) == 0 early return with plain return 0, despite PREVENTED being set — contradicting the documented "returns 1 if the cancel is responsible" contract.

ng7-side leaks (ng_callout oitem; ng_uncallout flag-only free) are source-certain and were fixed + built + survived the 20-run stress (together with DF-3514's fix) — but their individual leak counts were not separately instrumented (the surviving-run criterion is absence of the DF-3514 panic; slow noderef pinning from the still-broken callout_stop return remains until the kernel-side fix lands, as expected and documented).

Classification: status=reproduced, reproduced=1, impact=dos, confidence=certain (behavior) / certain (kern_timeout root cause by trace + source), fix_status=fixed (ng7 half, module-validated) / not_testable (kern_timeout half, requires full kernel rebuild).

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.