DF-3453 VERDICT — pfil COW free vs non-netisr walkers

Bottom line

netmsg_service_sync() in pfil_add_hook_dispatch() / pfil_remove_hook_dispatch() (sys/net/pfil.c:238, :346) only synchronizes with the netisr threads — netreglist is populated exclusively with the netisr_threads[i] msgports (sys/net/netisr.c:229, the only caller of netmsg_service_port_init), and netmsg_service_sync (netisr.c:292-303) iterates exactly that list. But pfil_run_hooks() walkers also execute on threads that are NOT in that list:

1. softclock: tbf_reprocess_q callout (sys/net/ip_mroute/ip_mroute.c:1874, TBF_REPROCESS = hz/100) → tbf_process_q (:1844) → tbf_send_packet (:1926) → ip_output(m, ..., IP_FORWARDING, ...) (:1931/:1948) → pfil_run_hooks(&inet_pfil_hook, ..., PFIL_OUT) (ip_output.c:529). Callouts execute on the per-CPU softclock N LWKT threads (kern_timeout.c:423 lwkt_create(softclock_handler, ...)).
2. driver interrupt threads with IFF_IDIRECT: ether_dispatch() calls ether_input_handler() directly in the caller's context (if_ethersubr.c:1457) → ether_input_oncpu → bridge_input → bridge_pfil(PFIL_IN) → pfil_run_hooks (if_bridge.c:4097).

An in-flight walk on one of those threads holds the OLD list pointer across the COW swap; pfil_list_free(old_list) (pfil.c:244-247, :351-354) then recycles packet_filter_hook entries and the list head while the walk's TAILQ_FOREACH (pfil.c:110) dereferences them and calls pfh->pfil_func — a use-after-free with an indirect call.

What was proven on the guest

(a) The walker class is real — stock INVARIANTS kernel PANICS from the plain feeder (no test modules): with only kldload ip_mroute + MRT_INIT/MRT_ADD_VIF(rate-limited)/MRT_ADD_MFC + a multicast UDP flood, the stock kernel (DragonFly 6.5-DEVELOPMENT #0 ... X86_64_GENERIC) panicked:

panic: thread 0xfffff800895bf040 cpu0 is not within netisr_ncpus 6
ip_output() at ip_output+0x64e
ip_output() at ip_output+0x64e
tbf_send_packet() at tbf_send_packet+0xac
tbf_process_q() at tbf_process_q+0x56
tbf_reprocess_q() at tbf_reprocess_q+0x2b
softclock_handler() at softclock_handler+0x1b8

(ip_output.c:185 ASSERT_NETISR_NCPUS(mycpuid) — the kernel's own assertion documents that this ip_output/pfil caller is not a netisr thread.) This is a self-contained root-configurable panic on default INVARIANTS builds; see panic.txt.

(b) On a production (non-INVARIANTS) kernel, the softclock pfil walk runs: the identical flood on a PRNOINV kernel (same source, INVARIANTS disabled) forwarded ~2.3kpps through tbf_send_packet → ip_output → pfil_run_hooks on softclock for minutes with no assert (asserts compiled out). With the prhook module's softclock-only delay, [softclock 0] burned 97% CPU inside pfil_run_hooks/hook A (net.prhook_n_delay counting hundreds of parked walks).

(c) The freed hook chunks are immediately reclaimable: the instrumented toggle (net.prhook_n_spray_hit vs net.prhook_n_walk_probe) returned 2/2: immediately after pfil_remove_hook() returns on netisr0, the very next kmalloc(36, M_IFADDR) on netisr0 hands back one of the just-freed packet_filter_hook chunks (slab fast path pops z_LChunks, kern_slaballoc.c:1008-1017 / kfree push :1584). Any concurrent same-zone allocation on the freeing CPU lands on the list a parked walk is still iterating — the poisoning step is trivially effective, as designed in the PoC.

Why the end-to-end wild-call crash did not trigger on this guest

Both sides of the race are CPU-pinned to CPU0 here: - The mutation always executes on netisr0 (PFIL_CFGPORT = netisr_cpuport(0), pfil.c:47), pinned to CPU0. - The mroute tbf callout is armed by tbf_control on the netisr CPU that processes the forwarding packet (ip_mroute.c:1792) and then re-arms itself from its own softclock handler (:1883). On this 6-vCPU guest the flood's loopback-hash flow landed on netisr0, so the ONLY sustainable walker was softclock 0 — the same CPU as netisr0. DFly LWKT threads are non-preemptive: while softclock-0 is parked inside the hook, netisr0 cannot run, and vice versa — the free can never interleave with the parked walk when both share CPU0. - Walking on softclock N (N!=0) starves instead: while the softclock-N walk parks, netisr-N's inline tbf_process_q (invoked per packet arrival, ip_mroute.c:1806) consumes every token the bucket accrues, so the softclock-N callout fires with zero tokens and never walks (net.prhook_n_delay stayed 0 for non-zero-hash flows — measured).

So on this specific configuration the same-CPU coincidence closes the window; on other flow mixes / CPU layouts (multi-flow forwarding where the callout-bearing CPU differs from CPU0, or the IFF_IDIRECT bridge walker on a driver interrupt thread whose IRQ CPU != 0), the mutation on netisr0/CPU0 proceeds concurrently with the parked walk. The constituent facts (walker on a non-netreglist thread; COW free; immediate reclaim of the freed chunks) are each individually proven above; the composition is blocked here by CPU pinning, not by any synchronization in pfil.c — netmsg_service_sync() never waits for softclock or interrupt threads.

Impact / exploitability

Both triggers are root-gated: enabling multicast routing (MRT_INIT via a raw IGMP socket) and loading/unloading pfil-hooking modules (ipfw/ipfw3/pf). There is no unprivileged trigger — this is a kernel reliability/UAF-class defect (root racing the kernel), not an escalation. Severity Low. The INVARIANTS panic (a) is an unconditional root-configured DoS on default INVARIANTS builds.

Fix

fix.diff: add a pfil_token (LWKT token); pfil_run_hooks() takes it around the walk, and both dispatch functions take it around the swap→free section. Token blocking/release semantics keep this deadlock free (the walker that holds the token during a long hook simply delays the mutation; netisr threads replying to netmsg_service_sync() never need the token). Not applied to sys/ (read-only audit tree); not runtime-validated on the guest since the end-to-end race did not trigger there (fix_status: not_testable, honestly recorded).

Follow-up observation (out of batch scope)

During cleanup on the non-INVARIANTS kernel, kldunload ipfw3_basic panicked in rn_flush+0x1d (NULL page fault, Fatal trap 12, fault va 0x28) right after "ipfw3 module basic unloaded" — a teardown bug in the ipfw3 table subsystem (sys/net/ipfw3_basic/ip_fw3_table.c) worth a dedicated audit pass.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.