DF-3436 — ipfw3_basic module unload races its own netmsg consumers (cleanup callout / table radix walk) — NULL-context deref panic

ip_fw3_state_fini() (sys/net/ipfw3_basic/ip_fw3_state.c:683-692) tears down per-CPU state contexts via netisr_domsg and then does a bare callout_stop() on a handler (ip_fw3_state_cleanup, :584-595) that unconditionally re-arms itself with callout_reset — callout_stop neither waits for a running handler nor cancels a re-arm that lands after it. ip_fw3_state_cleanup_dispatch (:539-582) also dereferences fw3_state_ctx[mycpuid] with no NULL guard, although ip_fw3_state_fini_dispatch (:676-677) sets it to NULL — a cleanup netmsg queued behind a fini netmsg on any CPU lands on a NULL context. The module's MOD_UNLOAD path (ip_fw3_basic.c:651-657) has no in-use guard at all — it tears the module down while its netmsg consumers and the 1-second callout are live (and while firewall rules still reference it).

Reproduce (root, stock kernel #0)

bounce_basic.sh: loop kldunload ipfw3_basic; kldload ipfw3_basic — no other activity required (the session had ipfw3 rules installed and a quiet tap; the "oops" lines in panic.txt are dmesg backlog from the DF-3434 runs).

Observed (< 90 s, < 4000 bounce cycles)

Fatal trap 12: page fault while in kernel mode
cpuid = 0; lapic id = 0
fault virtual address = 0x28
fault code = supervisor read data, page not present
current process = Idle          <- netisr0
kernel: type 12 trap, code=0
Stopped at rn_flush+0x1d: movq 0x28(%rbx),%rax

rn_flush (radix-node flush, from the module's teardown/table netmsg family) walks a NULL/freed tree head on netisr0 — a teardown-vs-consumer race in exactly the unsynchronized unload path this finding describes. (rbx == NULL → read at 0x28.)

Fix (fix.diff): callout_terminate() (wait for the self-re-arming handler) + NULL-guard in ip_fw3_state_cleanup_dispatch + the module should refuse MOD_UNLOAD while the firewall is enabled (guard left as upstream discussion; the two concrete races are fixed).

DF-3436 — VERDICT

Status: reproduced (panic; < 90 s of module bouncing). Fix validated on the second iteration (see below).

Run (baseline, stock kernel #0)

bounce_basic.sh (root): while n<4000: kldunload ipfw3_basic; kldload ipfw3_basic. Guest otherwise idle; ipfw3 rules installed (fwsetup).

Fatal trap 12: page fault while in kernel mode
cpuid = 0; lapic id = 0
fault virtual address = 0x28
fault code = supervisor read data, page not present
current process = Idle                  (netisr0)
kernel: type 12 trap, code=0
Stopped at rn_flush+0x1d: movq 0x28(%rbx),%rax

Root cause (resolved after the first fix attempt failed)

The crash is in ip_fw3_table_fini_dispatch (sys/net/ipfw3_basic/ip_fw3_table.c:576-590), reached from the module's unguarded MOD_UNLOAD (ip_fw3_basic.c:651-658):

1. ipfw3_basic_modevent runs ip_fw3_state_modevent(MOD_UNLOAD) and ip_fw3_table_modevent(MOD_UNLOAD) unconditionally — even when ip_fw3_basic_fini() returns an error (it returns 1 while any rule still references the basic module, ip_fw3.c:189-231). The module subsystem then refuses the unload — but the state/table contexts are already freed, leaving the loaded module pointing at freed memory (fw3_ctx[cpu]->table_ctx is never cleared).
2. ip_fw3_table_fini_dispatch calls rn_flush(table_ctx->node, ...) for all IPFW_TABLES_MAX ids, but only configured tables ever ran rn_inithead (ip_fw3_table.c:100-105); unconfigured ids have node == NULL, and rn_flush() (radix.c:1332-1343) dereferences head->rnh_walktree with no NULL check — read at NULL+0x28 = the observed fault VA.

Repeated cycles mix both: a refused-but-executed fini frees table_ctx, later cycles (or sockopt flushes) walk the freed/NULL heads until one lands on unmapped/zeroed memory — fatal.

Fix validation (two iterations)

• Attempt 1 (kernel #2, 2026-09-12 16:27): callout_terminate() in ip_fw3_state_fini + NULL guard in ip_fw3_state_cleanup_dispatch (fixes the state-module races that are real per code). The identical bounce still panicked with the same rn_flush signature — the state fixes do not touch the table/basic lifecycle. Recorded as a genuine fix_failed attempt.
• Attempt 2 (kernel #3, fix.diff v2 = state fixes + table NULL guards + fw3_ctx->table_ctx = NULL after free + ipfw3_basic_modevent returning EBUSY without running the sub-fini when the unload was refused): the identical bounce completed 4000 cycles with the module cleanly refused/unloaded and no panic (run.patched.log).

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.