# DF-3453 on-guest session log (reconstructed from the live ssh sessions, full outputs)
# Guest: DragonFly 6.5-DEVELOPMENT, 6 vCPU, two kernels used (see env.txt)

=== [1] STOCK INVARIANTS KERNEL: plain feeder -> panic (walker-class proof) ===
# kldload ip_mroute; ./pfilrace_feeder 10.0.2.15 4000   (no test modules loaded)
login: Warning: objcache(mbuf cluster) exhausted on cpu2!
panic: thread 0xfffff800895bf040 cpu0 is not within netisr_ncpus 6
cpuid = 0
Trace beginning at frame 0xfffff8008974b810
ip_output() at ip_output+0x64e 0xffffffff807b387e
ip_output() at ip_output+0x64e 0xffffffff807b387e
tbf_send_packet() at tbf_send_packet+0xac 0xffffffff826004ac
tbf_process_q() at tbf_process_q+0x56 0xffffffff82600506
tbf_reprocess_q() at tbf_reprocess_q+0x2b 0xffffffff82600ebb
softclock_handler() at softclock_handler+0x1b8 0xffffffff80688438
Debugger("panic")
(full serial console capture in panic.txt)

=== [2] PRNOINV (non-INVARIANTS) KERNEL: walk + mutation + reclaim instrumentation ===
$ uname -v
DragonFly 6.5-DEVELOPMENT #0: Sun Sep 13 01:13:18 UTC 2026  root@dfbsd:/usr/obj/usr/src/sys/PRNOINV
$ sysctl debug.use_weird_array
sysctl: unknown oid 'debug.use_weird_array'          # INVARIANTS compiled out

$ kldload ip_mroute && kldload prhook.ko && sysctl -w net.prhook_delay_ms=120 && sysctl -w net.prhook_toggle=1
net.prhook_delay_ms: 150 -> 120
net.prhook_toggle: 0 -> 0
# (hook B armed: COW add mutation performed)

$ ./pfilrace_feeder 10.0.2.15 1000 &
feeder: mroute up (vif0 lo0, vif1 10.0.2.15 rate 1000, mfc set)
feeder: flooding UDP 239.200.200.200:4711 via lo0 (mcast loop)

$ top -S -b -o cpu | head
    -1 root  -64  0K 0K   SLEEP 0 0:05 0:05 96.73% [softclock 0]   <-- parked inside pfil walk
    -1 root  -64  0K 0K waitport 0 0:03 0:03  0.88% [netisr 0]

$ netstat -ss | grep -i forward
	16577 packets forwarded        (grows to 297k over the session)

# single-flow CPU pinning probes (all 6 source ports tried):
k=0: [softclock 0] 97.75%  (callout armed on CPU0 -> sustain)
k=1..5: no softclock burn, net.prhook_n_delay stays 0
  -> tbf callout arms on the CPU of the feeding flow; netisr-N inline
     tbf_process_q starves a softclock-N walker of tokens (see VERDICT.md)

# toggle (COW remove mutation) with instrumented free-reclaim probe:
$ sysctl -w net.prhook_toggle=1
net.prhook_toggle: 0 -> 0
$ sysctl net.prhook_n_toggle net.prhook_n_spray_hit net.prhook_n_walk_probe
net.prhook_n_toggle: 2
net.prhook_n_spray_hit: 1          <-- first kmalloc(36,M_IFADDR) after the
net.prhook_n_walk_probe: 1             remove returned a just-freed pfil hook chunk
$ sysctl -w net.prhook_toggle=1 && sysctl -w net.prhook_toggle=1   (add, remove)
net.prhook_n_spray_hit: 2          <-- 2/2 reclaim hits
net.prhook_n_walk_probe: 2
$ echo STILL_UP                     <-- no wild-call crash: same-CPU (CPU0)
                                      LWKT non-preemption serializes the only
                                      sustainable walker against the netisr0
                                      mutation on this guest (VERDICT.md)

# tsleep-park variant experiment (rejected): softclock-0 wedged forever in
# 'prpark' state inside its own callout (softclock cannot process the sleep
# timeout wheel while blocked inside a callout handler) -> reverted to DELAY.

=== [3] cleanup-time crash on PRNOINV kernel (out-of-batch observation) ===
ipfw3 initialized, default to accept
ipfw3 module basic loaded
ip_fw3_ctl invalid option 65535
ipfw3 module basic unloaded
Fatal trap 12: page fault while in kernel mode
fault virtual address = 0x28
instruction pointer = 0x8:0xffffffff80711f4d
Stopped at rn_flush+0x1d: movq 0x28(%rbx),%rax
-> ipfw3_basic table-teardown bug (ip_fw3_table.c), separate follow-up.