DF-3738 / run.log
--- DF-3738 emx callout AB-BA replica session (guest, 2026-09-15) ---
$ kldload /root/emxkmod/emx_callout_replica.ko && echo KLD_OK
KLD_OK
$ sysctl kern.emxreplica.arm=1 ; sleep 2
kern.emxreplica.arm: 0 -> 0
$ sysctl kern.emxreplica.stop_gc=1 # negative control
kern.emxreplica.stop_gc: 0 -> 0
$ (sysctl kern.emxreplica.stop_blk=1 &) # AB-BA trigger, backgrounded
$ sleep 6 ; dmesg | grep emxreplica | tail -8
--- ssh session WEDGED here (killed by host-side timeout) ---
--- serial console (dfbsd-qemu/vm.sh log) ---
emx_callout_replica: loaded (hz=100)
emxreplica: armed (gc every tick, blk every hz/4)
emxreplica[gc]: stop while INPROG (fired=199 ran=199)
emxreplica[gc]: callout_stop returned, pending=0
emxreplica[gc]: RESULT pending=0 (0 == re-arm did NOT survive callout_stop for the try-based emx_txgc_timer shape)
emxreplica[blk]: handler BLOCKED on our serializer (fired=14 ran=13) - entering callout_stop() [emx_stop shape]
emxreplica[blk]: IF THIS IS THE LAST emxreplica LINE, AB-BA DEADLOCK IS PROVEN
<--- console output ENDS here; the "callout_stop returned (NO deadlock)" line NEVER appears
--- wedge proof from a second session (guest still partially alive) ---
$ ps -o pid,stat,wchan,command -ax | grep stop_blk | grep -v grep
1105 D4 costp sysctl kern.emxreplica.stop_blk=1
^^ ^^^^^ the synchronous sleep INSIDE callout_stop() -- sys/kern/kern_timeout.c:920
$ ps -ax -o pid,stat,wchan,command | grep -i softclock
-1 B0 slize (softclock 0) <--- CPU0's softclock thread parked in
^^ ^^^^^ lwkt_serialize_enter() -- sys/kern/lwkt_serialize.c:234,
i.e. INSIDE the emx_timer-shaped callback, holding no way out
-1 B1 - (softclock 1)
-1 B2 - (softclock 2)
...
--- recovery ---
guest reset via `dfbsd-qemu/vm.sh reset with-src`; fresh boot clean
(no emx module loaded, ifconfig -l => vtnet0 lo0)
--- fixed-shape validation (fix.diff protocol replica) ---
$ kldload /root/emxkmod3/emx_callout_replica.ko && echo KLD_OK
KLD_OK
$ sysctl kern.emxreplica.arm=1 ; sleep 2
$ sysctl kern.emxreplica.stop_fixed=1 ; echo "STOP_FIXED_RC=$?"
kern.emxreplica.stop_fixed: 0 -> 0
STOP_FIXED_RC=0
$ kldunload emx_callout_replica && echo UNLOADED
UNLOADED
$ dmesg | grep "emxreplica\[fx\]"
emxreplica[fx]: stop while INPROG/parked (fired=8 ran=8)
emxreplica[fx]: callout_stop returned, pending=0
emxreplica[fx]: RESULT pending=0 (fix effective: emx_stop-shaped stop completes against fixed shape)
(an earlier fx-shape attempt panicked in MY test module only:
"assertion cc->flags & CALLOUT_DID_INIT failed in callout_reset" --
fx_c lacked callout_init_mp(); module bug, unrelated to the finding;
fixed and rerun as above.)