β¬’ DragonFlyBSD Kernel Audit
← triage Β· dashboard
DF-3776

jme_stop()/jme_miibus_statchg() vs jme_tick() AB-BA deadlock: blocking lwkt_serialize_enter in the self-re-arming CPU0-pinned tick callback vs synchronous callout_stop() under ifnet_serialize_all β€” permanent kernel wedge + CPU0 softclock callout blackout (DF-3715/3738/3760 family, jme(4) instance)

Summary

jme_tick is a 1 Hz self-re-arming MP callout pinned to CPU0 whose callback BLOCKS in lwkt_serialize_enter(&sc->jme_serialize) and re-arms inside the serialized section. jme_stop and jme_miibus_statchg call the SYNCHRONOUS callout_stop() while holding ALL driver serializers (entered via ifnet_serialize_all from detach, suspend, ioctl down/mtu/media, watchdog). callout_stop ssleeps on 'costp' until an in-progress callback completes; callouts run on the per-CPU softclock lwkt thread so the parked enter legitimately sleeps. A tick that fires while the stopper holds the serializer deadlocks both threads permanently - and the wedged CPU0 softclock thread stops delivering EVERY callout queued on CPU0. The jme_in_tick/skipmain dance only manages the additional serializers inside statchg and does not defuse the blocking enter. MECHANISM PROVEN on the guest via a protocol-replica KLD (jme's exact protocol: CPU0-pinned MP callout, blocking enter + skipmain array dance + self re-arm; stopper with full serializer array + callout_stop): callout_stop never returned, ps shows 'D5 costp', CPU0 softclock wedged; fix-shape control returns cleanly pending=0. Fix: try-enter and retry next tick.

Discussion (0)

No comments yet.

PoC verification

Evidence pack

findings/poc/DF-3776 Β· 14 files
FileTypeDescriptionSize
README.md β€” 2.8 KB ↓ raw
VERDICT.md β€” 7.4 KB ↓ raw
jme_callout_replica.c β€” 14.8 KB view raw
Makefile β€” 139 B ↓ download
build.sh β€” 171 B view raw
run.sh β€” 384 B view raw
build.log β€” 63 B view raw
run.log β€” 788 B view raw
run.2.log β€” 144 B view raw
fix_build.log β€” 616.0 KB ↓ download
env.txt β€” 448 B view raw
fix.diff β€” 1.0 KB view raw
manifest.json β€” 2.5 KB view raw
verdict.json β€” 5.2 KB view raw

DF-3776 β€” jme_stop()/jme_miibus_statchg() vs jme_tick() AB-BA deadlock (callout subsystem vs driver serializer)

File: sys/dev/netif/jme/if_jme.c (pass 2, GLM 5.3 re-audit) Class: DF-3715 (em) / DF-3738 (emx) / DF-3760 (alc) family β€” independent jme(4) instance Severity: Medium Β· CWE: CWE-833 (Deadlock) / CWE-667 Β· Confidence: certain (source), mechanism PROVEN on guest

What

jme_tick is a self-re-arming 1 Hz callout (pinned to CPU0 via JME_TICK_CPUID, if_jme.c:77/:2881) whose callback blocks in lwkt_serialize_enter(&sc->jme_serialize) (if_jme.c:2571). jme_stop() calls the synchronous callout_stop(&sc->jme_tick_ch) (if_jme.c:2907) while holding all driver serializers (ASSERT_IFNET_SERIALIZED_ALL, if_jme.c:2898). On this kernel callout_stop() β†’ _callout_cancel_or_stop(..., sync=1) ssleeps on "costp" (sys/kern/kern_timeout.c:920) until an in-progress callback finishes β€” so a tick that fired and parked on the serializer held by the stopper deadlocks both threads permanently, and the wedged CPU0 softclock thread stops delivering every callout queued on CPU0.

Trigger paths (all hold all serializers)

  • ifconfig jme0 down / up / mtu / media β†’ jme_ioctl (if_jme.c:1954/:1939/:1965) β†’ jme_stop/jme_initβ†’jme_stop (:2672)
  • jme_miibus_statchg callout_stop at :409 (link state change)
  • jme_detach (:1092-1093), jme_suspend (:1589-1590)
  • jme_watchdog β†’ jme_init (:1885/:1900)
  • kldunload/devd detach, system suspend β€” all root-triggered; window widens arbitrarily when the stopper blocks acquiring the ring serializers behind a busy RX interrupt handler.

Reproduce (guest, no jme hardware needed β€” protocol replica)

scp findings/poc/DF-3776/{Makefile,jme_callout_replica.c} dfbsd:/root/kmod/
vm.sh run_root 'cd /root/kmod && make && kldload ./jme_callout_replica.ko'
vm.sh run_root 'sysctl kern.jmereplica.arm=1; sleep 2; sysctl kern.jmereplica.stop_try=1'
vm.sh run_root '(sysctl kern.jmereplica.stop_blk=1 &) ; sleep 6; dmesg | grep "jmereplica\[blk\]" | tail -2; ps -o pid,stat,wchan,command -ax | grep stop_blk | grep -v grep'
#   -> "IF THIS IS THE LAST jmereplica LINE, THE if_jme.c AB-BA DEADLOCK IS PROVEN"
#   -> sysctl thread D5 wchan "costp"; callout_stop NEVER returned.
#   Guest is wedged: vm.sh reset with-src to recover.

Expected (observed 2026-09-16, see run.log): fix-shape control returns with pending=0; shipped-shape callout_stop() never returns; ps shows the caller asleep on costp (kern_timeout.c:920).

Fix

fix.diff β€” try-enter + 1-tick retry in jme_tick (family fix, mirrors DF-3715/DF-3738/DF-3760). Compile-validated: nativekernel rebuild of X86_64_GENERIC with the fix β†’ RC=0, no warnings (fix_build.log). Mechanism validated by the replica's stop_try control.

VERDICT.md
↓ download raw

DF-3776 β€” VERDICT

Status: mechanism PROVEN on the guest kernel by deterministic replica of if_jme.c's exact callout/serializer protocol; jme-specific runtime hardware-gated β†’ inconclusive per family convention (DF-3715/DF-3738/DF-3760 precedent), reproduced=0, impact=none-on-this-guest. Class of impact when it fires on JMicron JMC250/JMC260 hardware: permanent kernel deadlock (DoS) triggered by a privileged local user (ifconfig down/up/MTU/media, detach, suspend) and opportunistically by the watchdog path; collateral: the CPU0 softclock thread stops delivering ALL callouts queued on CPU0.

Root cause (path:line)

  1. sys/dev/netif/jme/if_jme.c:772 β€” callout_init_mp(&sc->jme_tick_ch); armed at :2881 / :483 with callout_reset_bycpu(..., hz, jme_tick, sc, JME_TICK_CPUID) (CPU0-pinned, if_jme.c:77 "DO NOT CHANGE THIS").
  2. sys/dev/netif/jme/if_jme.c:2571 β€” jme_tick() does a blocking lwkt_serialize_enter(&sc->jme_serialize) (the first/main serializer of sc->jme_serialize_arr[], if_jme.c:733); :2579 β€” self re-arm callout_reset(&sc->jme_tick_ch, hz, jme_tick, sc) inside the serialized section. The jme_in_tick / jme_serialize_skipmain dance (if_jme.c:362-363/:3971-3982) only manages the additional serializers inside jme_miibus_statchg; the initial blocking enter at :2571 is the deadlock site.
  3. sys/dev/netif/jme/if_jme.c:2907 — jme_stop() calls callout_stop(&sc->jme_tick_ch) while holding ALL driver serializers (ASSERT_IFNET_SERIALIZED_ALL at :2898; entered via ifnet_serialize_all from jme_detach :1092-1093, jme_suspend :1589-1590, and every jme_ioctl path — SIOCSIFFLAGS down :1954, SIOCSIFMTU→jme_init :1939→jme_stop :2672, SIOCSIFMEDIA→ jme_miibus_statchg's own callout_stop at :409, watchdog :1885/:1900).
  4. sys/kern/kern_timeout.c:1091 β€” callout_stop() is the synchronous variant on this kernel (_callout_cancel_or_stop(..., sync=1)); with the callback INPROG the caller ssleep(c, &c->spin, 0, "costp", 0) (sys/kern/kern_timeout.c:920) until the callback completes. Callouts execute on the per-CPU softclock N lwkt thread (kern_timeout.c:423), so the blocking lwkt_serialize_enter in the callback legitimately sleeps β€” that is what parks it.
  5. β‡’ A tick callback dequeued by the CPU0 softclock thread and parked in lwkt_serialize_enter (if_jme.c:2571) on the serializer held by the jme_stop() caller deadlocks against the stopper (AB-BA): both threads wedge permanently. Collateral: the interface's serializers are held forever (all later driver entry points block β€” ioctls, RX/TX interrupts, tick) and the wedged CPU0 softclock thread stops delivering every callout queued on CPU0 (other drivers' timers, protocol timers, taskqueue timeouts).

Race window: the tick must be INPROG at the moment the stopper is between its serializer acquire and callout_stop(). For jme_stop the direct window is small (flag updates at :2903-2905), but it is widened without bound whenever the stopper's ifnet_serialize_all blocks acquiring the ring serializers after taking main β€” e.g. behind a busy RX interrupt handler performing the unbounded jme_rxeof walk (DF-3778) β€” while the 1 Hz tick fires and parks. The jme_miibus_statchg path (:409) has a wider intrinsic window: it is reached from mii_mediachg after PHY register DELAY loops. Every trigger (ifconfig jme0 down|mtu|media, detach, suspend, watchdog reinit) is repeatable by root, so eventual occurrence on long-lived boxes is plausible (same analysis as DF-3760/alc).

This is the if_jme.c instance of the class proven for if_em.c (DF-3715), if_emx.c (DF-3738) and if_alc.c (DF-3760); identical primitives, independent file, not previously filed for jme.

Not a re-arm UAF

DF-3715's replica (same kernel) proved the negative control: after the callback returns, callout_stop's STOP post-processing unloads a racing re-arm (_callout_update_spinlocked). No post-stop re-arm UAF β€” the defect is purely the sync-wait deadlock. Re-verified for the jme/fix shapes by this pack's stop_try control (pending=0 after stop).

Verification performed (guest, DragonFly 6.5-DEVELOPMENT x86_64)

Presence probe (run.log): no JMicron PCI device (vtnet0 lo0 only) β€” driver present in the running kernel (kldstat -v: 102 jme/miibus, 103 pci/if_jme) but attach never runs without hardware β†’ runtime hardware-gated, honest inconclusive per family convention.

Mechanism, proven by jme_callout_replica.ko (jme_callout_replica.c β€” replica of jme's exact protocol: MP callout pinned to CPU0, callback = blocking enter of ser_main + jme_serialize_skipmain array dance (array_enter/array_exit from index 1, if_jme.c:3971-3982) + self re-arm at hz; stopper = enter of the full serializer array (all, index 0) + callout_stop):

  1. sysctl kern.jmereplica.arm=1 β€” arms shipped-shape and fix-shape replicas (dmesg: "armed (hz=100)").
  2. sysctl kern.jmereplica.stop_try=1 β€” fix-shape control (DF-3776 fix.diff protocol: lwkt_serialize_try + 1-tick retry): callout_stop returned, pending=0, no deadlock (run.log).
  3. sysctl kern.jmereplica.stop_blk=1 β€” shipped shape: the sysctl thread takes ALL serializers, waits until the tick handler is BLOCKED on the main serializer (fired=16 ran=15), then calls callout_stop(): * last dmesg line = "… THE if_jme.c AB-BA DEADLOCK IS PROVEN" β€” callout_stop() never returned; * ps -o pid,stat,wchan,command -ax shows the caller 1514 D5 costp sysctl kern.jmereplica.stop_blk=1 (run.log) β€” D-state, wchan costp == sys/kern/kern_timeout.c:920; * CPU0 softclock thread wedged executing the parked callback (run.log). Guest force-killed afterwards; vm.sh reset with-src restored a clean guest (verified: no /root/kmod, no kern.jmereplica sysctl).

Fix validation

  • fix.diff (git-apply clean against sys/) switches jme_tick to lwkt_serialize_try + 1-tick retry (pinned via callout_reset_bycpu to JME_TICK_CPUID), mirroring the DF-3715/DF-3738/DF-3760 family fix.
  • Mechanism validated live: the replica's fix-shape control (identical try-enter + retry protocol) shows callout_stop under ALL serializers returns with pending=0, no deadlock (run.log).
  • Compile-validated on the guest: all three jme fix.diff files (DF-3776/DF-3777/DF-3778) applied together to /usr/src, incremental nativekernel build of X86_64_GENERIC β†’ RC=0, zero warnings/errors (fix_build.log, 2026-09-16 04:41 UTC).
  • Full driver-level validation is not possible on a guest without JMicron hardware β†’ fix_status: not_testable (family convention).

Bottom line

The if_jme.c instance of the proven em/emx/alc AB-BA: jme_tick's blocking serializer acquire under a self-re-arming callout vs callout_stop() under ifnet_serialize_all in jme_stop/statchg. On hardware: permanent kernel deadlock + CPU0 callout blackout from a root-repeatable ioctl (and opportunistically the watchdog path). Fix is one hunk (try-enter + retry), compile-clean, mechanism-proven.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.

Fix verification

not_testable
baseline no→ patch + rebuild →patched clean

fix.diff (git-apply clean) switches jme_tick to lwkt_serialize_try + 1-tick retry pinned to JME_TICK_CPUID, mirroring the DF-3715/DF-3738/DF-3760 family fix. Mechanism validated live on the guest: the replica's fix-shape control (identical try-enter + retry protocol) shows callout_stop under the full serializer array returns with pending=0, no deadlock. Compile validated: all three jme fixes applied to /usr/src, incremental nativekernel build of X86_64_GENERIC -> RC=0, zero warnings (fix_build.log). Full driver-level validation is not possible on a guest without JMicron hardware.

run.log: 'jmereplica[try]: callout_stop returned, pending=0' / 'RESULT pending=0 (0 == try-shape fully stopped, no deadlock)'; fix_build.log (17952 lines): '>>> Kernel build for X86_64_GENERIC completed' RC=0, grep -cE 'error:|warning:' == 0.
↓ fix.diffper-fix-DF-3776

Confirmed kernel references

Detail

Evidence (decisive lines)

run.log: 'jmereplica[blk]: jme_tick handler BLOCKED on our main serializer (fired=16 ran=15) - entering callout_stop()' followed by NOTHING (callout_stop never returned) and '1514 D5 costp sysctl kern.jmereplica.stop_blk=1' in ps output; run.log also shows the fix-shape control 'jmereplica[try]: callout_stop returned, pending=0'; jme_callout_replica.c replicates if_jme.c:2566-2582/:2907/:1092 verbatim (serializer array incl. the skipmain dance); build.log shows -Werror-clean build; fix_build.log shows nativekernel RC=0 with all three jme fixes applied.

PoC changes

n/a (source-level finding; replica module written fresh for this pack, adapted from the DF-3760 alc replica to jme's exact protocol: CPU0-pinned callout, blocking enter of sc->jme_serialize, skipmain array dance, stop under the full serializer array; plus a jme_rxeof verbatim-loop replica shared with DF-3777)

Verified recommended fix

jme_tick: replace the blocking lwkt_serialize_enter(&sc->jme_serialize) with lwkt_serialize_try + callout_reset_bycpu(1 tick, JME_TICK_CPUID) retry (family fix, see fix.diff).

Verdict

Mechanism PROVEN on the guest kernel by deterministic replica of if_jme.c's exact callout/serializer protocol; jme-specific runtime hardware-gated (inconclusive per family convention, DF-3715/DF-3738/DF-3760 precedent). jme_tick (sys/dev/netif/jme/if_jme.c:2566-2582) is a CPU0-pinned MP callout whose callback BLOCKS in lwkt_serialize_enter(&sc->jme_serialize) (:2571) and self re-arms at hz (:2579); jme_stop (:2907) and jme_miibus_statchg (:409) tear the callout down with the SYNCHRONOUS callout_stop() while holding ALL driver serializers (ASSERT_IFNET_SERIALIZED_ALL :2898; ifnet_serialize_all at jme_detach :1092, jme_suspend :1589, jme_ioctl :1913). DragonFly callout_stop synchronously waits for an in-progress callback (sys/kern/kern_timeout.c:1091 -> _callout_cancel_or_stop sync=1, ssleep 'costp' at :920). A tick callback dequeued and parked on the serializer therefore deadlocks against the stopper (AB-BA): both threads wedge permanently (replica sysctl thread D5 wchan 'costp', callout_stop NEVER returned), the driver serializers are never released, and the wedged CPU0 softclock thread stops delivering every callout on CPU0. Replica on the guest: stop under the full serializer array with the jme_tick-shaped handler parked -> callout_stop NEVER returned; fix-shape control (try-enter + 1-tick retry, DF-3776 fix.diff shape) -> callout_stop returned, pending=0.