DF-2729 VERDICT — reproduced (panic, INVARIANTS kernel)

Status: reproduced. Impact: panic (INVARIANTS builds only). Confidence: certain.

How it was reproduced

Same KLD as DF-2728 (df2728.c, shared source in ../DF-2728/), mode 2, on the audit guest (DragonFly 6.5-DEVELOPMENT #0, X86_64_GENERIC with options INVARIANTS, 6 cpus, stock kernel):

1. MOD_LOAD on cpu0 creates a never-started kernel-thread analogue homed on cpu1 (lwkt_alloc_thread(NULL, LWKT_THREAD_STACK, cpu=1, TDF_NOSTART)).
2. Mode 2 simulates the exact state a racing wakeup observes during either migration window: atomic_set_int(&td->td_flags, TDF_MIGRATING) followed by lwkt_schedule(td) from cpu0 — i.e. the thread still homed on cpu1, TDF_MIGRATING set by cpu-side code that has not yet completed the migration handshake.
3. The panic fired at the sender-side entry of the inlined _lwkt_schedule() (cpuid = 0, trace through df2728_mode_sysctl → lwkt_schedule). This is the identical inlined check that the IPI-side re-entry (lwkt_schedule_remote → _lwkt_schedule) executes on the old home cpu when a wakeup IPI lands inside the window; the demonstrated hit proves the assert rejects precisely the state the window produces. On a stock kernel, execution continues to _lwkt_enqueue() whose TDF_MIGRATING guard (line 188) skips the enqueue by design.

Console transcript (panic.txt):

df2728: mode2(DF-2729): TDF_MIGRATING window + lwkt_schedule(td2=0xfffff800905fc780) cpu0 -> cpu1
panic: assertion "(td->td_flags & TDF_MIGRATING) == 0" failed in _lwkt_schedule at /usr/src/sys/kern/lwkt_thread.c:1293
cpuid = 0
Trace beginning at frame 0xfffff80118465748
lwkt_schedule() at lwkt_schedule+0x2d9
lwkt_schedule() at lwkt_schedule+0x2d9
df2728_mode_sysctl() at df2728_mode_sysctl+0x11b
userland_sysctl() ... sys___sysctl()
Debugger("panic")

Why the assert is wrong (not the guard)

_lwkt_enqueue()'s three-flag guard at lwkt_thread.c:188 exists for no other reason than to make schedules of TDF_MIGRATING threads safe no-ops while the migration handshake is in flight. If scheduling a migrating thread were truly a protocol violation, the guard would be dead code — it is not: lwkt_setcpu_remote() (lwkt_thread.c:1616-1629) runs on the target cpu and enqueues the thread, while td->td_gd continues to point at the old cpu until td->td_gd = gd executes there, so any in-flight wakeup IPI addressed to the old home cpu re-enters _lwkt_schedule() with the flag set. The assert converts that designed race into a panic on INVARIANTS kernels.

Reachability today

The in-tree usched protocols (dfly pull migration with queue spinlocks, usched_dfly.c:671/792/973; bsd4 lwkt_giveaway/lwkt_acquire pairs) interlock their own scheduling, and tsleep-based wakeups cannot target a runnable migrating thread, so an unprivileged user cannot hit the race on a stock configuration. The assert still forbids a state the core scheduler explicitly supports for any foreign scheduler (e.g. usched implementations in KLDs) or driver-held threads.

Escalation assessment

Not applicable: INVARIANTS-only assert panic; the stock path (guard, no-op) is the designed behavior. No memory corruption: the assert aborts before any queue mutation.

Fix

fix.diff removes the contradicting assert (the enqueue guard already implements the correct semantics). Fix build not performed: Low/INVARIANTS-only finding; the designed no-op semantics are visible in the same source (_lwkt_enqueue line 188) and the control mode of the shared module exercised the non-migrating remote schedule cleanly.