DF-2688 — OOM kill block operates on bigproc without p_token and without re-validating liveness

File: sys/vm/vm_pageout.c:1812-1838 (vm_pageout_scan_cache), helper vm_pageout_scan_callback at sys/vm/vm_pageout.c:1841-1891.

Status

UNTESTED (Low / speculative impact). Not verified on the QEMU guest: the finding is neither Critical/High nor in the memcorrupt/privesc bucket, and triggering requires full swap exhaustion (swap_pager_full) plus a microsecond-scale race against process exit at the pagedaemon's once-per-second kill cadence — not a deterministic local program. Pack contains a seed trigger for a future verify-mode run.

Build / run (seed, UNVERIFIED)

cc -O2 -o df2688_trigger df2688_trigger.c
./df2688_trigger          # as an unprivileged user, on a system with swap configured

Expected (if the race ever fires)

Kernel panic in dfly_resetpriority (NULL/garbage lp) called from vm_pageout_scan_cache, e.g. NULL-deref at lp->lwp_qcpu (sys/kern/usched_dfly.c:1108), or a torn FIRST_LWP_IN_PROC rb-tree read racing lwp_rb_tree_RB_REMOVE in lwp_exit() (sys/kern/kern_exit.c:773).

Why it is only Low/speculative

Analysis (VERDICT.md) shows the callback's own PHOLD keeps the proc, its master lwp, and the lwp's thread allocated through the race window (non-master lwps are unlinked from p_lwp_tree before being freed, and the master-exit lwp is never unlinked while the proc is held), so the most likely outcome of the missing lock is a benign priority update on an exiting lwp, with a panic only via a torn rb-tree walk. The lock-discipline defect itself is certain by inspection; the exploitable consequence is not demonstrated. See VERDICT.md for the full kill-chain analysis.

DF-2688 VERDICT — OOM kill block TOCTOU / missing p_token

Status: untested. Reproduced: no. Impact ceiling: local DoS (panic), speculative.

The defect (certain, by inspection)

vm_pageout_scan_cache() runs the out-of-swap OOM kill once per second when swap_pager_full && pass > 1 && isep == 0 && avail_shortage > 0 && vm_paging_target1() (sys/vm/vm_pageout.c:1812-1817):

allproc_scan(vm_pageout_scan_callback, &info, 0);        // :1826
if (info.bigproc != NULL) {
        kprintf("Try to kill process %d %s\n", ...);      // :1828  <- can block on serial console
        info.bigproc->p_nice = PRIO_MIN;                  // :1830  <- no p_token
        info.bigproc->p_usched->resetpriority(
                FIRST_LWP_IN_PROC(info.bigproc));         // :1831-1832 <- un-tokened rb-tree walk
        atomic_set_int(&info.bigproc->p_flags, P_LOWMEMKILL);
        killproc(info.bigproc, "out of swap space");      // :1834
        ...
}

The selecting callback (vm_pageout_scan_callback, :1856) explicitly takes lwkt_gettoken(&p->p_token) while examining each proc and filters on p_stat ∈ {SACTIVE, SSTOP, SCORE} (:1862), then takes its own PHOLD (:1883) so the proc survives the scan. But between the callback returning and the kill block executing, no lock is held and p_stat is never re-checked. The kprintf in between can itself block for milliseconds on a serial console, widening the window.

What races: - FIRST_LWP_IN_PROC(p) is RB_FIRST(lwp_rb_tree, &p->p_lwp_tree) (sys/sys/proc.h:413) read without p_token, concurrent with lwp_rb_tree_RB_REMOVE(&p->p_lwp_tree, lp) in lwp_exit() (sys/kern/kern_exit.c:773, under p_token) — a torn traversal is possible. - dfly_resetpriority(lp) (sys/kern/usched_dfly.c:1091) dereferences lp->lwp_qcpu, lp->lwp_proc, and writes lp->lwp_thread->td_upri (:1158) on the returned lwp with no token and no lwp reference.

Compare vm_daemon_callback (:2859-2890), which does the same class of per-proc work strictly under p_token — the kill block is the outlier.

Why the impact collapses to Low/speculative

Traced teardown paths (sys/kern/kern_exit.c):

1. PHOLD(p) from the callback keeps the proc struct allocated and on its list; wait*() cannot complete the reap.
2. Non-master lwps unlink themselves from p_lwp_tree (:773) before being placed on deadlwp_list for async reaping — so at any instant the tree only references allocated lwps; a torn read yields an exiting-but-still-allocated lwp, not freed memory.
3. The master-exit lwp is deliberately left on p_lwp_tree and disposed synchronously by the reaper (comment at :756-760), so under our PHOLD the tree retains a valid lwp and FIRST_LWP_IN_PROC should not return NULL for an ordinary single- or multi-threaded exit.

Residual hazards: (a) a torn rb-tree walk during concurrent removal could return a concurrently-exiting lwp whose lwp_qcpu/scheduler fields are in flux — dfly_resetpriority's remote-cpu spinlock loop (:1103-1113) then operates on a moving target; (b) edge cases where the tree is empty ("UNDEAD" state mentioned at sys/kern/kern_exit.c:156-158) would give resetpriority(NULL) → NULL deref → panic. Neither was demonstrated. killproc→ksignal is self-protecting ("Don't try to deliver a generic signal to an exiting process", sys/kern/kern_sig.c lwpsignal prologue).

Preconditions an attacker would need: fill all swap + memory (swap_pager_full) — feasible for an unprivileged user only by exhausting system swap; then win a µs-scale race against the once-per-second kill window. Realistic worst case: kernel panic (local DoS) under swap exhaustion. No path to controlled memory corruption was found because the concurrent-teardown objects stay allocated under PHOLD.

Fix

Retake p_token, revalidate p_stat, and NULL-check the lwp before touching it — see fix.diff (authored against the read-only sys/ tree; not applied anywhere).

Decision not to run on the guest

• Not Critical/High, not memcorrupt/privesc bucket → outside the mandatory verification set; the trigger is a timing race needing total swap exhaustion on the single-tenant guest, with a real chance of wedging it for downstream users (OOM killer firing at sshd/getty).
• Left the QEMU guest untouched (guest_dirty: 0, guest was up and clean).