DF-2677 — SysV shmat() vs IPC_RMID TOCTOU: VM object ref leak → vm_object_terminate2 panic

What

sys_shmat() (sys/kern/sysv_shm.c) holds shm_token, then can block (inside vm_object_hold() and vm_map_find()); LWKT releases the token across blocking points. The attach is only accounted at the very END of the syscall (shmseg->shm_nattch++, sysv_shm.c:~1055). A concurrent shmctl(seg, IPC_RMID, NULL) — serialized only by shm_token — running inside that dropped-token window sees shm_nattch == 0 (sysv_shm.c:~444), calls shm_deallocate_segment() → vm_object_deallocate() → vm_object_terminate(). The in-flight shmat then does vm_object_reference_locked() on the OBJ_DEAD object and maps it, and terminate (unconditional check, not INVARIANTS-gated) panics:

panic: vm_object_terminate2: object with references, ref_count=1

Worse interleavings (RMID after the reference but before nattch++) free the segment while a live mapping exists: the later shmdt decrements shm_nattch of a recycled shmseg slot (attacker re-creates segments via shmget), corrupting the new segment's accounting — bounded kernel-memory corruption; and shm_delete_mapping may double-deallocate a live segment's object.

Trigger

Unprivileged local users (two cooperating processes — or one process with two threads: shmat vs shmctl need no special rights beyond the segment's perms). Reliably reproduced with kern.ipc.shm_use_phys >= 2 (a documented, root-set performance tunable; the kernel's shmget pre-allocation loop holds the object token and massively widens shmat's blocking window). At the default (=1) the window still exists in principle but is far narrower; ~5 min of racing did not reproduce it (see panic.txt matrix).

Contents

Build & run (as any unprivileged user)

cc -O2 -Wall -o shm_rmid_race shm_rmid_race.c
# (root, once, widens the window dramatically:)
sysctl kern.ipc.shm_use_phys=2
su -m nobody -c ./shm_rmid_race

Expected (vulnerable kernel)

Within ~a minute at shm_use_phys=2: console shows panic: vm_object_terminate2: object with references, ref_count=1 with shm_deallocate_segment ← vm_object_deallocate ← sys_shmctl (or sys_shmdt) in the trace; guest dies at ddb.

Expected (patched kernel)

The race runs to completion (round complete, no panic), repeatedly; no terminate2 panic.

DF-2677 VERDICT — sysv_shm shmat vs IPC_RMID TOCTOU (object ref leak → panic)

Status: reproduced (unprivileged trigger; requires kern.ipc.shm_use_phys >= 2 for the reliable window; default-config reachability likely but not reproduced in the time budget). Impact: kernel panic (local DoS), with a credible escalation of the same race into recycled-shmseg accounting corruption (bounded int decrement at fixed offsets) in the unobserved interleavings.

Root cause (path:line)

• sys_shmat() takes shm_token (sys/kern/sysv_shm.c:~361), finds the segment, reserves a shmmap slot, and then executes blocking operations with the token implicitly released by LWKT across blocking points:
• vm_object_hold(shm_handle->shm_object) sysv_shm.c:~1025 (exclusive object hold; blocks whenever the object token is contended — with shm_use_phys >= 2 the shmget pre-allocation loop holds it for the whole loop; at default config any in-flight fault contends it),
• vm_map_find() sysv_shm.c:~1027 (can allocate/block).
• The attach count is incremented only at the very end: shmseg->shm_nattch++ sysv_shm.c:~1055.
• sys_shmctl() IPC_RMID (sysv_shm.c:~440-447), serialized only by shm_token, checks shmseg->shm_nattch <= 0 and, in the dropped-token window, sees 0 while an attach is in flight → shm_deallocate_segment() → vm_object_deallocate() → vm_object_terminate() while the in-flight shmat holds/will hold a reference.
• vm_object_terminate() panics unconditionally when ref_count != 0 after its pip waits (sys/vm/vm_object.c:~853): exactly the observed vm_object_terminate2: object with references, ref_count=1.

Observed twice on the guest (see panic.txt): once via the shmdt path, once via the shmctl path. Both with kern.ipc.shm_use_phys=2 (root-set documented tunable, default 1) and unprivileged trigger processes.

Why the prealloc loop widens the window

shmget_allocate_segment() holds the shm object exclusively for its entire pre-allocation loop (sysv_shm.c:576-590). Every shmat() on that segment blocks at vm_object_hold() for the loop's full duration (hundreds of microseconds to milliseconds per page × thousands of pages), which makes the dropped-token window enormous and the RMID-vs-shmat race trivially winnable. At shm_use_phys=1 (swap pager, no prealloc) the window is only as wide as incidental object-token contention; ~5 minutes of 8- and 16-faulter racing at default config produced no panic — honestly recorded as not reproduced at default within budget.

Threat model

• Attacker: local unprivileged user(s).
• Precondition (reliable): kern.ipc.shm_use_phys >= 2 — a documented performance tunable (loader.conf/sysctl) intended for database hosts; precisely the machines where SysV-shm races matter.
• Effect: kernel panic → system down (both captured traces show the unconditional panic, not an INVARIANTS-only assert).
• Secondary (unobserved but code-derived) interleavings: segment freed under a live mapping → later shmdt decrements shm_nattch of a recycled shmseg (fixed-offset int corruption of a new segment's accounting) and can double-deallocate a live segment's VM object.

Fix validation

fix.diff counts the attach (shmseg->shm_nattch++) before the first potentially-blocking operation, removes the late increment, and unwinds on the vm_map_find() failure path (mirroring shm_delete_mapping()'s deferred-deallocate logic so a RMID that raced a failed attach still frees the segment).

• baseline (stock kernel, sysctl=2): panic within ~100 s (panic.txt).
• patched kernel (DF-2676+DF-2677 fixes, make nativekernel KERNCONF=X86_64_GENERIC): same PoC, sysctl=2, multiple rounds → round complete, no panic, guest stays up. See run_fixed.log.

Severity rationale

Medium: unprivileged local kernel panic (DoS); reliable under a documented non-default tunable, racy at default config. The corruption interleavings would push higher but were not demonstrated.