sys_mq_open race-gated NULL-pointer lockmgr panic: the under-lock EMFILE branch jumps to the shared exit label while mq is still NULL, releasing a lock on a NULL-derived address
| Field | Value |
|---|---|
| ID | DF-2782 |
| Status | new |
| Severity | Medium |
| CVSS 3.1 | CVSS:3.1/AV:L/AC:H/PR:L/UI:N/S:U/C:N/I:N/A:H |
| CWE | CWE-476 NULL Pointer Dereference |
| File | sys/kern/sys_mqueue.c |
| Lines | 577-580 (goto with mq NULL), 595-596 (shared release) |
| Area | kern |
| Confidence | certain |
| Discovered | 2026-08-31 |
| Pass | 2 (GLM 5.3 second pass β surfaced during Phase V of this run) |
| Bucket | memcorrupt |
| Reported | pending |
| Known CVE | none |
| CVE match | novel |
Summary
In sys_mq_open(), when the queue is not found and O_CREAT is set, the
under-lock limit check executes error = EMFILE; goto exit; while mq
is still NULL (mq = mq_new happens only after the check). The shared
exit label then runs lockmgr(&mq->mq_mtx, LK_RELEASE) on
&((struct mqueue*)NULL)->mq_mtx (~0x100) β guaranteed page fault.
Reachable whenever p_mqueue_cnt reaches mq_open_max between the early
unlocked check and the under-lock check: arranged with 8
barrier-synchronized threads racing from a count of 511. INVARIANTS-
independent; found during Phase V of this run.
Proof of contest
VERIFIED on the stock guest (findings/poc/DF-2782/limit_race.c): seed
511, barrier-release 8 threads into mq_open(O_CREAT) β Fatal trap 12
... Stopped at lockmgr_release+0x11, guest down (panic.txt). Fix
(guard the shared release with if (mq)) in the pack.
Recommended fix
exit:
- lockmgr(&mq->mq_mtx, LK_RELEASE);
+ if (mq)
+ lockmgr(&mq->mq_mtx, LK_RELEASE);
lockmgr(&mqlist_mtx, LK_RELEASE);
Timeline
- 2026-08-31 Discovered during pass-2 verification of sys_mqueue.c (GLM 5.3); unpriv race panic reproduced same run.
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-2782 Β· 13 files| File | Type | Description | Size | |
|---|---|---|---|---|
| limit_race.c | β | 3.3 KB | view raw | |
| build.sh | β | 53 B | view raw | |
| run.sh | β | 166 B | view raw | |
| build.log | β | 36 B | view raw | |
| run.log | β | 1.1 KB | view raw | |
| run.ssh.log | β | 48 B | view raw | |
| panic.txt | β | 2.1 KB | view raw | |
| fixiter1_context.txt | β | 2.1 KB | view raw | |
| fix_validation.log | β | 129 B | view raw | |
| VERDICT.md | β | 1.9 KB | β raw | |
| verdict.json | β | 2.9 KB | view raw | |
| fix.diff | β | 406 B | view raw | |
| env.txt | β | 119 B | view raw |
VERDICT β DF-2782: sys_mq_open NULL-pointer lockmgr panic (race-gated limit path)
Status: REPRODUCED Β· impact: panic Β· confidence: certain
What was proven on the guest (stock kernel, build #0)
/tmp/limit_race 300 (8 threads, barrier-synchronized from a count of 511):
kernel panicked mid-run β serial console (panic.txt):
Fatal trap 12: page fault while in kernel mode current process = 842 kernel: type 12 trap, code=0 Stopped at lockmgr_release+0x11: movq 0x8(%rdi),%rsi
ssh dropped mid-run (Connection ... closed by remote host),
vm.sh status β down β the honest panic signature.
Root cause path: two (or more) threads pass the early unlocked limit check
at sys_mqueue.c:452 while p_mqueue_cnt == 511; the first completes its
open (count 512); the second reaches the under-lock check at :577, takes
the error = EMFILE; goto exit; branch β but mq is still NULL in this
branch (mq = mqueue_lookup(name) returned NULL; mq = mq_new happens
only at :583, AFTER the check) β and the shared exit label at :595-596
executes lockmgr(&mq->mq_mtx, LK_RELEASE) on a NULL-derived address
(~0x100) β page fault β panic.
Corroboration: the identical crash signature was independently produced when
a rejected first-version DF-2781 fix diff introduced a goto exit with
mq == NULL at the same site (fixiter1_context.txt) β same trap, same
instruction β confirming the exit-path shape is the hazard and that the
stock :577 check has exactly that shape.
Fix validation (patched kernel, build #1 04:57)
fix.diff: guard the shared release (if (mq)) β mq is non-NULL on every
other path reaching the label. Patched kernel: the same PoC ran 300
rounds Γ 8 threads ("round 299 survived ... survived 300 rounds without
the panic"), guest up (fix_validation.log).
fix_status: fixed (fix_baseline_reproduced=1, fix_patched_reproduced=0).
Fix verification
fixedPatched kernel (if (mq) guard): the same PoC ran 300 rounds x 8 threads without a panic ('survived 300 rounds'), guest up.
fix_validation.log
Confirmed kernel references
Detail
Exploit chain
seed p_mqueue_cnt to 511 -> barrier-release N threads into mq_open(O_CREAT) -> >=2 threads pass :452 at 511 -> first increments to 512 -> second hits :577 EMFILE -> goto exit with mq==NULL -> lockmgr on ~&((struct mqueue*)NULL)->mq_mtx (offset ~0x100) -> page fault -> panic. INVARIANTS-independent.
Evidence (decisive lines)
['panic.txt: Fatal trap 12, Stopped at lockmgr_release+0x11, current process 842, db> prompt', 'run.log: ssh dropped mid-run + vm status down', 'fixiter1_context.txt: identical signature via mq==NULL goto-exit', 'fix_validation.log: 300 rounds survived on patched kernel']
PoC changes
none vs plan (unique per-iteration queue names to force the CREATE path, barrier+seed-511 choreography; round cleanup of fds and unlinks).
Verified recommended fix
Guard the shared exit-path release with if (mq) (fix.diff) - mq is non-NULL on every other path reaching the label.
Verdict
Race-gated NULL-pointer lockmgr panic in sys_mq_open proven on the stock kernel: 8 barrier-synchronized threads racing past the early unlocked limit check (sys_mqueue.c:452) from p_mqueue_cnt=511; when the counter reaches 512 in-flight, a thread takes the under-lock EMFILE branch at :577-580 and goto-exits while mq is still NULL (mq=mq_new happens only at :583), so the shared exit label (:595-596) executes lockmgr(&mq->mq_mtx, LK_RELEASE) on a NULL-derived address - kernel panicked: 'Fatal trap 12 ... Stopped at lockmgr_release+0x11', guest down. Independently corroborated: an identical signature was produced by a rejected fix iteration that introduced the same mq==NULL goto-exit shape at the same label (fixiter1_context.txt).
No comments yet.