kqsort with caller-supplied es == 0 never terminates (partition loop advances by +0) and performs an out-of-contract 8-byte access at the zero-length array's base (swapcode do-while executes once at n == 0)
Summary
No es==0 guard: every pointer advance is +0 so the partition loop spins forever calling the comparator with pb frozen (5M+ calls demonstrated for n=64), and the swapcode do-while executes its body once even at n==0 - an 8-byte load/store at the base of a zero-length array (SIGSEGV with base in a PROT_NONE page, before any comparator call). n<=6 returns cleanly. All 5 in-tree call sites pass sizeof-constants - hardening. one-line-guard fix validated A/B.
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-5697 Β· 15 files| File | Type | Description | Size | |
|---|---|---|---|---|
| qsort_rig.c | β | 23.9 KB | view raw | |
| qsort_fixed7.c | β | 5.2 KB | view raw | |
| qsort_v.src.c | β | 5.1 KB | view raw | |
| shim/sys/param.h | β | 162 B | view raw | |
| shim/sys/libkern.h | β | 195 B | view raw | |
| build.sh | β | 66 B | view raw | |
| run.sh | β | 90 B | view raw | |
| build.log | β | 557 B | view raw | |
| run.log | β | 2.0 KB | view raw | |
| run_es0.log | β | 1.1 KB | view raw | |
| env.txt | β | 518 B | view raw | |
| fix.diff | β | 274 B | view raw | |
| verdict.json | β | 4.1 KB | view raw | |
| README.md | β | 2.5 KB | β raw | |
| VERDICT.md | β | 3.4 KB | β raw |
DF-5697 β kqsort with es == 0: non-termination plus an out-of-contract 8-byte access at the zero-length array's base
sys/libkern/qsort.c never validates the caller-supplied element size. With
es == 0 and n >= 7:
- Non-termination (the primary defect): every pointer advance is
Β± es == Β±0, so the partition loop (qsort.c:136-159) spins forever calling the comparator withpbfrozen β an unkillable kernel spin in any caller context (DoS). Demonstrated: comparator invoked 5,000,000+ times for n=64 (a correct 64-element sort needs <3000). - Out-of-contract 8-byte access: with es==0 the
n < 7insertion path (qsort.c:113-119) is skipped,pm = a + (n/2)*es == a, andswap(a, pm)(qsort.c:132) callsswapfunc(a, a, 0, 1); theswapcodedo-while (qsort.c:48-57) executes its body once even for n==0, doing an 8-byte load+store ata. For a zero-length array any access is out-of-bounds by definition; demonstrated by pointingainto a PROT_NONE page β SIGSEGV inside kqsort (first access of the run, before any comparator call). On a writable page the self-swap writes back the same value, so alone it is C-level UB rather than corruption β the hang is the practical kernel impact. - n β€ 6 returns immediately (no access) β boundary confirmed.
Severity Info (hardening): no in-tree caller passes es==0 β the census
(ext2_htree.c:587, linux_sort.c:36, vinumio.c:760, ng_ppp.c:1747,
ng_ppp7:2292) shows constant sizeof(...) element sizes everywhere. This is
a contract-violation landmine for future callers of a core kernel primitive.
Evidence
Same rig family as DF-5696 (verbatim source, sha-pinned). This pack builds
the guard-only variant qsort_fixed7.c (if (es == 0) return;) to prove
the guard alone fixes both symptoms.
run.logβ host run (zig cc): hang (rc=7), SIGSEGV (es0_segv), guard-only f7 returns cleanly for n=0..8 (ALL_EXPECTATIONS_MET)run_es0.logβ the same modes run on the DragonFly guest (gcc 8.3) during the DF-5696 sessionenv.txtβ guest environment + pinned sha256fix.diffβ two-line guard, git-apply-able against sys/libkern/qsort.c (also covers kqsort_r via the shared body)
Reproduce
sh build.sh && sh run.sh
Expected decisive lines:
ES0 HANG CONFIRMED: comparator called 5,000,000 times with es=0 β¦ ES0 ACCESS CONFIRMED (verbatim): SIGSEGV inside kqsort with es=0 n=7 β¦ es0_segv(2): child exited st=0 (no fault) OVERALL: ALL_EXPECTATIONS_MET
DF-5697 VERDICT β reproduced (userland rig, host + DragonFly guest)
Bottom line: kqsort/kqsort_r accept es == 0 without any guard. For
n >= 7 the algorithm never terminates (all pointer deltas are multiples of
es == 0; the partition loop at sys/libkern/qsort.c:136-159 spins with pb
frozen), and the very first pivot swap performs an out-of-contract 8-byte
load/store at the (zero-length) array base via the swapcode do-while
executing once at n == 0 (qsort.c:48-57 reached through qsort.c:132 with
pm == a because the n < 7 insertion path at qsort.c:113 is skipped for
n == 7). Both symptoms demonstrated against the verbatim kernel source
compiled in userland; the two-line guard (if (es == 0) return;) eliminates
both. No in-tree caller can pass es == 0 (all five call sites pass
sizeof-constants), so this is hardening for a core kernel primitive β
severity Info, confidence certain.
Demonstrations
- Hang (
es0_hang): n=64, es=0, comparator counts calls and exits 7 at 5,000,000. Guest (gcc 8.3) and host agree:ES0 HANG CONFIRMED β¦ (pb frozen at +0; a correct 64-element sort needs <3000), rc=7. In a kernel caller this is an unkillable spin inside the sort (no comparator progress is possible βpb += esis a no-op), i.e. local DoS if any caller ever computes es from untrusted input. - 8-byte access (
es0_segv): n=7, es=0, base = 16 bytes into a PROT_NONE page. The first memory access of the entire sort is the pivot swap's 8-byte load β child dies SIGSEGV before the first comparator call (comparator count 0), proving the access comes fromswap(a, pm)at qsort.c:132 βswapfunc(a, a, 0, 1)βswapcode(long, β¦, n=0)whose do-while runs its body once (long i = 0; do { β¦ } while (--i > 0);). Honest characterization: on a normal writable page this is a self-swap (same address source and destination β no value corruption) β the defect is the access itself against a zero-length array plus the C-level UB ofdo {} whilewith a zero iteration count. The hang is the practical impact. - Boundary: verbatim es=0 with n β€ 6 returns cleanly with 0 comparator
calls (insertion loop body never runs) β the trigger threshold is exactly
the
n < 7branch at qsort.c:113. - Fix A/B: guard-only variant
qsort_fixed7.c(if (es == 0) return;before theloop:label): returns immediately for n=0..8 (0 comparator calls, no fault βes0_segv_f7child exits 0). The guard alone (without DF-5696's type fix) is sufficient, proving the causality.
Why not a kernel build for fix validation
No in-tree caller passes es==0, so a patched kernel exhibits no observable behavioral delta; the fix is validated A/B on the identical verbatim source (bad behavior present without the guard, gone with it).
Environment
Host: zig cc 18.1.6 (clang) x86_64. Guest: DragonFly 6.5-DEVELOPMENT, gcc 8.3 [DragonFly], clean-source snapshot (es0 runs from the DF-5696 session; guest since reset). Both hosts agree on every symptom.
Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.
Fix verification
fixedA/B on identical verbatim source: without the guard, es=0 n>=7 hangs (5M+ comparator calls) and faults on the 8-byte access (SIGSEGV at the PROT_NONE base); with the 2-line guard only (qsort_fixed7.c), n=0..8 all return immediately with 0 comparator calls and no memory access. A patched-kernel build is not applicable: no in-tree caller can pass es==0, so a kernel build would show no observable delta.
run.log: hang rc=7 and SIGSEGV on verbatim; 'guarded f7 (es==0 guard only) es=0 n=0..8: returned, cmp_calls=0' and 'es0_segv(2): child exited st=0 (no fault)'; fix.diff
Confirmed kernel references
Detail
Evidence (decisive lines)
run.log (host): 'ES0 HANG CONFIRMED: comparator called 5,000,000 times with es=0', 'ES0 ACCESS CONFIRMED (verbatim): SIGSEGV inside kqsort with es=0 n=7', 'es0_segv(2): child exited st=0 (no fault)' for the guard-only fix, ALL_EXPECTATIONS_MET; run_es0.log (guest gcc 8.3, same symptoms); VERDICT.md (mechanism + boundary n<=6 vs n>=7); fix.diff (2-line guard)
PoC changes
Authored from scratch (no seed): fork+waitpid SIGSEGV proof for the 8-byte access (an ASan build was impossible on this host - no system cc and the zig/musl ASan runtime lacks __asan_stack_malloc; the PROT_NONE page proof is equally decisive and runs on the guest's native cc too), comparator-call watchdog exit for the hang proof, guard-only fixed variant qsort_fixed7.c to isolate the fix ingredient from DF-5696's type fix.
Verified recommended fix
Reject the degenerate element size at function entry: 'if (es == 0) return;' before the loop label in the shared qsort.c body (covers kqsort and kqsort_r).
Verdict
Reproduced against the verbatim sys/libkern/qsort.c in the userland rig (host zig cc 18.1.6 AND DragonFly guest gcc 8.3): with caller-supplied es == 0 and n >= 7 the sort never terminates - every pointer advance is a multiple of es == 0, so the partition loop (qsort.c:136-159) spins forever calling the comparator with pb frozen (5,000,000+ comparator calls observed for n=64, which needs <3000) - and the very first pivot swap (qsort.c:132, pm == a because the n<7 insertion path is skipped) drives swapfunc(a, a, 0, 1) whose swapcode do-while (qsort.c:48-57) executes its body once at n == 0, an out-of-contract 8-byte load/store at the zero-length array's base (proven by SIGSEGV with base inside a PROT_NONE page, before any comparator call). n <= 6 returns cleanly; the two-line 'if (es == 0) return;' guard eliminates both symptoms (guard-only variant f7 returns for n=0..8 with 0 comparator calls and no fault). No in-tree caller passes es == 0 (all five call sites use sizeof-constants: ext2_htree.c:587, linux_sort.c:36, vinumio.c:760, ng_ppp.c:1747, ng_ppp7:2292), so this is hardening of a core kernel primitive - severity Info; the demonstrated artifact is a non-terminating kernel-context spin (impact dos) should any future caller derive es from untrusted input.
No comments yet.