β¬’ DragonFlyBSD Kernel Audit
← triage Β· dashboard
DF-5697

kqsort with caller-supplied es == 0 never terminates (partition loop advances by +0) and performs an out-of-contract 8-byte access at the zero-length array's base (swapcode do-while executes once at n == 0)

Summary

No es==0 guard: every pointer advance is +0 so the partition loop spins forever calling the comparator with pb frozen (5M+ calls demonstrated for n=64), and the swapcode do-while executes its body once even at n==0 - an 8-byte load/store at the base of a zero-length array (SIGSEGV with base in a PROT_NONE page, before any comparator call). n<=6 returns cleanly. All 5 in-tree call sites pass sizeof-constants - hardening. one-line-guard fix validated A/B.

Discussion (0)

No comments yet.

PoC verification

Evidence pack

findings/poc/DF-5697 Β· 15 files
FileTypeDescriptionSize
qsort_rig.c β€” 23.9 KB view raw
qsort_fixed7.c β€” 5.2 KB view raw
qsort_v.src.c β€” 5.1 KB view raw
shim/sys/param.h β€” 162 B view raw
shim/sys/libkern.h β€” 195 B view raw
build.sh β€” 66 B view raw
run.sh β€” 90 B view raw
build.log β€” 557 B view raw
run.log β€” 2.0 KB view raw
run_es0.log β€” 1.1 KB view raw
env.txt β€” 518 B view raw
fix.diff β€” 274 B view raw
verdict.json β€” 4.1 KB view raw
README.md β€” 2.5 KB ↓ raw
VERDICT.md β€” 3.4 KB ↓ raw

DF-5697 β€” kqsort with es == 0: non-termination plus an out-of-contract 8-byte access at the zero-length array's base

sys/libkern/qsort.c never validates the caller-supplied element size. With es == 0 and n >= 7:

  • Non-termination (the primary defect): every pointer advance is Β± es == Β±0, so the partition loop (qsort.c:136-159) spins forever calling the comparator with pb frozen β€” an unkillable kernel spin in any caller context (DoS). Demonstrated: comparator invoked 5,000,000+ times for n=64 (a correct 64-element sort needs <3000).
  • Out-of-contract 8-byte access: with es==0 the n < 7 insertion path (qsort.c:113-119) is skipped, pm = a + (n/2)*es == a, and swap(a, pm) (qsort.c:132) calls swapfunc(a, a, 0, 1); the swapcode do-while (qsort.c:48-57) executes its body once even for n==0, doing an 8-byte load+store at a. For a zero-length array any access is out-of-bounds by definition; demonstrated by pointing a into a PROT_NONE page β†’ SIGSEGV inside kqsort (first access of the run, before any comparator call). On a writable page the self-swap writes back the same value, so alone it is C-level UB rather than corruption β€” the hang is the practical kernel impact.
  • n ≀ 6 returns immediately (no access) β€” boundary confirmed.

Severity Info (hardening): no in-tree caller passes es==0 β€” the census (ext2_htree.c:587, linux_sort.c:36, vinumio.c:760, ng_ppp.c:1747, ng_ppp7:2292) shows constant sizeof(...) element sizes everywhere. This is a contract-violation landmine for future callers of a core kernel primitive.

Evidence

Same rig family as DF-5696 (verbatim source, sha-pinned). This pack builds the guard-only variant qsort_fixed7.c (if (es == 0) return;) to prove the guard alone fixes both symptoms.

  • run.log β€” host run (zig cc): hang (rc=7), SIGSEGV (es0_segv), guard-only f7 returns cleanly for n=0..8 (ALL_EXPECTATIONS_MET)
  • run_es0.log β€” the same modes run on the DragonFly guest (gcc 8.3) during the DF-5696 session
  • env.txt β€” guest environment + pinned sha256
  • fix.diff β€” two-line guard, git-apply-able against sys/libkern/qsort.c (also covers kqsort_r via the shared body)

Reproduce

sh build.sh && sh run.sh

Expected decisive lines:

ES0 HANG CONFIRMED: comparator called 5,000,000 times with es=0 …
ES0 ACCESS CONFIRMED (verbatim): SIGSEGV inside kqsort with es=0 n=7 …
es0_segv(2): child exited st=0 (no fault)
OVERALL: ALL_EXPECTATIONS_MET
VERDICT.md
↓ download raw

DF-5697 VERDICT β€” reproduced (userland rig, host + DragonFly guest)

Bottom line: kqsort/kqsort_r accept es == 0 without any guard. For n >= 7 the algorithm never terminates (all pointer deltas are multiples of es == 0; the partition loop at sys/libkern/qsort.c:136-159 spins with pb frozen), and the very first pivot swap performs an out-of-contract 8-byte load/store at the (zero-length) array base via the swapcode do-while executing once at n == 0 (qsort.c:48-57 reached through qsort.c:132 with pm == a because the n < 7 insertion path at qsort.c:113 is skipped for n == 7). Both symptoms demonstrated against the verbatim kernel source compiled in userland; the two-line guard (if (es == 0) return;) eliminates both. No in-tree caller can pass es == 0 (all five call sites pass sizeof-constants), so this is hardening for a core kernel primitive β€” severity Info, confidence certain.

Demonstrations

  1. Hang (es0_hang): n=64, es=0, comparator counts calls and exits 7 at 5,000,000. Guest (gcc 8.3) and host agree: ES0 HANG CONFIRMED … (pb frozen at +0; a correct 64-element sort needs <3000), rc=7. In a kernel caller this is an unkillable spin inside the sort (no comparator progress is possible β€” pb += es is a no-op), i.e. local DoS if any caller ever computes es from untrusted input.
  2. 8-byte access (es0_segv): n=7, es=0, base = 16 bytes into a PROT_NONE page. The first memory access of the entire sort is the pivot swap's 8-byte load β†’ child dies SIGSEGV before the first comparator call (comparator count 0), proving the access comes from swap(a, pm) at qsort.c:132 β†’ swapfunc(a, a, 0, 1) β†’ swapcode(long, …, n=0) whose do-while runs its body once (long i = 0; do { … } while (--i > 0);). Honest characterization: on a normal writable page this is a self-swap (same address source and destination β€” no value corruption) β€” the defect is the access itself against a zero-length array plus the C-level UB of do {} while with a zero iteration count. The hang is the practical impact.
  3. Boundary: verbatim es=0 with n ≀ 6 returns cleanly with 0 comparator calls (insertion loop body never runs) β€” the trigger threshold is exactly the n < 7 branch at qsort.c:113.
  4. Fix A/B: guard-only variant qsort_fixed7.c (if (es == 0) return; before the loop: label): returns immediately for n=0..8 (0 comparator calls, no fault β€” es0_segv_f7 child exits 0). The guard alone (without DF-5696's type fix) is sufficient, proving the causality.

Why not a kernel build for fix validation

No in-tree caller passes es==0, so a patched kernel exhibits no observable behavioral delta; the fix is validated A/B on the identical verbatim source (bad behavior present without the guard, gone with it).

Environment

Host: zig cc 18.1.6 (clang) x86_64. Guest: DragonFly 6.5-DEVELOPMENT, gcc 8.3 [DragonFly], clean-source snapshot (es0 runs from the DF-5696 session; guest since reset). Both hosts agree on every symptom.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.

Fix verification

fixed
baseline reproduced→ patch + rebuild →patched clean

A/B on identical verbatim source: without the guard, es=0 n>=7 hangs (5M+ comparator calls) and faults on the 8-byte access (SIGSEGV at the PROT_NONE base); with the 2-line guard only (qsort_fixed7.c), n=0..8 all return immediately with 0 comparator calls and no memory access. A patched-kernel build is not applicable: no in-tree caller can pass es==0, so a kernel build would show no observable delta.

run.log: hang rc=7 and SIGSEGV on verbatim; 'guarded f7 (es==0 guard only) es=0 n=0..8: returned, cmp_calls=0' and 'es0_segv(2): child exited st=0 (no fault)'; fix.diff
↓ fix.diffper-fix-DF-5697

Confirmed kernel references

Detail

Evidence (decisive lines)

run.log (host): 'ES0 HANG CONFIRMED: comparator called 5,000,000 times with es=0', 'ES0 ACCESS CONFIRMED (verbatim): SIGSEGV inside kqsort with es=0 n=7', 'es0_segv(2): child exited st=0 (no fault)' for the guard-only fix, ALL_EXPECTATIONS_MET; run_es0.log (guest gcc 8.3, same symptoms); VERDICT.md (mechanism + boundary n<=6 vs n>=7); fix.diff (2-line guard)

PoC changes

Authored from scratch (no seed): fork+waitpid SIGSEGV proof for the 8-byte access (an ASan build was impossible on this host - no system cc and the zig/musl ASan runtime lacks __asan_stack_malloc; the PROT_NONE page proof is equally decisive and runs on the guest's native cc too), comparator-call watchdog exit for the hang proof, guard-only fixed variant qsort_fixed7.c to isolate the fix ingredient from DF-5696's type fix.

Verified recommended fix

Reject the degenerate element size at function entry: 'if (es == 0) return;' before the loop label in the shared qsort.c body (covers kqsort and kqsort_r).

Verdict

Reproduced against the verbatim sys/libkern/qsort.c in the userland rig (host zig cc 18.1.6 AND DragonFly guest gcc 8.3): with caller-supplied es == 0 and n >= 7 the sort never terminates - every pointer advance is a multiple of es == 0, so the partition loop (qsort.c:136-159) spins forever calling the comparator with pb frozen (5,000,000+ comparator calls observed for n=64, which needs <3000) - and the very first pivot swap (qsort.c:132, pm == a because the n<7 insertion path is skipped) drives swapfunc(a, a, 0, 1) whose swapcode do-while (qsort.c:48-57) executes its body once at n == 0, an out-of-contract 8-byte load/store at the zero-length array's base (proven by SIGSEGV with base inside a PROT_NONE page, before any comparator call). n <= 6 returns cleanly; the two-line 'if (es == 0) return;' guard eliminates both symptoms (guard-only variant f7 returns for n=0..8 with 0 comparator calls and no fault). No in-tree caller passes es == 0 (all five call sites use sizeof-constants: ext2_htree.c:587, linux_sort.c:36, vinumio.c:760, ng_ppp.c:1747, ng_ppp7:2292), so this is hardening of a core kernel primitive - severity Info; the demonstrated artifact is a non-terminating kernel-context spin (impact dos) should any future caller derive es from untrusted input.