Key-size validation is no-op KASSERT invalid keyBits silently yields Nr=0 and runaway loop / OOB read
Summary
rijndaelKeySetupEnc validates keyBits ONLY via KASSERT which in production/release kernels (INVARIANTS undefined) expands to do{}while(0). With invalid keyBits none of three size branches taken function returns Nr=0 leaving rk[4..] uninitialized. rijndael_set_key stores Nr=0 without checking. rijndaelEncrypt computes r=Nr>>1=0 loop if(--r==0)break sets r=-1 never re-reaches 0 so loop runs indefinitely advancing rk+=8 each iteration past ek/dk buffers reading OOB kernel memory until faults unmapped page -> kernel hang/panic. All current in-tree callers pre-validate keyBits so not directly exploitable today but latent DoS landmine. [VERIFIED pass2: reproduced - setkey 0 panics at the sole KASSERT on INVARIANTS, on production kernels returns 0 with Nr=0 and partially-written schedule, encrypt then walks rk +32B per iteration into fatal trap 12 at rijndaelEncrypt+0x1d9 4128 bytes past the 488-byte ctx; all in-tree callers pre-validate so latent KPI landmine; layered validation fix-validated - invalid sizes rejected, 16/16 KATs bit-identical]
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-2368 Β· 28 files| File | Type | Description | Size | |
|---|---|---|---|---|
| README.md | β | 4.6 KB | β raw | |
| VERDICT.md | β | 8.1 KB | β raw | |
| rijndael_harness.c | β | 18.2 KB | view raw | |
| rijndael_twin_tables.h | β | 17.9 KB | view raw | |
| rijnh.c | β | 2.9 KB | view raw | |
| build.sh | β | 168 B | view raw | |
| run_kat.sh | β | 277 B | view raw | |
| run_setkey.sh | β | 497 B | view raw | |
| run_poison.sh | β | 907 B | view raw | |
| run_idx.sh | β | 2.6 KB | view raw | |
| build_baseline.log | β | 5.6 MB | β download | |
| build_fixed.log | β | 5.6 MB | β download | |
| install_baseline.log | β | 80.4 KB | view raw | |
| install_fixed.log | β | 80.4 KB | view raw | |
| kat_baseline_dmesg.txt | β | 1.4 KB | view raw | |
| panic.setkey0.txt | β | 1.3 KB | view raw | |
| panic.poison.txt | β | 1.1 KB | view raw | |
| panic.direct.txt | β | 945 B | view raw | |
| fixrun_dmesg.txt | β | 45.3 KB | view raw | |
| idx_dmesg.txt | β | 42.6 KB | view raw | |
| idx_run.out | β | 10.3 KB | view raw | |
| idx_analysis.txt | β | 659 B | view raw | |
| idx_analysis2.txt | β | 94 B | view raw | |
| fix.diff | β | 2.0 KB | view raw | |
| env.txt | β | 365 B | view raw | |
| ttables.txt | β | 345 B | view raw | |
| verdict.json | β | 4.0 KB | view raw | |
| manifest.json | β | 1.8 KB | view raw |
DF-2368 β rijndael key-size validation is a no-op KASSERT (Nr=0 -> runaway round loop / OOB read)
File: sys/crypto/rijndael/rijndael-alg-fst.c (+ wrapper sys/crypto/rijndael/rijndael-api.c)
Verdict (see VERDICT.md for the full narrative)
REPRODUCED on the stock INVARIANTS guest kernel (impact: panic / latent kernel DoS; confidence: certain). Both halves of the finding were demonstrated through the REAL kernel functions:
panic: Invalid key size (0)βrijndael_set_key(bits=0)reaches the KASSERT atrijndael-alg-fst.c:738, the ONLY validation. On production (non-INVARIANTS) kernels this compiles todo {} while (0)andrijndaelKeySetupEncreturns0at line 814, so the wrapper storesctx->Nr = 0with a partially-written schedule (rk[0..7] written unconditionally, lines 740-743/762-763/784-785).- The Nr=0 consequence: the round loop (
r = Nr >> 1; for(;;){ ... rk += 8; if (--r == 0) break; ... }, lines 951-1007) never terminates (--rruns -1, -2, ...) and walksrkforward 32 bytes per iteration, reading far pastek/dk. Demonstrated: 4128 bytes past the end of the 488-byterijndael_ctx, fatal kernel-mode read fault atrijndaelEncrypt+0x1d9.
NOT reachable by an unprivileged user today: every in-tree caller pre-validates the key size (see VERDICT.md reachability section). This is a latent landmine for any future/unprivileged-fed caller β hence the original Low severity, which this verification confirms.
Fix validated A/B (fix.diff): invalid sizes rejected with zeroed schedules,
Nr=0 encrypts return immediately, and all 16 FIPS-197 / SP800-38A / OpenSSL-derived
KATs remain bit-identical.
Rig
Kernel-compiled harness (runtime kldload GPFs on this guest) driving the REAL
rijndael_set_key / rijndael_encrypt / rijndaelKeySetupEnc through sysctl
kern.rijnh.test. Build/run below; all ops log with the RIJNH: tag (dmesg).
Build
On the guest (as root, from this directory):
./build.sh # installs rig into /usr/src, nativekernel build + install
# (buildkernel fails: "You must buildworld before
# buildkernel" on this snapshot; nativekernel is the
# documented native-tools path, same obj tree)
reboot
Run
./run_kat.sh # control: 16/16 KAT PASS + twin validation ./run_setkey.sh 0 # DEMO A: panic "Invalid key size (0)" ./run_poison.sh poison # DEMO B: fatal trap 12, OOB read walk ./run_poison.sh direct # DEMO B via raw rijndaelEncrypt ./run_idx.sh # DF-2367 side-channel evidence (shared rig)
After a panic the guest sits in DDB: recover with vm.sh reset harness-kernel
(snapshot taken right after the rig kernel was installed).
Fix validation: patch -p1 < fix.diff in /usr/src, rebuild, reboot, re-run the
same commands β see VERDICT.md for the recorded A/B.
Expected output (stock kernel)
run_kat.sh:RIJNH: KAT summary 16 pass 0 fail of 16 vectors,twinval 600 vectors, 0 mismatches, guest stays up.run_setkey.sh 0: kernel panic, ssh dies; serial console showspanic: Invalid key size (0).+ backtrace rijndaelKeySetupEnc <- rijndael_set_key <- rijnh_run_setkey (panic.setkey0.txt).run_poison.sh poison|direct:Fatal trap 12: page fault while in kernel mode,fault virtual address = 0x800475000,Stopped at rijndaelEncrypt+0x1d9: xorl -0x8(%rdi),%ecx(panic.poison.txt, panic.direct.txt).
Expected output (fixed kernel, fix.diff applied)
run_setkey.sh 0|1|127|257: no panic;ctx.Nr=0 ek[0]=00000000 ... dk[0]=00000000(rejected, schedules zeroed).run_poison.sh poison|direct:RIJNH: poison: SURVIVED Nr=0 encrypt; ct=00000000.run_kat.sh:16 pass 0 failβ bit-identical to baseline (fixrun_dmesg.txt vs kat_baseline_dmesg.txt).
Artifacts
- rijndael_harness.c / rijndael_twin_tables.h β kernel rig (twin tables are a verbatim copy of the Te tables from rijndael-alg-fst.c lines 56-386,718-722)
- rijnh.c β userland driver (mmaps the guard-page layout for DEMO B)
- build.sh, run_*.sh β exact commands
- build_baseline.log / build_fixed.log / install_*.log β full kernel build logs
- kat_baseline_dmesg.txt β stock-kernel KAT + twinval control (16/16, 600/600)
- panic.setkey0.txt / panic.poison.txt / panic.direct.txt β serial-console panics
- idx_dmesg.txt / idx_run.out / idx_analysis*.txt β DF-2367 evidence (shared)
- fixrun_dmesg.txt β fixed-kernel A/B log
- fix.diff β git-apply-able fix (validated on guest; never applied to sys/ here)
- env.txt, ttables.txt β guest environment, T-table kernel addresses
DF-2368 VERDICT β no-op keyBits validation -> Nr=0 -> runaway round loop / OOB read
Status: REPRODUCED (impact: panic β kernel DoS landmine; confidence: certain)
Rig: DragonFly 6.5-DEVELOPMENT x86_64, stock X86_64_GENERIC with
options INVARIANTS (config line 56) and device crypto (line 299), QEMU/KVM,
kernel-compiled sysctl harness (runtime kldload GPFs on this guest).
Guest: DragonFly dfbsd 6.5-DEVELOPMENT #1/#2 ... x86_64 (env.txt).
What the finding claims and what was proven
Claim 1 β validation is KASSERT-only (CWE-617)
rijndaelKeySetupEnc (rijndael-alg-fst.c:734-815) validates keyBits only via
KASSERT(keyBits == 128 || keyBits == 192 || keyBits == 256, ...) at lines
738-739. The public kernel KPI wrapper rijndael_set_key
(rijndael-api.c:37-43) passes bits through with no check of its own.
REPRODUCED on the stock INVARIANTS kernel (kernel #1 + rig):
RIJNH: cmd="setkey 0"
RIJNH: setkey: calling real rijndael_set_key(bits=0) now (ctx pre-zeroed)
panic: Invalid key size (0).
rijndaelKeySetupEnc() at rijndaelKeySetupEnc+0x3ed
rijndaelKeySetupEnc() at rijndaelKeySetupEnc+0x3ed
rijndael_set_key() at rijndael_set_key+0x1f
rijnh_run_setkey() ... rijnh_sysctl() ...
Debugger("panic") (panic.setkey0.txt)
The panic IS the assertion firing β proof that the assert is the only gate. On a
production (INVARIANTS-free) build the same call path compiles the KASSERT to
do { } while (0) and falls through: none of the three if (keyBits == ...)
branches is taken (lines 744/764/786) and the function returns 0 at line 814.
The wrapper stores ctx->Nr = 0. Note rk[0..7] are written unconditionally
before the branches (lines 740-743, 762-763, 784-785), so the stored state is a
partially-written schedule with Nr=0. (rijndaelKeySetupDec has the same
KASSERT-only shape and inherits the fix through rijndaelKeySetupEnc.)
Claim 2 β Nr=0 makes the round loop run away and read OOB
rijndaelEncrypt (line 862) compiles the #else /* !FULL_UNROLL */ path
(FULL_UNROLL is defined nowhere in the tree β verified): the loop
r = Nr >> 1; /* line 951: 0 when Nr==0 */
for (;;) {
... 4 lookups x 4 words ... /* reads rk[4..7] */
rk += 8; /* line 978: +32 bytes per iteration */
if (--r == 0) break; /* line 979: r was 0 -> -1, -2, ... */
... reads rk[0..3] ...
}
never breaks (--r starting at 0 goes negative; it would need 2^32-1 decrements
to wrap to 0 again) and walks rk forward 32 bytes per iteration, past the
60-word ek/dk schedules.
REPRODUCED β the rig builds a REAL rijndael_ctx with a valid 128-bit
rijndael_set_key at the end of a mapped (pre-faulted) user page followed by an
unmapped page, forces ctx->Nr = 0 (byte-identical to the state the broken
rijndael_set_key stores), and calls the real rijndael_encrypt
(rijndael-api.c:53-57) β i.e. the full public API, not a synthetic call:
RIJNH: poison: ctx built (Nr was 10), ctx->Nr forced to 0; entering rijndael_encrypt(ctx) now Fatal user address access from kernel mode from rijnh at ffffffff809f1c09 Fatal trap 12: page fault while in kernel mode fault virtual address = 0x800475000 fault code = supervisor read data, page not present Stopped at rijndaelEncrypt+0x1d9: xorl -0x8(%rdi),%ecx (panic.poison.txt)
The faulting VA is 0x1200 (4608) bytes past ek β i.e. the loop executed ~144
iterations and read 4128 bytes past the end of the entire 488-byte ctx
(DragonFly services kernel-mode demand faults on user VA while the surrounding
mapping exists, so the walk crossed the ctx page and stopped at the first truly
non-present page). In the real in-kernel scenario the ctx lives in kernel memory
(wlan_ccmp ctx, cryptoapi session ctx), where the same walk would consume
adjacent kernel objects and only trap β if ever β at an unmapped KVA hole up to
gigabytes later: exactly the claimed kernel hang/panic. Identical result via a
direct rijndaelEncrypt(ek, 0, ...) call (panic.direct.txt), confirming the
runaway is in the core function, not the wrapper.
Control (not a rig bug)
16/16 KAT vectors (FIPS-197 App. C.1-C.3, SP800-38A ECB, and an OpenSSL-derived extended set) PASS through the same real functions with correct encrypt+decrypt round-trip (kat_baseline_dmesg.txt); the instrumented twin used for the DF-2367 evidence matches the real function byte-exact on 600 random vectors.
Reachability honesty (who can actually hit this)
Not reachable from unprivileged userspace today β every in-tree caller pre-validates the key size:
sys/netproto/802_11/wlan_ccmp/ieee80211_crypto_ccmp.c:130-141βccmp_setkeyrejectswk_keylen != 16; keys arrive via privileged ioctls / association paths that fix 128-bit CCMP keys.sys/crypto/cryptoapi/cryptoapi.c:305-317(aes_cbc_setkey) and:399-411(aes_xts_setkeyviaaes_xts_valid_keysize_in_bits, 256/512 only) β the only cryptoapi consumer issys/dev/disk/dm/crypt/dm_target_crypt.c(dm-crypt), whose table configuration is privileged.sys/crypto/rijndael/rijndael-api-fst.c:48-52(rijndael_makeKey) validates.- opencrypto/IPsec do NOT use this rijndael at all (AES-NI/xform path).
So the finding stands as filed: a latent DoS landmine at the kernel KPI boundary (one future caller that forgets to pre-validate turns invalid keyBits into a kernel hang/panic), Low severity, CWE-617. The guest harness reaches it only because the rig is compiled into the kernel for verification.
Privesc assessment (team directive: crypto-core class β close or flag)
CLOSED as non-privesc with reasoning: the runaway loop performs OOB reads only (round-key words are XOR-ed into on-stack cipher state; nothing OOB is written), the walked data never reaches userspace, and the terminal behavior is a fatal trap. No write primitive, no controlled pointer, no info return path β there is no escalation chain from this bug. Impact ceiling: kernel DoS.
Fix validation (A/B, same guest, same rig)
fix.diff (generated against the read-only sys/ tree; applied only inside the
guest /usr/src with patch -p1; git apply --check clean on the host):
rijndaelKeySetupEnc/rijndaelKeySetupDec: runtime check returning-1for keyBits not in {128,192,256} (KASSERT kept for INVARIANTS builds).rijndael_set_key: rejects invalidbitsβ storesNr = 0and zeroesek/dk(no stale schedule reuse) without calling the setup functions.rijndaelEncrypt/rijndaelDecrypt: guardNr != 10 && != 12 && != 14-> immediate return, killing the runaway even for direct callers.
Baseline (kernel #1) vs fixed (kernel #2):
| input / op | stock kernel | fixed kernel |
|---|---|---|
| setkey 0/1/127/257 | panic (assertion) | no panic; Nr=0, ek/dk zeroed |
| setkey 128/192/256 | Nr=10/12/14 | Nr=10/12/14, identical schedules |
| poison (Nr=0 + encrypt) | Fatal trap 12 in rijndaelEncrypt | SURVIVED (guard returns), ct untouched |
| direct rijndaelEncrypt | Fatal trap 12 | SURVIVED |
| KAT 16 vectors | 16/16 PASS | 16/16 PASS, bit-identical ciphertexts |
| twinval | 600/600 match | 300/300 match |
Full log: fixrun_dmesg.txt (fixed), kat_baseline_dmesg.txt (baseline). Bug inputs now rejected/clamped and all FIPS-197 vectors bit-identical β fix objective met.
Notes
- The guest was returned to the pristine
with-srcsnapshot after verification. - PoC sources: rijndael_harness.c (kernel rig), rijnh.c (userland driver); sha256 in verdict.json.
Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.
Fix verification
not_testableConfirmed kernel references
- sys/crypto/rijndael/rijndael-alg-fst.c:734
- sys/crypto/rijndael/rijndael-alg-fst.c:738
- sys/crypto/rijndael/rijndael-alg-fst.c:814
- sys/crypto/rijndael/rijndael-alg-fst.c:862
- sys/crypto/rijndael/rijndael-alg-fst.c:951
- sys/crypto/rijndael/rijndael-alg-fst.c:978
- sys/crypto/rijndael/rijndael-alg-fst.c:979
- sys/crypto/rijndael/rijndael-api.c:37
- sys/crypto/rijndael/rijndael-api.c:41
- sys/crypto/cryptoapi/cryptoapi.c:305
- sys/crypto/cryptoapi/cryptoapi.c:399
- sys/netproto/802_11/wlan_ccmp/ieee80211_crypto_ccmp.c:130
Detail
Evidence (decisive lines)
panic.setkey0.txt (panic: Invalid key size (0) with rijndael_set_key -> rijndaelKeySetupEnc backtrace); panic.poison.txt and panic.direct.txt (Fatal trap 12 page fault while in kernel mode, fault VA 0x800475000 = 4128 bytes past the ctx, Stopped at rijndaelEncrypt+0x1d9 xorl -0x8(%rdi),%ecx); kat_baseline_dmesg.txt (16/16 KAT control); fixrun_dmesg.txt (fixed kernel A/B: setkey 0/1/127/257 rejected, poison/direct SURVIVED, 16/16 KAT identical); fix.diff; build_baseline.log/build_fixed.log
PoC changes
Original filing had no runnable PoC (poc_path null). Rig authored from scratch: kernel-compiled sysctl harness because runtime kldload GPFs on this guest; make buildkernel fails on this snapshot without buildworld so the documented nativekernel path was used; three compile iterations were needed (Werror sequence-point on index logging, kernel __va_list instead of va_list).
Verified recommended fix
Runtime keyBits validation returning -1 in rijndaelKeySetupEnc/Dec plus rejection (Nr=0, zeroed schedules) in rijndael_set_key and an Nr sanity guard in rijndaelEncrypt/rijndaelDecrypt, exactly as in fix.diff
Verdict
Both halves reproduced on the stock INVARIANTS guest through the real kernel API via a kernel-compiled sysctl rig. (1) rijndael_set_key with keyBits outside 128/192/256 panics at the KASSERT in rijndaelKeySetupEnc (rijndael-alg-fst.c:738) - the only validation present; on production INVARIANTS-free kernels that assert compiles out and the function returns 0, so rijndael_set_key stores Nr=0 with a partially written schedule. (2) With the Nr=0 state the rijndaelEncrypt round loop (r = Nr>>1 = 0, break test on --r) never terminates and walked rk forward 32 bytes per iteration, reading 4128 bytes past the end of the 488-byte rijndael_ctx before a fatal kernel-mode read fault inside rijndaelEncrypt+0x1d9 - exactly the claimed OOB runaway. Not unprivileged-reachable today: all in-tree callers pre-validate key size (wlan_ccmp 16 bytes, cryptoapi 128/192/256, rijndael_makeKey), so this is a latent kernel-DoS landmine at the KPI boundary as filed (Low). Privesc closed: the loop performs OOB reads only, no write primitive, no data returned to userspace - no escalation chain. Fix validated A/B on the same guest: invalid sizes rejected with zeroed schedules, Nr=0 encrypts return immediately, and all 16 FIPS-197/SP800-38A KATs bit-identical.
No comments yet.