DF-5756 — asymmetric unpaired-surrogate validation in iconv_ucs: lone U+DC00–DFFF accepted and re-encoded as CESU-8 (invalid UTF-8)

File audited (pass 2): sys/libiconv/iconv_ucs.c Reserved ID: DF-5756 Guest: DragonFly 6.5-DEVELOPMENT x86_64 (see logs/env.txt)

Root cause (one line)

In the UTF-16→UTF-8 direction iconv_ucs_conv() only recognizes the high-surrogate prefix on the leading unit (sys/libiconv/iconv_ucs.c:255, (*q & 0xfc) == 0xd8); a lone low surrogate (U+DC00–U+DFFF) in leading position falls to the plain 2-byte branch (:264-266) and the second half (:293-312) has no surrogate check at all before ucs4_to_utf8() (:481-522), which happily emits a 3-byte sequence (w=3 for anything < 0x10000, :495-497) — i.e. CESU-8: "\xDC\x00" → ED B0 80. The UTF-8 input direction, by contrast, rejects surrogates at :208.

Threat / reachability

• Any kernel consumer converting attacker-controlled UTF-16 with a ≥3-byte output buffer receives invalid UTF-8 that many downstream parsers reject (or worse, pair up into WTF-8/CESU-8 confusion). In-tree e2e: a crafted FAT image whose LFN contains a lone DC00 unit reaches win2unixchr() (msdosfs_conv.c:946) — its 2-byte staging buffer makes ucs4_to_utf8 fail the dstlen < w check (iconv_ucs.c:505-506), so msdosfs silently substitutes ? instead of rejecting invalid Unicode (userland iconv(3) returns EILSEQ — see logs/difftest.log).
• The raw CESU-8 emission (out buffer ≥3) is proven directly by the cesu_witness kernel module (also used by DF-5755) through the exact converter pair a KICONV mount opens.

Evidence (baseline)

• witness.log (kernel): lowsur "\xDC\x00" → ret=0 out=ed b0 80 (CESU-8 emitted as a successful conversion), while highsur-lone "\xD8\x3D" → ret=-1 — the asymmetry in one output.
• run.log (crafted image): LFN unit patched to U+DC00 → dirent abcd?fghij.k (3f substitution); unit patched to U+D83D → ?xyzabcd.k.
• logs/difftest.log: userland iconv(3) UTF-16BE→UTF-8 on "\xDC\x00" → EILSEQ.

Files

Build / Run

cc -O0 -g -Wall -I/usr/src/sys -o kread5756 kread5756.c
cc -O0 -g -Wall -o craft5756 craft5756.c
sh run.sh

Expected output (baseline)

• witness: witness[lowsur]: convchr ret=0 … out=ed b0 80
• dirents: abcd?fghij.k and ?xyzabcd.k

On the patched kernel (fix.diff)

• witness: witness[lowsur]: convchr ret=-1 … out=aa aa aa aa (rejected, nothing emitted)
• valid surrogate pair (D8 3D DE 48) still converts: f0 9f 99 88
• msdosfs-visible behavior unchanged by design (both accept-then-fail and reject-at-input surface as ? in win2unixchr); the converter itself now refuses invalid Unicode instead of laundering it.

DF-5756 — VERDICT

Status: reproduced (certain). Fix authored and validated at the converter level (fix_status: fixed).

Reproduced manifestations (baseline)

Honest scope

The 3-byte CESU-8 bytes themselves do not escape through msdosfs (the per-character staging buffer is 2 bytes, so the emission fails the dstlen < w check at iconv_ucs.c:505-506 and msdosfs substitutes ?). They do escape through any consumer with a ≥3-byte output buffer calling the same kernel converter — proven in-kernel by cesu_witness.ko via the exported iconv_open/iconv_convchr API. The msdosfs-visible defect is the silent acceptance (no EILSEQ-equivalent) of unpaired surrogates. Severity Low: invalid-Unicode laundering / parser-confusion input, no memory corruption, no info leak.

Fix validation (fix.diff: leading-DC rejection at the input stage +

surrogate check before UTF-8 emission)

Applied stacked with DF-5755's fix; libiconv.ko rebuilt (-Werror clean), reloaded before msdos_iconv.ko:

fix_status: fixed (converter-level behavior delta decisive via witness; no regression on valid pairs).

PoC notes

• craft5756.c initially searched for the full UTF-16LE name across the LFN entry — LFN entries split names at wePart1/wePart2 with 3 metadata bytes between, so the search was fixed to per-part patterns with an attr==0x0f cross-check.
• PoC fixes vs seed: none applicable (authored fresh); msdosfs mount dirmask and DF dirent (_DIRENT_DIRSIZ) quirks as in DF-5755.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.