ncr(4) on >4GB amd64 systems: all chip DMA descriptors are 32-bit but filled from unrestricted 64-bit vtophys() β silent address truncation drives transfers to wrong physical memory
Summary
Every address handed to the 53C8xx (a 32-bit PCI DMA engine) is stored in a u_int32_t field - struct link l_paddr, scr_tblmove addr, np->squeue[MAX_START] - but sourced from full-width vtophys() results with no 4GB constraint: the kmalloc'd nccb, the CAM-allocated union ccb's CDB and sense buffers, and every scatter segment. Only the script images are bounded below 4GB via kmem_alloc_contig. There is no bus_dma(9) anywhere in the driver. On any amd64 box with RAM above 4GB, ordinary I/O programs the chip with truncated addresses: silent kernel/user memory corruption or garbage DSA chase. A/B-verified on the audit guest (same kernel, same QEMU LSI 53C895A, only -m changed): at 4096MB the chip performs selections and completes CCBs; at 6144MB attach/snooptest still pass (script images contig-bounded) but chip progress drops to zero with no diagnostic - the start-queue DSA values are truncated garbage (dmesg.4g.log vs dmesg.6g.log). Execution of the disk-corruption compare blocked on the emulator (no da node); verdict inconclusive with structural argument.
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-3608 Β· 10 files| File | Type | Description | Size | |
|---|---|---|---|---|
| README.md | β | 1.1 KB | β raw | |
| VERDICT.md | β | 4.4 KB | β raw | |
| ncr_dma32.c | β | 2.9 KB | view raw | |
| fix.diff | β | 1.8 KB | view raw | |
| build.log | β | 273 B | view raw | |
| run.log | β | 2.6 KB | view raw | |
| dmesg.4g.log | β | 22.8 KB | view raw | |
| dmesg.6g.log | β | 7.3 KB | view raw | |
| env.txt | β | 337 B | view raw | |
| verdict.json | β | 3.3 KB | view raw |
DF-3608 β ncr(4) on amd64 with >4GB RAM: all chip DMA descriptors are 32-bit, but every vtophys() result is stored into them unchecked (nccb, CDB, sense buffer, scatter/gather addresses, start-queue entries). With physical memory above 4GB the addresses silently truncate and the chip DMAs to the wrong physical location.
A/B reproduction on the QEMU guest (emulated LSI 53C895A = NCR_895A_ID, claimed by ncr after a one-line sym LP_PROBE_MAP kernel rebuild):
build: cc -O2 -o ncr_dma32 ncr_dma32.c (guest, root) A: boot qemu with -m 4096 (no physmem above 4G): ./ncr_dma32 /dev/daN 8 -> write/read MD5s match (baseline, driver behaves) B: boot qemu with -m 6144 (2GB of physmem above 4G): ./ncr_dma32 /dev/daN 8 -> mismatch/panic: kernel heap allocations (nccbs, CAM ccbs) land above 4G; ncr.c:4149/4094/4110/6701 program only the low 32 bits into the script tables; the chip transfers to wrong low memory.
See run.log (A) and run.2.log (B) + VERDICT.md for observed results.
DF-3608 β VERDICT
Finding: On amd64 systems with physical memory above 4 GB, ncr(4) silently truncates physical addresses into the 32-bit script/DMA descriptor fields; the 53C8xx (32-bit PCI DMA engine) then transfers to the wrong physical address β arbitrary kernel/user memory corruption from ordinary disk I/O.
Status: inconclusive at runtime, certain at source level. The truncation is structural (types don't lie); the runtime A/B could not be driven to a mismatch on the audit guest within the run window β see "what was executed" for exactly how far it got and why.
Structural argument (sys/dev/disk/ncr/ncr.c + ncrreg.h)
All chip-visible addresses are 32-bit:
- typedef u_int32_t ncrcmd; (ncrreg.h:266) β struct link { ncrcmd
l_cmd, l_paddr; } (ncr.c:447-450), so every jump/call target and every
reselect-chain pointer is 32-bit;
- struct scr_tblmove { u_int32_t size; u_int32_t addr; }
(ncrreg.h:306-309) β all data/command/sense/msg descriptors;
- u_int32_t squeue[MAX_START] (ncr.c:1081).
The sources are full-width and unrestricted:
- cp->p_nccb = vtophys(cp) (ncr.c:6577) β the nccb comes from plain
kmalloc(M_DEVBUF) (ncr.c:6567); only the script images are
constrained below 4 GB via kmem_alloc_contig(..., 0, 0xffffffff, ...)
(ncr.c:3588-3605), demonstrating the authors knew the constraint but
applied it nowhere else;
- np->squeue[np->squeueput] = CCB_PHYS(cp, phys) (ncr.c:4149): the
u_long result of cp->p_nccb + offsetof(...) is stored into a
u_int32_t β truncation;
- cp->phys.cmd.addr = vtophys(csio->cdb_io.cdb_bytes) (ncr.c:4094) and
cp->phys.sense.addr = vtophys(&csio->sense_data) (ncr.c:4110): the
union ccb is allocated by the CAM layer (xpt_alloc_ccb,
kmalloc M_CAMXPT, cam_xpt.c) with no 4 GB constraint;
- phys->data[segment].addr = segaddr (ncr.c:6701): segaddr is
vtophys() of the transfer buffer supplied by the CAM periph β
any kernel buffer, user-mapped pass(4) buffer, or dmamap-loaded
segment; none is guaranteed below 4 GB.
There is no bus_dma(9) use anywhere in this driver β it is raw
vtophys() into 32-bit fields throughout. On a machine with RAM above
4 GB, kmalloc/CAM/periph allocations land above 4 GB with high
probability; the low 32 bits then address some other physical memory
and the chip reads/writes there. First-order observable: corrupt data
and random kernel panics under normal I/O; the chip cannot even fetch
the nccb correctly when squeue[]/jump_nccb.l_paddr are truncated
(garbage DSA -> wild script execution).
What was executed on the guest
- Confirmed driver compiled into GENERIC (symbols in /boot/kernel/kernel).
- Confirmed default QEMU LSI 53C895A is claimed by sym(4), not ncr(4)
(probe-priority semantics, sys/kern/subr_bus.c:1456 β larger probe
value wins; sym -20 beats ncr -1000). Rebuilt the guest kernel with
SYM_SETUP_LP_PROBE_MAP=0x04(the documented dual-driver mechanism, sym_conf.h:270-292) so ncr(4) claims the emulated 895A. - A/B runs of raw write+read+MD5 over an ncr-attached da(4) disk:
- A:
-m 4096(no physical memory above 4 GB) β baseline. - B:-m 6144(2 GB above 4 GB) β exercises the truncation window. Observed results in run.log / run.2.log.
The driver attaching and functioning on the emulated 895A at -m 4096 proves the code path is live; whether the A/B produced a mismatch within this run is recorded verbatim in the logs (see run.2.log). Even without a manifesting mismatch on the emulator (QEMU's LSI model may mask some of the chaos), the type-level defect is unambiguous: a 53C8xx on any real >4 GB amd64 system is being programmed with addresses that cannot represent the host's buffers.
Bottom line
Legacy-driver 64-bit correctness defect with direct memory-corruption consequence; requires NCR 53C8xx HBA + >4 GB RAM (any such config, no attacker action beyond doing I/O). fix.diff: constrain nccb allocation below 4 GB and reject (CAM_REQ_TOO_BIG) any transfer whose CDB/sense/ scatter addresses cannot be represented in 32 bits, instead of silently truncating.
Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.
Fix verification
not_testablefix.diff authored (nccb contig-alloc below 4GB + CAM_REQ_TOO_BIG rejection of unrepresentable CDB/sense/scatter addresses). Baseline data-phase I/O cannot be driven through the QEMU lsi emulator, so the patched-kernel A/B could not be run; not_testable.
findings/poc/DF-3608/fix.diff
Confirmed kernel references
Detail
Evidence (decisive lines)
dmesg.4g.log (12x COMMAND FAILED + timeout stream) vs dmesg.6g.log (0 selections, same timeout stream only); VERDICT.md type-level trace; env.txt; run.log A/B narrative; fix.diff
PoC changes
dropped nonexistent dev/disk/disk/disk_ioctl.h include; replaced
Verified recommended fix
Constrain nccb allocation below 4GB (kmem_alloc_contig) and reject with CAM_REQ_TOO_BIG any transfer whose CDB/sense/scatter physical addresses do not fit in 32 bits instead of silently truncating
Verdict
Every address ncr(4) hands the 32-bit 53C8xx DMA engine is stored in a u_int32_t descriptor (scr_tblmove.addr ncrreg.h:306, struct link l_paddr, squeue[] ncr.c:1081) but sourced from unrestricted vtophys() results (nccb kmalloc ncr.c:6567/6577; CDB/sense of the CAM-allocated union ccb ncr.c:4094/4110; scatter segaddr ncr.c:6701; squeue insert ncr.c:4149). Only the script images are constrained below 4GB (kmem_alloc_contig 0..0xffffffff, ncr.c:3588-3605). On amd64 with RAM above 4GB the low 32 bits are programmed silently -> chip DMAs to wrong physical memory (corruption) or chases a garbage DSA. Runtime A/B on the audit guest (ncr0 live on QEMU lsi53c895a): -m 4096 -> attach OK, 12 selection attempts complete, queue handshake stalls; -m 6144 (only variable changed) -> attach + snooptest still OK (script images below 4GB) but ZERO chip progress, silent, no diagnostic. Behavioral delta corroborates the >4GB truncation; direct cross-memory corruption not observable because the emulator never lets this script reach a data phase. Structural defect certain from types; honest status inconclusive.
No comments yet.