DF-4198 — get_bytes_per_element() returns 0 for dm_444_8/dm_mono_8 and unhandled formats → integer divide-by-zero (#DE) in get_surf_rq_param + dml_log2(0)=-127 UB-shift family

File under audit: sys/dev/drm/amd/display/dc/dml/dml1_display_rq_dlg_calc.c (pass 2)

What

get_bytes_per_element() (dml1_display_rq_dlg_calc.c:40-65) handles only dm_444_16/32/64 and dm_420_8/10. The legal enum value dm_444_8 (= dm_mono_8, display_mode_enums.h:42-43) — and any out-of-set enum — yields ret_val = 0. With bytes_per_element == 0 the file then:

• live #DE: get_surf_rq_param() line 641, linear surfaces: blk256_width = 256 / bytes_per_element; → integer division by zero → kernel #DE → panic (SIGFPE in the model harness).
• dead-code #DE (source-level): line 745 meta_blk_width = meta_blk_bytes * 256 / bytes_per_element / meta_blk_height; — meta_blk_width is never read, so gcc -O2 eliminates it (harness confirmed: tiled mono8 does NOT trap), but the source division is still against the same zero.
• UB shift family: dml_log2(0) returns -127 (dcn_bw_log bit-trick, calcs/dcn_calc_math.c:112-126: log2f-mantissa approximation of 0.0 gives -127), so every downstream 1 << log2_* / >> log2_* (lines 636/652-653/678/684/708-709/719/736-739/739) and log2_xxx arithmetic gets -127-based garbage; even non-linear mono8 silently programs wrong RQ/DLG registers (harness: sw=1919 h=2, stored_swath=0 — nonsense for a 1-byte format).

Reachability (honest)

The only in-tree caller of dml1_rq_dlg_get_rq_params is dcn_bw_calc_rq_dlg_ttu() (calcs/dcn_calcs.c:487), and its format mapping (calcs/dcn_calcs.c:353-377) maps VIDEO_420→dm_420_8/10, ARGB16161616→dm_444_64, everything else→dm_444_32 — it never produces dm_444_8 today. The defect is therefore latent*: one new format mapping (or a DCN1 caller passing dm_mono_8, which the enum explicitly defines as an alias) turns it into a kernel panic on modeset validation. Additionally all of it sits behind DCN1 (Raven) hardware. Arithmetic certainty, trigger currently absent — hence Low severity.

How it is proven here

dml1_harness.c compiles the REAL, unmodified dml1_display_rq_dlg_calc.c (plus real dcn_calc_math.c, dml_common_defs.c, real headers; only dm_services.h is a stub providing the non-fatal WARN_ON-style ASSERT, matching os_types.h:76 release semantics) on the DragonFly guest and drives it with a realistic DCN1 config (1920x1080, det_buffer 164 KB, vmm_page 4096):

$ ./dml1_harness normal        # dm_444_32 (what dcn_calcs maps) — OK, sane regs
$ ./dml1_harness mono8         # dm_444_8, tiled — NO trap (line 745 is dead
                               #   code at -O2) but silently WRONG values
$ ./dml1_harness mono8_linear  # dm_444_8 + dm_sw_linear — SIGFPE at :641
Floating point exception (core dumped)   rc=136   (3/3 runs)

dml1_harness_fixed is the SAME harness compiled with fix.diff applied to the real dml1 file: all three modes complete rc=0 and mono8 now produces sane values (bpe=1).

Reproduce

./build.sh          # copies the real files from /usr/src, builds baseline+fixed
./run.sh            # baseline mono8_linear -> SIGFPE; fixed -> clean

Fix

fix.diff: (1) handle dm_444_8 → 1 byte/element in get_bytes_per_element (mono 8bpc is 1 byte per element, matching the enum alias); (2) defense-in-depth: bail out of get_surf_rq_param when bytes_per_element == 0 (ASSERT is only WARN_ON in release builds, so the explicit return is what prevents the fault).

DF-4198 VERDICT

Status: inconclusive (hardware-gated: guest has no AMD/DCN GPU, and the in-tree caller never passes dm_444_8 today) — the divide-by-zero itself is proven EXACTLY on the DragonFly guest by compiling and running the REAL, unmodified dml1_display_rq_dlg_calc.c with crafted inputs; the fault is a SIGFPE (userland #DE analog). Impact class if reached in-kernel: integer divide exception → kernel panic (DoS); also silent wrong register programming for non-linear unhandled formats. Not memory corruption; uid=0 chain N/A.

Root cause (path:line)

• sys/dev/drm/amd/display/dc/dml/dml1_display_rq_dlg_calc.c:40-65 — get_bytes_per_element() has no case for dm_444_8/dm_mono_8 (display_mode_enums.h:42-43) nor for out-of-set values → returns 0.
• :641 — blk256_width = 256 / bytes_per_element; (surf_linear branch) → live integer division by zero (blk256_width is read at :652/:676/:684/ :688-695, so gcc -O2 cannot remove it — confirmed: harness traps).
• :745 — meta_blk_width = meta_blk_bytes * 256 / bytes_per_element / meta_blk_height; — second zero divisor in source; dead code at -O2 (meta_blk_width never read → eliminated; harness proves tiled mono8 does NOT trap), and meta_blk_height = blk256_height * 64 is 0 as well for bpe=0 (get_blk256_size:77-95 leaves 0 for unknown bpe).
• dml_log2(0) == -127 (calcs/dcn_calc_math.c:112-126 bit-trick) → log2_bytes_per_element = -127 (:636) poisons 1 << … / >> … expressions (UB shifts; x86 masks the count, producing garbage rather than a fault) — this is why even the non-crashing paths compute nonsense (harness mono8: sw=1919 h=2 stored_swath=0).

Reachability analysis (what keeps this Low)

1. Sole caller: dcn_bw_calc_rq_dlg_ttu() at calcs/dcn_calcs.c:487, fed by pipe_ctx_to_e2e_pipe_params() whose format switch (calcs/dcn_calcs.c:353) maps VIDEO_420_→dm_420_8/10, ARGB16161616→dm_444_64, default→dm_444_32. dm_444_8 is never produced today; the enum alias dm_mono_8 exists precisely because the format class is part of the DML input vocabulary, so any future mapping addition (upstream DCN reworks, new callers) arms the panic.
2. Everything sits behind DCN1 (Raven) hardware: dcn_calcs runs during dcn10_validate_bandwidth; no AMD GPU can be attached to this guest (ls /dev/dri → ENOENT).

Trigger when armed: modeset validation with an 8-bit mono/444 plane in linear layout → #DE in task context → panic.

Reproduction (guest, DragonFly 6.5-DEVELOPMENT, cc 8.3, -O2)

Baseline (real, unmodified file from sys/):

$ ./dml1_harness normal          # source_format=dm_444_32
OK: rq_l sw=1928 h=8 dpte_row_h=128 meta_row_h=64 ... stored_swath=123392 (assert-warns=0 dtrace=13)   rc=0
$ ./dml1_harness mono8           # dm_444_8, tiled — no trap (dead :745), wrong values
OK: rq_l sw=1919 h=2 dpte_row_h=32 ... stored_swath=0 ...    rc=0
$ ./dml1_harness mono8_linear    # dm_444_8 + dm_sw_linear — LIVE :641 division
Floating point exception (core dumped)    rc=136        (3/3 runs, run.2.log)

Patched (fix.diff applied to the same real file, dml1_display_rq_dlg_calc.fixed.c):

$ ./dml1_harness_fixed normal        rc=0  (identical regs to baseline normal)
$ ./dml1_harness_fixed mono8         rc=0  sw=1936 h=16 dpte_row_h=256 ... stored_swath=61952  (sane 1-byte-format values)
$ ./dml1_harness_fixed mono8_linear  rc=0  sw=2304 h=1 dpte_row_h=128 ... stored_swath=4608

The fixed binary's mono8* outputs are plausible for bpe=1 (swath width doubles vs bpe=4? no — narrows in bytes; the printed sanity: h=16/h=1 are power-of-two swath heights as required).

Killed negatives (dce_calcs checklist applied to this file)

• Divisions by mode/socBB clocks: t_calc_us = 24/min_dcfclk_mhz (:1146), line_time_in_us = htotal / pclk (:1206), delay ratios (:1231-1232), line_wait / line_time (:1293), prefetch_bw = ... / t_pre_us (:1392), flip_bw / total_flip_bytes (:1393-1394) — all double divisions; a zero/NaN operand yields inf/NaN → (unsigned int) cast → 0x80000000-style garbage (cvttsd2si), NOT a trap. No #DE from the double sites.
• / vratio_l/c (:1546/:1550), / dpte_groups_per_row_ub (:1518/:1523), / meta_chunks_per_row_ub (:1528), hratios_cur0 (:1840) — doubles, same as above.
• data_pitch divisions (:505/:832): data_pitch comes from pipe->plane_res.scl_data.viewport.width (calcs/dcn_calcs.c:285), i.e. a validated non-degenerate plane viewport, not a raw UABI pitch; zero would require a 0-width visible plane, which atomic validation rejects — speculative only, not filed.
• Array indexing: the file contains no arrays — struct walks only; no OOB writes (tq/surf structs are fixed-size members).
• u32 fixed-point accumulation overflow (:305/:331/:727/:1366-1383): bounded by validated mode dimensions (viewport ≤ 16k, bpe ≤ 8, lsw ≤ ~64) — max ~10^8, no overflow.
• meta_blk_width dead store + dml_round_to_multiple(...)/64.0 truncation (:261-266): logic quirks upstream-inherited, no fault.

Fix validation

fix.diff (sys/dev/drm/amd/display/dc/dml/dml1_display_rq_dlg_calc.c): adds the missing dm_444_8→1 mapping and an explicit bpe==0 bail-out in get_surf_rq_param (ASSERT alone is WARN_ON in release builds, os_types.h:76). Validated end-to-end on the REAL source file in the harness: baseline mono8_linear SIGFPE 3/3 vs fixed 0/3 with sane outputs. Kernel-level build is not possible on this guest (no AMD hardware to load amdgpu DC; building the amdgpu module is possible but cannot be exercised), so fix_status: inconclusive — semantic validation is at real-source model level.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.