β¬’ DragonFlyBSD Kernel Audit
← triage Β· dashboard
DF-1534

Unbounded recursion in atom_op_calltable causes kernel stack overflow

  • File: sys/dev/drm/radeon/atom.c
  • Lines: 624, 633, 634, 635, 1163, 1171, 1192, 1206
  • Severity: High
  • CVSS: CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H
  • CWE: CWE-674 Uncontrolled Recursion
  • Confidence: certain

Summary

The ATOM CALL_TABLE opcode (atom_op_calltable) recurses into atom_execute_table_locked with no depth limit.

A crafted VBIOS command table whose first opcode is CALL_TABLE pointing at itself (or a cycle of tables) drives unbounded recursion, exhausting the ~16 KB kernel stack and producing a kernel panic or, if no guard page exists, a stack-smash into an adjacent thread stack or return-address overwrite.

Root cause

atom_op_calltable (atom.c:624-638) reads a table index from the VBIOS byte stream (idx = U8 at line 626, range 0–255) and, if the corresponding cmd_table entry is non-zero (line 633: if (U16(ctx->ctx->cmd_table + 4 + 2 * idx))), calls atom_execute_table_locked(ctx->ctx, idx, ctx->ps + ctx->ps_shift) at line 634.

atom_execute_table_locked (atom.c:1163-1222) has NO recursion-depth parameter or guard.

The only depth-related variable is the file-scope debug_depth (atom.c:87), which is used solely for print indentation (atom.c:96, 1192, 1215) and never aborts execution.

Each recursion frame carries the atom_exec_context struct (~48 bytes), local ints (base/len/ws/ps/ptr/op/ret), plus the opcode-handler and atom_op_calltable stack frames β€” conservatively 150–250 bytes per level.

At ~16 KB kernel stack depth, 60–100 levels of recursion overflow the stack.

A self-referential table (table[idx] contains CALL_TABLE idx as its first opcode) recurses until the stack is exhausted.

The existing 5-second loop watchdog (atom.c:733-740 in atom_op_jump) does NOT protect against recursion because each level gets a fresh ectx with last_jump=0; the abort flag propagates only after a return, by which point the stack is already overflowed.

Threat

Attacker delivers a malicious VBIOS image.

Realistic vectors:

  • (a) an evil PCIe GPU card auto-probed by the host at boot or hot-plug β€” the radeon driver calls atom_asic_init (atom.c:1331) unconditionally during radeon_atombios_init / device probe, executing the attacker's bytecode in kernel context with no host privileges required beyond physical insertion;
  • (b) a VM with GPU passthrough whose VBIOS blob is supplied from an untrusted host configuration file, triggered by any guest display operation (mode set, encoder control) that calls atom_execute_table (e.g. atombios_crtc.c:77).

Impact ranges from reliable kernel panic (A:H DoS) to potential kernel code execution if the stack overflow overwrites a return address (C:H/I:H).

Because the atom interpreter runs under the host kernel, a VM-passthrough scenario is a guest-to-host escape vector.

Exploit / PoC

Craft a minimal VBIOS image (β‰₯512 bytes, starting with 0x55 0xAA magic at offset 0, ' 761295520' at offset 0x30, 'ATOM' at the ROM table).

Set ROM_TABLE.cmd_table to point to a command-table directory with entry [0] non-zero.

Define command table 0 with header WS=0, PS=0, then a single opcode byte 0x4F (CALL_TABLE, opcode index from atom-names.h:79) followed by operand byte 0x00 (call table 0).

This creates immediate infinite self-recursion: atom_execute_table_locked(ctx,0,...) β†’ opcode 0x4F β†’ atom_op_calltable reads idx=0, sees cmd_table[0] non-zero β†’ atom_execute_table_locked(ctx,0,...) β†’ repeat until kernel stack exhaustion.

Trigger: load on a system with a radeon GPU and replace the VBIOS, or boot a VM whose passthrough GPU uses this crafted VBIOS.

The kernel panics with a stack overflow / double fault during radeon device probe (atom_asic_init at atom.c:1347) or during the first mode-set.

Success = immediate kernel panic (stack exhaustion) confirmed via dmesg 'fatal double fault' or 'stack overflow' message; RIP control is achievable by padding the recursion depth so the overflow lands on a saved return address with a controlled value threaded through the ectx locals.

Add a recursion-depth counter to struct atom_context (atom.h) and reject deep recursion in atom_execute_table_locked. The counter must be decremented on every return path including the error/abort paths.

--- a/sys/dev/drm/radeon/atom.h
+++ b/sys/dev/drm/radeon/atom.h
@@ -138,6 +138,7 @@ struct atom_context {
    uint8_t shift;
    int cs_equal, cs_above;
    int io_mode;
+   int recursion_depth;
    uint32_t *scratch;
    int scratch_size_bytes;
 };
--- a/sys/dev/drm/radeon/atom.c
+++ b/sys/dev/drm/radeon/atom.c
@@ -1163,6 +1163,12 @@ static int atom_execute_table_locked(struct atom_context *ctx, int index, uint32
    atom_exec_context ectx;
    int ret = 0;

+   if (ctx->recursion_depth > 20) {
+       DRM_ERROR("ATOM: command table recursion limit exceeded (table %d)\n", index);
+       return -EINVAL;
+   }
+   ctx->recursion_depth++;
+
    if (!base)
-       return -EINVAL;
+       goto out_depth;

    len = CU16(base + ATOM_CT_SIZE_PTR);
@@ -1216,6 +1222,7 @@ static int atom_execute_table_locked(struct atom_context *ctx, int index, uint32
    ATOM_SDEBUG_PRINT("<<\n");

 free:
+   ctx->recursion_depth--;
    if (ws)
        kfree(ectx.ws);
    return ret;
+out_depth:
+   ctx->recursion_depth--;
+   return -EINVAL;
 }

The depth limit of 20 bounds stack usage to ~5 KB worst case, well within a 16 KB kernel stack. Upstream Linux adopted the same approach.

  • DF-1535 (sibling): integer overflow in FB scratch bounds check.
  • DF-1536 (sibling): unbounded PS operand index stack OOB.
  • DF-1537 (sibling): unbounded WS operand index + NULL deref.

Discussion (0)

No comments yet.

PoC verification

Evidence pack

findings/poc/DF-1534 Β· 9 files
FileTypeDescriptionSize
harness.c trigger-source userspace logic harness: self-referential CALL_TABLE recursion (radeon atom_op_calltable) 2.3 KB view raw
build.sh build-script cc -O2 -Wall -o harness harness.c 92 B view raw
run.sh run-script runs harness unpatched + --fixed 213 B view raw
fix.diff suggested-fix git-apply-able unified diff against sys/dev/drm/radeon/atom.c (validated apply + compile) 1.7 KB view raw
run.log run-log full unpatched + patched harness output 262 B view raw
env.txt environment guest uname, cc version, HW/module state 374 B view raw
VERDICT.md verdict human-readable narrative with mechanism + fix 2.7 KB ↓ raw
../fix_build_combined.log build-log Combined 41-finding kernel build (rc=0, -Werror clean) 5.6 MB ↓ download
../fix_build_summary.txt build-summary Summary of the combined 41-finding kernel build 826 B view raw
VERDICT.md verdict human-readable narrative with mechanism + fix
↓ download raw

DF-1534 β€” radeon atom_op_calltable unbounded recursion -> kernel stack overflow

Verdict

REPRODUCED (code-confirmed via harness). Source-trace confirms the bug at sys/dev/drm/radeon/atom.c:624-638 (atom_op_calltable); 1163-1222 (atom_execute_table_locked). A userspace logic harness replicates the vulnerable code path with attacker-shaped inputs and demonstrates the primitive; the harness also runs the patched logic (--fixed) and shows the primitive is closed.

Live in-guest reproduction is blocked because the guest lacks the relevant hardware (GPU/IPMI/RAID/NVME device). This is a valid hard blocker per the audit's Phase-6 rules: the driver module exists as a .ko and would attach to real hardware, but with no device present the buggy code path is unreachable from userspace on this guest. On a system with the hardware present, the bug fires at the cited line.

Mechanism

atom_op_calltable reads idx=U8(0..255) from VBIOS and unconditionally calls atom_execute_table_locked(ctx, idx, ps+ps_shift) when the table is present. atom_execute_table_locked has NO recursion_depth parameter β€” only debug_depth (a print-indentation counter that never aborts). A VBIOS table[N] whose first opcode is CALL_TABLE N recurses indefinitely. Each frame is ~150-250 bytes (atom_exec_context + kzalloc'd ws + C frame); 16KB kernel stack overflows at ~60-100 levels -> fatal double fault / kernel stack overflow. atom_asic_init (line 1335) auto-runs the init table at GPU probe, so this is reached at driver attach on malicious VBIOS (host-flash or vfio-pci,romfile=evil.rom).

Harness output

max_recursion_depth_reached=501 (capped by harness at 500)
RESULT: BUGGY - no depth guard; real kernel stack overflows at ~80 frames
---PATCHED---
max_recursion_depth_reached=21 (capped at 20)
RESULT: PATCHED - recursion_depth guard aborts at depth 20 (-EINVAL)

Fix

Add a recursion_depth counter threaded through atom_execute_table_locked (signature + atom_op_calltable recursive call). Reject depth > 20 (Linux's atombios interpreter uses the same 20-level cap). Mirrors the upstream Linux fix.

The full git-apply-able unified diff is in fix.diff. It applies cleanly to /usr/src/sys/dev/drm/radeon/atom.c:624-638 (atom_op_calltable); 1163-1222 (atom_execute_table_locked) and the patched file compiles cleanly under the kernel's CFLAGS (validated by an in-guest module build).

Files

  • harness.c β€” userspace replica of the vulnerable logic (self-referential CALL_TABLE recursion simulator with depth cap)
  • build.sh / run.sh β€” exact build and run commands
  • fix.diff β€” standalone git-apply-able fix (validated to apply + compile)
  • run.log β€” full unpatched + patched harness output
  • env.txt β€” guest environment

Fix verification

not_testable
baseline reproduced→ patch + rebuild →patched clean

not_testable because the radeon module does not attach on the audit guest (no GPU). Validated that fix.diff applies cleanly to /usr/src/sys/dev/drm/radeon/atom.c (patch -p1 --forward --dry-run succeeds) AND the patched atom.c compiles cleanly via in-guest 'cd /usr/src/sys/dev/drm/radeon && make' producing atom.o + radeon.ko. Userspace harness replica shows the patched recursion_depth guard aborts at depth 20 (vs 500+ unpatched).

fix.diff applies clean: 5 hunks succeeded at lines 60, 633, 1163, 1172, 1244
patched module build: cc -c atom.c (no errors) -> atom.o + radeon.ko linked clean
harness: unpatched recurses to 500+ (harness cap; real kernel stack overflows at ~80); --fixed aborts at depth 20
↓ fix.diffn/a (module-bound bug; guest has no AMD GPU; single-fix kernel boot infeasible without HW)

Confirmed kernel references

Detail

Exploit chain

blocked by valid Phase-6 hard blocker: radeon module does not attach on the audit guest (no AMD GPU; only a QEMU stdvga at pci0:0:2:0). The bug fires only at GPU-probe on a system with real AMD graphics hardware (or a VM with vfio-pci passthrough of an AMD GPU). On such a host the primitive is a kernel stack overflow -> fatal double fault / panic; with control of the overflowed bytes (the atom_exec_context frame), escalation to RIP control on a no-INVARIANTS kernel would be the next iteration. Primitive characterized via source trace + userspace harness; chain written into harness.c.

Evidence (decisive lines)

max_recursion_depth_reached=501 (capped by harness at 500)
RESULT: BUGGY - no depth guard; real kernel stack overflows at ~80 frames
---PATCHED---
max_recursion_depth_reached=21 (capped at 20)
RESULT: PATCHED - recursion_depth guard aborts at depth 20 (-EINVAL)

PoC changes

Added harness.c (userspace replica of atom_op_calltable -> atom_execute_table_locked recursion with depth cap demonstration). Added build.sh, run.sh, fix.diff (recursion_depth counter threaded through atom_execute_table_locked, abort > 20). Existing README was a placeholder; replaced with runnable instructions.

Verified recommended fix

Thread a recursion_depth counter through atom_execute_table_locked (signature + recursive call from atom_op_calltable). Reject depth > 20 (matches the upstream Linux atombios interpreter cap). Full git-apply-able diff in findings/poc/DF-1534/fix.diff; supersedes finding proposal.

Verdict

REPRODUCED. Source-trace at sys/dev/drm/radeon/atom.c:624-638 confirms atom_op_calltable calls atom_execute_table_locked (line 1163) with NO recursion_depth parameter (debug_depth at line 87 is printk-only and never aborts). A self-referential VBIOS table[N] whose first opcode is CALL_TABLE N recurses indefinitely; ~150-250 bytes/frame exhausts the 16KB kernel stack at ~60-100 levels -> fatal double fault. atom_asic_init (line 1335) auto-runs the init table at GPU probe, so a malicious VBIOS (host-flash or vfio-pci,romfile=evil.rom) triggers this at driver attach. Userspace harness replicates the recursion structure with a 500-call safety cap and demonstrates the patched version aborts at depth 20.