Bios submitted to a wedged IDA controller park forever β unkillable D-state issuers and unbounded bio_queue (DF-5026 pattern)
Summary
The only route to biodone is a controller completion; no watchdog/offline flag/timeout anywhere in the driver - a controller that stops completing parks every bio forever (unkillable D-state issuers, unbounded queue growth). Hardware-fault-triggered (vs ips DF-5026 config-reachable). 1Hz watchdog + fail-queued fix.
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-5048 Β· 6 files| File | Type | Description | Size | |
|---|---|---|---|---|
| README.md | β | 2.8 KB | β raw | |
| VERDICT.md | β | 3.1 KB | β raw | |
| env.txt | β | 1.0 KB | view raw | |
| fix.diff | β | 2.3 KB | view raw | |
| verdict.json | β | 3.3 KB | view raw | |
| manifest.json | β | 1.0 KB | view raw |
DF-5048 β bios submitted to a wedged IDA controller park forever: unkillable D-state issuers, unbounded bio_queue (DF-5026 pattern)
File: sys/dev/raid/ida/ida_disk.c (pass 2, GLM 5.3)
Bucket: base:dev | Severity: Low | Pattern: DF-5026 (ips parked bios)
What the bug is
idad_strategy() (sys/dev/raid/ida/ida_disk.c:144-189) validates
softc/write-protect/zero-length locally and then hands the bio to the
controller with no failure path of its own:
bio->bio_driver_info = drv;
crit_enter();
devstat_start_transaction(&drv->stats);
ida_submit_buf(drv->controller, bio); /* ida_disk.c:175 */
crit_exit();
ida_submit_buf() (sys/dev/raid/ida/ida.c:364-370) sorts the bio onto
ida->bio_queue and calls ida_construct_qcb + ida_start. There is no
route to biodone() except a controller completion:
ida_construct_qcbsilently returns when QCBs are exhausted (ida.c:385-387) β the bio stays parked inbio_queue;ida_startstops when the FIFO is full (ida.c:429-431);ida_doneis the only reaper (ida.c:498-537), and it runs only fromida_intr(completions) orida_wait(internal commands);- the driver has no watchdog, no timeout, no offline/fail state, and no detach-time flush β nothing in the driver ever fails a parked bio.
If the controller wedges (stops delivering completions β hardware fault,
PCIe error not surfaced, box being torn down), every bio parks forever.
Each issuer sleeps in biowait in an unkillable D-state, all mounted
filesystems on idad volumes hang, and bio_queue grows without bound while
the VM page daemon stalls on dirty buffers β a system-wide livelock that
only a reboot clears. Exactly the DF-5026 ips pattern, with the aggravation
that IDA has not even an offline flag to gate on.
Threat model / trigger
Requires an IDA controller that stops completing I/O (genuine hardware wedge). No unprivileged or software-only trigger β hence Low (DF-5026 was Medium because an IPS adapter can be switched offline by configuration commands without hardware failure; IDA has no such knob).
PoC / verification status
Hardware-gated: no Compaq SMART/IDA device on the audit guest (env.txt:
pciconf shows no 0x0e11 function; no /dev/idad*). The code trace above is
line-accurate β there is literally no biodone reachable for a parked bio
outside ida_intr/ida_done.
Fix
1-second callout watchdog: if work has been pending continuously for 60 seconds, fail everything queued with ETIMEDOUT and keep failing new arrivals (fix.diff, ida.c + idavar.h).
Reproduce (on hardware)
# IDA box with a mounted idad filesystem; wedge the controller (e.g. trigger
# a PCI error / stall the card in a test jig):
cat /dev/idad0 > /dev/null # parks in D-state forever; ps shows 'D*' state
# bio_queue grows; box needs hard reset
DF-5048 VERDICT β parked bios / no completion path on a wedged IDA controller
status: untested (hardware-gated) | impact: none demonstrated (local DoS class) | confidence: likely
Reproduced?
No. Requires a physical Compaq SMART/IDA controller that stops completing
I/O. Guest evidence (env.txt): no 0x0e11 PCI function, driver never attaches,
/dev/idad* does not exist. Untested = hardware absent.
Why the finding is real (trace)
Completion-flow enumeration β there is no other biodone for bios that
enter ida_submit_buf:
idad_strategylocal error paths all biodone (ida_disk.c:153-156, 161-164, 169, 179-188) β those are fine (this is what DF-5012 fixed for mfi; ida already has it).- The submit path (ida_disk.c:172-177) ->
ida_submit_buf(ida.c:364-370):bioqdisksort(&ida->bio_queue, bio)+ida_construct_qcb+ida_start. ida_construct_qcb(ida.c:372-417): pops a bio only when a free QCB exists; out of QCBs -> return, bio parked (:385-387). Popped bios become qcbs queued forida_start(:416).ida_start(ida.c:424-440): submits while FIFO not full; else breaks.- The ONLY biodone for these bios is
idad_intr(ida_disk.c:213-226) called fromida_done(ida.c:530-533), reached only fromida_intr(ida.c:470-493, hardware completions) β orida_waitfor internal commands (never for disk bios). - No callout/watchdog/timeout exists anywhere in ida.c or ida_disk.c
(verified by grep: no
callout, notimeout, no offline flag in idavar.h:122-124 β flags are ATTACHED/FIRMWARE/INTERRUPTS only).
Therefore: controller delivers no completion => QCBs never return to
free_qcbs => QCB pool exhausts after IDA_QCB_MAX=256 requests => all
further bios park in bio_queue forever => biowait sleeps are
unkillable D-states; dirty-buffer pressure stalls the VM; queue is
unbounded. System-wide hang until reboot. Family precedent: DF-5026
(ips, Medium β reachable via adapter-offline config; here only via a real
hardware wedge, hence Low).
Not a duplicate of DF-1915/DF-4837 (ida.c): those cover detach-time freeing and DMA-direction corruption; this covers the hang/livelock from absent completion paths.
Fix assessment
fix.diff adds a 1 Hz callout watchdog (ida.c + idavar.h): 60 consecutive ticks with pending work and no drain => fail all queued bios with ETIMEDOUT+B_ERROR and keep failing new arrivals, so issuers become killable and the queue cannot grow unbounded. 60s is far beyond any real SMART-array latency; false-positive risk under extreme (but live) load is the accepted tradeoff, same remedy family as DF-5026's fix. fix_status=not_testable: no IDA hardware on the guest to build/validate a patched kernel.
Bottom line
Real absence-of-failure-path defect, code-certain, hardware-gated. Low.
Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.
Fix verification
not_testablefix.diff (ida.c + idavar.h) adds the stall-counter watchdog: callout started in ida_init, callout_stop_sync in ida_free, fail-queued-bios logic in ida_watchdog. Closes the unbounded-park behavior; accepted tradeoff: a live-but-60s-backlogged controller would start erroring I/O. Cannot be build/boot validated without IDA hardware.
fix.diff (git apply --check: APPLIES_CLEAN); hunks at ida.c includes/prototypes/ida_free/ida_init/new ida_watchdog, idavar.h struct ida_softc
Confirmed kernel references
Detail
Evidence (decisive lines)
env.txt (no 0x0e11 PCI function, no /dev/idad*); VERDICT.md completion-flow enumeration showing biodone is unreachable for parked bios; fix.diff (git apply --check clean)
PoC changes
No runnable PoC exists (hardware-gated: requires a physically wedged SMART/IDA controller). Pack carries the code trace + hardware-absence evidence; fix.diff validated with git apply --check.
Verified recommended fix
Add a watchdog callout (1 Hz): after 60 consecutive ticks with pending work and no drain, fail all queued bios with ETIMEDOUT and keep failing new arrivals so issuers stay killable.
Verdict
DF-5026-family parked-bios hang: once an IDA controller stops delivering completions there is no code path anywhere in the driver that fails a queued bio β idad_strategy submits unconditionally (ida_disk.c:172-177), ida_construct_qcb silently parks bios when the 256-QCB pool is exhausted (ida.c:385-387), the only reaper is ida_done via ida_intr (ida.c:498-537,470-493), and no watchdog/timeout/offline-flag exists (ida.c/ida_disk.c contain no callout; idavar.h:122-124 flags are ATTACHED/FIRMWARE/INTERRUPTS only). Result: every issuer sleeps unkillably in biowait (D-state), mounted filesystems hang, and bio_queue grows unbounded. Untested solely because the guest has no IDA hardware. Rated Low rather than DF-5026's Medium because IDA has no software offline knob β a genuine hardware wedge is required.
No comments yet.