β¬’ DragonFlyBSD Kernel Audit
← triage Β· dashboard
DF-4841

Post-reset OBFL refill is a no-op: sense_bufs[].posted survives the controller soft reset, so tws_init_obfl_q() re-run by the recovery path posts nothing β€” firmware left with zero response frames, permanent I/O hang after any successful reset recovery

Summary

init_obfl_q only posts frames whose posted flag is false, but the flags are cleared only at attach and one overrun path - never on the reset path that must re-post every frame (the soft reset wiped the fw's free list). Recovery completes and every post-recovery command still times out forever (init_connect's own 60s poll runs through the queue it failed to refill) - a recoverable hiccup becomes a permanent adapter hang. Clear-flags-on-reset-path fix. Hardware-gated (no 9750 on guest).

Discussion (0)

No comments yet.

PoC verification

Evidence pack

findings/poc/DF-4841 Β· 8 files
FileTypeDescriptionSize
README.md β€” 3.3 KB ↓ raw
VERDICT.md β€” 3.2 KB ↓ raw
trigger.c β€” 2.2 KB view raw
run.sh β€” 132 B view raw
guest_probe.log β€” 94 B view raw
env.txt β€” 641 B view raw
fix.diff β€” 1.9 KB view raw
verdict.json β€” 4.3 KB view raw

DF-4841 β€” post-reset OBFL refill is a no-op: sense_bufs[].posted survives the soft reset, leaving the firmware with zero response frames

sys/dev/raid/tws/tws_hdm.c:118-148 (tws_init_obfl_q) + the reset/recovery path tws_cam.c:1198-1266.

What

sense_bufs[i].posted tracks "this response frame is currently in the firmware's outbound free list". At attach, tws_init_reqs() clears all flags (tws.c:620) and tws_init_ctlr() β†’ tws_init_obfl_q() (tws_hdm.c:111) posts all 256 frames.

The reset recovery path (tws_reset β†’ tws_reset_cb β†’ tws_reinit, tws_cam.c:1198-1266) exists precisely because a soft reset reinitializes the controller's message unit β€” the firmware's copy of the free list is gone. tws_reinit() resets obfl_q_overrun, re-runs tws_init_connect() and calls tws_init_obfl_q() (tws_cam.c:1262-1266) to refill the queue. But posted is still true for every frame β€” nothing on the reset path clears it (grep: cleared only at attach, tws.c:620, and on an OBFL-overrun repost failure, tws_cam.c:543). The "refill" loop

tws_hdm.c:127-144
    while ( i < tws_queue_depth ) {
        if ( !sc->sense_bufs[i].posted ) {   /* never taken */
            ... post frame ...
            sc->sense_bufs[i].posted = true;

posts nothing. The firmware is left with zero outbound free frames:

  • tws_init_connect() itself polls up to 60 s (TWS_POLL_TIMEOUT) for a response it can never receive β†’ recovery burns a full minute then fails (and per DF-1926 also leaks the internal request slot).
  • Even if recovery "completes", every subsequent command submission cannot be answered β€” all I/O to the adapter times out permanently. The adapter and everything on it is dead until reboot.

One reset β€” triggered by any controller error that reaches tws_reset() (bus reset, firmware fault, the unguarded re-entry of DF-4401, DF-4590's callout storm) β€” converts a recoverable hiccup into a permanent hang of the storage path.

Verification status

Not reproduced on the audit guest β€” hardware-gated. The audit guest has no LSI/AVC 3ware 9750 PCI device, QEMU does not emulate one, and tws is compiled into X86_64_GENERIC with nothing attached (guest_probe.log: NO_3WARE_PCI, dev.tws absent). The defect is certain from the driver's own code shape: the recovery path resets obfl_q_overrun and re-runs the refill function intending to re-post, while the state that makes the refill a no-op is left untouched β€” the two cannot both be correct, and if the flags were correct as-is (queue survives reset) the refill call would be pointless.

Fix

fix.diff β€” clear the posted flags on the reset path (tws_reset_obfl_q() in tws_hdm.c, called from tws_reinit() before tws_init_obfl_q()). Deliberately NOT done inside tws_init_obfl_q() itself: its other caller (tws_submit_command() on OBFL overrun, tws_hdm.c:282-284) must not double-post frames still queued. git-apply clean; not runtime-testable (no controller).

Run (for a maintainer with hardware)

  1. Put the adapter into reset recovery (e.g. camcontrol reset on the tws sim, or a controller fault).
  2. Stock kernel: watch the recovery spin in tws_init_connect's poll for TWS_POLL_TIMEOUT, then all I/O hang (iostat flatlines, every ccb times out).
  3. Patched kernel: recovery completes and I/O resumes.
VERDICT.md
↓ download raw

DF-4841 β€” VERDICT

Finding: post-reset OBFL refill no-op β€” sc->sense_bufs[i].posted survives the controller soft reset, so tws_init_obfl_q() re-run by the recovery path posts nothing and the firmware is left with zero response frames β†’ permanent I/O hang after any reset recovery.

Reproduced? NO β€” hardware-gated on this guest (defect certain from source)

guest_probe.log: no 3ware (13c1) PCI device on the guest, dev.tws absent, tws not a loadable module here (compiled into X86_64_GENERIC). QEMU cannot emulate a 9750 controller. Same verification posture as the other tws hardware findings (DF-1924/1925/1926/4590/4591).

Why the defect is certain from the driver's own code

  1. The reset path re-runs the refill intending to re-post: tws_reinit() resets sc->obfl_q_overrun = false and calls tws_init_obfl_q(sc) (tws_cam.c:1262-1266) β€” a function whose only job is putting frames into the firmware's outbound free list.
  2. The refill posts only frames with !posted (tws_hdm.c:128).
  3. posted is set true for all 256 frames during normal operation (tws_hdm.c:141) and is cleared at exactly two sites: attach (tws_init_reqs, tws.c:620) and a failed repost on OBFL overrun (tws_cam.c:543). Neither runs on the reset path (tws_reset β†’ tws_reset_cb β†’ tws_reinit, tws_cam.c:1198-1266).
  4. Therefore, after a soft reset wipes the firmware's queue state, tws_init_obfl_q() posts nothing. Invariants (2)+(3) and the intent (1) are mutually exclusive β€” the recovery path is self-defeating: tws_init_connect() polls 60 s (TWS_POLL_TIMEOUT, tws.h:74) for a response through the very queue it failed to refill, then every subsequent command times out forever. (If one instead assumed the queue SURVIVES the reset, the obfl_q_overrun=false + refill call in tws_reinit would be dead code.)

Impact ceiling: local DoS of the storage adapter and all I/O bound to it (system-wide stall for root/boot disks on it), reachable from any controller error that enters tws_reset() β€” including DF-4401's unguarded re-entry and DF-4590's stale-callot storms. Root/privileged or hardware-fault trigger; Low severity per the audit rubric (aligned with DF-1924/1925/1926).

Distinct from known findings: DF-1924/1925 (unbounded spins β€” hang inside the spin loops), DF-1926 (request-slot leak on init_connect failure). This finding is the successful-reset case: reset completes, recovery runs to completion, and the adapter is still dead because the refill was a no-op.

Fix

fix.diff: tws_reset_obfl_q() (tws_hdm.c) clears all posted flags; tws_reinit() (tws_cam.c) calls it before tws_init_obfl_q(). Not inside tws_init_obfl_q() itself β€” the OBFL-overrun caller (tws_hdm.c:282-284) must not double-post frames still queued. git-apply clean against the tree (dry-run verified). Not runtime validated: no controller on the audit guest.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.

Fix verification

not_testable
baseline no→ patch + rebuild →patched clean

Fix not validated at runtime: no 3ware controller on the audit guest and QEMU cannot emulate one. fix.diff is git-apply clean (dry-run verified), places the clearing on the reset path only (not inside init_obfl_q, whose OBFL-overrun caller must not double-post frames still queued), and is validated by inspection against the four-step invariant argument in VERDICT.md.

findings/poc/DF-4841/fix.diff
↓ fix.diffper-fix-DF-4841

Confirmed kernel references

Detail

Evidence (decisive lines)

VERDICT.md (four-step invariant argument with line receipts); trigger.c (hardware-gated trigger recipe); guest_probe.log (no 3ware PCI dev, dev.tws absent); fix.diff (tws_reset_obfl_q clearing posted flags on the reset path only - deliberately NOT inside init_obfl_q, whose OBFL-overrun caller must not double-post; git-apply dry-run clean)

PoC changes

No seed pack. Authored pack: trigger recipe + hardware-absence probe + fix.diff.

Verified recommended fix

Clear sense_bufs[].posted on the reset path before re-running tws_init_obfl_q() (tws_reset_obfl_q() called from tws_reinit; see fix.diff).

Verdict

tws_init_obfl_q() (tws_hdm.c:118-148) only posts response frames whose sense_bufs[i].posted is false; the flags are cleared at exactly two sites - attach (tws_init_reqs, tws.c:620) and an OBFL-overrun repost failure (tws_cam.c:543) - neither of which runs on the reset/recovery path (tws_reset -> tws_reset_cb -> tws_reinit, tws_cam.c:1198-1266). tws_reinit() nevertheless resets obfl_q_overrun and re-runs the refill (tws_cam.c:1262-1266), which is therefore a complete no-op after the soft reset wipes the firmware's copy of the outbound free list: the firmware is left with zero response frames, init_connect's own 60s poll (tws.h:74) cannot be answered, and every post-recovery command times out forever - a recoverable controller hiccup becomes a permanent adapter hang (local DoS of the storage path; root/hardware-fault trigger). The defect is certain from the driver's own invariant structure (the refill intent and the posted flags cannot both be correct), but hardware semantics of the soft reset were inferred, not measured - hence 'likely'. Not reproduced: the audit guest has no 3ware 13C1 PCI device, dev.tws is absent, and QEMU cannot emulate a 9750 (guest_probe.log); same posture as DF-1924/1925/1926/4590/4591. Distinct from DF-1924/1925 (hang inside spin loops) and DF-1926 (init_connect slot leak): this is the successful-reset case where recovery completes and the adapter is still dead because the refill posted nothing.