# DF-4613 — ips_timeout forces adapter reinit while holding queue_lock and then sleeps forever on a completion only the (lock-blocked) interrupt handler can deliver

## What
`sys/dev/raid/ips/ips.c:331-371` (`ips_timeout`), `ips.c:355` (forced reinit),
`ips.c:585` / `ips.c:769` (interrupt handlers take the same lock),
`sys/dev/raid/ips/ips_commands.c:722,743` (`ips_timed_wait(..., 0)` = infinite
tsleep) via `ips_clear_adapter()` called from `ips_morpheus_reinit(sc, 1)`
(`ips.c:516`) / `ips_copperhead_reinit(sc, 1)` (`ips.c:738`).

Timer-path twin of DF-2163 (which filed the identical pattern reached from the
IPS_USER_CMD ioctl in ips_ioctl.c:125-139). This path needs **no user action**:
it fires automatically whenever any card command is stuck for ~100 s.

## Sequence (morpheus; copperhead is identical in shape)
1. A command hangs on the card. `ips_timeout` (10 s callout, softclock thread)
   decrements `timeout` each tick; at 0 it sets `IPS_TIMEOUT`, error-completes
   the command via its callback (ips.c:343-349) — all under
   `lockmgr(&sc->queue_lock, LK_EXCLUSIVE)` taken at ips.c:337.
2. Once no command still has a pending timeout (`state == 0`), ips.c:354-355
   calls `sc->ips_adapter_reinit(sc, 1)` **still holding queue_lock**.
3. `ips_morpheus_reinit(force=1)` resets the card (5 s DELAY + up to 45+240
   1-second DELAY busy-waits — themselves under the lock, starving every other
   callout on the CPU), unmasks interrupts (`ips_write_4(OIMR, 0)`,
   ips.c:515), then `ips_clear_adapter(sc)` (ips.c:516).
4. `ips_clear_adapter` → `ips_send_config_sync_cmd` (ips_commands.c:707-725)
   issues the sync command and calls
   `ips_timed_wait(command, "ipssyn", 0)` — **timo = 0 = sleep forever**
   (ips_commands.c:43-58; plain `tsleep`, which does not release lockmgr
   locks).
5. The card completes the command → IRQ → `ips_morpheus_intr` →
   `lockmgr(&sc->queue_lock, LK_EXCLUSIVE|LK_RETRY)` (ips.c:585) — **blocks
   forever**: the sleeping softclock thread owns the lock.
6. The wakeup never happens; the softclock thread never releases the lock;
   every subsequent IPS I/O, every ioctl, and every future `ips_timeout` tick
   blocks on queue_lock. Permanent wedge of the controller plus two kernel
   threads.

`LK_CANRECURSE` (lockinit at ips_pci.c:154) does not help: the interrupt
handler is a different thread.

## Build / run
Requires an IBM ServeRAID (ips) controller: PCI ids 0x1014:0x01BD (morpheus),
0x1014:0x002E / 0x1014:0x0250 / Adaptec 0x9005 (copperhead/marco). The audit
guest is QEMU/virtio and has no such device (`env.txt`); QEMU has no ServeRAID
device model, so the path cannot be instantiated. Verdict rests on the static
trace above (certain) plus the mechanically identical DF-2163 pattern.

## Expected behavior (if hardware were present)
~100 s after any command hangs, the console prints "WARNING: command timeout…
toaster mode", then "resetting adapter…"; when the reset finishes and
`ips_clear_adapter` issues the config sync, the system wedges: `ps` shows the
softclock thread in `tsleep` ("ipssyn") holding `ipslk` and the IPS ithread
blocked on `ipslk`; all I/O to ips(4) logical drives hangs forever; boot
device on ips ⇒ hard hang requiring power cycle.

## Contents
- `VERDICT.md`, `env.txt`, `fix.diff`, `manifest.json`, `verdict.json`