ahci_pm_softreset leaks the error-CCB reservation on final-retry second-FIS failure: stuck AP_F_ERR_CCB_RESERVED + permanently zeroed ap_active/ap_sactive -> port I/O wedge and KKASSERT panic on next error recovery/probe
Summary
The second-FIS failure branch calls put_err_ccb only when it retries; on the final retry it jumps to err: which never releases (contrast the first-FIS branch and the non-PM twin which always release). After the leak the one-shot reservation flag stays set forever and the port's active masks stay zeroed - the port wedges, and the next get_err_ccb (hotplug, camcontrol rescan, CAM error escalation) trips the INVARIANTS KKASSERT. Primed by a PM target failing the 2nd SRST FIS twice; detonated by any port reset. Trigger builds/runs on guest; runtime hardware-gated (no AHCI on guest). Release-unconditionally fix.
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-4502 Β· 11 files| File | Type | Description | Size | |
|---|---|---|---|---|
| README.md | β | 1.4 KB | β raw | |
| VERDICT.md | β | 6.8 KB | β raw | |
| trigger_DF4502.c | β | 2.5 KB | view raw | |
| build.sh | β | 118 B | view raw | |
| run.sh | β | 133 B | view raw | |
| build.log | β | 604 B | view raw | |
| run.log | β | 618 B | view raw | |
| env.txt | β | 1.1 KB | view raw | |
| fix.diff | β | 772 B | view raw | |
| manifest.json | β | 1.4 KB | view raw | |
| verdict.json | β | 4.3 KB | view raw |
DF-4502 β ahci_pm_softreset error-CCB reservation leak
ahci_pm_softreset() (sys/dev/disk/ahci/ahci_pm.c:805-817) releases the reserved error CCB only when it retries; on the FINAL second-FIS failure it jumps to err: with the reservation still held. AP_F_ERR_CCB_RESERVED stays set forever and ap_active/ap_sactive/ap_active_cnt stay zeroed β the port wedges (no new commands issued) and the next ahci_get_err_ccb() on the port KKASSERT-panics INVARIANTS kernels (X86_64_GENERIC included).
Reproduce (PM hardware required)
build.sh # cc -O2 -Wall -o trigger_DF4502 trigger_DF4502.c run.sh <cam-bus> # 64 x camcontrol rescan <bus> (root)
- Primed by: a PM target failing the 2nd SRST FIS twice in a row (device-driven; console shows "Soft-reset(2) through PM failed, retrying" then "... giving up").
- Detonated by: any later port reset/probe (hotplug, rescan) β
panic: assertion "(ap->ap_flags & AP_F_ERR_CCB_RESERVED) == 0" failed.
Expected output (vulnerable, primed)
Kernel panic as above; without INVARIANTS the port silently hangs all I/O ("DELAY CCB" console spew) instead.
On the audit guest
No AHCI controller exists (PIIX3 IDE + virtio only) and QEMU does not emulate port multipliers β build + CAM plumbing verified (build.log, run.log), bug path not runtime-reachable. See VERDICT.md for the full line-cited trace.
DF-4502 β ahci_pm_softreset leaks the error-CCB reservation on final-retry second-FIS failure
Verdict: INCONCLUSIVE at runtime (feature absent on guest); code defect CERTAIN by trace
- status:
inconclusiveβ the audit QEMU guest has no AHCI controller (PCI: PIIX3 IDE + virtio only, seeenv.txt), and QEMU does not emulate SATA port multipliers, so the priming condition (a PM target failing the second SRST FIS twice in a row) cannot be created on any available emulation. The defect itself is a plain, mechanical code-path asymmetry that is fully provable by reading the source; nothing about it is speculative. - expected impact on real hardware: kernel panic (KKASSERT, INVARIANTS kernels) or permanently wedged port I/O (non-INVARIANTS kernels) β a local DoS on machines with a port multiplier and a misbehaving/flaky target.
Root cause (line-cited)
ahci_pm_softreset() (sys/dev/disk/ahci/ahci_pm.c:679) acquires the
reserved error CCB at ahci_pm.c:719:
ccb = ahci_get_err_ccb(ap);
ahci_get_err_ccb() (ahci.c:3331) does the one-shot reservation:
KKASSERT((ap->ap_flags & AP_F_ERR_CCB_RESERVED) == 0); /* ahci.c:3352 */
ap->ap_flags |= AP_F_ERR_CCB_RESERVED; /* ahci.c:3353 */
/* save outstanding command state */ /* ahci.c:3356-3358 */
ap->ap_active = ap->ap_active_cnt = ap->ap_sactive = 0; /* ahci.c:3364 */
and only ahci_put_err_ccb() (ahci.c:3379) restores the saved state and
clears the flag (ahci.c:3407-3412).
The softreset has TWO failure sites:
-
First FIS poll (ahci_pm.c:752-767) β releases the CCB unconditionally before deciding to retry or give up:
c ahci_put_err_ccb(ccb); /* ahci_pm.c:756 β always */ if (--count) { ... goto retry; } goto err; /* already released: OK */ -
Second FIS poll (ahci_pm.c:805-817) β releases the CCB only when retrying:
c if (ahci_poll(ccb, 1000, ahci_ata_cmd_timeout) != ATA_S_COMPLETE) { kprintf("%s: Soft-reset(2) through PM failed, %s\n", ...); if (--count) { ahci_os_sleep(1000); ahci_put_err_ccb(ccb); /* ahci_pm.c:811 β retry only */ ... goto retry; } goto err; /* ahci_pm.c:816 β NOT released */ }
and the common exit err: (ahci_pm.c:862-877) never calls
ahci_put_err_ccb().
So when the second FIS fails on the final retry (count 2β1 on an earlier failure, then 1β0 here; or the mirror sequence), the function returns with:
AP_F_ERR_CCB_RESERVEDstill set, forever;ap->ap_active,ap->ap_sactive,ap->ap_active_cntstuck at 0 (their saved values are never restored);at->at_probe = ATA_PROBE_FAILED(ahci_pm.c:876).
Compare the non-PM twin ahci_port_softreset() (ahci.c:1233), which does
it correctly: its err: label releases unconditionally
(ahci.c:1393-1394). The asymmetry inside ahci_pm.c between its own two
failure sites (756 vs 811/816) shows the second site is simply missing
the unconditional release.
Failure cascade after the leak
While the flag is stuck:
ahci_issue_pending_commands()refuses to issue anything (ahci.c:2319-2322, "DELAY CCB" spew) β the port stops doing I/O; queued requests hang until CAM timeouts.- The next
ahci_get_err_ccb()on the port tripsKKASSERT((ap->ap_flags & AP_F_ERR_CCB_RESERVED) == 0)(ahci.c:3352) βpanic("assertion ... failed")on INVARIANTS kernels (X86_64_GENERIC is INVARIANTS, sys/config/X86_64_GENERIC:56; KKASSERT panics per sys/sys/systm.h:98-101). On non-INVARIANTS kernels the KKASSERT is a no-op (systm.h:118) and the port silently re-saves its already-zeroed active state, permanently corrupting completion accounting. - Re-entry sites that call
ahci_get_err_ccb(): ahci_pm_port_probe()ahci_pm.c:188 β reached from any port-level hard reset: hotplug event βahci_port_state_machine()βahci_port_reset(ap, NULL, 1)(ahci.c:818-819) βahci_port_hardresetβahci_pm_port_probe(ahci.c:1710);camcontrol rescan <bus>/ wildcard XPT_PATH_INQ βahci_port_state_machine()(ahci_cam.c:920-923) β same chain;ahci_port_read_ncq_error()ahci.c:3441 (NCQ error recovery);ahci_port_softreset()ahci.c:1307 (any port error recovery).ahci_put_err_ccb()on a port with real CI bits after bookkeeping corruption also hard-panics at ahci.c:3393/3399 (plainpanic, not KKASSERT).
Trigger
- Priming (device-driven): a target behind the PM fails the second SRST FIS on two consecutive softreset attempts. The code explicitly plans for this ("Soft-reset(2) through PM failed, retrying/giving up").
- Detonation: any later port reset/probe/error recovery on that port
(hotplug,
camcontrol rescan, CAM timeout escalation).
trigger_DF4502.c implements the detonation half (64 Γ wildcard rescan).
It builds and runs on the guest (see build.log/run.log); the guest has no
ahci controller, so the ahci sink is absent β no panic is possible there
by construction.
Why not reproduced (honest accounting)
- Guest PCI inventory contains no AHCI controller (class 0x010601) β atapci0 is PIIX3 IDE; ahci.ko has nothing to attach to.
- Even with QEMU
-device ahci(possible viaQEMU_EXTRAin vm.sh), QEMU has no port-multiplier emulation:ahci_pm_port_probe()would fail to detect a PM at target 15 and none of the PM target paths (ahci_pm_softreset,ahci_pm_read/writeunder FBSS) would ever run. - The bug is therefore classified
inconclusive(code-certain, runtime-untestable on available hardware), consistent with how this audit records hardware-gated storage findings.
Fix validation
Not performable: requires a PM-capable ahci controller and a device that
fails the second SRST FIS deterministically; no such emulation exists.
fix.diff is authored against the read-only sys/ tree (never applied) β
it mirrors the first-FIS pattern (release before the count check) so the
CCB is released on every failure exit, matching ahci.c:1393-1394's
err-label discipline.
References
- sys/dev/disk/ahci/ahci_pm.c:719,752-767,805-817,862-877
- sys/dev/disk/ahci/ahci.c:2319-2322,3331-3413 (esp. 3352-3353, 3364, 3407-3412)
- sys/dev/disk/ahci/ahci.c:1307,1710,3441 (re-entry sites)
- sys/dev/disk/ahci/ahci_cam.c:920-923 (wildcard PATH_INQ β state machine)
- sys/sys/systm.h:98-101,118 (KKASSERT panic/no-op)
- sys/config/X86_64_GENERIC:56 (INVARIANTS)
Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.
Fix verification
not_testableFix authored (fix.diff, git-apply --check clean) but not kernel-buildable into a testable configuration: the bug requires PM hardware absent from the only emulation available (no AHCI controller on the guest; QEMU has no port-multiplier emulation), so baseline-vs-patched comparison cannot be performed.
fix.diff (git apply --check OK on the read-only sys/ tree; never applied); VERDICT.md 'Fix validation' section.
Confirmed kernel references
- sys/dev/disk/ahci/ahci_pm.c:719
- sys/dev/disk/ahci/ahci_pm.c:752-767
- sys/dev/disk/ahci/ahci_pm.c:805-817
- sys/dev/disk/ahci/ahci_pm.c:862-877
- sys/dev/disk/ahci/ahci.c:3352
- sys/dev/disk/ahci/ahci.c:3364
- sys/dev/disk/ahci/ahci.c:3407-3412
- sys/dev/disk/ahci/ahci.c:2319-2322
- sys/dev/disk/ahci/ahci.c:1710
- sys/dev/disk/ahci/ahci_cam.c:920-923
- sys/sys/systm.h:98-101
- sys/config/X86_64_GENERIC:56
Detail
Evidence (decisive lines)
VERDICT.md (full line-cited trace incl. the 752-767 vs 805-817 asymmetry and the re-entry chain); run.log (detonator runs clean on guest's CAM buses); env.txt (no AHCI controller on guest); fix.diff (git-apply --check OK)
PoC changes
Seed sketch did not exist; wrote the detonator from scratch. First variant used
Verified recommended fix
Release the error CCB unconditionally in the second-FIS failure branch before the retry decision, mirroring the first-FIS branch (ahci_pm.c:756) and ahci_port_softreset's err-label discipline (ahci.c:1393-1394); see fix.diff.
Verdict
Code defect is certain by mechanical trace: ahci_pm_softreset() releases the reserved error CCB only when it retries (ahci_pm.c:811) and jumps to err: without ahci_put_err_ccb() on the final second-FIS failure (ahci_pm.c:805-817), leaving AP_F_ERR_CCB_RESERVED set forever and ap_active/ap_sactive/ap_active_cnt zeroed (saved only at ahci.c:3356-3364, restored only at ahci.c:3407-3412). The port then wedges (ahci.c:2319-2322 refuses to issue anything) and the next ahci_get_err_ccb() on the port -- e.g. via camcontrol rescan -> wildcard XPT_PATH_INQ -> ahci_port_state_machine (ahci_cam.c:920-923) -> ahci_pm_port_probe (ahci.c:1710, ahci_pm.c:188) -- trips KKASSERT at ahci.c:3352, a real panic on INVARIANTS kernels (X86_64_GENERIC:56; sys/sys/systm.h:98-101). Runtime reproduction was impossible: the audit QEMU guest has no AHCI controller (PCI inventory: PIIX3 IDE atapci0 + virtio only, env.txt) and QEMU does not emulate SATA port multipliers, so the priming condition (PM softreset double-failure) cannot occur on any available emulation. The detonator trigger builds and runs on the guest (build.log/run.log: 64 wildcard rescans against the ata CAM buses complete without error -- the ahci sink is simply absent). Expected real-hardware impact: kernel panic / permanent port I/O hang (local DoS) on machines with a port multiplier and a flaky target.
No comments yet.