DF-4744 — ahci_pci_detach() lacks an interlock against an in-flight ahci_intr() on another CPU

File under audit: sys/dev/disk/ahci/ahci_attach.c (pass 2)

What the bug is

ahci_pci_detach() (sys/dev/disk/ahci/ahci_attach.c:580-647) tears a live AHCI controller down in this order:

1. sc->sc_flags &= ~AHCI_F_INT_GOOD (ahci_attach.c:592)
2. ahci_write(sc, AHCI_REG_GHC, 0) (ahci_attach.c:594)
3. bus_teardown_intr(...) (ahci_attach.c:596-599)
4. free every port: ahci_cam_detach(ap); ahci_port_free(sc, i) — kfree of ap, its CCB array, DMA maps, at array (ahci_attach.c:604-610 → ahci.c:979-1044)
5. bus_release_resource(SYS_RES_IRQ) and bus_release_resource(SYS_RES_MEMORY) — the ABAR MMIO mapping is destroyed (ahci_attach.c:615-628)

ahci_intr() (sys/dev/disk/ahci/ahci.c:2472-2502) guards itself with a single non-atomic check of AHCI_F_INT_GOOD and then walks sc->sc_ports[], reads ABAR registers and processes CCBs.

The interlock between (1) and (4) is missing. DragonFly's bus_teardown_intr() → pci_teardown_intr() (sys/bus/pci/pci.c:3384, masks INTx/MSI at the PCI config level) → nexus_teardown_intr() → unregister_int() (sys/kern/kern_intr.c:416-497) unlinks the handler under a local critical section and returns; it never waits for a handler that is already executing on another CPU. Masking the vector at the interrupt controller does not affect a handler that has already been entered.

So a CPU that entered ahci_intr() just before step (1) can still be inside the handler body while steps (4)/(5) free every structure the handler touches and unmap the register window it is about to read:

• ahci_read(sc, AHCI_REG_IS) after (5) → read of unmapped MMIO,
• sc->sc_ports[port] / ahci_port_intr(ap, ...) after (4) → use-after-free of struct ahci_port (kfree'd at ahci.c:1041) in hard-interrupt context,
• ahci_os_lock_port_nb(ap) on a kfree'd/freed lock (lockuninit'd in ahci_os_stop_port, ahci_dragonfly.c:297).

The driver authors know: XXX interlock last interrupt? at ahci_attach.c:589-591.

Reachability / threat

Trigger requires a controller detach while the controller is generating interrupts:

• kldunload ahci.ko (or devctl detach ahciN) — root only, or
• physical surprise-removal (express-card/thunderbolt-style AHCI HBA) — physical access.

There is no unprivileged-local or remote trigger. Race window is the duration of the handler body vs. the teardown sequence — needs an interrupt to land during detach of an active controller (I/O in flight).

Impact ceiling: kernel use-after-free in hardirq context (panic; potentially worse given a sufficiently controllable heap), but only under privileged or physical action → Low per the audit rubric.

Why this pack has no run logs (status: untested)

The audit QEMU guest has no AHCI controller: pciconf -l shows no SATA/AHCI class device (root disk is if=virtio on an i440fx machine; verified in env.txt). ahci_pci_detach therefore cannot be entered on the guest at all. The race itself additionally requires interrupt traffic during the teardown window. A deterministic demonstration needs (a) a real/instantiated AHCI controller with a drive under I/O and (b) an instrumented kernel (artificial delay in ahci_intr after the flag check) — a recipe is given in VERDICT.md for the team to run on iron that has the hardware. Per contract, Low-severity privileged-trigger findings do not mandate guest verification; nothing here is claimed as reproduced.

Contents

• VERDICT.md — full narrative + line-by-line trace + repro recipe
• trigger_devctl_detach.sh — the detach-race exerciser for real hardware (NOT RUN on this guest; no AHCI controller present)
• fix.diff — proposed driver-side interlock (NOT applied to sys/)
• env.txt — guest environment proof (no ahci device)
• manifest.json, verdict.json

DF-4744 VERDICT — ahci_pci_detach() / in-flight ahci_intr() interlock missing

Machine verdict: untested (reproduced=0, impact=none observed). Classification of the vulnerability itself (source-proven): race → hard-interrupt-context use-after-free / unmapped-MMIO read; trigger is privileged (root detach) or physical (surprise removal) → severity Low, confidence likely (mechanism fully provable from source; runtime window not demonstrable on this guest — no AHCI hardware).

1. Full source trace

The handler side — what an in-flight ahci_intr() touches

ahci_intr() — sys/dev/disk/ahci/ahci.c:2472:

void ahci_intr(void *arg)
{
    struct ahci_softc *sc = arg;
    ...
    if ((sc->sc_flags & AHCI_F_INT_GOOD) == 0)   // ahci.c:2482 — the ONLY guard
        return;
    is = ahci_read(sc, AHCI_REG_IS);             // ahci.c:2484 — ABAR MMIO read
    if (is == 0 || is == 0xffffffff) return;
    is &= sc->sc_portmask;
    ahci_write(sc, AHCI_REG_IS, is);             // ABAR MMIO write
    for (ack = 0; is; is &= ~(1 << port)) {
        port = ffs(is) - 1;
        ap = sc->sc_ports[port];             // ahci.c:2497 — walks port array
        if (ap == NULL) continue;
        if (ahci_os_lock_port_nb(ap) == 0) {  // lockmgr on ap->ap_lock
            ahci_port_intr(ap, 0);        // CCBs, DMA maps, port regs...
            ahci_os_unlock_port(ap);
        } else {
            ahci_pwrite(ap, AHCI_PREG_IE, 0);   // ap MMIO
            ahci_os_signal_port_thread(ap, AP_SIGF_PORTINT); // ap fields
        }
    }
}

Everything after the single flag check reads the ABAR mapping (sc_iot/sc_ioh out of sc_regs) and the per-port heap objects. Note the else branch also dereferences ap — grabbing the port lock first does not shield the teardown.

The teardown side

ahci_pci_detach() — sys/dev/disk/ahci/ahci_attach.c:580:

    sc->sc_flags &= ~AHCI_F_INT_GOOD;            // :592
    if (sc->sc_regs)
        ahci_write(sc, AHCI_REG_GHC, 0);     // :594

    if (sc->sc_irq_handle) {
        bus_teardown_intr(dev, sc->sc_irq, sc->sc_irq_handle);  // :597
        sc->sc_irq_handle = NULL;
    }

    for (i = 0; i < AHCI_MAX_PORTS; i++) {       // :604
        ap = sc->sc_ports[i];
        if (ap) {
            ahci_cam_detach(ap);
            ahci_port_free(sc, i);           // kfrees ap and everything in it
        }
    }
    ... bus_release_resource(irq) ...            // :615
    ... bus_release_resource(SYS_RES_MEMORY) ... // :624 — ABAR unmapped

Why bus_teardown_intr() does not save you (the crux)

Chain on DragonFly:

• bus_teardown_intr → pci_teardown_intr (sys/bus/pci/pci.c:3384): masks INTx (PCIM_CMD_INTxDIS) or disables MSI at the PCI config level, then delegates to bus_generic_teardown_intr → nexus.
• unregister_int (sys/kern/kern_intr.c:416-497): under crit_enter() on the calling CPU it unlinks the intrec, calls machintr_intr_teardown() if this was the last handler (masks the vector — ioapic_abi.c:681 / icu_abi.c:282), crit_exit(), then kfree(rec).

There is no wait anywhere in this chain for a handler already executing on another CPU. A critical section only guards the local CPU; masking a vector does not affect an entered handler. (Compare FreeBSD's intr_event teardown, which drains active handlers — DFly has no equivalent.)

The interleaving that breaks it

CPU0 (detach thread)                     CPU1 (hardirq)
---------------------------------------  ------------------------------------
  <intr fires; ahci_intr entered>
                                          sc_flags & AHCI_F_INT_GOOD != 0  (pass)
sc_flags &= ~AHCI_F_INT_GOOD   (:592)     ...
ahci_write(GHC, 0)            (:594)     ...
bus_teardown_intr             (:597)     ...
  (no drain; returns)                    is = ahci_read(sc, AHCI_REG_IS)
ahci_port_free() x N          (:604-610)   ap = sc->sc_ports[port]      ← UAF
  kfree(ap), free maps/CCBs               ahci_port_intr(ap, 0) / pwrite(ap,...)
bus_release_resource(MEMORY)  (:624)      ahci_read/ahci_write ...        ← unmapped MMIO

The in-source XXX interlock last interrupt? (ahci_attach.c:589-591) shows this is a known-but-unhandled gap.

What is NOT affected (checked and killed)

• The attach-failure unwind paths that call ahci_pci_detach mid-attach (ahci_attach.c:303, 317, 328, 352, 431, 523): in every one of them either bus_setup_intr has not run yet (no handler registered → no race) or no ports exist yet. Verified individually.
• Port-thread lifetime vs ahci_port_free: ahci_os_stop_port (ahci_dragonfly.c:279-298) signals AP_SIGF_STOP and waits for ap->ap_thread to clear before the kfrees — thread side is interlocked.
• The interrupt-vs-port-init race at attach is handled correctly: bus_setup_intr at :515 runs before AHCI_F_INT_GOOD is set at :546 and before GHC_IE is enabled at :545; the port-thread sync loop (:533-538) precedes the master enable. No window.
• MSI release on early failure: pci_alloc_1intr at :296 can allocate MSI before bus_alloc_resource_any fails; ahci_pci_detach :621-622 calls pci_release_msi — covered.
• Partial DMA-tag failure unwind: softc is zero-allocated; detach NULL-checks and NULLs every tag (:630-645) — idempotent, no double destroy.
• 1 << i port map walk with i<32 (AHCI_MAX_PORTS=32, ahci.h:400): fits.
• Per-port sysctl tree (link_pwr_mgmt, link_pwr_state, ahci_dragonfly.c:250-267) is sysctl_ctx_free'd in ahci_os_stop_port before kfree(ap) — no dangling sysctl handler (unlike the DF-4647 pattern).
• ahci_lookup_device (ahci_attach.c:128-161) matches listed VID/DID entries without checking PCI class — binds ahci to e.g. IDE-mode MCP controllers; hardware-selection quirk, not a security boundary (a malicious PCIe device is already a DMA attacker). Not filed.
• Adjacent-file observation (not this file, ahci.c already audited pass 2): ahci_init leaks its temporary ap (ahci.c:186 alloc, ahci.c:197-201 return (1) without kfree) when bus_space_subregion fails — one-time few-KB boot-time leak on broken hardware.

2. Why untested (not run on the guest)

pciconf -l in the audit guest (env.txt) contains no SATA/AHCI class device — the guest boots from if=virtio storage on an i440fx machine, so ahci_pci_attach/ahci_pci_detach are never even entered. Even with a controller, the window requires an interrupt to land inside the microseconds-scale teardown of an active controller.

3. Reproduction recipe for hardware that has AHCI (for the team)

1. Instrument a debug kernel: in ahci_intr (ahci.c:2482), after the AHCI_F_INT_GOOD check, add DELAY(100000); (100 ms) to hold the handler inside the window deterministically.
2. Boot it on a machine whose boot disk is NOT on the ahci controller under test; put a secondary AHCI disk under streaming I/O (dd from the raw device).
3. devctl detach ahciN (or kldunload ahci) while the I/O runs.
4. Expected on an unpatched kernel: panic in hardirq context (NULL/UAF deref in ahci_intr/ahci_port_intr, or a fault on the unmapped ABAR) within seconds. With fix.diff applied: clean detach, no panic.

trigger_devctl_detach.sh in this pack encodes steps 2-3.

4. Fix

fix.diff — driver-side interlock (authoritative recommendation is to also give unregister_int() a real drain, FreeBSD-intr_event-style; the driver patch stands alone and is self-contained):

• ahci.h: add u_int sc_intr_active; u_int sc_intr_wakeup; to the softc.
• ahci.c ahci_intr(): after passing the flag check, atomically increment sc_intr_active, re-check the flag, bail (decrement + wakeup) if teardown began; on body exit decrement and wakeup.
• ahci_attach.c ahci_pci_detach(): after bus_teardown_intr(), issue a full memory fence (cpu_mfence()) and wait for sc_intr_active to drain before freeing ports / releasing the register mapping.

Fix validation was not possible on this guest (no AHCI controller; no baseline reproduction possible): fix_status: not_testable, fix_baseline_reproduced: 0, fix_patched_reproduced: 0.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.