hpt_timeout()/os_cmddone() completion race: non-synchronizing callout_stop() frees the PCOMMAND while the timeout handler dereferences it (UAF -> wild ldm_reset_vbus -> panic)
Summary
Every queued read/write arms a 20s callout. The completion path calls the non-synchronizing callout_stop, then frees the extension and the command; hpt_timeout runs in softclock with NO lock and re-validates nothing - a timeout firing on another CPU at the completion instant reads pCmd->priv of a freed-and-recyclable command. A recycled command (flush sets priv=&done on the kernel stack; internal LDM commands set NULL/other) turns ext->vbus_ext into stack garbage and ldm_reset_vbus receives a wild pointer -> fatal trap. Local DoS on SMP RocketRAID 27xx systems when I/O completes at the instant its deadline expires (stalling disk). Disassembly appendix also KILLED the suspected get_logical_devices stack-smash (the binary lib clamps to min(total,maxcount)). Lock + re-validate fix (callout_stop_sync deliberately avoided - would deadlock in intr context).
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-4369 Β· 10 files| File | Type | Description | Size | |
|---|---|---|---|---|
| README.md | β | 2.6 KB | β raw | |
| VERDICT.md | β | 5.8 KB | β raw | |
| race_proof.txt | β | 5.5 KB | view raw | |
| verify_env.sh | β | 677 B | view raw | |
| run.sh | β | 164 B | view raw | |
| run.log | β | 1.0 KB | view raw | |
| env.txt | β | 302 B | view raw | |
| fix.diff | β | 1.4 KB | view raw | |
| manifest.json | β | 2.8 KB | view raw | |
| verdict.json | β | 4.5 KB | view raw |
DF-4369 β hpt27xx: hpt_timeout / os_cmddone completion race β use-after-free of PCOMMAND
What this is
A pass-2 (GLM 5.3) audit finding in sys/dev/raid/hpt27xx/hpt27xx_osm_bsd.c:
the per-command timeout hpt_timeout() races the completion path
os_cmddone(). os_cmddone() uses callout_stop(), which on DragonFly
explicitly does not interlock against an in-progress callout, and then
frees the command (ldm_free_cmds(pCmd)) and its extension. If the 20-second
timeout fires on another CPU at the same instant the command completes,
hpt_timeout() dereferences pCmd->priv after the free β a use-after-free
read that becomes a wild-pointer dereference when the freed PCOMMAND is
immediately recycled (e.g. as a flush command whose priv points at a kernel
stack variable).
Build / Run / Expected
This bug is hardware-gated: it requires a HighPoint RocketRAID 27xx SAS controller (PCI vendor 0x1103) attached and processing I/O.
- There is nothing to build on the audit guest: the driver is already
compiled into its
X86_64_GENERICkernel (sys/config/X86_64_GENERIC:117). ./run.sh(orbash verify_env.sh) probes the guest and prints: kernel/uname, the full PCI device list, and the hpt27xx module state.- Expected on real hardware (not available here): under SMP with I/O that
completes in the same instant its
HPT_OSM_TIMEOUT(20 s) deadline expires,hpt_timeout()readspCmd->privof a command already released to the LDM freelist; if the command block was recycled (flush:priv = &donekernel stack at hpt27xx_osm_bsd.c:303; internal ldm commands:privper blob),ext->vbus_extresolves to a stack/garbage pointer andldm_reset_vbus(<wild>)panics the kernel (fatal trap inhpt_timeout/ldm_reset_vbus). - Observed on the audit guest: no HighPoint PCI device exists
(
pciconf -lshows only i440FX/PIIX4/bochs-VGA/virtio), so the driver'shpt_probe()never attaches,hpt_final_init()prints "no controller detected", no/dev/hpt27xxnode and no CAM sim are created, and the racing code paths are unreachable. Seerun.log.
Contents
VERDICT.mdβ full narrative with path:line proof chain and the honest verification classification (not reproducible without the controller).race_proof.txtβ annotated source walk of the race.verify_env.sh/run.shβ guest environment probe.run.logβ actual probe output from the audit guest (2026-09-19).env.txtβ guest uname, kernel, module state.fix.diffβ git-apply-able fix (guarded reset under the vbus lock).manifest.json,verdict.json.
DF-4369 β VERDICT
Status: not_reproduced (hardware-gated β no RocketRAID 27xx controller on the audit guest).
Static analysis: the race is real; confidence likely.
The bug
sys/dev/raid/hpt27xx/hpt27xx_osm_bsd.c arms a per-command timeout whenever
a CAM read/write is queued:
- scatter/gather fast path:
callout_reset(ccb->ccb_h.timeout_ch, HPT_OSM_TIMEOUT, hpt_timeout, pCmd)β hpt27xx_osm_bsd.c:730-731 - busdma path: same call from
hpt_io_dmamap_callbackβ hpt27xx_osm_bsd.c:530-531
HPT_OSM_TIMEOUT is 20*hz (os_bsd.h:190).
The completion path os_cmddone() (hpt27xx_osm_bsd.c:421-466) runs from the
interrupt / taskqueue context (always under the vbus lock, taken by
hpt_pci_intr() at :836 or hpt_do_tasks() at :386):
static void os_cmddone(PCOMMAND pCmd)
{
POS_CMDEXT ext = (POS_CMDEXT)pCmd->priv; /* :423 */
union ccb *ccb = ext->ccb;
callout_stop(ccb->ccb_h.timeout_ch); /* :428 β NOT synchronizing */
...
cmdext_put(ext); /* :463 β ext back on free list */
ldm_free_cmds(pCmd); /* :464 β pCmd back on LDM freelist */
xpt_done(ccb); /* :465 */
}
The timeout handler runs in softclock context without any lock:
static void hpt_timeout(void *arg) /* :411 */
{
PCOMMAND pCmd = (PCOMMAND)arg;
POS_CMDEXT ext = (POS_CMDEXT)pCmd->priv; /* :414 β UAF read if freed */
ldm_reset_vbus((PVBUS)ext->vbus_ext->vbus); /* :418 β wild deref if recycled */
}
DragonFly's callout_stop() explicitly does not protect against this:
"Stops a callout if it is pending or queued, does not block. This function does not interlock against a callout that is in-progress." β sys/kern/kern_timeout.c:1087-1088 (contract in sys/sys/callout.h:131)
So the following interleaving is legal on an SMP box:
| CPU 0 (interrupt, vbus lock held) | CPU 1 (softclock) |
|---|---|
os_cmddone(pCmd) entered |
20 s deadline expires; hpt_timeout(pCmd) dequeued and starts running |
callout_stop(timeout_ch) β returns; cannot stop the in-progress callback |
|
cmdext_put(ext) |
|
ldm_free_cmds(pCmd) β pCmd on the LDM freelist |
|
(new I/O on another thread) ldm_alloc_cmds() recycles pCmd |
reads pCmd->priv (:414) β stale or foreign |
reads ext->vbus_ext (:418) |
|
ldm_reset_vbus(<wild>) β fatal trap |
priv is a dedicated field of struct _COMMAND (him.h:417), not overlaid by
freelist linkage, so after a plain freeβidle it still points at the old (still
mapped, contents-intact) OS_CMDEXT and the outcome is a benign spurious bus
reset. The crash needs the freed PCOMMAND to be recycled before :414 runs:
- recycled as a flush command (
hpt_flush_vdev, hpt27xx_osm_bsd.c:291-303):pCmd->priv = &doneβ a kernel stack address;ext->vbus_extthen reads stack garbage andldm_reset_vbus()gets a wild pointer; - recycled as an LDM-internal command (rebuild/verify, blob code):
privper the blob's internal convention (can be NULL β NULL-page deref).
Why it was not reproduced
The audit guest (QEMU/KVM, DragonFly 6.5-DEVELOPMENT X86_64_GENERIC) has the
driver compiled in (sys/config/X86_64_GENERIC:117; kldstat -v shows
86 pci/hpt27xx) but no HighPoint controller: pciconf -l lists only
i440FX/PIIX4/bochs-VGA/virtio devices, dmesg says
hpt27xx: no controller detected. (hpt_final_init, hpt27xx_osm_bsd.c:1021),
there is no /dev/hpt27xx and no CAM sim. The racing code paths
(hpt_scsi_io β callout_reset β os_cmddone/hpt_timeout) are only
reachable with a real RocketRAID 27xx doing I/O. See run.log.
No amount of guest-side setup can synthesize a PCI 0x1103 SAS controller, so
this stays honestly not_reproduced (device absent), with the static proof
above carrying the finding.
Field-trigger scenario (for the maintainer)
Any RocketRAID 27xx system under SMP with storage that completes a command in
the same instant its 20-second deadline expires (failing/detaching disk,
firmware stall recovering at ~20 s, heavy VM stall). Observed field symptom
would be a Fatal double memory fault / panic in hpt_timeout on storage
boxes, previously attributed to the controller.
Exploitability
Panic/DoS only: the corrupted dereference is a read; ldm_reset_vbus then
chases a wild pointer. There is no attacker-controlled write primitive.
Not a privilege-escalation path.
Fix
fix.diff β serialize hpt_timeout() with the completion path by taking the
vbus lock (it is recursive: lockinit(..., LK_CANRECURSE),
hpt27xx_osm_bsd.c:1040) and re-validating pCmd->priv before acting. If the
command was recycled, priv no longer equals the captured ext and the reset
is skipped; if it merely completed and sits idle on the freelist, the reset is
equivalent to today's behavior but now race-free. callout_stop_sync() was
deliberately NOT used: os_cmddone runs in interrupt context and
hpt_timeout needs the same vbus lock, so a synchronizing stop would
deadlock. Fix validation is not_testable on this guest (no hardware).
Verified-negative side work (same file, killed candidates)
See race_proof.txt appendix: the hpt_get_logical_devices count shuffle is
clamped by the blob (disassembly: the lib's get-logical-devices returns
min(total, maxcount)); the cmdext NULL-deref cannot happen (devq openings ==
cmdext pool size == os_max_queue_comm); the flush/ioctl sleep loops are
lost-wakeup-safe because lksleep drops the lock atomically and completions
run under the same lock.
Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.
Fix verification
not_testablefix.diff authored (hpt_timeout takes the recursive vbus lock and re-validates pCmd->priv==ext and pCmd->done==os_cmddone before resetting). Cannot be build-validated on the audit guest: no HighPoint controller, and reproducing the baseline race itself requires one. callout_stop_sync() was deliberately avoided β os_cmddone runs in interrupt context while hpt_timeout needs the same vbus lock, so a synchronizing stop would deadlock.
fix.diff; VERDICT.md 'Fix' section
Confirmed kernel references
- sys/dev/raid/hpt27xx/hpt27xx_osm_bsd.c:411
- sys/dev/raid/hpt27xx/hpt27xx_osm_bsd.c:414
- sys/dev/raid/hpt27xx/hpt27xx_osm_bsd.c:418
- sys/dev/raid/hpt27xx/hpt27xx_osm_bsd.c:421
- sys/dev/raid/hpt27xx/hpt27xx_osm_bsd.c:428
- sys/dev/raid/hpt27xx/hpt27xx_osm_bsd.c:463
- sys/dev/raid/hpt27xx/hpt27xx_osm_bsd.c:464
- sys/dev/raid/hpt27xx/hpt27xx_osm_bsd.c:530
- sys/dev/raid/hpt27xx/hpt27xx_osm_bsd.c:730
- sys/dev/raid/hpt27xx/hpt27xx_osm_bsd.c:303
- sys/dev/raid/hpt27xx/os_bsd.h:190
- sys/dev/raid/hpt27xx/him.h:417
- sys/kern/kern_timeout.c:1087
Detail
Evidence (decisive lines)
race_proof.txt (arm/completion/timeout interleaving table with path:line); VERDICT.md; run.log line 'hpt27xx: no controller detected.' proving the guest cannot reach the code; fix.diff (lock+revalidate fix)
PoC changes
No seeded PoC existed. Wrote an environment-probe script (verify_env.sh/run.sh) and a disassembly-verified negative-analysis appendix (race_proof.txt) covering the blob's GET_LOGICAL_DEVICES clamping (min(total,maxcount) at .text 0x77330/0x7f1c0), which killed a candidate stack-overflow theory.
Verified recommended fix
In hpt_timeout(), take the (recursive) vbus lock and re-validate that pCmd still belongs to this completion path (pCmd->done == os_cmddone && pCmd->priv == ext) before calling ldm_reset_vbus(), so a timeout racing os_cmddone's callout_stop/free cannot dereference a recycled PCOMMAND.
Verdict
Real race, hardware-gated: hpt_timeout() (hpt27xx_osm_bsd.c:411-419, softclock, no lock) dereferences pCmd->priv after the completion path os_cmddone() (:421-466) callout_stop()s the timer and frees both the OS_CMDEXT (:463) and the PCOMMAND (:464). DragonFly callout_stop() explicitly 'does not interlock against a callout that is in-progress' (sys/kern/kern_timeout.c:1087-1088), so a timeout firing on another CPU at the completion instant reads a freed/recycled command: if the block was recycled as a flush command priv points at a kernel stack variable (&done, :303) and ldm_reset_vbus() receives a wild pointer -> fatal trap (panic/DoS, no write primitive). Requires a RocketRAID 27xx controller; the audit guest has none (pciconf shows no vendor 0x1103 device; dmesg: 'hpt27xx: no controller detected.'), so the race could not be executed there β classification is not_reproduced with the static proof chain in the evidence pack carrying the finding.
No comments yet.