ips_morpheus_poll deadline mixes units: seconds + watchdog ticks (10 s each) β crash-dump poll window 10Γ too short, dump proceeds on unacknowledged writes
Summary
command->timeout is 10 and decremented once per 10-SECOND callout tick (~100s watchdog allowance), but the poll computes time_uptime + timeout - a 10-SECOND deadline. The poll is the only completion mechanism in the crash-dump path (interrupts off): a controller slower than 10s makes the dump continue with the write still in flight - silent crash-dump corruption. Scale-by-10 fix.
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-4616 Β· 6 files| File | Type | Description | Size | |
|---|---|---|---|---|
| README.md | β | 2.1 KB | β raw | |
| VERDICT.md | β | 2.3 KB | β raw | |
| env.txt | β | 273 B | view raw | |
| fix.diff | β | 671 B | view raw | |
| manifest.json | β | 677 B | view raw | |
| verdict.json | β | 2.5 KB | view raw |
DF-4616 β ips_morpheus_poll computes its deadline in seconds but command->timeout counts 10-second watchdog ticks: the crash-dump poll window is 10Γ too short
What
sys/dev/raid/ips/ips.c:607-616:
void
ips_morpheus_poll(ips_command_t *command)
{
uint32_t ts;
ts = time_uptime + command->timeout;
while (command->timeout != 0 &&
ips_morpheus_check_intr(command->sc) == 0 &&
(ts > time_uptime))
DELAY(1000);
}
command->timeoutis set to10at issue time (ips_issue_morpheus_cmd, ips.c:601) and is decremented once per 10-second watchdog tick byips_timeout(callout every10 * hz, ips.c:369, decrement at ips.c:342). Its unit is therefore "10-second ticks": a command is allowed ~100 s by the watchdog.time_uptimeis seconds.ts = time_uptime + 10yields a deadline 10 seconds in the future β a 10Γ unit mismatch.
ips_morpheus_poll is the synchronous-completion engine of the crash-dump
path: ipsd_dump_map_sg (ips_disk.c:322) calls sc->ips_poll_cmd(command)
immediately after issuing the write, with interrupts disabled β polling is
the only completion mechanism during a dump. When the controller takes
longer than 10 s (large dump block through a busy cache-flushing controller β
exactly the state a machine is in when it is crashing), the poll bails out
with the command still outstanding:
ipsd_dump_map_sgreturns as if the block had been written;- the dump proceeds on an unacknowledged write β missing/garbage blocks in the crash dump (silent evidence corruption), or a dump that fails downstream when the command eventually completes into a dead kernel.
Also in the same function: ips_morpheus_check_intr is invoked without
queue_lock here (the real handler takes it, ips.c:585) β harmless in the
dump context where interrupts are off, but a latent race if any other caller
ever appears.
Build / run
Requires a morpheus ServeRAID controller (0x1014:0x01BD) and a kernel crash
on a machine whose dump device is an ips volume. The audit guest has no such
hardware (env.txt).
Contents
VERDICT.md, env.txt, fix.diff, manifest.json, verdict.json.
VERDICT β DF-4616
Status: not_reproduced β requires a morpheus ServeRAID controller as the
dump device of a crashing kernel; the audit guest has no ServeRAID hardware
(env.txt). The unit mismatch itself is arithmetically certain from the
source; the dump-corruption consequence is the likely downstream effect
(confidence: certain for the bug, likely for the worst-case consequence).
Trace
- Unit of
timeout:ips_issue_morpheus_cmdsetscommand->timeout = 10(ips.c:601).ips_timeoutruns every10 * hz(ips.c:369) and decrements each outstanding command's timeout once per run (ips.c:342) β 1 unit β 10 s; the watchdog reaps at ~100 s. - Unit of the deadline:
time_uptimeis seconds (kern_clock); sots = time_uptime + command->timeout(ips.c:611) = now + 10 s. - Poll exit:
(ts > time_uptime)fails after 10 s regardless ofcommand->timeout != 0(which stays 10: during a crash dump no callouts run to decrement it).
Caller: ipsd_dump_map_sg (ips_disk.c:321-322) β sc->ips_issue_cmd(command);
sc->ips_poll_cmd(command); with the kernel dumping (interrupts disabled;
ips_copperhead_poll for the other family just prints "not implemented",
which is its own statement about how little this path is exercised).
After ips_morpheus_poll returns early:
ipsd_dump_map_sgreturns βipsd_dumptreats the block as written;- the controller may still be mid-transfer when the dump moves on β unacknowledged/interleaved blocks in the crash dump β silent corruption of forensic evidence; or a hang if the dump loop later waits on the same command's completion.
Honest ceiling
Crash-dump integrity on morpheus ServeRAID hardware; not a runtime privilege or memory-safety issue. Low severity, filed because it is a plain arithmetic defect in an evidence-critical path.
Fix
Scale the deadline by the watchdog period: ts = time_uptime +
(uint32_t)command->timeout * 10; (and ideally bound the loop regardless).
See fix.diff.
Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.
Fix verification
not_testableNo ServeRAID dump device on the guest; A/B kernel run impossible. fix.diff is a one-line unit correction with no behavioral risk.
fix.diff (ips.c:611 deadline scaled by the 10 s watchdog period).
Confirmed kernel references
Detail
Evidence (decisive lines)
VERDICT.md (unit proof: ips.c:601 sets timeout=10, ips.c:342+369 decrement once per 10s tick, ips.c:611 deadline = seconds + ticks, ips_disk.c:321-322 sole caller is the dump path); env.txt (no ServeRAID hardware)
PoC changes
none - no PoC possible; requires morpheus ServeRAID dump device on crashing hardware.
Verified recommended fix
Scale the poll deadline to seconds: ts = time_uptime + (uint32_t)command->timeout * 10.
Verdict
ips_morpheus_poll (ips.c:611) computes its poll deadline as time_uptime + command->timeout, but command->timeout is in watchdog ticks of 10 s (set to 10 at ips.c:601, decremented once per 10 s callout at ips.c:342/369) while time_uptime is seconds - the synchronous completion window is 10 s instead of the intended 100 s. This poll is the crash-dump completion engine (ips_disk.c:322, interrupts disabled), so a controller slower than 10 s (cache-flush-heavy during a crash) makes the dump proceed on unacknowledged writes -> silently corrupted crash dumps or a wedged dump. Arithmetic is certain from source; not reproduced on the audit guest because it requires a morpheus ServeRAID controller (0x1014:0x01BD) as the dump device of a crashing kernel and the guest has none (env.txt). Secondary note: the poll calls ips_morpheus_check_intr without queue_lock (fine in dump context, latent race if ever reused).
No comments yet.