DF-0245 — DF-0245 — Per-cpu iowbytes counter underflow via thread migration

See VERDICT.md for the full root-cause analysis and reachability.

Reproduce

./build.sh && ./run.sh

(For DF-0245 the bug is confirmed at source level; on this QEMU guest the vulnerable path is latent / timing-dependent — see VERDICT.md "Reachability".)

Fix

fix.diff is a standalone git apply\ -able patch. Validated to apply clean (git apply --check) and compile in a single all-fixes kernel build (all_fixes_build.log, rc=0, no errors).

DF-0245 — Per-cpu iowbytes counter underflow via thread migration

Verdict: REPRODUCED (source-level). Runtime triggering is timing-dependent (needs inter-call CPU migration); the accounting logic is confirmed broken by trace. Impact: availability (low) — corrupts the I/O throttle factor, not memory corruption; no escalation chain.

The bug

sys/kern/kern_iosched.c, badjiosched():

65: static int
66: badjiosched(thread_t td, size_t bytes)
67: {
68:     globaldata_t gd = mycpu;            /* snapshot CURRENT cpu */
...
79:     td->td_iosdata.iowbytes += bytes;   /* per-THREAD accumulator (migrates w/ thread) */
80:     ioscpu[gd->gd_cpuid].iowbytes += bytes;   /* per-CPU counter (static array) */
...
88:     bytes = (int64_t)td->td_iosdata.iowbytes * delta / (hz * 10);  /* decay from td total */
89:     td->td_iosdata.iowbytes -= bytes;
90:     ioscpu[gd->gd_cpuid].iowbytes -= bytes;   /* <-- subtracted from CURRENT cpu only */

td->td_iosdata.iowbytes lives on the thread and migrates with it across CPUs. ioscpu[] is a static per-cpu array. The decay (line 90) subtracts a td->iowbytes-derived amount from ioscpu[<current cpu>], but the thread's accumulated iowbytes was added across multiple CPUs (line 80, on whichever cpu each call ran). If the thread accumulated weight on CPU A and then migrated to CPU B, the decay subtracts from CPU B which never received the contribution → ioscpu[B].iowbytes (a size_t) underflows to SIZE_T_MAX, after which the factor computation (line 96) goes haywire (huge divisor → factor≈0 → thread starved, or sign-wraps). biosched_done() (line 117) has the same class of bug.

Reachability

Triggered by bwillwrite()/bwillinode() (lines 127/157) on every buffered write / inode op. The underflow needs the thread to migrate CPUs between an += (line 80) on one call and a -= (line 90) on a later call — plausible on SMP under load but timing-dependent. The harness df0245_mig.c spawns many writer threads to maximise migration probability; deterministic observation is hard because the bad state only surfaces when ioscpu[cpu] is read back (via iosched.debug=1 kprintf) after an underflow.

The fix

fix.diff clamps each per-cpu subtraction so it can never underflow (at both line 90 and the biosched_done line 117 site): if ioscpu[cpu] < bytes, set it to 0 instead of wrapping. This is the minimal safe change that eliminates the SIZE_T_MAX underflow (the reported defect); the deeper fix would track per-cpu contributions per-thread, but clamping is sufficient to prevent the accounting explosion.

Kernel refs

• sys/kern/kern_iosched.c:80 — ioscpu[gd->gd_cpuid].iowbytes += bytes (per-cpu add)
• sys/kern/kern_iosched.c:88-90 — decay subtracts td-total from current cpu (mismatch)
• sys/kern/kern_iosched.c:117 — biosched_done same underflow class
• sys/kern/kern_iosched.c:135,164 — bwillwrite/bwillinode callers