# DF-3168 — VERDICT

**Status: reproduced** (stock INVARIANTS kernel #0 Jul 2 06:02:54) —
**fix validated** (patched kernel #1 Sep 7 13:18:26).

## The bug

`hammer_ioc_rebalance()` (`sys/vfs/hammer/hammer_rebalance.c:53`) is the
backend of `hammer rebalance` / `hammer cleanup` (HAMMERIOC_REBALANCE,
gated by `caps_priv_check(SYSCAP_NOVFS_IOCTL)` at `hammer_ioctl.c:72`).
For every internal node visited on the way up, `rebalance_node()`
(:266) locks the node's children (`hammer_btree_lock_children`,
hammer_btree.c:2401) which stores each child's **on-disk** `count`
straight into `item->count` (hammer_btree.c:2473) — node load validates
only the CRC (`hammer_load_node`, hammer_ondisk.c:1313-1325); the count
is an unvalidated `int32_t` (hammer_btree.h:225: "maximum 62 for
INTERNAL, 63 for LEAF" — convention only).

`rebalance_node` then consumes it with no bounds check:

- `count += item->count` (:330) — signed-int accumulation of unvalidated
  int32s (sum can wrap negative; `KKASSERT(avg_elms >= 0)` :334 is
  INVARIANTS-only).
- `avg_elms = howmany(count, lockroot.count)` (:333) — avg can exceed the
  63-element `elms[]` array of the 4096-byte kmalloc'd node copies
  (`hammer_btree_lock_copy`, hammer_btree.c:2505).  The saturation clamp
  in the ioctl (:75-78) bounds `rebal->saturation` to [31,62]; it does
  NOT bound `avg_elms` when the forged counts already exceed saturation.
- pack loop `for (i = 0; i < item->count; ++i)` (:371) with
  `elm = &base_item->copy->elms[base_count]; *elm = node->ondisk->elms[i];`
  (:406-407) — `base_count` runs up to `avg_elms-1` → **64-byte-granule
  writes past the end of the 4KB copy** (heap OOB WRITE), and
  `node->ondisk->elms[i]` for i ≥ 63 reads past the node's 4KB slice
  (OOB READ, in-buffer up to +16KB, then adjacent kernel heap).
- `rebalance_closeout()` stores `copy->count = base_count` > 63 (:541-543)
  which `hammer_btree_sync_copy()` (:2519-2537) writes back to the
  on-disk node — persistent filesystem corruption; every later
  `btree_search` into that leaf OOB-reads (KKASSERT at hammer_btree.c:1278
  is INVARIANTS-only).
- the scan loop also consumes the count: `cursor.index = count - 1`
  (:174) then `elm = &cursor.node->ondisk->elms[cursor.index].leaf`
  (:194) → OOB read whose 40 bytes are returned to the ioctl caller in
  `rebal->key_cur` (:195).

This is the DF-0776 family (missing on-disk count validation) at THIS
file's sinks — DF-0776 covered the btree-core search/shift loops in
hammer_btree.c; the rebalance pack loop is a distinct set of sinks that
DF-0776's analysis did not cover and that needs its own validation (or a
global load-time bounds check).

## Threat model

Attacker crafts a HAMMER1 image (all CRCs recomputed — the CRC is not a
MAC) and gets an administrator to mount it and run `hammer rebalance` /
`hammer cleanup` on it (standard maintenance of attacker-supplied or
attacker-corrupted volumes).  The ioctl itself is privileged, so the
unprivileged-user→root chain requires that admin interaction — same
preconditions as DF-0776/DF-3167.

## Reproduction (baseline, stock kernel)

1. `mkimage.sh` — 12G vn image, ~2500 dirents → B-Tree: root
   `0x2103f000`, mid-level internals, 156 leaves (count=32 each).
2. `patch_leaf_rebal walk` — selects parent `0x21001000` (first child at
   the root) and its 14 conforming non-first leaf children (slice 0/1 of
   their 16K buffers, in-buffer `elms[127]`).
3. `patch_leaf_rebal patch` — forges `count=128` on those 14 leaves,
   re-CRCs each node.  Expected runtime math: 31 children, total =
   17*32 + 14*128 = 2336 → `avg_elms = ceil(2336/31) = 76` > 63.
4. `trigger.sh` — mount rw, `hammer rebalance /mnt`.

Observed (console_full.log / panic.txt):

    rebalance_node: lockroot 0xfffff801166db308 count 31
    rebalance_node: add count 32   (x17)
    rebalance_node: add count 128  (x14)   <- unvalidated disk counts
    rebalance_closeout: 8000000021000000: (count update) (parent bound update 1)
    ... (x31, includes the forged leaves 0x21004000 etc.)
    panic: assertion "slab->magic == KMALLOC_SLAB_MAGIC && delta != slab->ncount"
           failed in _kfree_obj at /usr/src/sys/kern/kern_kmalloc.c:942
    _kfree_obj() -> fdrop() -> closef() -> kern_close() -> syscall2()

The pack loop wrote elms[63..75] (13 granules x 64B = 832B) past the end
of each of ~30 kmalloc'd 4KB node copies; the overflow smashed kmalloc
slab metadata and was detected when an unrelated close() syscall freed a
neighboring object.  This is a **direct, observed out-of-bounds kernel
heap write**, not a speculative one.  The guest stayed in DDB (down).

Note: the panic interrupted the run before umount, so the >63 count
sync-to-disk is evidenced by the `(count update)` closeout lines and by
the fixed-kernel readback comparison below rather than by a baseline
readback.

## Primitive characterization (why this matters beyond the panic)

- **Length control**: overflow extent = `avg_elms - 63` granules; the
  image author chooses each child count (up to INT32_MAX; with 62
  children the sum itself overflows int at :330), so the write length is
  attacker-chosen (bytes to megabytes).
- **Content control**: the copied granules are `node->ondisk->elms[i]` —
  for i ≤ 62 these are the leaf's own on-image elements (fully
  attacker-controlled 64-byte blocks); for 63 ≤ i ≤ ~254 they are bytes
  of the same 16KB hammer buffer (on-image data of neighboring nodes or
  zeroed free space — still attacker-controlled at image-authoring
  time); beyond that, adjacent kernel heap (uncontrolled).
- **Placement**: sequential forward overflow from a 4096-byte kmalloc
  from the hammer m_misc pool; on this INVARIANTS guest the smash was
  detected by the slab-magic check; on production kernels there is no
  such check (silent corruption of adjacent heap objects).
- **Gating**: the sink requires a privileged ioctl caller (root runs
  `hammer rebalance`/`hammer cleanup` on the attacker's volume), so an
  *unprivileged* local user cannot reach it alone — the honest impact
  ceiling here is High (kernel heap corruption from a crafted volume
  during routine maintenance), matching DF-0776/DF-3167, not an
  unpriv→uid0 chain.

## Fix validation

`fix.diff` adds two bounds checks using the existing
`hammer_node_max_elements()` helper (hammer_btree.h:283):

1. scan loop (:146ff): validate `cursor.node->ondisk->count` and type
   before the count is consumed (:174/:194) — EIO aborts the scan.
2. `rebalance_node` accumulation (:355ff): validate each `item->count`
   before `count += item->count` — EIO, no packing.

Kernel #1 (Sep 7 13:18:26) built in-guest (`make nativekernel
KERNCONF=X86_64_GENERIC`), same crafted image, same patch:

    hammer_ioc_rebalance: rebalance: bad node 8000000021004000 type L count 128
    Rebalance /mnt failed: Input/output error
    Rebalance: 0 B-Tree nodes scanned, 0 rebalanced
    VERIFY: 14 out-of-bounds counts found   <- ONLY the leaves we forged

vs. baseline (31 leaves rewritten with count=76 + slab-smash panic).
Bad behavior GONE; guest healthy; clean umount.

## Kernel references

- sys/vfs/hammer/hammer_rebalance.c:75-78 (saturation clamp — bounds
  saturation, not avg_elms)
- sys/vfs/hammer/hammer_rebalance.c:174,194-195 (scan consumes count)
- sys/vfs/hammer/hammer_rebalance.c:326-334 (unvalidated accumulation +
  int overflow)
- sys/vfs/hammer/hammer_rebalance.c:371,406-407 (copy loop: OOB read
  source + OOB write dest)
- sys/vfs/hammer/hammer_rebalance.c:541-543 (count > 63 synced to disk)
- sys/vfs/hammer/hammer_btree.c:2473 (item->count = child->ondisk->count)
- sys/vfs/hammer/hammer_ondisk.c:1313-1325 (load validates CRC only)
- sys/vfs/hammer/hammer_btree.h:225 (count: int32, max by convention)

## Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest `make -j6 nativekernel KERNCONF=X86_64_GENERIC` completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.