# DF-2645 — flush-driven indirect collapse vs concurrent child modify → repchange KKASSERT / orphaned subtree

**Status: not_reproduced (2 stress attempts, ~35 min of racing; race is
code-proven but was not won; attempt 2 degraded into an ENOSPC flush
storm — DF-2633 family behavior — before the window could be hit).**

## Root-cause analysis (code-proven)

`hammer2_flush_core()` calls `hammer2_chain_indirect_maintenance(parent,
chain)` for every INDIRECT child with UPDATE set
(sys/vfs/hammer2/hammer2_flush.c:1039-1042).  The collapse loop in
maintenance *knows* about concurrent modification races and skips children
whose in-memory bref no longer matches the parent's media entry:

```c
		if (bcmp(&bsave, &sub->bref) ||          /* chain.c:4202 */
		    sub->parent != chain ||
		    (sub->flags & HAMMER2_CHAIN_DELETED)) {
			hammer2_chain_unlock(sub);       /* skip, not moved */
			...
			continue;
		}
```

…but the code after the loop assumes the loop emptied the chain:

```c
	hammer2_chain_repchange(parent, chain);          /* chain.c:4228 */
```
```c
hammer2_chain_repchange(...)
{
	KKASSERT(chain->core.live_count == 0 && RB_EMPTY(&chain->core.rbtree));
	                            /* chain.c:2314 — UNCONDITIONAL KKASSERT */
```

The two are mutually inconsistent under exactly the race the skip guard
anticipates: a child that was flushed by this pass's downward scan
(flush_recurse, hammer2_flush.c:1163-1257) and then **modified again by
the frontend before the bottom-up phase reaches maintenance** has a fresh
COW bref (new data_off from hammer2_chain_modify, chain.c:1437+) that no
longer matches the parent's media blockref; the skip guard leaves it in
the rbtree with live_count ≥ 1; repchange's KKASSERT fires → panic on
INVARIANTS kernels (DragonFly KKASSERT is unconditional) and mis-migrated
reptracks + silently orphaned subtree on others (the chain's own bref was
already deleted from the grandparent at chain.c:4160, so the skipped
child's media entry becomes unreachable → silent data loss).

Race window: from the end of flush_core(sub)'s bottom-up (its parent
blocktable update, hammer2_flush.c:1116-1134) to maintenance's
combined_find/bcmp (chain.c:4179-4210) — the remainder of B's RB_SCAN
plus B's own CRC/flush work.  An unprivileged local user running
append-write loops against files whose inode chains live under a
collapsible indirect block races the syncer flush.

## Experiments (guest, stock kernel #0)

1. run_df2645.sh 12 min mode 1: 6000-file dir, 70% pre-deleted,
   4 append writers (~130M appends total), churn create/delete children,
   `sync` loop at 20 Hz.  Result: no panic, clean umount
   (full_run.log).  Post-mortem: the pre-stress sparsification syncs had
   already collapsed the sparse indirects, so maintenance had no
   eligible work during the race phase.
2. Same with continuous key-space-wide churn (full_run2.log): writers
   + create/delete churn filled the 512MB volume mid-run; dmesg filled
   with `hammer2_flush: ... error=00000020` (HAMMER2_ERROR_ENOSPC from
   the chain_modify(parent) failure path, hammer2_flush.c:1057-1064) and
   the teardown `sync` wedged in `h2coll` (ENOSPC retry storm — DF-2633
   family, not this finding).  No `LOST CHILD*`, no `debug repchange`,
   no `excessive loops` markers: the maintenance skip was never taken
   with a live child before ENOSPC took over.

## Why it stays `likely` (not demoted)

Every link in the chain is individually verifiable in source: the skip
guard exists and is reachable with a non-deleted, rbtree-resident child
(bcmp mismatch via COW modify); repchange unconditionally asserts
emptiness; the call order guarantees the assert runs after any skip.
What was not achieved is winning the scheduler interleaving inside the
window on this guest within the time budget (and the volume-fill side
effect of the churn harness sabotaged attempt 2).

## Suggested fix

```diff
--- a/sys/vfs/hammer2/hammer2_chain.c
+++ b/sys/vfs/hammer2/hammer2_chain.c
@@ repchange caller (indirect_maintenance, chain.c:4226-4228)
-	hammer2_chain_unex ... spin release ...
-	hammer2_chain_repchange(parent, chain);
+	if (chain->core.live_count == 0 && RB_EMPTY(&chain->core.rbtree)) {
+		hammer2_chain_repchange(parent, chain);
+	} else {
+		/* children were skipped due to concurrent modification;
+		 * leave the chain linked (not deleted from the parent's
+		 * rbtree/blocktable) and let the next flush retry the
+		 * collapse.  Requires undoing the chain delete at
+		 * chain.c:4160 (defer it to this point). */
+		...
+	}
```

The robust structure is to make the collapse loop's skip path count
skips and, if any occurred, abort the collapse *before*
hammer2_chain_delete(parent, chain) performs any destructive step
(move the delete after a successful full sweep, or pre-flight-verify
like the DF-2632 fix does for key-range overlaps).