DF-2660 — hammer2_sync_insert() re-lookup TOCTOU: concurrent same-key

create on a mounted rw cluster trips KKASSERT(chain == NULL) (panic on

INVARIANTS) or duplicate-key chain_create on production kernels

sys/vfs/hammer2/hammer2_synchro.c:736-758.

Medium / hammer2 bucket. Code defect: certain (the race window is structural). Reproduction: not attempted beyond bounded opportunity (status: untested) — see below.

Root cause (path:line)

The sync thread scans with the parent locked SHARED and the inode UNLOCKED (hammer2_inode_unlock at synchro.c:458), so VOP writers on a mounted rw cluster run concurrently against the scan. When the scan finds a key missing locally (n > 0 path, synchro.c:555-577), hammer2_sync_insert() relocks the parent exclusively and re-runs the lookup as a TOCTOU re-check:

sys/vfs/hammer2/hammer2_synchro.c:741-758:
    ...unlock child/parent...
    hammer2_chain_unlock(*parentp);
    hammer2_chain_lock(*parentp, HAMMER2_RESOLVE_ALWAYS);
    chain = hammer2_chain_lookup(parentp, &dummy,
                                 focus->bref.key, focus->bref.key,
                                 &error, ...);
    KKASSERT(chain == NULL);            /* fires on the race */
    chain = NULL;
    error = hammer2_chain_create(parentp, &chain, ...);

If a concurrent create/rename xop (which writes ALL columns, including the sync thread's own idx) inserts the same key between the shared iteration and the exclusive re-lock — e.g. a user does rm X; echo > X (same dirent key = same name hash) while the sync thread is inserting an older X from the focus — the KKASSERT panics the INVARIANTS kernel. On production kernels chain_create would encounter an existing overlapping chain (hammer2_chain_cmp treats overlap as match), with duplicate-key handling by the RB tree left to chance.

Note: on STOCK 2-master clusters this window is unreachable because the sync threads park forever in the DF-2657 quorum wedge before the scan; it is live on stock for >=3-master clusters (quorum reachable among 2 participants) and on any kernel carrying the DF-2657 fix.

Fix

if (chain != NULL) {
        /* raced a concurrent modify on our column: adopt it */
        hammer2_chain_unlock(chain);
        hammer2_chain_drop(chain);
        return (HAMMER2_ERROR_EAGAIN);
}
(replace the KKASSERT; the caller already handles EAGAIN by
re-running the pass — synchro.c:190-191.)

PoC status

untested — a deliberate hammer (tight rm X; echo > X loop against a diverged merged cluster across several 5 s sync passes) was planned for the fix kernel, but every multi-session cluster run on this guest was terminated early by the pre-existing DF-2620-family teardown UAF (see DF-2657 panic_serial.log), so no honest reproduction attempt completed. The window and the panic assertion are structural, line-cited above.