HAMMER1 mirror-write: wire delete_tid == create_tid panics hammer_delete_at_cursor via hammer_mirror_update (duplicate-record state on production kernels)
Summary
hammer_mirror_check (:777-786) approves any wire delete_tid != 0 differing from the on-disk one without comparing to create_tid, and hammer_mirror_update (:814-833, only guards delete_tid==0) passes it straight into hammer_delete_at_cursor(ADJUST) which stores the tid first (hammer_object.c:2468) and then KKASSERTs delete_tid != create_tid (:2501-2502) - a wire delete_tid equal to the existing record's create_tid panics INVARIANTS kernels; production kernels store delete_tid==create_tid, the duplicate-record B-Tree invariant violation the code itself documents. One PASS mrec against any existing slave record whose create_tid the master knows (mirror reads disclose it) panics the slave kernel - no crafted record creation needed. VERIFIED stock INVARIANTS: REC mrec (well-formed) + PASS mrec {same base, delete_tid=create_tid} -> 'panic: assertion "delete_tid != create_tid" failed in hammer_delete_at_cursor at :2502'. Worse than DF-3148 on production kernels: stored duplicate-record state degrades B-Tree integrity. Fix validated (reject delete_tid <= create_tid -> EINVAL): record still created, mrec rejected, no panic.
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-3149 Β· 12 files| File | Type | Description | Size | |
|---|---|---|---|---|
| README.md | β | 2.1 KB | β raw | |
| VERDICT.md | β | 3.3 KB | β raw | |
| mirror_poc.c | β | 6.6 KB | view raw | |
| build.sh | β | 92 B | view raw | |
| run.sh | β | 152 B | view raw | |
| run.log | β | 800 B | view raw | |
| run.fixed.log | β | 309 B | view raw | |
| panic.txt | β | 749 B | view raw | |
| env.txt | β | 651 B | view raw | |
| fix.diff | β | 1.2 KB | view raw | |
| verdict.json | β | 3.4 KB | view raw | |
| manifest.json | β | 725 B | view raw |
DF-3149 β HAMMER1 mirror-write: wire delete_tid == create_tid panics hammer_delete_at_cursor
What
hammer_mirror_check() (hammer_mirror.c:777-786) approves any wire
delete_tid != 0 that differs from the on-disk one, and
hammer_mirror_update() (hammer_mirror.c:814-833) passes it unvalidated
into hammer_delete_at_cursor(ADJUST). When the wire delete_tid equals the
existing record's create_tid, the assignment at hammer_object.c:2468 creates
the forbidden state and the INVARIANTS assertion fires:
KKASSERT(elm->leaf.base.delete_tid != elm->leaf.base.create_tid);
/* hammer_object.c:2501-2502 */
One ioctl with two mrecs reproduces it: (1) REC creating a well-formed record
(create_tid=T, delete_tid=0, btype='R'), (2) PASS with the same base and
delete_tid=T. Reachable through both hammer_ioc_mirror_write_rec
(mirror.c:625-626) and hammer_ioc_mirror_write_pass (mirror.c:696-697).
On non-INVARIANTS production kernels there is no panic: the record is stored
with delete_tid == create_tid β the exact duplicate-record B-Tree invariant
violation the code's own comment warns about (hammer_object.c:2496-2502).
Threat model
Same as DF-3148: untrusted mirror master -> slave kernel panic; local root can trigger directly. Note this variant needs NO crafted record on the slave β any existing record whose create_tid the master knows (mirror reads disclose it) can be hit with a single PASS mrec.
Reproduce (baseline, stock INVARIANTS kernel)
dd if=/dev/zero of=/build/h.img bs=1m count=2048 vnconfig -c /dev/vn0 /build/h.img newfs_hammer -f -L MIRRORTEST /dev/vn0 mkdir -p /mnt && mount_hammer /dev/vn0 /mnt && touch /mnt/f1 cc -O -o /tmp/mirror_poc mirror_poc.c /tmp/mirror_poc /mnt/f1 duptid # -> kernel panic (console): assertion "elm->leaf.base.delete_tid != # elm->leaf.base.create_tid" failed in hammer_delete_at_cursor # at hammer_object.c:2502
Fix validation
Apply fix.diff in /usr/src, make nativekernel && make installkernel,
reboot, re-run: no panic; record 1 is created and the stream aborts with
head.error=EINVAL on mrec 2. See VERDICT.md.
VERDICT β DF-3149
Status: REPRODUCED (panic) / fix VALIDATED
Baseline (stock INVARIANTS kernel #0, 2026-07-02)
Single HAMMERIOC_MIRROR_WRITE ioctl, two mrecs:
1. REC: btype='R', create_tid=0x1000, delete_tid=0, data_len=0 β creates a
well-formed record through the normal mirror path.
2. PASS: same base, delete_tid=0x1000 (== create_tid).
Result: kernel panic (serial console):
panic: assertion "elm->leaf.base.delete_tid != elm->leaf.base.create_tid"
failed in hammer_delete_at_cursor
at /usr/src/sys/vfs/hammer/hammer_object.c:2502
hammer_delete_at_cursor() at hammer_delete_at_cursor+0x729
hammer_ioc_mirror_write() at hammer_ioc_mirror_write+0x860 (write_pass/update inlined)
hammer_ioctl() at hammer_ioctl+0xebe
hammer_vop_ioctl() at hammer_vop_ioctl+0x48
vop_ioctl() at vop_ioctl+0x63
(The KKASSERT spans hammer_object.c:2501-2502 in the audited tree; the guest build reports :2502.)
Root cause chain (line-accurate):
- hammer_mirror_check() (hammer_mirror.c:777-786) approves an update whenever
the wire delete_tid differs from the on-disk one and is nonzero β it never
compares against create_tid.
- hammer_mirror_update() (hammer_mirror.c:814-833) only guards
delete_tid == 0, then passes the wire tid straight to
hammer_delete_at_cursor(cursor, HAMMER_DELETE_ADJUST, mrec->leaf.base.delete_tid, ...)
(mirror.c:828-831).
- ADJUST stores it first (hammer_object.c:2467-2470), THEN asserts
delete_tid != create_tid (hammer_object.c:2501-2502). Wire
delete_tid == existing create_tid β guaranteed assertion failure.
- Reachable from both hammer_ioc_mirror_write_rec (mirror.c:625-626) and
hammer_ioc_mirror_write_pass (mirror.c:696-697). NOTE: unlike DF-3148 this
needs no crafted record creation β ANY existing slave record whose
create_tid the master knows (mirror reads disclose create_tid) is killable
with one PASS mrec.
Non-INVARIANTS production kernels: no panic; the record is stored with
delete_tid == create_tid β the exact duplicate-record B-Tree invariant
violation the code comment at hammer_object.c:2496-2502 warns about
(subsequent update chains can create duplicate keys).
Fix validation (patched kernel #1, 2026-09-07, fix.diff applied)
fix.diff rejects mrec->leaf.base.delete_tid <= cursor->leaf->base.create_tid
in hammer_mirror_update (hammer_mirror.c:832-841 after patch).
Re-run of the identical PoC on a fresh fs:
=== duptid on patched kernel === MIRROR_WRITE rc=0 head.error=22 head.flags=00008000 count=80 => kernel REJECTED the stream (error 22)
No panic; count=80 proves mrec 1 was still created normally, and mrec 2 was
rejected with EINVAL exactly at the update path. Guest stayed up.
Build: make -j6 nativekernel RC=0, make installkernel RC=0.
Bottom line
Certain. Single-ioctl kernel panic (INVARIANTS) / on-disk duplicate-record state (production) from an unvalidated wire delete_tid; malicious-master remote DoS + mirror-target B-Tree integrity corruption. Fix proven.
Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.
Fix verification
fixedSame patched build as DF-3148 (fix.diff includes the update-path guard): identical PoC on fresh fs creates mrec1's record (count=80) and aborts with head.error=22 on mrec2; no panic; guest up.
run.fixed.log; patched-kernel uname #1 (2026-09-07)
Confirmed kernel references
Detail
Exploit chain
gated ioctl or malicious mirror-master stream -> HAMMERIOC_MIRROR_WRITE -> write_rec/write_pass -> hammer_mirror_check approves -> hammer_mirror_update -> hammer_delete_at_cursor(ADJUST, wire delete_tid) -> stores delete_tid==create_tid (hammer_object.c:2468) -> KKASSERT (hammer_object.c:2501-2502) panic
Evidence (decisive lines)
['panic.txt: console panic + backtrace hammer_delete_at_cursor<-hammer_ioc_mirror_write', 'run.log: baseline two-mrec trigger (ssh dies, guest down)', 'run.fixed.log: patched kernel creates mrec1 (count=80) then rejects mrec2 with error 22, no panic', 'fix.diff: validated kernel fix']
PoC changes
Same harness as DF-3148; duptid mode sends REC(btype='R') + PASS(delete_tid==create_tid) in one 160-byte ubuf
Verified recommended fix
In hammer_mirror_update reject mrec->leaf.base.delete_tid <= cursor->leaf->base.create_tid (see fix.diff).
Verdict
One HAMMERIOC_MIRROR_WRITE with two mrecs (REC creating a well-formed record create_tid=T delete_tid=0; PASS with same base and delete_tid=T) panics the stock INVARIANTS kernel at hammer_delete_at_cursor() KKASSERT(delete_tid != create_tid) (hammer_object.c:2501-2502). hammer_mirror_check (mirror.c:777-786) approves any wire delete_tid != 0 differing from the on-disk one and hammer_mirror_update (mirror.c:814-833) passes it unvalidated; the ADJUST path stores the tid first, then asserts. Production (non-INVARIANTS) kernels store delete_tid==create_tid - the duplicate-record B-Tree invariant violation hammer_object.c:2496-2502 explicitly warns about. Needs no crafted creation: any existing slave record whose create_tid the master knows is killable with a single PASS mrec. fix.diff (reject delete_tid <= cursor leaf create_tid) validated: record 1 still created (count=80), mrec 2 rejected EINVAL, no panic.
No comments yet.