mount -u ronly→rw transition: RW internal state, FWRITE device opens, and recovered-volume-header media writes are not rolled back when stage2 recovery fails
Summary
On MNT_UPDATE ronly->rw, hammer_vfs_mount() clears hmp->ronly and reopens every device vnode FWRITE (:478-482) BEFORE recovery, and hammer_recover_flush_buffers(...,1) (:485) writes the stage1-recovered root volume header to media BEFORE stage2 runs. When hammer_recover_stage2 fails (:486 - corrupt undo/blockmap, e.g. 'Undo size is absurd' EIO), the error is returned (:509) but nothing rolls back hmp->ronly=0 or the device modes: sys_mount() restores only mp->mnt_flag/mnt_kern_flag. The mount stays presented as read-only while internally RW: new inodes load without HAMMER_INODE_RO, every hmp->ronly-gated write ioctl (mirror/prune/reblock/dedup) becomes usable, the flusher runs RW against the FWRITE devices, and the 'failed' mount already mutated media (half-applied recovery). Operator mounts suspect media RO (succeeds), attempts mount -u -o rw (fails visibly), then operates a filesystem the kernel reports as MNT_RDONLY but which accepts write ioctls and flusher writes into half-recovered on-disk state - integrity destruction of evidence-bearing media plus a misled operator. Not executed (deterministic stage2-only failure needs REDO_SYNC-dependent crafting; state machine fully line-proven in pack VERDICT.md). Fix: roll back ronly/device modes on stage2 failure + don't adopt the failed-recovery blockmap.
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-3096 · 3 files| File | Type | Description | Size | |
|---|---|---|---|---|
| VERDICT.md | — | 4.6 KB | ↓ raw | |
| verdict.json | — | 2.3 KB | view raw | |
| manifest.json | — | 363 B | view raw |
DF-3096 — mount -u ronly→rw: RW internal state and media writes are not rolled back when stage2 recovery fails
Target: sys/vfs/hammer/hammer_vfsops.c:475-509 (the MNT_UPDATE
transition), interacting with sys/kern/vfs_syscalls.c:399-410 (flag
rollback) and sys/vfs/hammer/hammer_recover.c:1427-1467 (media write
of the recovered volume header).
Mechanism (all lines verified against the audit tree)
hammer_vfs_mount() MNT_UPDATE ronly→rw path:
478: if (hmp->ronly && (mp->mnt_kern_flag & MNTK_WANTRDWR)) {
479: hkprintf("read-only -> read-write\n");
480: hmp->ronly = 0; /* (a) cleared FIRST */
481: RB_SCAN(... hammer_adjust_volume_mode, NULL); /* (b) devices reopened FWRITE */
483: rootvol = hammer_get_root_volume(hmp, &error);
484: if (rootvol) {
485: hammer_recover_flush_buffers(hmp, rootvol, 1); /* (c) recovered root vol header WRITTEN TO MEDIA */
486: error = hammer_recover_stage2(hmp, rootvol); /* (d) may FAIL */
487: bcopy(rootvol->ondisk->vol0_blockmap, hmp->blockmap, ...);
490: hammer_rel_volume(rootvol, 0);
491: }
...
508: lwkt_reltoken(&hmp->fs_token);
509: return(error); /* (e) error returned, NO state rollback */
- (c) with
final=1flushes the stage1-recovered root volume header to media:hammer_recover_flush_buffers()→hammer_io_flush(&root_volume->io, 0)(hammer_recover.c:1460-1466) — the write happens before stage2 is known to succeed. - (d)
hammer_recover_stage2fails on corrupt undo/blockmap data (e.g. the "Undo size is absurd" EIO at hammer_recover.c:~610, or REDO-term scan errors). - (e) the error propagates to
sys_mount(), which rolls back only the flags:mp->mnt_flag = flag; mp->mnt_kern_flag = flag2;(vfs_syscalls.c:405-408) — restoringMNT_RDONLYon the mount — whilehmp->ronlystays 0 and every device vnode stays open read-write.
Consequences on the still-mounted, RO-presenting filesystem
hammer_get_inode()stampsHAMMER_INODE_ROonlyif (hmp->ronly)(hammer_inode.c:469-470) — new inodes are created writable.- Every HAMMER ioctl that gates on
hmp->ronly(mirror-write, prune, reblock, dedup, snapshot ops — hammer_ioctl.c) is write-enabled on a mount the administrator believes is read-only after seeing the failedmount -u. - The flusher runs in RW mode against the FWRITE-opened devices.
- Media state: the recovered volume header was already rewritten (half-applied recovery) even though mount reported failure — the filesystem on disk is mutated by a mount operation that "failed".
Threat model
Privileged: requires mount -u -o rw (root) on a filesystem whose
stage2 recovery fails — corrupt/crafted media (post-incident forensics
scenario: mount -o ro succeeds, the subsequent RW update attempt
fails halfway) or a device developing I/O errors mid-recovery. The
resulting state misleads the operator and enables writes into a
half-recovered filesystem.
Why Phase V was not executed (status: untested)
Triggering requires an image that passes stage1 at RO mount but fails stage2 at the update — stage2's reachable error returns sit behind REDO_SYNC-record state ("Undo size is absurd" only fires when stage1 found REDO records; term-scan errors need crafted REDO chains), which makes a deterministic crafted image a research project of its own (DF-3067/3068-era crafts fail at stage1 or panic instead). The state machine itself is fully code-proven above. Risk/effort for a Medium, root-triggered integrity bug was judged disproportionate this run.
Suggested fix
--- a/sys/vfs/hammer/hammer_vfsops.c
+++ b/sys/vfs/hammer/hammer_vfsops.c
@@ rootvol = hammer_get_root_volume(hmp, &error);
if (rootvol) {
hammer_recover_flush_buffers(hmp, rootvol, 1);
error = hammer_recover_stage2(hmp, rootvol);
+ if (error) {
+ /* roll the RW transition back */
+ hmp->ronly = 1;
+ RB_SCAN(hammer_vol_rb_tree, &hmp->rb_vols_root,
+ NULL, hammer_adjust_volume_mode, NULL);
+ }
bcopy(rootvol->ondisk->vol0_blockmap, hmp->blockmap,
sizeof(hmp->blockmap));
(the bcopy after a failed stage2 should also be dropped — the
blockmap of a failed recovery must not replace the live one).
Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.
(No fix.diff for this item: documented recommendation only - see Fix above.)
Fix verification
not_testableConfirmed kernel references
Detail
Evidence (decisive lines)
['findings/poc/DF-3096/VERDICT.md - line-cited mechanism and suggested rollback fix']
PoC changes
n/a - no PoC authored; a deterministic trigger requires an image that passes RO-mount stage1 but fails stage2 (REDO_SYNC-dependent error paths), judged disproportionate for a Medium root-triggered integrity bug this run
Verified recommended fix
On stage2 failure in the MNT_UPDATE ronly->rw path, restore hmp->ronly=1 and re-run hammer_adjust_volume_mode before returning the error; do not bcopy the failed-recovery blockmap
Verdict
Not exercised on the guest this run (conditional-verify criteria not met: Medium severity, base:vfs bucket, privileged trigger). The state machine is fully code-proven: on a MNT_UPDATE ronly->rw transition hammer_vfs_mount() clears hmp->ronly and reopens all device vnodes FWRITE (vfsops:478-482) BEFORE running stage2 recovery, and hammer_recover_flush_buffers(...,1) at vfsops:485 writes the stage1-recovered root volume header to MEDIA (hammer_recover.c:1460-1466) before stage2 is known to succeed. When stage2 then fails, the error is returned (vfsops:509) but hmp->ronly stays 0 - sys_mount() rolls back only mp->mnt_flag/mnt_kern_flag (vfs_syscalls.c:405-408), leaving a kernel-RO-flagged mount whose internal state is RW: inodes load without HAMMER_INODE_RO (hammer_inode.c:469-470), all hmp->ronly-gated write ioctls (mirror/prune/reblock/dedup) become usable, and media was already mutated by the 'failed' update.
No comments yet.