blkmap_lock leaked on hammer_bnew error paths β goto failed jumps past hammer_unlock permanently deadlocking HAMMER filesystem
Summary
hammer_blockmap.c:285 hammer_lock_ex(&hmp->blkmap_lock). :372-377 if((next_offset&HAMMER_BUFMASK)==0){hammer_bnew_ext(...); if(*errorp){result_offset=0; goto failed}}. failed: label at :393 is PAST hammer_unlock at :392 = lock permanently held. Same pattern hammer_blockmap_reserve :546 lock acquired :604 hammer_bnew() :606 goto failed past unlock :611. Additionally hammer_blockmap_alloc :303 if reservation found :315 ++resv->refs never decremented on error path = refcount leak. Caller hammer_ip_add_bulk (hammer_object.c:974) checks resv==NULL NOT *errorp KKASSERT only INVARIANTS so production proceeds with leaked-lock reservation. Trigger: memory pressure (mmap+memset) concurrent with 16K-aligned HAMMER writes when hammer_bnew fails. blkmap_lock held = ALL HAMMER metadata ops block forever unkillable D-state. Fix: hammer_unlock before goto failed + release reservation ref.
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-0819 Β· 13 files| File | Type | Description | Size | |
|---|---|---|---|---|
| blkmap_lock_trace.c | trigger-source | code-level analysis harness proving lock-acquire-without-matching-release | 4.9 KB | view raw |
| hammer_blkmap_trigger.c | runtime-trigger | runtime trigger: mounts hammer fs, fills to ENOSPC under memory pressure, checks for deadlock | 4.2 KB | view raw |
| build.sh | build-script | exact cc build commands | 191 B | view raw |
| run.sh | run-script | runs the code analysis harness | 461 B | view raw |
| run.log | run-log | code analysis PoC output on patched kernel | 3.5 KB | view raw |
| fix.diff | suggested-fix | git-apply-able fix: add hammer_unlock before goto failed at lines 377 and 606 | 620 B | view raw |
| fix_build.log | fix-build-log | full nativekernel build output for single-fix kernel (rc=0) | 5.6 MB | β download |
| env.txt | environment | uname, cc version, sysctls | 270 B | view raw |
| VERDICT.md | verdict | full narrative: mechanism, reachability, fix validation | 5.8 KB | β raw |
| manifest.json | manifest | this file | 3.1 KB | view raw |
| build.log | build-log | kernel build log excerpt proving -Werror clean compile of patched source | 157 B | view raw |
| ../fix_build_combined.log | build-log | Combined 41-finding kernel build (rc=0, -Werror clean) | 5.6 MB | β download |
| ../fix_build_summary.txt | build-summary | Summary of the combined 41-finding kernel build | 826 B | view raw |
DF-0819 β blkmap_lock leaked on hammer_bnew error paths
Verdict: REPRODUCED (code-level) / LATENT β fix VALIDATED
The bug is a real, confirmed code defect: two goto failed statements in
hammer_blockmap_alloc() and hammer_blockmap_reserve() jump past the
hammer_unlock(&hmp->blkmap_lock) call, permanently leaking the lock on the
error path. If that error path is ever reached, the entire HAMMER filesystem
deadlocks permanently (every subsequent metadata operation blocks forever in
hammer_lock_ex() on blkmap_lock, unkillable D-state).
However, the error path is effectively dead code under normal operation:
hammer_bnew()/hammer_bnew_ext() call through to hammer_io_new() which
always returns 0 (getblk either succeeds or panics). Runtime testing on a
mounted HAMMER v1 filesystem confirmed the path cannot be triggered β the
filesystem returns clean ENOSPC (handled upstream by _hammer_checkspace)
and never deadlocks.
Classification: latent lock-leak bug β the code pattern is objectively wrong, the deadlock impact is certain IF the path is reached, but the path requires filesystem corruption / hardware I/O error / crafted image to trigger (it cannot be reached from a normally-functioning filesystem).
Mechanism (confirmed by code trace)
Bug 1: hammer_blockmap_alloc() (sys/vfs/hammer/hammer_blockmap.c:95)
285: hammer_lock_ex(&hmp->blkmap_lock); // LOCK ACQUIRED
...
372: if ((next_offset & HAMMER_BUFMASK) == 0) {
373: hammer_bnew_ext(trans->hmp, next_offset, bytes,
374: errorp, &buffer3);
375: if (*errorp) {
376: result_offset = 0;
377: goto failed; // <<<< JUMPS PAST UNLOCK
378: }
379: }
...
392: hammer_unlock(&hmp->blkmap_lock); // UNLOCK (skipped by goto)
393: failed: // label is AFTER the unlock
goto failed at line 377 targets label at line 393, which is AFTER the unlock
at line 392. The lock acquired at line 285 is NEVER released on this path.
Bug 2: hammer_blockmap_reserve() (sys/vfs/hammer/hammer_blockmap.c:419)
546: hammer_lock_ex(&hmp->blkmap_lock); // LOCK ACQUIRED
...
602: if (bytes < HAMMER_BUFSIZE && (next_offset & HAMMER_BUFMASK) == 0) {
603: if (!vm_paging_min_dnc(HAMMER_BUFSIZE / PAGE_SIZE)) {
604: hammer_bnew(hmp, next_offset, errorp, &buffer3);
605: if (*errorp)
606: goto failed; // <<<< JUMPS PAST UNLOCK
607: }
608: }
...
611: hammer_unlock(&hmp->blkmap_lock); // UNLOCK (skipped by goto)
613: failed: // label is AFTER the unlock
Same pattern: goto failed at line 606 targets line 613, past the unlock at 611.
Impact if triggered
blkmap_lock serializes ALL HAMMER blockmap (metadata + data) allocation.
A leaked lock means every subsequent hammer_blockmap_alloc/reserve/finalize/
free/dedup call blocks forever in hammer_lock_ex(). This is an unkillable
D-state deadlock affecting the entire mounted HAMMER filesystem.
Trigger reachability analysis
The error path requires hammer_bnew()/hammer_bnew_ext() to set *errorp.
Tracing the call chain:
hammer_bnew_ext (hammer_ondisk.c:1178)
β _hammer_bread (hammer_ondisk.c:1116)
β hammer_get_buffer (hammer_ondisk.c:696)
β hammer_load_buffer(buffer, isnew=1) (hammer_ondisk.c:881)
β hammer_io_new() (hammer_io.c:437)
β getblk(devvp, ...) β return(0) // ALWAYS returns 0
hammer_io_new() at hammer_io.c:437 always returns 0 (line 460). It
calls getblk() which either succeeds or panics (NULL deref at bp->b_ops
on line 444 if getblk returns NULL). So the isnew=1 path through
hammer_load_buffer cannot produce a non-zero error.
hammer_get_buffer() CAN set *errorp via hammer_get_volume() (returns
ENOENT if zone2_offset decodes to a non-existent vol_no β filesystem
corruption) or hammer_load_volume() (I/O error on backing device). These
are realistic on a corrupted/worn device or crafted image but NOT on a
normally-functioning filesystem.
Runtime confirmation: mounted a 12GB HAMMER v1 filesystem, wrote 68,760 16K-aligned blocks under memory pressure (3GB mmap+memset), then filled to 100% ENOSPC and continued writes. The filesystem returned clean ENOSPC errors in every case β never deadlocked. The error path was never reached.
Fix
Add hammer_unlock(&hmp->blkmap_lock) before each goto failed that occurs
after the lock is acquired. See fix.diff.
- Line 377 (hammer_blockmap_alloc): added unlock before
goto failed - Line 606 (hammer_blockmap_reserve): added braces + unlock before
goto failed
The fix is a pure safety addition: it only adds an hammer_unlock call on an
error path that was previously missing it. On the success path (which is the
only path reachable in practice), nothing changes.
Fix validation
- Unpatched baseline (#0): code trace confirms
goto failedat lines 377/606 skiphammer_unlockat lines 392/611. Runtime: filesystem works correctly (error path is dead code). - Patched kernel (#1):
hammer_unlock(&hmp->blkmap_lock)now present at lines 377 and 607 (error paths). Built single-fix kernel, booted, verified HAMMER v1 filesystem still works correctly (newfs, mount, write, read, stress, ENOSPC β all clean, no regression). - Result: fix is correct and introduces no regression. The lock is now released on all paths including the previously-leaking error path.
PoC changes
Created two files:
- blkmap_lock_trace.c β code-level analysis harness that prints the exact
line ranges proving the lock-acquire-without-matching-release.
- hammer_blkmap_trigger.c β runtime trigger that mounts a HAMMER fs, fills
it, applies memory pressure, and checks for deadlock (confirms the error
path is dead code at runtime).
Fix verification
fixedVALIDATED the fix: on the unpatched #0 baseline, code trace confirms goto failed at lines 377/606 skips hammer_unlock at lines 392/611 (lock-leak pattern present). On the single-fix #1 kernel, hammer_unlock is now present at lines 377 and 607 on the error paths (lock-leak pattern eliminated). Built single-fix kernel (nativekernel rc=0), installed, rebooted, re-ran HAMMER v1 regression test: newfs, mount, 47K+ writes, ENOSPC -- all clean, no deadlock, no regression. Fix closes the code defect without affecting the success path.
baseline (#0): hammer_unlock NOT present between lock(285) and goto failed(377) -> lock leak. After fix (#1): line 377 `hammer_unlock(&hmp->blkmap_lock); goto failed;` and line 607 `hammer_unlock(&hmp->blkmap_lock); goto failed;` -> lock correctly released. Runtime on #1: HAMMER_FS_WORKING=yes, stress=47376 writes/0 errors, ENOSPC handled cleanly.
Confirmed kernel references
- sys/vfs/hammer/hammer_blockmap.c:285
- sys/vfs/hammer/hammer_blockmap.c:375
- sys/vfs/hammer/hammer_blockmap.c:377
- sys/vfs/hammer/hammer_blockmap.c:392
- sys/vfs/hammer/hammer_blockmap.c:393
- sys/vfs/hammer/hammer_blockmap.c:546
- sys/vfs/hammer/hammer_blockmap.c:605
- sys/vfs/hammer/hammer_blockmap.c:606
- sys/vfs/hammer/hammer_blockmap.c:611
- sys/vfs/hammer/hammer_blockmap.c:613
- sys/vfs/hammer/hammer_io.c:437
- sys/vfs/hammer/hammer_io.c:460
Detail
Exploit chain
none -- this is a lock-leak -> permanent filesystem deadlock (DoS), not memory corruption. No escalation chain applies. The leaked blkmap_lock causes all subsequent HAMMER metadata ops to block forever in hammer_lock_ex() (unkillable D-state). Impact ceiling: permanent DoS of the mounted HAMMER filesystem. The error path is latent (dead code under normal operation) so runtime exploitation requires a corrupted/worn device.
Evidence (decisive lines)
Code trace (hammer_blockmap.c): line 285 `hammer_lock_ex(&hmp->blkmap_lock)` acquired; line 375 `if (*errorp) {` -> line 377 `goto failed;` targets label at line 393 which is AFTER `hammer_unlock` at line 392 -> lock NEVER released. Same pattern in hammer_blockmap_reserve(): line 546 lock, line 606 `goto failed;` -> label at 613, past unlock at 611. Runtime test (12GB HAMMER v1, 68760 writes + ENOSPC): filesystem responsive, 0 errors, clean ENOSPC -- error path not reached (hammer_io_new always returns 0). Patched kernel (#1): hammer_unlock now at lines 377 and 607 on error paths; HAMMER v1 fs works with no regression.
PoC changes
Created blkmap_lock_trace.c (code-level analysis harness proving the lock-acquire-without-matching-release at the exact line ranges), hammer_blkmap_trigger.c (runtime trigger: mounts HAMMER fs, writes 16K blocks under memory pressure, fills to ENOSPC, checks for deadlock -- confirms error path is dead code), build.sh, run.sh. Authored fix.diff adding hammer_unlock before goto failed at both sites.
Verified recommended fix
Add hammer_unlock(&hmp->blkmap_lock) before goto failed at line 377 (hammer_blockmap_alloc) and line 606 (hammer_blockmap_reserve, with added braces). This ensures the lock is released on the hammer_bnew error path. Matches finding proposal. The full git-apply-able diff is in findings/poc/DF-0819/fix.diff.
Verdict
REPRODUCED (code-level). The bug is a confirmed code defect in sys/vfs/hammer/hammer_blockmap.c: in hammer_blockmap_alloc() the goto failed at line 377 jumps PAST hammer_unlock(&hmp->blkmap_lock) at line 392 (label failed: is at line 393), and in hammer_blockmap_reserve() the goto failed at line 606 jumps PAST hammer_unlock at line 611 (label at 613). Both lock acquisitions (lines 285 and 546) are never released on these error paths. blkmap_lock serializes ALL HAMMER metadata allocation, so a leaked lock deadlocks the entire filesystem permanently. The error path (hammer_bnew failure) is effectively dead code under normal operation because hammer_io_new() (hammer_io.c:437) always returns 0 -- confirmed by mounting a 12GB HAMMER v1 filesystem, writing 68K+ 16K blocks under 3GB memory pressure, then filling to 100% ENOSPC: the filesystem returned clean ENOSPC every time, never deadlocking. The path could only be triggered by filesystem corruption or hardware I/O error causing hammer_get_volume() to return ENOENT.
No comments yet.