Backtrace fault outside an established DDB session longjmps into the BSS-zeroed global db_jmpbuf β on the vkernel the longjmp is host libc's from inside the SIGSEGV handler: restores rsp=0/rbp=0, vkernel process dies of SIGILL inside longjmp+115, core dumped, zero further diagnostics (DF-5471 twin)
Summary
db_read_bytes arms db_nofault=&db_jmpbuf but the global is only setjmp'd inside an established DDB session - every vkernel panic runs print_backtrace BEFORE Debugger() with the buffer zeroed, and live debug sites call it with no debugger at all. vkernel flavor: setjmp/longjmp are host libc's taken from inside a signal handler; the zeroed buffer restores rsp=0/rbp=0 and the process dies of SIGILL at libc longjmp+115, never emitting anything again - the trace_on_panic forensic record is annihilated. Reproduced 2x deliberate + 1x accidental (harness logs the zeroed panic-path state, then Fatal trap 12 -> longjmp -> core dump shows rsp=0 rbp=0); walk-side fix validated, full fix wants a local setjmp at print_backtrace entry.
Discussion (0)
PoC verification
Evidence pack
findings/poc/DF-5473 Β· 23 files| File | Type | Description | Size | |
|---|---|---|---|---|
| README.md | β | 4.8 KB | β raw | |
| VERDICT.md | β | 4.1 KB | β raw | |
| verdict.json | β | 4.5 KB | view raw | |
| fix.diff | β | 2.2 KB | view raw | |
| src/vk_trace_harness.c | β | 3.8 KB | view raw | |
| src/vktrigger.c | β | 2.2 KB | view raw | |
| src/run_vk.sh | β | 1.9 KB | view raw | |
| src/build_image.sh | β | 2.1 KB | view raw | |
| vktrigger.c | β | 2.2 KB | view raw | |
| run_vk.sh | β | 1.9 KB | view raw | |
| build_image.sh | β | 2.1 KB | view raw | |
| vkrun_clean.log | β | 2.2 KB | view raw | |
| vkrun_bad.log | β | 2.1 KB | view raw | |
| vkrun_fixed.log | β | 2.3 KB | view raw | |
| vkrun_smoke.log | β | 3.7 KB | view raw | |
| vkexit_clean | β | 10 B | β download | |
| vkexit_bad | β | 10 B | β download | |
| vkexit_fixed | β | 10 B | β download | |
| core_gdb.txt | β | 496 B | view raw | |
| env.txt | β | 250 B | view raw | |
| vk_build2.log | β | 25.6 KB | view raw | |
| vk_build3.log | β | 502 B | view raw | |
| vk_build5.log | β | 298.8 KB | β download |
DF-5473 β vkernel64 backtrace fault outside a DDB session longjmps into the BSS-zeroed global db_jmpbuf β libc longjmp restores rsp=0/rbp=0 β vkernel process dies of SIGILL inside longjmp (pc64 DF-5471 twin, vkernel flavor)
File: sys/platform/vkernel64/x86_64/db_trace.c (pass 2; mechanism spans
sys/platform/vkernel64/x86_64/db_interface.c, sys/ddb/db_command.c)
Severity: Medium Β· Confidence: certain Β· Verified: reproduced on the QEMU guest (in-vkernel rig), twice
What
Every dereference the vkernel backtracer performs goes through
db_get_value() β db_read_bytes(), which arms the global no-fault recovery
db_nofault = &db_jmpbuf (vkernel db_interface.c:236) around a raw pointer
dereference. On a fault, the vkernel signal path
(exc_segfault β kern_trap β trap_pfault β trap_fatal β kdb_trap,
trap.c:959) executes longjmp(*no_fault, 1) (db_interface.c:138-142).
db_jmpbuf (sys/ddb/db_command.c:62, BSS-zeroed) is only initialized by
setjmp() inside an established DDB session (db_command.c:467
db_command_loop, db_trap.c:68). But the vkernel runs the backtracer:
- On every panic, before DDB exists β VKERNEL64 config has
options DDB_TRACE(sys/config/VKERNEL64:59), sotrace_on_panic=1andpanic()callsprint_backtrace(6)(kern_shutdown.c:880-881) beforeDebugger("panic")β withdb_jmpbufstill all-zero. The harness prints the state right before the walk:db_active=0 db_jmpbuf[0]=0 db_jmpbuf[3]=0. - From live kernel debug print sites with no debugger involved β the
same unguarded
print_backtrace(-1)sites as pc64 (uipc_mbuf.c:1339, kern_jail.c:785, indefinite2.h:173, busdma_machdep.c:205, ...).
The vkernel is a host userland process, so this longjmp/setjmp pair is
host libc's, and the fault semantics differ from pc64 in the worst way:
the fault arrives as an asynchronous host SIGSEGV inside libc's memcpy-ish
loop in db_read_bytes, and the recovery longjmps out of a signal handler
into a buffer that has never been set.
Observed crash chain (guest-proven)
db_stack_trace_cmd adopts guest-user VA 0x100000000 (DF-5472)
β db_get_value/f_retaddr read (db_trace.c:231) faults
β host SIGSEGV β exc_segfault (exception.c:263) β kern_trap (trap.c:583)
β T_PAGEFLT β trap_pfault(FALSE) β vm_fault(kernel_map, 0x100000008) fails
β trap_fatal (trap.c:887) β kdb_trap (trap.c:959)
β prints "kernel: type 12 trap, code=0" (db_interface.c:136)
β db_nofault armed β longjmp(db_jmpbuf, 1) on the ZEROED buffer
β libc longjmp restores rsp=0, rbp=0 (rbx=0) β faults at longjmp+115
β process killed by SIGILL: pid (kernel.stripped), uid 0: exited on
signal 4 (core dumped) [guest host dmesg]
β core: #0 0x102903f3 <longjmp+115> in /lib/libc.so.8, rsp=0x0, rbp=0x0
After the longjmp the vkernel produces zero further output: no DDB
prompt, no second "Fatal trap", no reboot β the original panic's evidence
(the whole point of trace_on_panic) is annihilated and the process leaves
an 11 MB core instead of a backtrace. Reproduced twice (accidental first
run, then deliberate); the isolated mechanism was confirmed with a standalone
longjmp-on-zeroed-jmp_buf test on the guest (exit 139, SIGSEGV β in the
vkernel the surrounding context turns it into SIGILL inside libc longjmp).
Proof (this pack)
vkrun_clean.logβ control (chain stays in kernel KVA): walk completes, sysctl returns, vkernel survives (VK_EXIT=0, clean halt).vkrun_bad.logβ the crash run: 54 lines, ends atkernel: type 12 trap, code=0; nothing after. Guest host dmesg:exited on signal 4 (core dumped).core_gdb.txt:#0 <longjmp+115> in /lib/libc.so.8, rsp=0x0, rbp=0x0.vkrun_fixed.logβ withfix.diff(walk refuses non-kernel frames) the identical trigger prints two frames and returns; no fault, no signal, clean halt. The walk-side fix removes the reachable trigger; the full remediation additionally wantsdb_interface.cto arm a jmp_buf that is valid wheneverdb_nofaultcan fire (e.g.setjmpa local inprint_backtrace()/db_stack_trace_cmd()entry, or fall back todb_global_jmpbufwhen no DDB loop ever ran) β noted as out-of-file scope.
Build / run
# on the guest (see README steps in full in run_vk.sh/build_image.sh): sysctl -w vm.vkernel_enable=1 sh /root/vkrig/build_image.sh # UFS root image with vktrigger sh /root/vkrig/run_vk.sh clean # control: survives sh /root/vkrig/run_vk.sh bad # crash: SIGILL in libc longjmp sh /root/vkrig/run_vk.sh fixed # after fix.diff: survives
The vkernel binary is make nativekernel KERNCONF=VKERNEL64 (requires the
DF-5474 build fix first) with the rig harness
sys/platform/vkernel64/x86_64/vk_trace_harness.c compiled in.
DF-5473 VERDICT β reproduced (in-vkernel rig, twice + 1 accidental)
Bottom line: A fault taken by the vkernel backtracer outside an
established DDB session longjmps into the never-initialized global
db_jmpbuf (sys/ddb/db_command.c:62) β because db_read_bytes arms
db_nofault = &db_jmpbuf (vkernel db_interface.c:236) but the buffer is only
setjmp'd inside db_command_loop (db_command.c:467) / db_trap
(db_trap.c:68). On the vkernel the longjmp is host libc's, run from
inside a signal handler, into a BSS-zeroed buffer: it restores rsp=0/rbp=0
and the process dies of SIGILL inside libc longjmp+115 (core-dumped) β
destroying the vkernel, the panic backtrace it was printing, and any chance
of DDB. State proven identical to the panic path: db_active=0,
db_jmpbuf zeroed (harness print), which is exactly the state during
print_backtrace(6) at kern_shutdown.c:881 (runs BEFORE Debugger("panic"))
and during every live print_backtrace(-1) debug site.
Evidence chain (all files in this pack)
vkrun_bad.log(also in DF-5472 pack): walk adopts guest-user VA βFatal trap 12: page fault while in kernel mode fault virtual address = 0x100000008 <- db_trace.c:231 f_retaddr read instruction pointer = 0x2b:0x71ca40 <- movzbl (%rdi),%ecx = db_read_bytes copy loop (db_interface.c:239-240) current process = 3 (vktrigger) <- live syscall context, not DDB kernel: type 12 trap, code=0 <- kdb_trap print (db_interface.c:136), immediately before the longjmp <log ends β line 54 of 54>Nothing further is ever printed: no DDB prompt, no second Fatal trap, no reboot.- Guest host dmesg:
pid (kernel.stripped), uid 0: exited on signal 4 (core dumped)(two entries: accidental + deliberate runs). core_gdb.txt+ first-session symbolization (binary matching at crash time):#0 0x102903f3 in longjmp () from /lib/libc.so.8,rip 0x102903f3 <longjmp+115>,rsp 0x0,rbp 0x0,Cannot access memory at address 0x0. (The regenerated core_gdb.txt against the rebuilt binary shows the same rip=0x102903f3, signal 4; the guest's libc is stripped so the symbol came from the matching-build session.)- Isolated mechanism check on the guest:
longjmpon a memset-0 jmp_buf β SIGSEGV exit 139 (ljtest) β inside the vkernel's signal-handler context the same restore dies as SIGILL at longjmp+115. vkrun_clean.log: identical rig with the chain kept in kernel KVA β walk completes and returns; the vkernel survives. (The crash is caused by the out-of-DDB fault recovery, not by the walk harness itself.)
Exploit chain
No host privilege escalation (vkernel = unprivileged host process). In-vkernel
impact: any vkernel panic whose frame chain is corrupted (the moment the
backtrace is needed most), or any live print_backtrace() debug site
(uipc_mbuf.c:1339, kern_jail.c:785, indefinite2.h:173, ...) hitting an
unmapped frame, converts a survivable diagnostic print into the death of the
kernel process with total loss of forensic output.
Fix validation
The walk-side fix.diff (KvaStart/KvaEnd + forward-progress validation in
db_trace.c, shared with DF-5472) removes the demonstrated trigger: identical
bad run prints two frames, DFVK: walk RETURNED without crashing, clean halt
(vkrun_fixed.log). fix_status: fixed (walk-side); the complete remediation
additionally requires db_interface.c to arm a jmp_buf valid whenever
db_nofault can fire (e.g. setjmp a local in print_backtrace()/db_stack_trace_cmd()
entry, or fall back to db_global_jmpbuf when no DDB loop has run) β flagged
as out-of-this-file scope, mirroring pc64 DF-5471's note.
Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.
Fix verification
fixedpatched db_trace.c (KvaStart/KvaEnd walk validation) removes the demonstrated trigger: identical bad run prints 2 frames, 'DFVK: walk RETURNED without crashing', sysctl returns, clean halt, no signal-4 death (vkrun_fixed.log). db_interface.c-level hardening noted as the complete remediation.
findings/poc/DF-5473/vkrun_fixed.log lines 34-46 vs findings/poc/DF-5473/vkrun_bad.log; fix.diff applies clean to repo tree (git apply --check)
Confirmed kernel references
- sys/platform/vkernel64/x86_64/db_trace.c:231
- sys/platform/vkernel64/x86_64/db_trace.c:443
- sys/platform/vkernel64/x86_64/db_interface.c:136
- sys/platform/vkernel64/x86_64/db_interface.c:138
- sys/platform/vkernel64/x86_64/db_interface.c:236
- sys/ddb/db_command.c:62
- sys/ddb/db_command.c:467
- sys/ddb/db_trap.c:68
- sys/kern/kern_shutdown.c:881
- sys/platform/vkernel64/x86_64/trap.c:583
- sys/platform/vkernel64/x86_64/trap.c:959
- sys/platform/vkernel64/x86_64/exception.c:263
Detail
Exploit chain
any vkernel panic with a corrupted frame chain (or a live debug-print site hitting an unmapped frame) -> SIGSEGV in db_read_bytes -> exc_segfault -> kern_trap -> trap_pfault -> trap_fatal -> kdb_trap -> longjmp(never-initialized db_jmpbuf) -> libc longjmp restores rsp=0/rbp=0 -> SIGILL at longjmp+115 -> vkernel process dies, core dumped, panic backtrace and DDB both lost. No host privesc (vkernel is an unprivileged host process).
Evidence (decisive lines)
vkrun_bad.log (54 lines, ends at 'kernel: type 12 trap, code=0'); core_gdb.txt (rip=0x102903f3, rsp=0x0, rbp=0x0, signal 4) plus matching-build first-session symbolization 'longjmp+115 in /lib/libc.so.8' recorded in VERDICT.md; guest dmesg 'pid (kernel.stripped), uid 0: exited on signal 4 (core dumped)'; vkrun_clean.log (control survives β crash is the recovery path, not the harness)
PoC changes
authored fresh in-vkernel rig (no seed): sysctl-driven db_stack_trace_cmd call from live syscall context reproducing the exact panic-path state; VKERNEL64 kernel built on the guest (required DF-5474 defsym workaround to link) + minimal UFS root image
Verified recommended fix
walk-side: validate frames (fix.diff, shared with DF-5472). complete fix: db_interface.c must arm a jmp_buf that is valid whenever db_nofault can fire (setjmp a local in print_backtrace()/db_stack_trace_cmd() entry, or fall back to db_global_jmpbuf when no DDB loop ever ran)
Verdict
The pc64 DF-5471 defect is present in the vkernel fork with worse mechanics: db_read_bytes (vkernel db_interface.c:236) arms db_nofault=&db_jmpbuf, but db_jmpbuf is only setjmp'd inside an established DDB session (db_command.c:467, db_trap.c:68); every panic runs print_backtrace(6) BEFORE Debugger() (kern_shutdown.c:880-883, DDB_TRACE default) and live print_backtrace(-1) debug sites run with no debugger at all. On the vkernel the longjmp is host libc's, taken from inside the SIGSEGV handler, into a BSS-zeroed buffer: guest-proven twice β after 'kernel: type 12 trap, code=0' (kdb_trap's own print at db_interface.c:136) the vkernel produces zero further output and the host records 'exited on signal 4 (core dumped)'; the core shows #0 0x102903f3
No comments yet.