β¬’ DragonFlyBSD Kernel Audit
← triage Β· dashboard
DF-3640

bnx_reset acquires the NVRAM software-arbitration lock (SET1/GNT1) and never releases it β€” flash arbiter held for the driver's lifetime, locking out APE/management firmware

Summary

bnx_reset() requests the on-chip NVRAM arbitration (SET1, polls GNT1 up to 160ms, prints and CONTINUES on timeout) and BGE_NVRAMSWARB_CLR1 appears zero times in if_bnx.c (guest-side grep against byte-identical source: CLR1-in-bnx == 0 vs bge == 1). bnx_reset runs on every attach/init/stop/watchdog/error-recovery cycle, so the host holds the flash arbiter for the driver's lifetime - the APE management firmware (NC-SI/DASH) and any other arbiter agent are locked out of NVRAM until power cycle. The bge code this driver derives from pairs acquire/release; upstream tg3/FreeBSD bge release after chip reset; bce had the same family as DF-3605. No kernel memory-safety impact; permanent loss of NVRAM arbitration by the management plane. Fix: release after bnx_ape_unlock(GRC).

Discussion (0)

No comments yet.

PoC verification

Evidence pack

findings/poc/DF-3640 Β· 10 files
FileTypeDescriptionSize
README.md β€” 1.5 KB ↓ raw
VERDICT.md β€” 4.1 KB ↓ raw
trigger.c β€” 2.5 KB view raw
build.sh β€” 117 B view raw
run.sh β€” 204 B view raw
build.log β€” 74 B view raw
run.log β€” 777 B view raw
env.txt β€” 974 B view raw
fix.diff β€” 396 B view raw
verdict.json β€” 3.6 KB view raw

DF-3640 β€” bnx_reset() NVRAM arbitration lock leak β€” evidence pack

What: bnx_reset() (sys/dev/netif/bnx/if_bnx.c:2585-2592) requests the on-chip NVRAM software arbitration (BGE_NVRAMSWARB_SET1 / polls GNT1) and never writes BGE_NVRAMSWARB_CLR1 β€” zero occurrences in the file (grep executed on the guest's byte-identical source). Every attach/init/stop/watchdog/error-recovery cycle re-asserts it; the host holds the flash arbiter for the driver's lifetime.

Impact (hardware-conditional): APE management firmware (NC-SI/DASH) and any other arbiter agent are locked out of NVRAM until power cycle. The timeout path also prints-and-continues without the grant (2591-2592). No kernel memory-safety impact β†’ Low / base:dev.

Reproduce (audit guest β€” honest negative):

  cc -O -Wall -o trigger trigger.c     # guest; builds clean
  ./run.sh                             # via dfbsd-qemu/vm.sh run_root

run.log shows: pci/if_bnx in kernel, NO_BROADCOM_DEVICE, sysctl hw.bnx unknown oid, src md5 == audited tree, and the decisive grep: CLR1 count in bnx: 0 (bge: 1) β†’ runtime verdict inconclusive (source-level certainty).

On real hardware: after driver load, BGE_NVRAM_SWARB GNT1 stays set forever (kgdb/pciconf register read).

Fix: fix.diff β€” release the arbitration after bnx_ape_unlock(GRC) in bnx_reset(), matching bge's paired idiom (if_bge.c:588).

Files: VERDICT.md (full narrative), trigger.c, build.sh/run.sh, build.log/run.log (untrimmed guest transcripts), env.txt, manifest.json, verdict.json.

VERDICT.md
↓ download raw

DF-3640 β€” bnx_reset() holds the NVRAM software-arbitration lock forever

File: sys/dev/netif/bnx/if_bnx.c (pass 2, GLM 5.3) Class: shared-resource lock leak / management-plane interop (CWE-667) Severity filed: Low, bucket base:dev, confidence certain (source fact) Verdict: inconclusive at runtime β€” no Broadcom hardware on the audit guest; the lock state itself is chip-internal and not observable without the device.

1. Root cause (path:line)

bnx_reset(): - sys/dev/netif/bnx/if_bnx.c:2585 β€” CSR_WRITE_4(sc, BGE_NVRAM_SWARB, BGE_NVRAMSWARB_SET1); - sys/dev/netif/bnx/if_bnx.c:2586-2590 β€” poll up to 8000Γ—20Β΅s (160 ms) for BGE_NVRAMSWARB_GNT1 - sys/dev/netif/bnx/if_bnx.c:2591-2592 β€” on timeout: print and continue anyway

and then performs the GRC core-clock reset, PCI state restore, APE lock/unlock (bnx_ape_unlock(sc, BGE_APE_LOCK_GRC) at 2715) β€” but never writes BGE_NVRAMSWARB_CLR1. Verified: grep -c NVRAMSWARB_CLR1 if_bnx.c β†’ 0 (executed against the guest's byte-identical /usr/src copy, see run.log).

Callers of bnx_reset(): bnx_attach() (2092), bnx_init() (3604 β€” i.e. every ifconfig bnx0 up, MTU change at 3857, watchdog reset at 3941, and the error-recovery path bnx_handle_status() β†’ bnx_init() at 2983), and bnx_stop() (4000). The arbitration request/grant is therefore asserted continuously for the whole lifetime of the driver.

2. Why this is a bug (sibling/upstream comparison)

  • The code this driver derives from pairs the lock: bge(4) bge_nvram_getbyte() β€” sys/dev/netif/bge/if_bge.c:551-558 acquires (SET1/GNT1) and if_bge.c:588-589 releases (BGE_NVRAMSWARB_CLR1 + read-back).
  • Upstream Broadcom tg3 and FreeBSD bge release the NVRAM arbitration when they are done resetting the chip (tg3_chip_reset: "Release NVRAM arbitration"; FreeBSD bge_reset similarly) β€” i.e. the chip reset does NOT implicitly return the grant, otherwise those drivers would not bother.
  • bce(4) had the same family of defect filed as DF-3605 (arbitration + access enable left held on an error path).

Note the bce case (DF-3605) broke the driver's own later NVRAM reads. bnx cannot self-break because bnx_read_nvram() is a stub returning 1 (if_bnx.c:405-409) β€” the victim is instead every other agent of the arbiter, principally the APE management firmware (NC-SI/DASH) these chips run for out-of-band management, and any other driver instance sharing the part.

3. Impact

No kernel memory-corruption. The host permanently holds the flash/NVRAM arbitration on the device: - APE/management firmware NVRAM access (persistent configuration) fails from driver load until power cycle β†’ out-of-band management plane breakage. - Timeout handling is also wrong (print-and-continue at 2591-2592 proceeds to touch NVRAM-adjacent state without the grant), compounding the interop hazard on malfunctioning arbiters.

Rated Low: requires the specific hardware; no direct kernel memory-safety impact; consequence is management-plane availability.

4. Guest evidence (why inconclusive)

run.log (executed on the guest against its /usr/src): - pci/if_bnx in kernel, NO_BROADCOM_DEVICE, sysctl hw.bnx unknown oid β†’ bnx_reset() unreachable at runtime on this guest. - md5 if_bnx.c = 49a18c4b7642b9b2ebd1d02ad11542e4 == audited tree. - grep -c NVRAMSWARB_CLR1 if_bnx.c β†’ 0; only NVRAMSWARB uses are 2585/2587 (acquire/poll). bge comparison: 1 CLR1.

The lock is chip-internal state; with no device, nothing further is observable β†’ honest inconclusive / reproduced: 0.

5. Fix

fix.diff β€” release the arbitration (CLR1 + read-back, matching bge's idiom) right after bnx_ape_unlock(sc, BGE_APE_LOCK_GRC) in bnx_reset(), before the firmware-handshake poll. Not runtime-validatable on this guest: fix_status: not_testable.

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.

Fix verification

not_testable
baseline no→ patch + rebuild →patched clean

fix.diff adds the paired CLR1 release at the point upstream drivers release (post-reset, before the firmware handshake poll). Cannot be runtime-validated: the guest has no Broadcom device, so the arbiter state cannot be observed before or after the patch (same reason the PoC is inconclusive). The patch touches only register writes on an existing path.

fix.diff; run.log (NO_BROADCOM_DEVICE => neither baseline nor patched behavior observable)
↓ fix.diffper-fix-DF-3640

Confirmed kernel references

Detail

Evidence (decisive lines)

run.log (guest: driver in kernel, no Broadcom device, no attach, src md5 49a18c4b7642b9b2ebd1d02ad11542e4, grep CLR1-in-bnx=0 vs bge=1, NVRAMSWARB lines 2585/2587 only), VERDICT.md (full path:line trace + upstream comparison), trigger.c (annotated hardware-precondition trigger), fix.diff (CLR1 release after bnx_ape_unlock in bnx_reset)

PoC changes

No seed existed (new pass-2 finding); trigger written as an annotated hardware-precondition skeleton, built clean (-O -Wall) and executed on the guest; the decisive evidence is the guest-side grep of the byte-identical source.

Verified recommended fix

In bnx_reset, write BGE_NVRAMSWARB_CLR1 (+ read-back) after bnx_ape_unlock(BGE_APE_LOCK_GRC) to release the NVRAM arbitration, matching bge's paired idiom (see fix.diff).

Verdict

Source-level certain: bnx_reset (sys/dev/netif/bnx/if_bnx.c:2585-2592) requests the on-chip NVRAM software arbitration (BGE_NVRAMSWARB_SET1, polls GNT1 up to 160ms, prints and CONTINUES on timeout) and never releases it β€” BGE_NVRAMSWARB_CLR1 appears zero times in if_bnx.c (grep executed on the guest's byte-identical /usr/src copy, run.log). bnx_reset runs on every attach/init/stop/watchdog/error-recovery cycle, so the host holds the flash arbitration for the driver's lifetime, locking out the APE management firmware (NC-SI/DASH) and any other arbiter agent until power cycle. The bge code this driver derives from pairs acquire/release (if_bge.c:551-590) and upstream tg3/FreeBSD bge release after chip reset; bce had the same family filed as DF-3605. Runtime verification honestly inconclusive: no Broadcom PCI device on the audit guest (NO_BROADCOM_DEVICE, sysctl hw.bnx unknown oid), and the lock state is chip-internal, unobservable without hardware.