# DF-3539 — v1 netgraph ng_async: control/data-path race → kernel heap corruption (unbounded write past `sc->sbuf`)

## File
`sys/netgraph/async/ng_async.c` (v1 netgraph, 607 LOC)

## Root cause

v1 netgraph dispatches control messages and data **directly in the sender's
context with no per-node serialization**:

- `ng_send_msg()` (sys/netgraph/netgraph/ng_base.c:1226) calls the target
  node's `rcvmsg` synchronously — for a control socket this runs in cpu0's
  netisr thread (`so->so_port = netisr_cpuport(0)`, sys/kern/uipc_socket.c:259).
- `ng_send_data()` (ng_base.c:1687) calls the peer node's `rcvdata`
  synchronously — for the async hook this runs wherever the data arrived,
  e.g. an ng_ksocket upcall (`so_upcall = ng_ksocket_incoming`,
  ng_ksocket.c:605) executing inside udp/ip input on any CPU's netisr.

`ng_async` keeps mutable per-node state — `sc->abuf`, `sc->sbuf`, `sc->slen`,
`sc->fcs`, `sc->amode`, `sc->cfg` — with **zero locking**:

- `NGM_ASYNC_CMD_SET_CONFIG` (ng_async.c:303-341) reallocates both buffers:
  `kmalloc(new); kfree(sc->sbuf); sc->sbuf = buf; … sc->cfg = *cfg;`
  — the `kfree` can execute while a decoder/encoder on another CPU is
  still writing through the old pointer.
- The decode guard `if (sc->slen >= SYNC_BUF_SIZE(sc->cfg.amru))`
  (ng_async.c:595) reads `cfg.amru` and `sc->sbuf`/`sc->slen` without any
  consistency guarantee; once the race corrupts `sc->cfg.amru` (or
  `sc->slen`) the guard no longer bounds the write and the loop
  `sc->sbuf[sc->slen++] = ch` (ng_async.c:600) walks past the allocation
  without limit.

The ng7 twin kills exactly this with `NG_HOOK_FORCE_WRITER()` on both hooks
(sys/netgraph7/async/ng_async.c:257-275, comment: *"We use a static buffer
here so only one packet at a time can be allowed to travel in this
direction. Force Writer semantics."*) plus the cpu0 netgraph-thread funnel
(ng_snd_item → `lwkt_sendmsg(ng_cpuport(0))`, ng7 ng_base.c:1953). v1 has
neither.

## Reproduction (all as root on the stock guest)

Graph: `[ng control socket S] –sync– [async node A] –async– [ksocket K,
udp/127.0.0.1:<port>]`.

- 3 flood processes sendto() 8000-byte UDP datagrams of
  `[0x7e]['A'×63]…` → ksocket upcall → `nga_rcvdata(async hook)` →
  `nga_rcv_async` decode loop writing `sc->sbuf[sc->slen++]` continuously
  in ip-input context on CPUs 1–5.
- The control process loops `SET_CONFIG amru 8191 ↔ 8192` (~70k msg/s
  measured); every "grow" executes `kfree(sc->sbuf)` on cpu0 while the
  decoders run.

```
cc -O2 -Wall -o /root/race2 /root/race2.c
/root/race2 all config        # flood + SET_CONFIG realloc storm
```

## Observed (unpatched)

1. Run 1 & 2: guest hard-locked < 20 s — serial console shows only
   `Warning: objcache(mbuf cluster) exhausted on cpuN`, then silence;
   no panic banner, no dump; guest unresponsive; vm.sh's DDB heuristic
   force-killed QEMU (`guest not answering (likely DDB on panic)`).
2. Run 3: guest degraded over ~8 min; captured via the QEMU monitor with
   the vCPUs frozen:
   - CPU0 RIP `0xffffffff82656737` — inside ng_async.ko text, **not
     halted, CPL=0**; CPUs 1–5 idle-halted.
   - Disassembly at the RIP is exactly `nga_rcv_async`'s byte loop:
     `movq 0x28(%rbx),%rcx` (= `sc->sbuf`), `movl %esi,0x30(%rbx)`
     (= `sc->slen++`), `movb %al,(%rcx,%rdx)` (= the write),
     `movzwl 0x1a(%rbx)` (= `sc->fcs`, PPP_FCS) — struct-offset-proven.
   - **RDX = 0x5655 = 22101 = `sc->slen`** — 14 KB past the 8202-byte
     `sbuf` allocation (guard limit is `amru+10 ≤ 8202`), i.e. a
     runaway, self-amplifying kernel heap overflow writing attacker
     bytes at monotonically increasing offsets. This flattens the
     kernel heap, which is why the machine dies silently instead of
     panicking.
   - Stack return address `0xffffffff826671d5` lies in ng_ksocket.ko
     (`ng_ksocket_incoming` → `NG_SEND_DATA` → `nga_rcvdata`) — the
     exact analyzed delivery chain.

Control runs (no race): flood-only 60 s — stable; SET_CONFIG-only at
70k msg/s — stable, zero mbuf growth. The crash requires the
*concurrency* of the two paths.

## Fix (validated)

`fix.diff`: per-node `struct spinlock sc_lock`;
- decode guard+write and every `abuf` write take the lock (the guard and
  the write observe a consistent `(cfg.amru, sbuf, slen)` tuple),
- `SET_CONFIG` performs the realloc/swap/`cfg` copy under the lock and
  frees the retired buffers only after unlocking,
- alloc-failure paths keep the old ERROUT semantics (cfg is never applied
  against a smaller buffer).

Baseline vs patched, same trigger (`race2 all config`):
- unpatched: dead < 20 s (twice), wedge ~8 min (third run, register-dumped).
- patched: alive & ssh-responsive at 2/4/6/8/10 min soak; mbufs stable;
  `cycle` mode (graph create/destroy storm) also survived. No wedge, no
  crash, no corruption.

## Threat model / reachability

The trigger needs a v1 netgraph **control socket**, which requires
`caps_priv_check(SYSCAP_RESTRICTEDROOT)` (ng_socket.c:172) — i.e. root (or
a root-built PPP-era graph plus local root for the config side). The
decode side alone is remotely reachable in a PPP deployment, but the
*reconfiguring* side is local-root, hence Medium, not High. The primitive
itself is unbounded kernel heap corruption with attacker-influenced data.

## Residual issues (documented, not fixed here)

`nga_shutdown` (ng_async.c:368-382) frees `sc` and the node while an
in-flight `rcvdata` may still hold `sc` — the same architectural v1
netgraph absence of node-lifetime serialization; a per-node lock cannot
fix a free of the lock itself. That needs reference counting in the v1
core and affects every v1 node type, not just ng_async.

## Fix status (batch validation)
Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest `make -j6 nativekernel KERNCONF=X86_64_GENERIC` completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.