DF-5369 — ums sc_lock <-> evdev ev_mtx lock-order inversion (unkillable open deadlock)

What

ums_intr_callback() runs holding sc->sc_lock (the xfer priv lock, ums.c:606-608) and calls ums_evdev_push() (ums.c:363, also ums.c:207 from the SBU callout) which takes the evdev state lock (ev_mtx, because attach uses evdev_register(), ums.c:743 -> evdev.c:345-347). Lock order A: sc_lock -> ev_mtx.

evdev_register_client() (evdev.c:984-991, from cdev open, cdev.c:132 under EVDEV_LOCK) and evdev_unregister() -> evdev_dispose_client() (evdev.c:375-388, evdev.c:1004-1008, detach) call the driver methods ums_ev_open()/ums_ev_close() which take sc->sc_lock (ums.c:964, ums.c:983). Lock order B: ev_mtx -> sc_lock.

Two sleepable lockmgr locks acquired in opposite orders -> classic AB-BA. When an interrupt report and an open()/close() (or detach) hit the µs-crossing simultaneously, BOTH sides block forever: the opener is stuck in an unkillable D-state (lockmgr sleep has no PCATCH), the USB callback thread is wedged holding sc_lock, and the input device is dead.

Reproduced (2/2 stock runs)

# stock module, root open/close loop of the ums0 evdev node while
# the QEMU mouse streams (~90 reports/s):
t= 5s opens=6200
t=10s opens=12600
t=15s opens=19200
t=20s opens=21200     <- froze; stream still flowing
... (counter never moves again)
kill check: pids=31041 ; kill -9 ineffective ; loop never exits

# ps forensics:
PID   STAT WCHAN      COMMAND
31041 D1   ums lock   /bin/sh /tmp/ocloop.sh

WCHAN "ums lock" is sc->sc_lock (lockinit'ed as "ums lock", ums.c:593) — the opener is blocked exactly in ums_ev_open()'s lockmgr(&sc->sc_lock) under ev_mtx while the callback holds sc_lock and waits for ev_mtx. Unkillable (D1), survived kill -9.

Second stock run froze even faster (1400 opens).

Rig

Same as DF-5368 (QEMU usb-mouse on uhci, X86_64_GENERIC guest, EVDEV_SUPPORT). ocprobe.py drives everything.

Run

python3 ocprobe.py                       # stock /boot/kernel/ums.ko
UMS_KO=/usr/src/.../ums.ko python3 ocprobe.py   # patched build

Fix validation

fix.diff: register the evdev with the driver's lock — evdev_register_mtx(sc->sc_evdev, &sc->sc_lock) — so both orders become one recursive lock (sc_lock is LK_CANRECURSE, ums.c:593).

Patched run: 171,473 open/close cycles under continuous stream, counter monotonic, loop exits on time, process killable, guest healthy (fix_run.log).

DF-5369 VERDICT — REPRODUCED (unkillable kernel deadlock, local DoS)

Result

Mechanism (path:line)

Order A (data path, per delivered report): ums_intr_callback holds sc->sc_lock (xfer lock from ums.c:606-608) -> ums_evdev_push ums.c:363 -> evdev_push_rel (evdev.h:199) -> evdev_push_event -> EVDEV_ENTER = lockmgr(ev_mtx) (evdev_private.h:147-155; ev_mtx because ums.c:743 used evdev_register, evdev.c:345-347).

Order B (control path, open/close/detach): cdev open cdev.c:132 EVDEV_LOCK -> evdev_register_client evdev.c:984-991 -> ums_ev_open -> lockmgr(&sc->sc_lock) ums.c:964; detach: evdev_unregister evdev.c:375 -> evdev_dispose_client evdev.c:1004-1008 -> ums_ev_close -> lockmgr(&sc->sc_lock) ums.c:983.

A(then)B(then) crossing = both sleepable lockmgr locks held mutually -> permanent wedge. lockmgr blocking sleeps are not signal-interruptible here -> kill -9 cannot reap the opener.

Reproduction detail

• Run 1 (from session log, ocprobe against stock): counter 6200 / 12600 / 19200 / 21200 then frozen from t=20s through t=140s; loop's own 150s deadline passed without exit; kill -9 ineffective (pgrep still lists the pid 1s later).
• Run 2 (run.log, stock): frozen at 1,400 opens; identical kill failure.
• Forensics: ps -o pid,stat,wchan,command -p 31041 -> STAT D1, WCHAN "ums lock" — the exact lock from ums.c:593 (lockinit(&sc->sc_lock, "ums lock", ...)), i.e. blocked inside ums_ev_open/ums_ev_close under ev_mtx.
• Guest otherwise alive (ssh fine); the ums device and its USB callback thread are dead for the session — matches the wedge being device-scoped, not system-wide.

Trigger reachability

Opening the ums0 evdev node requires root or group input (/dev/input/eventN crw-rw---- root:input, cdev.c:714 make_dev 0600->0660 via devfs rules). Real-world: an X11/wayland/libinput session (root or seat-privileged) opening/closing or re-scanning the mouse while it streams — routine on VT switch, resume, device re-enumeration. The detach-side variant additionally overlaps with the DF-5368 window.

Fix validation

fix.diff: evdev_register_mtx(sc->sc_evdev, &sc->sc_lock) — the evdev state lock becomes sc_lock itself; both former orders collapse into recursive acquisition of one LK_CANRECURSE lock. - Patched module: 171,473 open/close cycles under continuous stream, monotonic progress every 5s sample, loop exited at its deadline, process gone (loop-gone), guest healthy. - No behavioral regressions: events still flow (client open/push/close exercised 171k times), detach cycles during DF-5368 work unaffected.

Artifacts

• ocprobe.py — the probe (stock vs UMS_KO=... patched)
• run.log — stock run 2 (froze at 1400)
• fix_run.log — patched run (171,473 opens, clean)
• fix.diff — evdev_register_mtx change
• ps_forensic.txt — D1 / "ums lock" evidence
• run1 session excerpt embedded in README/VERDICT

Fix status (batch validation)

Applied with the 36 sibling fix.diffs of this batch in findings/poc/_batch41/all_fixes.patch; single in-guest make -j6 nativekernel KERNCONF=X86_64_GENERIC completed RC=0 with zero errors under -Werror (full log: findings/poc/_batch41/fix_build8.log). Behavior A/B intentionally skipped for this Info-tier item.