DF-2819 — kmalloc_obj zone-limit accounting uses cacheline-aligned objsize while callers derive limits from unaligned sizeof → tmpfs inode quota exhausted at ~85% of configured cap

What this is

Reproduction of a resource-accounting defect in the kmalloc_obj zone allocator (sys/kern/kern_kmalloc.c). The allocator accounts per-object memory use as the cache-line-aligned ks_objsize (__VM_CACHELINE_ALIGN(objsize), applied at zone creation in _kmalloc_create_obj, sys/kern/kern_slaballoc.c:610; accounting at sys/kern/kern_kmalloc.c:899 use->memuse += size), but callers cannot learn ks_objsize (no accessor exists), so the only caller that derives a zone limit from an object count — tmpfs — computes it from the unaligned sizeof(struct tmpfs_node) * tm_nodes_max (sys/vfs/tmpfs/tmpfs_vfsops.c:206-207).

sizeof(struct tmpfs_node) = 272, aligned objsize = 320 (measured on the guest with a KLD probe: SZPROBE node=272 nodealign=320). The zone therefore "fills" at 272/320 = 85% of the promised inode count.

How to reproduce (as performed on the QEMU guest)

1. Build szprobe KLD (optional, proves sizes): cd /root/szprobe && make → kldload ./szprobe.ko → dmesg shows SZPROBE node=272 nodealign=320 dirent=88 direntalign=128.
2. Mount a tmpfs whose raised limit exceeds the default floor (kmem_lim_size()/10 = 390 MiB on this guest): mount -t tmpfs -o size=8g,inodes=1600000 none /mnt/tfill → raised limit = 1,600,000 × 272 = 435,200,000 B = 415.04 MiB.
3. As an unprivileged user (nobody), create empty files until failure: /mnt/tfill/fill /mnt/tfill 1500000 (see fill.c).
4. Expected (observed): ENOSPC at 1,360,369 files — exactly the first k where k × 320 ≥ 435,200,000 (k = 1,360,000) — i.e. 85.02% of the 1,600,000-inode cap. vmstat -m at that moment shows tmpfs_node MemUse 415M Limit 415M (memuse == limit).
5. tmpfs's own inode cap never comes into play (tm_nodes_inuse = 1.36M < 1.6M), so the filesystem reports ENOSPC while it still advertises ~240K free inodes.

Files

• fill.c — file-creation loop (the trigger), built with cc -O2
• szprobe.c — KLD printing sizeof(struct tmpfs_node) and its cacheline alignment
• run.log — decisive run: progress lines + stopped at 1360369: No space left on device
• vmstat.txt — vmstat -m after the run: tmpfs_node MemUse == Limit
• env.txt — guest uname, mount, compiler, sysctls
• fix.diff — one-line fix (tmpfs raises the limit using the aligned size)
• VERDICT.md, manifest.json, verdict.json

Impact

Quota-correctness / availability: any tmpfs (including default-mounted /tmp — the guest's /tmp has a default node cap of ~3.14M and would ENOSPC at ~2.67M files) delivers only ~85% of its configured/default inode budget. The allocation failure path is graceful (M_NULLOK at sys/vfs/tmpfs/tmpfs_subr.c:110-111 → ENOSPC), so there is no panic and no memory-safety impact; severity Low.

DF-2819 VERDICT

Status: reproduced (impact: none/minor availability — quota correctness; no panic, no memory-safety impact) Severity: Low · Confidence: certain

Root cause

The kmalloc_obj zone allocator accounts memory use per object as the cache-line-aligned object size:

• _kmalloc_create_obj() (sys/kern/kern_slaballoc.c:610) sets type->ks_objsize = __VM_CACHELINE_ALIGN(objsize);
• _kmalloc_obj() (sys/kern/kern_kmalloc.c:899) does use->memuse += size with size == type->ks_objsize (KKASSERT at kern_kmalloc.c:715);
• the limit gate (sys/kern/kern_kmalloc.c:694-708) refuses allocation once Σ ks_use[].memuse >= type->ks_limit (NULL with M_NULLOK, else panic("malloc limit exceeded")).

ks_objsize is private to the allocator — there is no accessor — so the one caller that derives a zone limit from an object count, tmpfs (sys/vfs/tmpfs/tmpfs_vfsops.c:206-207), uses the unaligned sizeof(struct tmpfs_node) * tm_nodes_max. On this kernel (guest-measured via KLD probe):

• sizeof(struct tmpfs_node) = 272
• accounted per node (ks_objsize) = 320 (cacheline 64)

Every node therefore burns 320 B of quota against a 272 B budget: the zone limit binds at 272/320 = 85% of the configured inode cap.

Reproduction (QEMU guest, DragonFly 6.5-DEVELOPMENT #0, INVARIANTS kernel)

1. mount -t tmpfs -o size=8g,inodes=1600000 none /mnt/tfill → zone limit raised to 1,600,000 × 272 = 435,200,000 B = 415.04 MiB (exceeds the malloc_init floor kmem_lim_size()/10 = 390 MiB, so the raise binds).
2. Predicted refusal point: first k with k × 320 ≥ 435,200,000 → k = 1,360,000 (85.0% of cap).
3. Unprivileged user (nobody) runs fill.c creating empty files.
4. Observed: stopped at 1360369: No space left on device (run.log) — within 369 files of the exact prediction (offset = root node + ks_loosememuse flush granularity KMALLOC_LOOSE_SIZE = 512 KiB, kern_kmalloc.c:901).
5. vmstat -m at the stop point: tmpfs_node 1.30M 415M 428M 415M — MemUse (415M) == Limit (415M); Count 1.30M == requests 1.30M (vmstat.txt). The tmpfs_node row of an idle 1-node tmpfs (/var/run/shm) shows 1 320 — per-object accounting is 320 B, direct confirmation of the aligned-unit accounting.
6. tmpfs's own cap never engaged: tm_nodes_inuse = 1,360,369+1 < tm_nodes_max = 1,600,000; zero data pages used. The ENOSPC came solely from the zone limit (M_NULLOK at tmpfs_subr.c:110-113).

Why ENOSPC and not a panic: tmpfs allocates nodes with M_WAITOK | M_ZERO | M_NULLOK, so the over-reached limit returns NULL → ENOSPC. (Callers without M_NULLOK would panic("malloc limit exceeded") at kern_kmalloc.c:706 — no practical unprivileged path to that exists today: the remaining kmalloc_obj zones either are never raised (hammer2 mchain/mio, nfs mnode — default 390 MiB floor, needs ~1.3M live kernel objects) or pass M_NULLOK.)

Impact

Any tmpfs — including default-mounted /tmp (default node cap ≈ 3.14M on this 4 GiB guest → real capacity only ≈ 2.67M files) — delivers ~85% of its advertised inode quota. This is a quota/availability correctness bug (users get fewer inodes than configured), not a memory-safety bug; it is also a silent API trap for every future caller that computes zone limits from sizeof.

Fix

fix.diff (authoritative, caller-side one-liner): tmpfs raises the node zone limit using the same cacheline alignment the allocator will apply:

-   kmalloc_obj_raise_limit(tmp->tm_node_zone,
-               sizeof(struct tmpfs_node) * tmp->tm_nodes_max);
+   kmalloc_obj_raise_limit(tmp->tm_node_zone,
+               __VM_CACHELINE_ALIGN(sizeof(struct tmpfs_node)) *
+               tmp->tm_nodes_max);

Arithmetic check of the fix: limit becomes 1.6M × 320 = 512 MiB; the k-th allocation checks ttl = (k-1) × 320 ≥ 512 MiB ⟺ k ≥ N+1, and tmpfs refuses at tm_nodes_inuse ≥ N first (tmpfs_subr.c:107), so the zone limit can never bind before the filesystem's own cap. (Longer-term: export ks_objsize — e.g. a kmalloc_obj_objsize() — or make limit accounting use the requested size.)

Fix validation: not performed (kernel rebuild + 1.6M-file re-run is disproportionate for a Low, non-corruption finding); the fixed threshold arithmetic above is exact. fix_status: not_testable with this justification.

Negative result context

The panic variant of this limit gate (kern_kmalloc.c:706) was chased and is not reachable through tmpfs (M_NULLOK); see the audit JSON negative notes for the full list of killed candidates in this file.