AI & VRAM guides

Follow the model through its memory tiers

An AI memory problem can begin during loading, during a larger request, or after repeated work. Those phases can have different allocation patterns. Record the model, representation, device placement, framework version, and input settings so a comparison is meaningful.

The AI VRAM overview explains the boundary between discrete GPU memory, host RAM, and supported offloading. The offloading article shows how to evaluate that resource path without confusing a model that loads with a model that serves useful requests. The CUDA article focuses on identifying live allocations, cached memory, and retained references before applying a workaround.

Use the AI memory planning page for the broader budget. Weights are not the only resource, and a single short request is not a complete capacity test. Include realistic lengths, repeated requests, and the intended level of concurrency.

Keep output quality in the acceptance criteria when testing a smaller model or representation. The configuration should meet the application's actual task, not merely reduce a memory counter. Also keep inference-specific features separate from training strategies unless the relevant software documents both uses.