LLM VRAM calculator

How it works

Weights are total parameters × average bits per weight ÷ 8. Parameter counts are entered in billions. The architecture presets supply only attention dimensions from the official config files. The initial 7 billion parameters and 1 GiB overhead are illustrative user assumptions, not measured model requirements.

KV bytes = 2 × layers × KV heads × head dimension × context × concurrent sequences × bytes per cache element Total bytes = weight bytes + KV bytes + reserved overhead Max context = floor((available bytes − weights − overhead) ÷ KV bytes per context token)

The factor 2 represents keys and values, cached independently per layer as described in Transformers’ cache documentation. Context includes both prompt and generated tokens. GQA uses KV heads, not query heads. Enter the explicit head_dim from custom model configs; hidden size ÷ attention heads is only a fallback when the architecture defines it that way. Qwen3-4B, for example, uses head dimension 128 despite hidden size 2,560 and 32 attention heads. Maximum context is a memory-only ceiling and is not capped to a model’s supported positional range. If weights plus overhead exceed capacity, no context fits.

The ggml structures include block scales: Q2_K uses 2.625 bits per weight, Q3_K 3.4375, and Q6_K 6.5625. These pure block sizes are not whole-model recipe averages. The Q6_K selection uses pure block storage. The Q2_K, Q3_K_M, Q4_K_M and Q5_K_M selections instead use approximate Llama-3.1-8B benchmark averages of 3.1593, 3.9960, 4.8944 and 5.7036 bits per weight, respectively, from llama.cpp. They depend on architecture and quantizer options. Any format can use a measured override. q8_0 cache uses 34 bytes per 32 elements; q4_0 uses 18 per 32. Block padding may increase actual memory.

GB means 1,000,000,000 bytes; GiB means 1,073,741,824 bytes. GPU comparison rows are illustrative capacity tiers in GiB. Calculations retain intermediate precision. Displayed memory rounds to three decimals and context rounds down. The overhead reserve is fixed for this estimate; actual scratch memory can grow with context and workload.

Worked examples

Illustrative 7 billion weights with Qwen2.5-7B attention dimensions

FP16 weights occupy 14,000,000,000 bytes. With 28 layers, 4 KV heads, head dimension 128, context 4,096 and one sequence, FP16 cache uses 234,881,024 bytes (0.21875 GiB). Add a user-assumed 1 GiB reserve: total 15.309 GB, or 14.257 GiB.

Block quantization and a memory-only context ceiling

For the same illustrative 7 billion weights, Q6_K uses 5,742,187,500 bytes. q8_0 cache at 4,096 tokens uses 124,780,544 bytes. Including 1 GiB overhead, total is 6.941 GB, or 6.464 GiB. An 8 GiB capacity allows a memory-only maximum of 58,232 tokens per sequence; model context support must be checked separately.

Frequently asked questions

Sources

• llama.cpp — Llama-3.1-8B quantization benchmark averages (checked 2026-11-14)
• Qwen3-4B — explicit head dimension (checked 2026-11-14)
• llama.cpp — ggml block structures and scale storage (checked 2026-11-14)
• llama.cpp — model-dependent mixed quantization recipes (checked 2026-11-14)
• Qwen2.5-7B-Instruct — official architecture config (checked 2026-11-14)
• Qwen2.5-14B-Instruct — official architecture config (checked 2026-11-14)
• Hugging Face Transformers — cache tensor shapes (checked 2026-11-14)