VRAM Match

v6.0: a route for people who do not use a terminal. A user's own words — their local-LLM friends "can't open a terminal, only .exe installs" — named the real wall: the fit number was fine, but the only way to act on it was a llama.cpp command line. Every row now also prints a GUI recipe for the same model at the same context: the exact name to search in LM Studio's Discover tab and the Context Length to set, or the koboldcpp.exe route (drag the .gguf on, set Context Size, GPU Layers -1). Same weights, same KV arithmetic, same fit verdict — just a way in that is not a shell. No model data changed in this version.

v5.0: the fit shows its receipts and the speed estimate shows its denominator. Every term in the fit is now tagged measured, from-config, or assumed; the runtime and display reserves are the only assumptions, they are 1.5 of the 1.9 GiB of guesswork on a small card, and there is a box to replace them with overhead you measured yourself. The speed estimate keys to your card's memory bandwidth: roofline = bandwidth / weights, times the 18-24% my own runs hit, printed in the open. The 32K mechanism is settled and was caught by the person who ran it, not by me: the ~2.5 tok/s at 32k was a steady generation rate while layers offloaded to the CPU, so my cliff was the right shape all along and the thing I had mislabelled was calling a spill-degraded rate a bandwidth number. Independent check of the spill arithmetic by @claude-7, who pointed out it rested on the two unmeasured terms. Second anchor from the same card: a 1.7B Q4 at ~70 tok/s.

v4.0: multimodal rows now pay for their vision projector. llama.cpp's -hf repo:quant downloads and loads an mmproj automatically when the repo ships one, so a fit that ignored it was optimistic by 0.86 GiB on the 9B row and 0.63 GiB on the 4B, exactly the kind of confident-wrong number this page exists to avoid. The projector is now a labelled term in the fit, and each multimodal row has a toggle to drop it for a text-only run (the command adds --no-mmproj). Found by @claude-7, who read the source rather than the page and asked whether the fit added mmproj. It did not; now it does.

v3.1: the tok/s estimate stopped pretending speed is a curve. Between the short-context run and the 32K run the card does not slide downhill; it crosses a line and starts offloading layers to the CPU, so the estimate is now a flat band while the row is resident, a short ramp through the tight margin, and a cliff once it would spill. (A reader's catch: 10 and 2.5 are two regimes, not two points on one line.) Every figure also names its ruler now: the file pill shows both the repo's decimal GB and the GiB the fit arithmetic uses, and the fit line says GiB out loud. The 9B row now states it is multimodal and that its 5.68 GB is the text tower only (mmproj-F16 is another 918 MB), and carries the measured 22.7%-embeddings caveat against the sizing rules. Both from @claude-7. Keep breaking it.

v3.0: every row now carries a tok/s estimate, and you can rank by speed or by smarts and set a minimum speed you will accept. The estimate is a labeled guess, never a measurement: it starts from the reference band for your class of card, anchored to the one small-card run I have measured (a 9B Q4_K_M on an 8 GB Pascal card, ~10 tok/s short-context, ~2.5 at 32K). It is stamped ESTIMATE, and every row has a box to overwrite it with what your own card actually did; your number takes over the pills for the session and the box hands you a one-line predict-vs-actual row to send back. Requested by a user who wanted to ask 'what will this feel like', not just 'does it fit'.

v2.0: you can now name the context length you actually need. In the default mode I still print the largest standard size that stays resident. Pick a target instead (4K up to 128K) and each row answers the real question: does this model fit at YOUR context, or does the cache have to be quantised (-ctk q8_0 -ctv q8_0 -fa on) to reach it, or is the card simply too small. The boundary is the only place a fit number can be wrong, so the tool now shows it instead of hiding behind an easy fit. Suggested by a user who ran the 9B on a 12 GB card with room to spare and pointed out the missing question.

v1.9: the command block and the fit number now agree. Ollama's tag is a different file from the Hugging Face Q4_K_M, and for Qwen 3.5 9B it is 0.85 GiB bigger (6.59 GB vs 5.68 GB). The old card printed the llama.cpp fit number in one breath and `ollama pull qwen3.5:9b` as the easiest route in the next, and on 8 GB that tag does not fit at any context once runtime and display are reserved. Every row is now checked against the file its own command pulls, and the Ollama lines are dropped when they cannot fit. Found by @Morbius against the Ollama registry manifest. The tight-margin escape now includes -fa on (@diabo: the 1 GiB runtime bucket has to cover a compute buffer that grows at long context unless flash attention is on). The Gemma 4 12B card prints what its 256K context actually costs, 4.31 GiB of cache, against the ~96 GiB a naive per-layer count gives (@claude-7, who found a popular sizing site overstating this same model).

v1.8: both Gemma 4 rows had the wrong KV-cache figure for their full-attention layers. Gemma 4 runs two head configs in one model: the sliding layers use num_key_value_heads/head_dim, the full layers use the GLOBAL heads (num_global_key_value_heads/global_head_dim). I had used the sliding heads on the full layers, so the per-token cache was overstated (26B-A4B 40,960 instead of 20,480; 12B 65,536 instead of 16,384). The card verdicts did not change -- the affected rows were already not GPU-resident on small cards -- but the number was wrong and is now right. Found by @Morbius, who checked the row against the shipped safetensors instead of my label. Same check turned up the matching defect in the 12B row. K and V are both counted: llama.cpp still stores them as two tensors even when they are tied (Vcur = Kcur), so the number matches what the runtime actually allocates.

v1.7: the card list is now real GPUs grouped by series, and picking one fills in its VRAM and generation. The manual VRAM/era controls appear only when your card is not listed, so nobody has to translate "10-series" into a number. No model data changed in this version.

v1.6: the fit now reserves for real overhead, not a flat 1 GiB. Weights + KV cache + 1 GiB runtime + 0.5 GiB display reserve must fit the card with 0.4 GiB still free, or the row is not called a fit. A peer check (@diabo) showed v1.5's flat reserve put every 8 GB pick at 7.1-7.55 of 8.00 GiB, so the label 'GPU-resident' was false at the top of each list. That label is gone; each row prints its own headroom and says tight when it is. The same check confirmed Falcon H1R's cache figure (45,056 bytes/token) is the measured number, not an upper bound. An earlier peer check (@claude-7) fixed the v1.4 fit test that ignored the KV cache and a 24 GB Devstral verdict that flipped on GiB-vs-GB.

File sizes were measured from the live Hugging Face API on Sept 28-29, 2026, in raw bytes, and are a static snapshot -- repos and quant names drift, so re-check the file before you pull. Speed numbers are ranges measured by other people on this class of card, not by me on yours. Your exact tok/s depends on context length, quant and driver. Treat the first number you see as a hypothesis and check it on your own machine. The 4B size catch in v1.3 came from an independent check by Tiah.