Best Local LLM for Coding in 2026: What Actually Fits on Your Machine

IBM’s Granite 4.2 8B is a 5.35GB download. On a 16GB MacBook it needs 10.72GB to run, and it does not fit.

That gap is the KV cache, and almost no guide to local coding models publishes it. They quote the file size, which is the one number that cannot tell you whether a model will run.

This is what actually fits at 8GB, 16GB, 24GB and 64GB, with every weight read from the model’s own repository rather than estimated. It is the compatibility companion to our Apertus 1.5 measurement, built on the same method. Figures checked 10 September 2026, refreshed monthly.

The Short Version

How Much Memory Does a Coding LLM Actually Need?

Three numbers, and most guides publish only the first.

Weights come to roughly 0.6GB per billion parameters at Q4_K_M. The KV cache stores a key-value pair for every token the model has read, in every layer: 2 × layers × kv_heads × head_dim × context × bytes. At 32,000 tokens, which is a few files open, that adds about 1.8GB to 5.4GB depending on the attention design. For Granite 4.2 8B the raw product is ~5.2GB at FP16; the 5.37GB used below includes allocator padding. Then add about 15% headroom on top of weights plus cache.

What Your Hardware Actually Gives You

Subtract all of that from what the machine actually gives you, which is always less than the number on the box. On Apple silicon the figures below use about two-thirds of listed unified memory, not 75%. The 75% “wired limit” number is a community rule of thumb; in practice Activity Monitor and Metal leave less. That is why a 16GB Mac is treated as 10.6GB here, not 12GB.

The two-thirds figure is a community rule of thumb, not an Apple specification. It can be raised with sudo sysctl iogpu.wired_limit_mb=, which resets on reboot and risks swapping. Raising it is how a 16GB Mac can be forced toward the 12GB the 75% rule implies. On Nvidia there is no equivalent override. VRAM is a hard ceiling.

Why File Size Misleads

Granite pays that price because it is a dense 40-layer model with no sliding-window attention, so its cache costs 5.37GB, as much as the model itself. Gemma 3 12B is 2GB heavier on disk and fits at 9.78GB, because sliding-window attention holds its cache to 2.48GB. Drop Granite to Q4_K_S at 5.09GB and it lands at 10.46GB, inside the limit. File size predicts none of this.

Two more traps sit in the table below. gpt-oss-20b cannot be quantised down to fit 16GB. OpenAI’s model card says it runs within 16GB of memory, which is true of system RAM and of a 16GB NVIDIA card at moderate context, but not of the default Metal cap (~10.6GB usable). Its mixture-of-experts weights already ship in native MXFP4. Every GGUF from Q2_K to Q8_0 lands between 11.5GB and 12.1GB. There is no smaller fallback. Qwen3-Coder-Next activates 3B parameters and still needs 48.4GB for the weights alone, because active-parameter counts tell you speed, not memory. All 80B stay resident. That is a 64GB-class workstation or an 80GB-class GPU, not a 48GB Mac.

What Fits at Each Memory Tier

Weights read from repository file listings on 8–10 September 2026. KV calculated at 32,000 tokens.

Two licence traps are worth naming. Qwen2.5-Coder is Apache 2.0 at 1.5B and 7B but non-commercial at 3B, because model families do not always share a licence. And Hugging Face tags are not the licence. Gemma 3 and earlier Gemma releases used Google’s custom Gemma Terms of Use. Gemma 4 is Apache 2.0; Google said so on the Open Source Blog in April 2026, and the Gemma terms page now points Gemma 4 traffic to that Apache text. The same discipline applies across every open-weight family, as our Chinese LLM map found: the licence travels with the release, not with the company.

Which Model to Run at Each Tier

8GB. Qwen2.5-Coder-1.5B for autocomplete, with native fill-in-the-middle at 1.12GB. Granite 4.2 3B if you want chat as well. IBM reports LiveCodeBench v6 69.71 for it, self-reported and unverified.

12GB. Qwen2.5-Coder-7B is a January 2025 release, old for this field, but nothing newer at this size has displaced it on published evidence. Gemma 4 E4B scores higher on LiveCodeBench at 52.0%, though its fill-in-the-middle support appears broken, with an unresolved issue open on Hugging Face. Treat it as a chat model.

16GB. Gemma 4 12B. Its 1,024-token sliding window keeps the KV cache far below a dense model of the same size, and quantisation-aware-trained builds hold up better at 4-bit than post-hoc ones. It is not a specialist coder, which is the trade-off at this tier.

24GB. Devstral Small 2 24B, around 68% on SWE-bench. This is the first tier where a local model does real multi-file work. Note that Mistral publishes no official GGUF for it, so you are on a community build.

32GB and up. GLM-4.7-Flash adds MIT licensing and an unusually small KV cost, 1.77GB at 32k, from multi-head latent attention. Community GGUFs run it in 24GB at 4-bit; 32GB is the comfortable tier once context and headroom are included. gpt-oss-20b also belongs here on a Mac. Qwen3-Coder-Next is the next step up, not a 32GB or 48GB-Mac model: 262,144-token context, Apache 2.0, 48.4GB of resident weights, and Qwen’s reported 70.6% on SWE-bench Verified, 44.3% on SWE-bench Pro and 36.2% on TerminalBench 2.0. Plan for a 64GB-class workstation with the wired limit raised, or an 80GB-class GPU. Active parameters are 3B. Memory is still 80B.

Where the Quality Cliff Is

No model that fits in roughly 6GB has ever been run on SWE-bench Verified or Aider polyglot. Not one. The nearest published data point is Qwen2.5-Coder 32B, four times larger than anything in the 8 to 12GB tiers, scoring 16.4% on Aider polyglot.

Since that measurement, IBM has published 47.67% SWE-bench Verified for Granite 4.2 8B. That is vendor-reported, unverified here, and still well below Devstral Small 2’s 68%. It does not move the 8–12GB models into the agent tier. It does mean the cliff is no longer “nothing below 24B has a SWE-bench number.

In short, the 1.5B to 12B range is useful for autocomplete, single functions, small refactors and reading a stack trace. It is not useful for multi-file agentic work, and prompting does not fix that. If the job is fixing a failing test across three files, you need the 24B tier, which means 24GB of memory or an API.

How We Checked

Every weight figure was read from the model’s own repository file listing on Hugging Face on 8–10 September 2026, not derived from the 0.6GB-per-billion shorthand. KV figures are calculated at 32,000 tokens from each model’s published layer, key-value head and head-dimension counts. Usable memory assumes about two-thirds of unified memory on macOS and near-full VRAM on NVIDIA.

We did not run these models. This is a compatibility guide, not a benchmark. Every score here belongs to the organisation that published it, and vendor figures with no independent verification are marked as such. We do not quote HumanEval, which is saturated and no longer separates current models. Where we do measure, the method is the one in our Apertus test.

Author: Akos Szima

This article is for information only and is not deployment advice. Model availability, sizes, licences and scores change frequently. Every figure was checked against its primary source on 8–10 September 2026 and this page is refreshed monthly. Performance scores are vendor-published unless stated otherwise. Corrections welcome and published inline.

See Also:

What Is the Best Open-Source LLM in 2026?

European LLMs in 2026: The Complete Map

Chinese LLMs in 2026: The Complete Map