The 2026 hardware reality check: what it takes to run local LLMs

The belief that you need a $3,000 graphics card to run AI at home is out of date. Two shifts — 4-bit quantization and unified memory — quietly moved capable local models onto ordinary laptops. Here is the honest math.

Ask a beginner what it takes to run a language model at home and you will usually hear some version of "a very expensive NVIDIA GPU." That was true a couple of years ago. It is not true now, and the reason is two unglamorous engineering shifts — quantization and unified memory — that between them turned "needs a data-centre card" into "runs on the laptop you already own." This guide is the math, without the marketing.

01The one formula that explains everything

Almost every hardware question reduces to a single rule of thumb: the memory a model needs is roughly its parameter count times the bytes used per parameter, plus some overhead for context. In full 16-bit precision each parameter takes 2 bytes, so a 7-billion-parameter model wants about 14 GB just for weights.¹ That is what scared everyone off. The trick is that you rarely need full precision.

# Memory for weights ≈ parameters × bytes-per-parameter (+ context overhead)
# FP16 : 2.0  bytes/param  ->  7B ≈ 14 GB
# 8-bit: 1.0  byte/param   ->  7B ≈  7 GB
# 4-bit: 0.5  byte/param   ->  7B ≈  3.5 GB  (+ ~1-1.5 GB context ≈ ~5 GB)

02The quantization magic

Quantization stores each weight at lower precision — most popularly 4-bit integers instead of 16-bit floats. Going from 16-bit to 4-bit is a 4× size reduction (about 75% smaller), and the widely-used Q4_K_M format from llama.cpp is considered the sweet spot precisely because it delivers that shrink with only minor quality loss.² That is the whole game: 4-bit quantization is what lets a 7B model that wanted 14 GB run in roughly 5 GB, and a 14B model — which would need ~28 GB at full precision — sit comfortably inside a 12 GB GPU.² You are trading a sliver of accuracy for three-quarters of the memory, and for local use that trade is almost always worth it.

03VRAM vs unified memory: why Apple Silicon changed the value equation

A discrete NVIDIA GPU keeps the model in its own dedicated VRAM, which is very fast but fixed and expensive — a 24 GB card is a serious purchase, and once the model exceeds that VRAM it simply will not load. Apple Silicon (M3/M4/M5) takes a different approach: unified memory, a single pool the CPU, GPU, and Neural Engine all address directly, with no copy across a PCIe bus.³ Because that pool is the machine's whole RAM, a Mac with 64 or 128 GB of unified memory can load models that no consumer GPU can fit in VRAM at any sane price.

The honest trade-off: a top discrete GPU still has far higher memory bandwidth, so it generates tokens faster on models that fit in its VRAM.⁴ But for the specific goal of running large models locally without a data-centre budget, high-memory Apple Silicon is currently the best value going — you buy capacity (fit the model at all) where the GPU buys speed (run a smaller model faster). Pick the axis your workload actually cares about.

04A buyer's guide by memory tier

Because it all comes back to memory, you can shop by tier. Assume 4-bit models and leave a couple of gigabytes of headroom for context:

• 8 GB (VRAM or unified) — the entry tier. Runs 7–8B models at 4-bit comfortably (Qwen3 8B, Mistral 7B, Phi-4-mini), which is enough for a genuinely useful local assistant and coding helper.⁵
• 16 GB — the sweet spot. Runs 14B-class models at 4-bit with room to spare, and smaller models fast. This is where most people should aim.
• 24 GB — the enthusiast tier. Comfortably runs ~32B models at 4-bit, the point where local quality starts to feel genuinely strong.
• 64–128 GB unified (Apple Silicon) — the "run the big ones" tier: 70B-class and large Mixture-of-Experts models that no single consumer GPU can hold.³

You are not buying teraflops; you are buying gigabytes. The question that decides what you can run locally is almost never "how fast is the chip" — it is "how big is the memory, and how small can I make the model."

05Where OcxlyDev lands

Start with the machine you own. A modern laptop with 16 GB can already run a capable 4-bit model, and you should confirm you like running models locally before spending anything. If you then want to go bigger, the decision is clean: buy a used 24 GB NVIDIA card if you want maximum speed on models that fit it, or a high-memory Apple Silicon machine if you want to run large models at all. Either way, the $3,000-GPU myth is dead — quantization and unified memory killed it, and the rest of this series is about what to actually do with the capable, affordable local models that resulted.