01
Qwen3 30B-A3BTight fit
ollama run qwen3:30b-a3bA field guide to private AI
Run the right model.
On the hardware you own.
Choose your memory and platform. Get a conservative model fit, the right inference server, and a command you can actually run.
Primary-source linked16 languagesEvidence refreshed 2026-08-31
8 GB
24 GB
80GB
ollama runqwen3:8b
LOCAL / 24G
MEMORY MAP01—80 GB
RECOMMENDED ENVELOPE6 models
Comfortable Tight fit
02
Gemma 3 27BTight fit
ollama run gemma3:27b03
Devstral Small 2 24BTight fit
ollama run <model-tag>04
gpt-oss-20bComfortable
ollama run gpt-oss:20b05
Qwen3 14BComfortable
ollama run qwen3:14b06
Gemma 3 12BComfortable
ollama run gemma3:12b02 / EVIDENCE FIRST
A map, not a leaderboard
We separate total parameters, active MoE parameters, quantized weight fit, runtime support, and measured performance. Unknown speed stays unknown.
Weight fit ≠ speedTotal ≠ active parametersDocs ≠ benchmark
03 / HARDWARE MAP
From a quiet laptop to an 80 GB accelerator
Each device class has a different memory budget, runtime path, and honest upper edge.
8GB
Everyday laptop
8 GB CPU laptop
Small text models with short context. Expect patient, private inference rather than speed.
Open guide16GB
No discrete GPU
16 GB CPU desktop
Comfortable with 3B–8B Q4 models; 12B is possible only with reduced context and patience.
Open guide16GB
M-series notebook
Apple silicon · 16 GB
A polished 4B–8B local experience when the OS and apps have enough headroom.
Open guide8GB
RTX 4060 class
NVIDIA · 8 GB
The mainstream 4B–8B tier. Some 12B INT4 builds fit tightly with modest context.
Open guide12GB
RTX 3060 / 4070 class
NVIDIA · 12 GB
Strong 8B–14B Q4 territory for a single user.
Open guide16GB
RTX 4080 / 5080 class
NVIDIA · 16 GB
14B models are comfortable; 20B-class low-bit MoE models are a tight upper edge.
Open guide04 / MODEL FIELD NOTES
Representative open-weight models
Google
Gemma 3 1B
1BINT4 / GGUF Q4
- Estimated memory
- ~1.4 GB
- Context
- 32K
Alibaba Qwen
Qwen3 4B
4BGGUF Q4
- Estimated memory
- ~3.6 GB
- Context
- 32K+
Alibaba Qwen
Qwen3 8B
8BGGUF Q4
- Estimated memory
- ~6.8 GB
- Context
- 32K+
OpenAI
gpt-oss-20b
21B3.6B active · MXFP4
- Estimated memory
- ~16 GB
- Context
- 128K
Alibaba Qwen
Qwen3 30B-A3B
30B3B active · GGUF Q4 MoE
- Estimated memory
- ~22.5 GB
- Context
- 32K+
05 / SERVING LAYER
Pick the serving layer for the job
Fastest first run
Ollama
One-command local chat and app integration
Open guideWidest hardware reach
llama.cpp
GGUF models, CPU/GPU offload, embedded and unusual hardware
Open guideBest visual desktop
LM Studio
Discovering, downloading and testing models without a terminal
Open guideApple silicon specialist
MLX LM
Native Apple silicon inference, experimentation and fine-tuning
Open guideThroughput server
vLLM
Linux GPU servers, concurrency and OpenAI-compatible production APIs
Open guideNVIDIA optimized
TensorRT-LLM
Maximum NVIDIA throughput after engine tuning
Open guide06 / METHOD
How the estimates work
We start with documented model sizes and quantization, reserve space for the OS or runtime, and treat context cache as extra. MoE fit uses total parameters—not active parameters.
CONSERVATIVE FITweights + runtime + cache
total params×bits / 8+headroom
No universal tokens/s claims.07 / FAQ
Questions before you download 40 GB
Does a 24 GB GPU run a 30B model?+
Often at Q4/INT4 with a conservative context. Qwen3 30B-A3B and Gemma 3 27B are representative fits, but cache and runtime overhead still matter.
Are active MoE parameters the memory requirement?+
No. Active parameters affect compute per token; total parameters still need to be stored in memory or offloaded.
Which runtime should a beginner choose?+
Ollama for a terminal-first setup or LM Studio for a visual desktop. llama.cpp is the portable fallback; vLLM is for higher-throughput GPU serving.
Do you benchmark speed?+
Not yet. Launch recommendations are transparent memory-fit estimates backed by primary documentation. We do not invent tokens-per-second numbers.