Start with the smallest parameter range likely to meet the workflow.
Local model planning
Size the hardware around the workload.
A practical guide to common parameter classes, approximate memory requirements, and the kinds of local hardware that can support them.
Planning estimates · Not a performance guarantee4-bit runs are more compact; 8-bit runs need more memory but preserve more numerical detail.
Reserve capacity for context, software, concurrent users, and future expansion.
Reference table
Model size to hardware class
Memory figures are broad planning ranges for inference, not training. Actual requirements depend on architecture, quantization format, context length, and runtime.
| Model size | Approx. 4-bit memory | Approx. 8-bit memory | Suggested system RAM | Potential hardware | Typical fit |
|---|---|---|---|---|---|
| 1–3B parameters | 2–4 GB | 4–7 GB | 8 GB minimum · 16 GB comfortable | Recent laptop or mini PC; integrated graphics; entry-level GPU with 4–6 GB VRAM | Light assistants, classification, extraction, and constrained edge tasks |
| 7–8B parameters | 6–8 GB | 10–14 GB | 16 GB minimum · 32 GB comfortable | Modern desktop CPU; 8–12 GB GPU; or a system with 16+ GB unified memory | General chat, document help, summarization, and everyday local workflows |
| 13–14B parameters | 10–14 GB | 18–24 GB | 32 GB minimum · 48 GB comfortable | 12–16 GB GPU for compact runs; 24 GB GPU or 32+ GB unified memory for more headroom | Stronger writing, reasoning, coding, and private knowledge workflows |
| 20–24B parameters | 16–20 GB | 28–38 GB | 48 GB minimum · 64 GB comfortable | 24 GB GPU; 48+ GB unified-memory workstation; or CPU inference with 64 GB RAM | Higher-quality business assistance and more demanding multi-step work |
| 30–35B parameters | 22–28 GB | 42–56 GB | 64 GB minimum · 96 GB comfortable | Single high-memory GPU above 24 GB; two coordinated GPUs; or 64–96 GB unified memory | Advanced analysis, coding, and higher-accuracy professional workflows |
| 65–70B parameters | 42–50 GB | 80–100 GB | 96 GB minimum · 128+ GB comfortable | Multiple GPUs totaling 48+ GB VRAM; 96–128 GB unified-memory workstation; or large RAM server | High-capability local deployments where quality matters more than compact hardware |
| 100–120B parameters | 65–85 GB | 130–180 GB | 192 GB minimum · 256+ GB comfortable | Multi-GPU server; high-memory accelerator platform; or enterprise workstation with very large unified memory | Specialized, costly installations with careful performance and power planning |
Planning principles
Capacity is more than parameter count.
These estimates create a useful starting boundary. A pilot should still validate the exact model, runtime, document size, response speed, and number of simultaneous users.
Model weights
The table estimates memory for quantized model weights plus practical runtime headroom.
Context matters
Long prompts and large documents increase the key-value cache and can materially raise memory use.
Speed varies
A model that fits in system RAM may still run slowly without enough memory bandwidth or GPU acceleration.
Workload first
The best appliance is the smallest configuration that reliably handles the intended private workflow.
From estimate to appliance
Start with the work that must stay local.
The final hardware choice should follow the workload, privacy boundary, response-time target, and expected number of users.