आप कहाँ से शुरू करते हैं?
अपने हार्डवेयर या इच्छित मॉडल से शुरू करें; तैनाती से पहले क्षमता, गति और प्रमाण देखें।
Select hardware to see model fit results.
Guides & methodology
Browse the complete catalogs and estimation method when you need them.
सभी हार्डवेयर21
हर डिवाइस की मेमोरी सीमा, रनटाइम रास्ता और ईमानदार ऊपरी सीमा अलग है।
Mac Studio M5 Ultra · 96 GB
The entry M5 Ultra configuration combines a 96 GB unified pool with 1.2 TB/s memory bandwidth. Apple announced it in August 2026; availability begins September 22.
गाइड खोलेंMac mini M6 · 24 GB
A compact current-generation Mac with 24 GB unified memory. Keep macOS, runtime, and KV-cache headroom separate from model weights.
गाइड खोलेंMacBook Pro M5 Max · 64 GB
A portable Apple-native tier for larger quantized models. Sustained inference still depends on thermals, runtime support, and context length.
गाइड खोलेंFramework Desktop · Ryzen AI Max+ 395 · 128 GB
A 128 GB Strix Halo desktop. AMD allows a large graphics-memory allocation, but usable capacity and speed depend on BIOS, driver, and runtime support.
गाइड खोलेंRadeon AI PRO R9700 · 32 GB
A 32 GB AMD workstation card for larger single-GPU quantized models. Check the exact operating-system and ROCm or Vulkan runtime path first.
गाइड खोलेंNVIDIA DGX Spark · 128 GB
A compact local AI system with 128 GB coherent memory. It can hold large low-bit models, but decode speed and software support remain model-specific.
गाइड खोलेंMac Studio M3 Ultra · 96 GB
A quiet single-box option for 70B-class Q4 models and Apple-native MLX workflows, with memory shared by macOS and applications.
गाइड खोलेंMac Studio M3 Ultra · 256 GB
Enough capacity for many 100B-plus low-bit models, provided the architecture is supported and optimized by the selected runtime.
गाइड खोलेंGeForce RTX 5090 · 32 GB
A fast single-GPU tier for 27B-class low-bit models, with limited remaining space for very long context or multiple concurrent users.
गाइड खोलेंGeForce RTX 4090 · 24 GB
A 27B Q4 checkpoint can be a tight fit. Leave room for runtime overhead and KV cache instead of filling every gigabyte with weights.
गाइड खोलें8 GB CPU laptop
Small text models with short context. Expect patient, private inference rather than speed.
गाइड खोलें16 GB CPU desktop
Comfortable with 3B–8B Q4 models; 12B is possible only with reduced context and patience.
गाइड खोलेंApple silicon · 16 GB
A polished 4B–8B local experience when the OS and apps have enough headroom.
गाइड खोलेंNVIDIA · 8 GB
The mainstream 4B–8B tier. Some 12B INT4 builds fit tightly with modest context.
गाइड खोलेंNVIDIA · 12 GB
Strong 8B–14B Q4 territory for a single user.
गाइड खोलेंNVIDIA · 16 GB
14B models are comfortable; 20B-class low-bit MoE models are a tight upper edge.
गाइड खोलेंApple silicon · 32 GB
24B–32B Q4 models become practical for one local user.
गाइड खोलेंNVIDIA · 24 GB
The consumer sweet spot for 27B and 30B-A3B INT4/Q4 models.
गाइड खोलेंApple silicon · 64 GB
32B models with long context or 70B Q4 with conservative headroom.
गाइड खोलेंWorkstation · 48 GB
70B Q4 serving begins here, with limited room for cache and concurrency.
गाइड खोलेंDatacenter · 80 GB
Large MoE models and high-throughput servers, still bounded by quantization and cache.
गाइड खोलेंसभी मॉडल24
Qwen3.8 Flash Next
English- अनुमानित मेमोरी
- ~108 GB
- कॉन्टेक्स्ट
- 262,144 native; 1M extended
Ling-3.0-flash
English- अनुमानित मेमोरी
- ~74.4 GB
- कॉन्टेक्स्ट
- Check selected runtime and checkpoint
Nex-N2.5-mini
English- अनुमानित मेमोरी
- ~21 GB
- कॉन्टेक्स्ट
- 262,144 (config max_position_embeddings)
Qwen3.6 27B
- अनुमानित मेमोरी
- ~18.5 GB
- कॉन्टेक्स्ट
- 262K native
Qwen3.8 27B
- अनुमानित मेमोरी
- ~19.5 GB
- कॉन्टेक्स्ट
- 262K native
Qwen3.8 Flash Next REAP-288
- अनुमानित मेमोरी
- ~40.57 GB
- कॉन्टेक्स्ट
- 7.6K recommended on measured 48 GB Mac
Qwen3.6 35B-A3B
- अनुमानित मेमोरी
- ~25 GB
- कॉन्टेक्स्ट
- 262K native
DeepSeek V4 Flash
- अनुमानित मेमोरी
- ~176 GB
- कॉन्टेक्स्ट
- 1M native
Gemma 3 1B
- अनुमानित मेमोरी
- ~1.4 GB
- कॉन्टेक्स्ट
- 32K
Llama 3.2 3B
- अनुमानित मेमोरी
- ~2.8 GB
- कॉन्टेक्स्ट
- 128K
Qwen3 4B
- अनुमानित मेमोरी
- ~3.6 GB
- कॉन्टेक्स्ट
- 32K+
Gemma 3 4B
- अनुमानित मेमोरी
- ~4.2 GB
- कॉन्टेक्स्ट
- 128K
Qwen3 8B
- अनुमानित मेमोरी
- ~6.8 GB
- कॉन्टेक्स्ट
- 32K+
Gemma 3 12B
- अनुमानित मेमोरी
- ~9.4 GB
- कॉन्टेक्स्ट
- 128K
Qwen3 14B
- अनुमानित मेमोरी
- ~11.2 GB
- कॉन्टेक्स्ट
- 32K+
gpt-oss-20b
- अनुमानित मेमोरी
- ~16 GB
- कॉन्टेक्स्ट
- 128K
Devstral Small 2 24B
- अनुमानित मेमोरी
- ~18.5 GB
- कॉन्टेक्स्ट
- 256K
Gemma 3 27B
- अनुमानित मेमोरी
- ~20.5 GB
- कॉन्टेक्स्ट
- 128K
Qwen3 30B-A3B
- अनुमानित मेमोरी
- ~22.5 GB
- कॉन्टेक्स्ट
- 32K+
Qwen3 32B
- अनुमानित मेमोरी
- ~24.5 GB
- कॉन्टेक्स्ट
- 32K+
Llama 3.1 70B
- अनुमानित मेमोरी
- ~46 GB
- कॉन्टेक्स्ट
- 128K
Llama 4 Scout
- अनुमानित मेमोरी
- ~78 GB
- कॉन्टेक्स्ट
- 10M advertised
gpt-oss-120b
- अनुमानित मेमोरी
- ~80 GB
- कॉन्टेक्स्ट
- 128K
Qwen-Image 2.1
English- अनुमानित मेमोरी
- ~33.1 GB
- कॉन्टेक्स्ट
- Not a token context: one prompt plus up to 10 reference images
काम के लिए सही सर्विंग लेयर15
Ollama
One-command local chat and app integration
गाइड खोलेंllama.cpp
GGUF models, CPU/GPU offload, embedded and unusual hardware
गाइड खोलेंLM Studio
Discovering, downloading and testing models without a terminal
गाइड खोलेंMLX LM
Native Apple silicon inference, experimentation and fine-tuning
गाइड खोलेंoMLX
Apple-silicon serving with continuous batching, memory guards, and supported SSD-backed paths
गाइड खोलेंvLLM
Linux GPU servers, concurrency and OpenAI-compatible production APIs
गाइड खोलेंTensorRT-LLM
Maximum NVIDIA throughput after engine tuning
गाइड खोलेंSGLang
Multi-request GPU serving with prefix caching and speculative decoding
गाइड खोलेंLocalAI
One OpenAI-compatible endpoint for text, vision, speech and image models, GPU optional
गाइड खोलेंLemonade
Ryzen AI, Radeon and Strix Halo PCs, including the XDNA2 NPU, behind one local API
गाइड खोलेंOpenVINO Model Server
Serving OpenVINO IR models on Intel CPUs, Arc GPUs and NPUs with one API
गाइड खोलेंKoboldCpp
One-file GGUF chat on Windows, macOS or Linux with no Python install
गाइड खोलेंllamafile
Distributing a whole GGUF model as one runnable executable on Linux, macOS, Windows and BSD
गाइड खोलेंKTransformers
Very large MoE models with hot experts on the GPU and cold experts in system RAM
गाइड खोलेंUnsloth
Running GGUF or MLX models from a desktop app that also trains and exports them
गाइड खोलेंअनुमान कैसे बनते हैं
आधिकारिक आकार और क्वांटाइज़ेशन से शुरू करके OS और रनटाइम के लिए जगह रखते हैं; कॉन्टेक्स्ट कैश अतिरिक्त है। MoE में कुल पैरामीटर गिने जाते हैं।
कुल पैरामीटर, सक्रिय MoE पैरामीटर, क्वांटाइज़्ड फ़िट, रनटाइम समर्थन और मापी गई गति को अलग रखते हैं।
- Memory fit is an estimate, not a speed measurement.
- Size MoE by total parameters; shared and distributed memory need additional checks.
- Vendor claims, community measurements, runtime conditions and unknowns are disclosed separately.
Catalog data updated: 2026-10-03
40GB डाउनलोड से पहले
Does a 24 GB GPU run a 30B model?
Often at Q4/INT4 with a conservative context. Qwen3 30B-A3B and Gemma 3 27B are representative fits, but cache and runtime overhead still matter.
Are active MoE parameters the memory requirement?
No. Active parameters affect compute per token; total parameters still need to be stored in memory or offloaded.
Which runtime should a beginner choose?
Ollama for a terminal-first setup or LM Studio for a visual desktop. llama.cpp is the portable fallback; vLLM is for higher-throughput GPU serving.
How do you report speed?
Exact device-and-model measurements are labeled measured. Relative manufacturer figures are labeled vendor claims. Every other combination stays unknown instead of receiving an invented tokens-per-second estimate.