RUNTIME / MEDIUM

llama.cpp

GGUF models, CPU/GPU offload, embedded and unusual hardware

Source primaire

DifficultyMediumSetup profile

Platforms5cpu · apple · nvidia · amd · intel

APIOpenAI-compatible server:9931 default

01 / INSTALL & RUN

Installation depends on your OS. These commands are examples or templates: replace placeholders and verify quantization, file format, drivers and runtime support. Memory fit does not guarantee deployment.

1
Installbrew install llama.cpp # or use upstream binaries
2
Start a modelllama-server -m <downloaded-model.gguf> --ctx-size <verified-context> --port 9931
3
Connect your app
OpenAI-compatible server · :9931 default. Keep the service bound to localhost unless you add authentication and network controls.

Attention

New architectures may require a recent build. Backend availability does not imply equal performance. Since v0.7.0 (2026-10-15) llama-server listens on port 9931 by default instead of 8080; the example commands pin that port explicitly. Older builds accept the same --port flag but default to 8080.

02 / COMPATIBLE MODELS