RUNTIME / MEDIUM

llama.cpp

GGUF models, CPU/GPU offload, embedded and unusual hardware

DifficultyMediumSetup profile

Platforms5cpu · apple · nvidia · amd · intel

APIOpenAI-compatible server:8080

01 / INSTALL & RUN

1
Installbrew install llama.cpp # or use upstream binaries
2
Start a modelllama-server -hf google/gemma-3-1b-it --port 8080
3
Connect your app
OpenAI-compatible server · :8080. Keep the service bound to localhost unless you add authentication and network controls.

Let op

New architectures may require a recent build. Backend availability does not imply equal performance.

02 / COMPATIBLE MODELS