RUNTIME / EASY

llamafile

Distributing a whole GGUF model as one runnable executable on Linux, macOS, Windows and BSD

Primaire bron

DifficultyEasySetup profile

Platforms5cpu · apple · nvidia · amd · intel

APIOpenAI + Anthropic-compatible:8080

01 / INSTALL & RUN

Installation depends on your OS. These commands are examples or templates: replace placeholders and verify quantization, file format, drivers and runtime support. Memory fit does not guarantee deployment.

1
InstallDownload a prebuilt .llamafile for your model, or the llamafile binary plus an external GGUF
2
Start a model./llamafile -m <downloaded-model.gguf> -ngl 999
3
Connect your app
OpenAI + Anthropic-compatible · :8080. Keep the service bound to localhost unless you add authentication and network controls.

Let op

Windows can only execute files under 4 GB, so larger models need the standalone binary with external GGUF weights. Vulkan covers Intel and other GPUs, but AMD multi-GPU offload can break.

02 / COMPATIBLE MODELS