RUNTIME / EASY

KoboldCpp

One-file GGUF chat on Windows, macOS or Linux with no Python install

Check which models this runtime can serve on your hardwarePick a device or GPU to see only the models this runtime can serve there.

DifficultyEasySetup profile

Platforms5cpu · nvidia · amd · intel · apple

APIOpenAI + Ollama + KoboldCpp-compatible:5001

01 / INSTALL & RUN

Installation depends on your OS. These commands are examples or templates: replace placeholders and verify quantization, file format, drivers and runtime support. Memory fit does not guarantee deployment.

1
InstallDownload the single-file release binary from the official GitHub releases (koboldcpp.exe on Windows)
2
Start a modelkoboldcpp --model <downloaded-model.gguf> --usevulkan
3
Connect your app
OpenAI + Ollama + KoboldCpp-compatible · :5001. Keep the service bound to localhost unless you add authentication and network controls.

Let op

The project warns that koboldcpp.com is an impersonation; since v1.123.1 (2026-10-13) the official community site is koboldcpp.net, while executable downloads remain only on the GitHub release page. GPU flags are not interchangeable: the portable command uses --usevulkan, which covers AMD, Intel and NVIDIA GPUs; --usecuda is NVIDIA-only, and macOS needs a Metal build rather than a flag; you can only use the accelerator libraries the release bundles. Since v1.122.1 (2026-09-26) the release also bundles its own agent harness (--agent) with MCP tool loading; its release notes ask for at least 28k context, an 8k generation budget and about 12 GB of VRAM, which is above what the small-memory lanes on this site can give it.

02 / COMPATIBLE MODELS