推論服務 / 簡單

KoboldCpp

One-file GGUF chat on Windows, macOS or Linux with no Python install

一手來源

用你的硬體檢查這個執行階段的適配模型選裝置或顯示卡，只顯示該執行階段在這台機器上的適配路徑。

難度簡單安裝難度

平台5cpu · nvidia · amd · intel · apple

APIOpenAI + Ollama + KoboldCpp-compatible:5001

01 / 安裝與執行

安裝方式取決於作業系統。以下指令是範例或範本，請先替換佔位符並核對量化版本、檔案格式、驅動與執行環境支援；記憶體適配不保證部署成功。

1
安裝Download the single-file release binary from the official GitHub releases (koboldcpp.exe on Windows)
2
啟動模型koboldcpp --model <downloaded-model.gguf> --usevulkan
3
連接你的應用
OpenAI + Ollama + KoboldCpp-compatible · :5001. 除非加入驗證與網路存取控制，否則請將服務綁定到 localhost。

注意

The project warns that koboldcpp.com is an impersonation; since v1.123.1 (2026-10-13) the official community site is koboldcpp.net, while executable downloads remain only on the GitHub release page. GPU flags are not interchangeable: the portable command uses --usevulkan, which covers AMD, Intel and NVIDIA GPUs; --usecuda is NVIDIA-only, and macOS needs a Metal build rather than a flag; you can only use the accelerator libraries the release bundles. Since v1.122.1 (2026-09-26) the release also bundles its own agent harness (--agent) with MCP tool loading; its release notes ask for at least 28k context, an 8k generation budget and about 12 GB of VRAM, which is above what the small-memory lanes on this site can give it.

02 / 相容模型