推理服务 / 简单

KoboldCpp

One-file GGUF chat on Windows, macOS or Linux with no Python install

一手来源

用你的硬件检查这个运行时的适配模型选设备或显卡，只显示该运行时在这台机器上的适配路径。

难度简单安装难度

平台5cpu · nvidia · amd · intel · apple

APIOpenAI + Ollama + KoboldCpp-compatible:5001

01 / 安装与运行

安装方式取决于操作系统。以下命令是示例或模板，请先替换占位符并核对量化版本、文件格式、驱动和运行时支持；内存适配不保证部署成功。

1
安装Download the single-file release binary from the official GitHub releases (koboldcpp.exe on Windows)
2
启动模型koboldcpp --model <downloaded-model.gguf> --usevulkan
3
接入你的应用
OpenAI + Ollama + KoboldCpp-compatible · :5001. 除非添加认证与网络访问控制，否则请将服务绑定到 localhost。

注意

The project warns that koboldcpp.com is an impersonation; since v1.123.1 (2026-10-13) the official community site is koboldcpp.net, while executable downloads remain only on the GitHub release page. GPU flags are not interchangeable: the portable command uses --usevulkan, which covers AMD, Intel and NVIDIA GPUs; --usecuda is NVIDIA-only, and macOS needs a Metal build rather than a flag; you can only use the accelerator libraries the release bundles. Since v1.122.1 (2026-09-26) the release also bundles its own agent harness (--agent) with MCP tool loading; its release notes ask for at least 28k context, an 8k generation budget and about 12 GB of VRAM, which is above what the small-memory lanes on this site can give it.

02 / 兼容模型