推理服务 / 进阶
oMLX
Apple-silicon serving with continuous batching, memory guards, and supported SSD-backed paths
一手来源难度进阶安装难度
平台1apple
APIOpenAI-compatible + Anthropic-compatible:8000
01 / 安装与运行
安装方式取决于操作系统。以下命令是示例或模板，请先替换占位符并核对量化版本、文件格式、驱动和运行时支持；内存适配不保证部署成功。
- 1安装
brew install jundot/omlx/omlx - 2启动模型
omlx serve --model-dir <downloaded-model-directory> --memory-guard-gb <verified-memory-budget> - 3接入你的应用
OpenAI-compatible + Anthropic-compatible · :8000. 除非添加认证与网络访问控制，否则请将服务绑定到 localhost。
注意
Apple silicon only. SSD-backed behavior is model-specific: the Qwen3.8 REAP build streams its sparse PLE table, not arbitrary model layers.
02 / 兼容模型