推論服務 / 進階
oMLX
Apple-silicon serving with continuous batching, memory guards, and supported SSD-backed paths
一手來源難度進階安裝難度
平台1apple
APIOpenAI-compatible + Anthropic-compatible:8000
01 / 安裝與執行
安裝方式取決於作業系統。以下指令是範例或範本，請先替換佔位符並核對量化版本、檔案格式、驅動與執行環境支援；記憶體適配不保證部署成功。
- 1安裝
brew install jundot/omlx/omlx - 2啟動模型
omlx serve --model-dir <downloaded-model-directory> --memory-guard-gb <verified-memory-budget> - 3連接你的應用
OpenAI-compatible + Anthropic-compatible · :8000. 除非加入驗證與網路存取控制，否則請將服務綁定到 localhost。
注意
Apple silicon only. SSD-backed behavior is model-specific: the Qwen3.8 REAP build streams its sparse PLE table, not arbitrary model layers.
02 / 相容模型