RUNTIME / ADVANCED
oMLX
Apple-silicon serving with continuous batching, memory guards, and supported SSD-backed paths
Source primaireDifficultyAdvancedSetup profile
Platforms1apple
APIOpenAI-compatible + Anthropic-compatible:8000
01 / INSTALL & RUN
- 1Install
brew install jundot/omlx/omlx - 2Start a model
omlx serve --model-dir ~/models --memory-guard-gb 48 - 3Connect your app
OpenAI-compatible + Anthropic-compatible · :8000. Keep the service bound to localhost unless you add authentication and network controls.
Attention
Apple silicon only. SSD-backed behavior is model-specific: the Qwen3.8 REAP build streams its sparse PLE table, not arbitrary model layers.
02 / COMPATIBLE MODELS