RUNTIME / MEDIUM

MLX LM

Native Apple silicon inference, experimentation and fine-tuning

Źródło pierwotne

DifficultyMediumSetup profile

Platforms1apple

APIOpenAI-compatible server + CLI:8080

01 / INSTALL & RUN

Installation depends on your OS. These commands are examples or templates: replace placeholders and verify quantization, file format, drivers and runtime support. Memory fit does not guarantee deployment.

1
Installpip install mlx-lm
2
Start a modelmlx_lm.server --model <verified-MLX-conversion>
3
Connect your app
OpenAI-compatible server + CLI · :8080. Keep the service bound to localhost unless you add authentication and network controls.

Uwaga

Only for Apple silicon. Use an MLX-converted model and tune KV cache size for long contexts. The bundled mlx_lm.server is an OpenAI-compatible HTTP endpoint (default 127.0.0.1:8080), but upstream warns it implements only basic security checks and is not recommended for production.

02 / COMPATIBLE MODELS