RUNTIME / EASY

Unsloth

Running GGUF or MLX models from a desktop app that also trains and exports them

Primaire bron

DifficultyEasySetup profile

Platforms5cpu · apple · nvidia · amd · intel

APIOpenAI + Anthropic-compatible:8888 (printed at launch)

01 / INSTALL & RUN

Installation depends on your OS. These commands are examples or templates: replace placeholders and verify quantization, file format, drivers and runtime support. Memory fit does not guarantee deployment.

1
Installcurl -fsSL https://unsloth.ai/install.sh | sh # or download the Unsloth Desktop app
2
Start a modelunsloth run --model <verified-GGUF-or-MLX-repo-tag>
3
Connect your app
OpenAI + Anthropic-compatible · :8888 (printed at launch). Keep the service bound to localhost unless you add authentication and network controls.

Let op

Unsloth also fine-tunes, so its published VRAM savings are training numbers and not an inference fit claim. The local endpoint is an authenticated llama-server instance and the generated API key is shown once.

02 / COMPATIBLE MODELS