RUNTIME / ADVANCED

vLLM

Linux GPU servers, concurrency and OpenAI-compatible production APIs

DifficultyAdvancedSetup profile

Platforms3nvidia · amd · intel

APIOpenAI-compatible server:8000

01 / INSTALL & RUN

1
Installuv pip install vllm --torch-backend=auto
2
Start a modelvllm serve google/gemma-3-1b-it --dtype auto
3
Connect your app
OpenAI-compatible server · :8000. Keep the service bound to localhost unless you add authentication and network controls.

Attention

CUDA, ROCm and XPU feature coverage differs. Driver, shared-memory and quantization compatibility are version-specific.

02 / COMPATIBLE MODELS