RUNTIME / MEDIUM

OpenVINO Model Server

Serving OpenVINO IR models on Intel CPUs, Arc GPUs and NPUs with one API

Birincil kaynak

DifficultyMediumSetup profile

Platforms2intel · cpu

APIOpenAI-compatible:8000

01 / INSTALL & RUN

Installation depends on your OS. These commands are examples or templates: replace placeholders and verify quantization, file format, drivers and runtime support. Memory fit does not guarantee deployment.

1
Installdocker pull openvino/model_server:latest-gpu
2
Start a modelovms --source_model <verified-OpenVINO-IR-model> --model_repository_path /models --rest_port 8000
3
Connect your app
OpenAI-compatible · :8000. Keep the service bound to localhost unless you add authentication and network controls.

Dikkat

`latest` is the Intel CPU image and `latest-gpu` adds Intel GPU/NPU; NPU execution needs a current Intel driver. Models are served as OpenVINO IR, so a checkpoint with no IR export or an unsupported architecture is not a fit.

02 / COMPATIBLE MODELS