RUNTIME / MEDIUM

OpenVINO Model Server

Serving OpenVINO IR models on Intel CPUs, Arc GPUs and NPUs with one API

一手来源

DifficultyMediumSetup profile

Platforms2intel · cpu

APIOpenAI-compatible:8000

01 / INSTALL & RUN

安装方式取决于操作系统。以下命令是示例或模板，请先替换占位符并核对量化版本、文件格式、驱动和运行时支持；内存适配不保证部署成功。

1
Installdocker pull openvino/model_server:latest-gpu
2
Start a modelovms --source_model <verified-OpenVINO-IR-model> --model_repository_path /models --rest_port 8000
3
Connect your app
OpenAI-compatible · :8000. Keep the service bound to localhost unless you add authentication and network controls.

注意

`latest` is the Intel CPU image and `latest-gpu` adds Intel GPU/NPU; NPU execution needs a current Intel driver. Models are served as OpenVINO IR, so a checkpoint with no IR export or an unsupported architecture is not a fit.

02 / COMPATIBLE MODELS