Pre-built docker images with vLLM and zentorch are available for quick deployment.
Pulling the Image
# Pull the Ubuntu image
docker pull amdih/zendnn_zentorch:vllm_v0.18.0_zentorch_v5.2.1_ubuntu22.04_2026_r5.2.1
# Pull the RHEL image
docker pull amdih/zendnn_zentorch:vllm_v0.18.0_zentorch_v5.2.1_rhel9.5_r5.2.1Running the Container
# Ubuntu Container
docker run -d --name vllm_zentorch-ubuntu \
-v /path/to/models:/models \
amdih/zendnn_zentorch:vllm_v0.18.0_zentorch_v5.2.1_ubuntu22.04_2026_r5.2.1 \
tail -f /dev/null
# Enter the container
docker exec -it vllm_zentorch-ubuntu bash
cd workspaceNote: Mount volumes (-v) for model
files and any datasets you need inside the container. Export
HF_TOKEN of your account and HF_HOME to
the model path Environment variables are pre-configured. Adjust VLLM_CPU_OMP_THREADS_BIND for your machine (example: 0-127 for Turin, 0-95
for Genoa).