Release Highlights - Release Highlights

Attention: In this document, we refer to the ZenDNN plugin for TensorFlow as zentf, and the ZenDNN plugin for PyTorch and vLLM as zentorch.

ZenDNN 6.0.0

Compatibility with Deep-learning Frameworks: Fully aligned with PyTorch 2.12.0 and TensorFlow 2.21.0, ensuring smooth upgrades and interoperability.
Java Interface: Java Interface to the TensorFlow plugin (zentf) compatible with TensorFlow Java v1.2.0-SNAPSHOT.
- TensorFlow-Java v1.2.0-SNAPSHOT supports TensorFlow v2.21.0.
- For TensorFlow-Java 1.2.0-SNAPSHOT, you must build from source.
zentorch.llm.optimize has been deprecated from the 5.2 release.
zentorch provides functional support for INT4 weight-only quantized models using TorchAO's (v0.17.0) IntxWeightOnlyConfig, supporting both per-channel and per-group quantization granularity.
Zentorch provides functional support for INT8 dynamic quantization using TorchAO's (v0.17.0) Int8DynamicActivationInt8WeightConfig with symmetric activation mapping, where activations are dynamically quantized to INT8 at runtime alongside INT8 quantized weights. These models can be executed seamlessly through the vLLM zentorch plugin.
Zentorch provides support for LLM Compressed W8A8 and W4A16 models.

Native Framework Support

The ZenDNN library can be used in the following frameworks through a plugin:
- TensorFlow v2.21.0 through v2.16.1
  Note: The zentf 2.21.0.0 plugin is optimized to give the best performance with TensorFlow v2.21.0.
- PyTorch v2.12.0 and v2.11.0.
  Note: The zentorch 2.12.0.2 and 2.11.0.2 plugins are optimized to give the best performance with PyTorch v 2.12.0 and v2.11.0, respectively.
- vLLM v0.20.0 to v0.23.0
  Note: vLLM support is added to zentorch. The vLLM-ZenTorch plugin integrates with vLLM's V1 engine, enabling plug-and-play acceleration of large language model inference on AMD EPYC™ CPUs.
In ZenDNN 5.0, the ZenDNN library is directly integrated with ONNX Runtime v1.19.2. Beginning with ZenDNN 5.1 and later, support for ONNXRT has been temporarily paused.

For the latest information on the ZenDNN release and installers, visit AMD Developer Central.

Previous Release (ZenDNN 5.2.1) Highlights

Compatibility with Deep-learning Frameworks: Fully aligned with PyTorch 2.11.0 and TensorFlow 2.21.0, ensuring smooth upgrades and interoperability.
Java Interface: Java Interface to the TensorFlow plugin (zentf) compatible with TensorFlow Java v1.2.0-SNAPSHOT.
- TensorFlow-Java v1.2.0-SNAPSHOT supports TensorFlow v2.20.0.
- For TensorFlow-Java 1.2.0-SNAPSHOT, you must build from source.
zentorch.llm.optimize has been deprecated from the 5.2 release.
zentorch provides functional support for INT4 weight-only quantized models using TorchAO's (v0.16.0) IntxWeightOnlyConfig, supporting both per-channel and per-group quantization granularity. These models can be executed seamlessly via torch.compile (backend='zentorch') or through the vLLM zentorch plugin.
Zentorch provides functional support for INT8 dynamic quantization using TorchAO's (v0.16.0) Int8DynamicActivationInt8WeightConfig with symmetric activation mapping, where activations are dynamically quantized to INT8 at runtime alongside INT8 quantized weights. These models can be executed seamlessly through the vLLM zentorch plugin.

Native Framework Support

The ZenDNN library can be used in the following frameworks through a plugin:
- TensorFlow v2.21.0 through v2.16.1
  Note: The ZenDNN 5.2.1 plugin for TensorFlow is optimized to give the best performance with TensorFlow v2.21.0.
- PyTorch v2.11.0 and v2.10.0.
  Note: The ZenDNN 5.2.1 plugin for PyTorch is optimized to give the best performance with PyTorch v2.11.0 and v2.10.0, respectively.
- vLLM v0.15.0 to v0.18.0
  Note: vLLM support is added to ZenDNN PyTorch plugin (zentorch). The vLLM-ZenTorch plugin integrates with vLLM's V1 engine, enabling plug-and-play acceleration of large language model inference on AMD EPYC™ CPUs.
In ZenDNN 5.0, the ZenDNN library is directly integrated with ONNX Runtime v1.19.2. As of ZenDNN 5.1 and 5.2, support for ONNXRT has been temporarily paused.

Release Highlights - Release Highlights - 57300