The highlights of this release are as follows:
JIT and Kernel Infrastructure Improvements
- Enabled JIT based code generation for GEMM and GEMV acrossFP32, BF16, INT8, and FP16 data types.
- Updated Xbyak JIT assembler from v7.05.0 to v7.28.
Performance Optimizations
- Improved F32, BF16 GEMM multi threaded performance.
- Major k = 1 optimizations across GEMM and GEMV.
- GEMV (M=1) JIT backend updates across all data types.
- Build time configurable kernel dispatch table sizing.
New APIs and Datatype Support
- Added new GEMM APIs: FP16FP16, F32FP16 (mixed precision).
- Extended quantized GEMM support: F32×S8, BF16×S8, BF16×U4 (WoQ).
- Improved Batch GEMM performance for all supported data types.
Misc
- Bug fixes.