AOCL-DLP provides multiple GEMM variants optimized for different precision requirements and use cases. Every GEMM call computes:
C = post_ops( alpha * op(A) * op(B) + beta * C )
Where op(X) is either X (no transpose) or X^T (transpose), and optional fused
post-operations can be applied to the result via dlp_metadata_t.
Naming Convention
Function names follow a consistent pattern: aocl_gemm_<A_type><B_type><accumulator_type>o<output_type>
For example, aocl_gemm_bf16bf16f32of32 means bfloat16 inputs, float32 accumulation, float32 output.
Supported Data Type Combinations:
Float Precision:
Input A |
Input B |
Accumulator |
Supported Outputs |
Min ISA |
|---|---|---|---|---|
f32 |
f32 |
f32 |
f32 |
AVX2 |
f16 |
f16 |
f16 |
f16 |
AVX512_FP16 |
BFloat16 Precision:
Input A |
Input B |
Accumulator |
Supported Outputs |
Min ISA |
|---|---|---|---|---|
bf16 |
bf16 |
f32 |
f32, bf16 |
AVX2 (*) |
bf16 |
s4 |
f32 |
f32, bf16 |
AVX512 |
bf16 |
u4 |
f32 |
f32, bf16 |
AVX512 |
bf16 |
s8 |
s32 |
s32, f32, bf16, s8, u8 |
AVX512_VNNI |
Float-Quantized Mixed Precision:
Input A |
Input B |
Accumulator |
Supported Outputs |
Min ISA |
|---|---|---|---|---|
f32 |
s8 |
s32 |
s32, f32, bf16, s8, u8 |
AVX512_VNNI |
Integer Quantized:
Input A |
Input B |
Accumulator |
Supported Outputs |
Min ISA |
|---|---|---|---|---|
u8 |
s8 |
s32 |
s32, s8, u8, f32, bf16 |
AVX512_VNNI |
s8 |
s8 |
s32 |
s32, s8, u8, f32, bf16 |
AVX512_VNNI |
Symmetric Quantization:
Input A |
Input B |
Accumulator |
Supported Outputs |
Min ISA |
|---|---|---|---|---|
s8 |
s8 (sym_quant) |
s32 |
f32, bf16 |
AVX512_VNNI |
(*) BFloat16 operations on hardware without native AVX512_BF16 automatically fall back to float32 kernels with transparent conversion. See BFloat16 Fallback Behavior.
Choosing a GEMM Variant:
Need |
Recommended Variant |
Why |
|---|---|---|
Maximum accuracy |
|
Full 32-bit precision throughout |
Good accuracy, less memory |
|
BF16 inputs save memory, f32 accumulation preserves range |
Quantized inference |
|
Integer math is fastest on VNNI hardware |
Weight-quantized inference |
|
BF16 activations with 4-bit weights |
Half-precision pipeline |
|
Native FP16 end-to-end (requires AVX512_FP16) |
For a complete list of GEMM variants, see the GEMM API Reference and the GEMM Guide Wiki.
Basic GEMM Call Pattern:
The basic pattern for calling AOCL-DLP GEMM functions follows this structure:
#include "aocl_dlp.h"
// Basic f32 GEMM call: C = alpha * A * B + beta * C
aocl_gemm_f32f32f32of32(
'R', // Storage format (R=row-major, C=column-major)
'N', // TransA (N=no transpose, T=transpose)
'N', // TransB
m, n, k, // Matrix dimensions
1.0f, // alpha scalar
a, lda, 'N', // Matrix A, leading dimension, memory format
b, ldb, 'N', // Matrix B, leading dimension, memory format
0.0f, // beta scalar
c, ldc, // Matrix C, leading dimension
NULL // Post-operations metadata (NULL = no post-ops)
);
Matrix Reordering for Performance:
For matrices that will be reused multiple times, reordering can significantly improve performance:
// Get buffer size needed for reordering matrix B
msz_t buffer_size = aocl_get_reorder_buf_size_f32f32f32of32(
'R', // Storage order (row-major)
'N', // TransB
'B', // Matrix to reorder ('A' or 'B')
k, n, // Dimensions (rows and cols of B)
NULL // Post-operations metadata
);
// Allocate buffer and reorder matrix B
float* reordered_b = (float*)malloc(buffer_size);
aocl_reorder_f32f32f32of32(
'R', // Storage order
'N', // TransB
'B', // Matrix to reorder
b, // Source matrix
reordered_b, // Destination buffer
k, n, // Dimensions
ldb, // Leading dimension of source
NULL // Post-operations metadata
);
// Use reordered matrix in GEMM calls
aocl_gemm_f32f32f32of32(
'R', 'N', 'N', m, n, k,
1.0f, a, lda, 'N',
reordered_b, ldb, 'R', // 'R' indicates reordered format
0.0f, c, ldc, NULL
);
// Clean up
free(reordered_b);
For detailed buffer size and reorder APIs, see the API Lifecycle documentation.
Batch GEMM Operations:
AOCL-DLP supports batch GEMM operations for processing multiple matrix multiplications efficiently:
// Batch f32 GEMM example
aocl_batch_gemm_f32f32f32of32(
order_array, // Array of storage layouts
transa_array, transb_array, // Arrays of transpose flags
m_array, n_array, k_array, // Arrays of dimensions
alpha_array, // Array of alpha scalars
(const float**)a_array, lda_array, // Arrays of A matrices
(const float**)b_array, ldb_array, // Arrays of B matrices
beta_array, // Array of beta scalars
c_array, ldc_array, // Arrays of C matrices
group_count, group_sizes, // Grouping parameters
mem_format_a_array, mem_format_b_array, // Memory format arrays
post_ops_array // Array of post-operations metadata
);
For more information, see aocl_batch_gemm_f32f32f32of32.
Function Parameters:
All GEMM functions share a common parameter pattern:
Parameter |
Values |
Description |
|---|---|---|
|
|
Row-major or column-major layout |
|
|
No transpose or transpose |
|
integers |
Matrix dimensions: A is m×k, B is k×n, C is m×n |
|
scalars |
C = alpha × A×B + beta × C |
|
integers |
Leading dimensions (stride between rows/columns) |
|
|
Normal, packed, or reordered format |
|
|
Post-operations (NULL for none) |
Error Handling:
GEMM functions validate parameters and report errors via dlp_metadata_t.error_hndl when a
metadata pointer is provided:
float alpha=1.0f, beta=0.0f;
dlp_metadata_t meta = {0};
aocl_gemm_f32f32f32of32('R', 'N', 'N', m, n, k,
alpha, a, lda, 'N', b, ldb, 'N', beta, c, ldc, &meta);
if (meta.error_hndl.error_code != DLP_CLSC_SUCCESS) {
fprintf(stderr, "Error code: %d\n", meta.error_hndl.error_code);
}
Common error codes (from dlp_clsc_err_t):
Code |
Meaning |
|---|---|
|
Operation completed successfully |
|
NULL pointer passed as argument |
|
Invalid m, n, or k |
|
Leading dimension too small |
|
Invalid memory layout character |
|
Invalid transpose character |
|
Invalid mem_format character |