4.1.7.1. Single-Thread AOCL-BLAS#

Complete the following steps to install a single-thread AOCL-BLAS:

1. Clone the AOCL-BLAS Git repository (amd/blis.git).

2. Configure the library as required:

   # CMake commands
   
   # GCC (Default)
   $ cmake . -DENABLE_CBLAS=ON -DCMAKE_INSTALL_PREFIX=<your-install-dir> \
     -DBLIS_CONFIG_FAMILY=auto
   
   # AOCC
   $ cmake . -DENABLE_CBLAS=ON -DCMAKE_INSTALL_PREFIX=<your-install-dir> \
     -DCOMPLEX_RETURN=intel CC=clang CXX=clang++ -DBLIS_CONFIG_FAMILY=auto
   
   # Alternatively, using configure
   
   # GCC (Default)
   $ ./configure --enable-cblas --prefix=<your-install-dir> auto
   
   # AOCC
   $ ./configure --enable-cblas --prefix=<your-install-dir> \
     --complex-return=intel CC=clang CXX=clang++ auto
   

3. To build the library, use the command:

   $ make
   

4. To install the library on build machine, use the command:

   $ make install
   

4.1.7.2. Multi-Thread AOCL-BLAS#

Complete the following steps to install a multi-thread AOCL-BLAS:

1. Clone the AOCL-BLAS Git repository (amd/blis.git).

2. Configure the library as required:

   # CMake commands
   
   # GCC (Default)
   $ cmake . -DENABLE_CBLAS=ON -DCMAKE_INSTALL_PREFIX=<your-install-dir> \
     -DENABLE_THREADING=[Mode] -DBLIS_CONFIG_FAMILY=auto
   
   # AOCC
   $ cmake . -DENABLE_CBLAS=ON -DCMAKE_INSTALL_PREFIX=<your-install-dir> \
     -DCOMPLEX_RETURN=intel CC=clang CXX=clang++ \
     -DENABLE_THREADING=[Mode] -DBLIS_CONFIG_FAMILY=auto
   
   # Alternatively, using configure
   
   # GCC (Default)
   $ ./configure --enable-cblas --prefix=<your-install-dir> \
     --enable-threading=[Mode] auto
   
   # AOCC
   $ ./configure --enable-cblas --prefix=<your-install-dir> \
     --enable-threading=[Mode] \
     --complex-return=intel CC=clang CXX=clang++ auto
   

   Mode indicates one of the options in {openmp, no}. no option implies disable multi-threading.

3. To build the library, use the command:

   $ make
   

4. To install the library on build machine, use the command:

   $ make install
   

4.1.8.1. Building AOCL-BLAS Using GUI#

4.1.8.2. Preparing Project with CMake GUI#

Complete the following steps in the CMake GUI:

1. Set the source (folder containing AOCL-BLAS source code) and build (folder in which the project files will be generated, for example, out) folder paths as shown in the following figure:

   

   It is not recommended to use the folder named build since build is used by Linux build system.

2. Click on the Configure button to prepare the project options.

3. Set the generator to Visual Studio 17 2022 and the compiler to ClangCl as shown in the following figure:

   

4. Update the options based on the project requirements. All the available options are listed in the following table:

   Table 4.5 CMake Config Options#

   Feature	CMake Parameter
   AMD CPU architecture	BLIS_CONFIG_FAMILY=zen / zen2 / zen3 / zen4 / zen5 / amdzen
   Shared library	BUILD_SHARED_LIBS=ON
   Static library	BUILD_SHARED_LIBS=OFF
   Debug/Release build type	CMAKE_BUILD_TYPE=Debug / Release
   Enable single threading (disables AOCL dynamic dispatch)	ENABLE_THREADING=no(default)
   Enable multi-threading (enables AOCL dynamic dispatch with OpenMP)	ENABLE_THREADING=openmp
   AOCL Dynamic (automatically selected depending on the value of ENABLE_THREADING)	ENABLE_AOCL_DYNAMIC=ON/OFF
   Enable BLAS/CBLAS support	ENABLE_BLAS=ON ENABLE_CBLAS=ON
   Enable 32-bit integer size in BLIS and BLAS APIs	INT_SIZE=32 and BLAS_INT_SIZE=32
   Enable 64-bit integer size in BLIS and BLAS APIs	INT_SIZE=64 and BLAS_INT_SIZE=64
   Absolute path to the OpenMP library, including the library name	OpenMP_libomp_LIBRARY

   Table 4.6 CMake Config Options (all variables and their default values)#

   CMake Parameter
   BUILD_SHARED_LIBS=ON(default)/OFF
   ENABLE_THREADING=no(default)/openmp
   INT_SIZE=auto(default)/32/64
   BLAS_INT_SIZE=32(default)/64
   ENABLE_BLAS=ON/OFF(default)
   ENABLE_CBLAS=ON/OFF(default)
   ENABLE_MIXED_DT=ON(default)/OFF
   ENABLE_SUP_HANDLING=ON(default)/OFF
   ENABLE_AOCL_DYNAMIC=ON(default)/OFF
   COMPLEX_RETURN=gnu(default)/intel
   ENABLE_NO_UNDERSCORE_API=ON/OFF(default)
   ENABLE_UPPERCASE_API=ON/OFF(default)
   ENABLE_SYSTEM=ON(default)/OFF
   THREAD_PART_JRIR=slab(default)/rr
   ENABLE_PBA_POOLS=ON(default)/OFF
   ENABLE_SBA_POOLS= ON(default)/OFF
   ENABLE_MEM_TRACING=ON/OFF(default)
   ENABLE_MIXED_DT=ON(default)/OFF
   ENABLE_MIXED_DT_EXTRA_MEM=ON(default)/OFF
   ENABLE_SUP_HANDLING=ON(default)/OFF
   ENABLE_TRSM_PREINVERSION=ON(default)/OFF
   FORCE_VERSION=no(default)/<user-defined>
   DISABLE_BLIS_ARCH_TYPE=ON/OFF(default)
   RENAME_BLIS_ARCH_TYPE=BLIS_ARCH_TYPE(default)/<user-defined>
   RENAME_BLIS_MODEL_TYPE=BLIS_MODEL_TYPE(default)/<user-defined>

   For the detailed documentation of all the options, configure CMake with PRINT_CONFIGURE_HELP=ON.

5. To generate the Microsoft Visual Studio project in the out folder, click on the Generate button as shown in the following figure:

   

4.1.8.3. Building the Project in Visual Studio GUI#

Complete the following steps in the Microsoft Visual Studio GUI:

1. Open the project generated by CMake (build folder) in Preparing Project with CMake GUI.

2. To generate AOCL-BLAS binaries, build the AOCL-LibBlis project or libs/libblis target. The library files will be generated in the out folder based on the project settings.

   For example, blis/out/Release/AOCL-LibBlis-Win-MT.dll or AOCL-LibBlis-Win-MT.lib

3. To install the binaries (or to build and install them), build the INSTALL project under CMakePredefinedTargets.

4.1.8.4. Building AOCL-BLAS Using Command-Line Arguments#

The project configuration and build procedures can be triggered from the command prompt as well. The corresponding steps are described in the following sections.

Configuring the Project in Command Prompt

In the AOCL-BLAS project folder, create a folder out. Open the command prompt in this directory and run the following command to configure the project:

$ cmake -S .. -B . -G "Visual Studio 17 2022"
-DCMAKE_BUILD_TYPE=Release
-DBLIS_CONFIG_FAMILY=amdzen -DBUILD_SHARED_LIBS=ON
-DENABLE_THREADING=openmp
-DCOMPLEX_RETURN=intel -DOpenMP_libomp_LIBRARY="C:\Program
Files\LLVM\lib\libomp.lib" -TClangCL

You can refer CMake Config Options and update the parameter options in the command according to the project requirements or run the following command for a detailed description of the available options:

$ cmake -S .. -B . -G "Visual Studio 17 2022" -DPRINT_CONFIGURE_HELP=ON

Building the Project in Command Prompt

Open command prompt in the blis\out directory. Invoke CMake with the build command with release or debug option. For example:

$ cmake --build . --config Release

For building the library using multiple threads, run the following command:

$ cmake --build . --config Release -j

The library files would be generated in the Release or Debug folder based on the project settings.

4.2.2.1. Using BLAS API in Fortran#

For example, the following Fortran code does a double precision general matrix-matrix multiplication. It calls the ‘DGEMM’ BLAS API function to accomplish this. A sample command to compile and link it with the AOCL-BLAS library is shown in the following code:

! File: BLAS_DGEMM_usage.f
! Example code to demonstrate BLAS DGEMM usage

program dgemm_usage

implicit none
EXTERNAL DGEMM

DOUBLE PRECISION, ALLOCATABLE :: a(:,:)
DOUBLE PRECISION, ALLOCATABLE :: b(:,:)
DOUBLE PRECISION, ALLOCATABLE :: c(:,:)
INTEGER I, J, M, N, K, lda, ldb, ldc
DOUBLE PRECISION alpha, beta

M=2
N=M
K=M
lda=M
ldb=K
ldc=M
alpha=1.0
beta=0.0

ALLOCATE(a(lda,K), b(ldb,N), c(ldc,N))
a=RESHAPE((/ 1.0, 3.0, &
             2.0, 4.0 /), (/lda,K/))
b=RESHAPE((/ 5.0, 7.0, &
             6.0, 8.0 /), (/ldb,N/))

WRITE(*,*) ("a =")
DO I = LBOUND(a,1), UBOUND(a,1)
    WRITE(*,*) (a(I,J), J=LBOUND(a,2), UBOUND(a,2))
END DO
WRITE(*,*) ("b =")
DO I = LBOUND(b,1), UBOUND(b,1)
    WRITE(*,*) (b(I,J), J=LBOUND(b,2), UBOUND(b,2))
END DO

CALL DGEMM('N','N',M,N,K,alpha,a,lda,b,ldb,beta,c,ldc)

WRITE(*,*) ("c =")
DO I = LBOUND(c,1), UBOUND(c,1)
    WRITE(*,*) (c(I,J), J=LBOUND(c,2), UBOUND(c,2))
END DO

end program dgemm_usage

A sample compilation command with gfortran compiler for the code above:

$ gfortran -ffree-form BLAS_DGEMM_usage.f /path/to/libblis.a

4.2.2.2. Using BLAS API in C#

Following is the C version of the Fortran code in section Using BLAS API in Fortran. It uses the standard BLAS API. The following process takes place during the execution of the code:

1. The matrices are transposed to account for the row-major storage of C and the column-major convention of BLAS (inherited from Fortran).

2. The function arguments are passed by address again to be in line with Fortran conventions.

3. There is a trailing underscore in the function name (dgemm_) as BLAS APIs require Fortran compilers to add a trailing underscore.

4. blis.h is included as a header. A sample command to compile it and link with the AOCL-BLAS library is also shown in the following code:

   // File: BLAS_DGEMM_usage.c
   // Example code to demonstrate BLAS DGEMM usage
   
   #include <stdio.h>
   #include "blis.h"
   
   #define DIM 2
   
   int main() {
   
      double a[DIM * DIM] = { 1.0, 2.0, 3.0, 4.0 };
      double b[DIM * DIM] = { 5.0, 7.0, 6.0, 8.0 };
      double c[DIM * DIM];
      int I, J, M, N, K, lda, ldb, ldc; double alpha, beta;
   
      M = DIM;
      N = M; K = M;
      lda = M; ldb = K; ldc = M;
      alpha = 1.0;
      beta = 0.0;
   
      printf("a = \\n");
      for ( I = 0; I < M; I ++ ) {
        for ( J = 0; J < K; J ++ ) {
          printf("%f\t", a[J * K + I]);
        }
        printf("\n");
      }
   
      printf("b = \\n");
      for ( I = 0; I < K; I ++ ) {
        for ( J = 0; J < N; J ++ ) {
          printf("%f\t", b[J * N + I]);
        }
        printf("\n");
      }
   
      dgemm_("N","N",&M,&N,&K,&alpha,a,&lda,b,&ldb,&beta,c,&ldc);
      printf("c = \\n");
   
      for ( I = 0; I < M; I ++ ) {
        for ( J = 0; J < N; J ++ ) {
          printf("%f\t", c[J * N + I]);
        }
        printf("\n");
      }
   
      return 0;
   }
   

A sample compilation command with a gcc compiler for the code above:

$ gcc BLAS_DGEMM_usage.c -I/path/to/include/aocl-blas/ \
  /path/to/libblis.a -lpthread -lm

4.2.2.3. Using CBLAS API in C#

This section contains an example application written in C code using the CBLAS API for DGEMM. The following process takes place during the execution of the code:

1. The CBLAS Layout option is used to choose row-major layout which is consistent with C.

2. The function arguments are passed by value.

3. cblas.h is included as a header. A sample command to compile it and link with the AOCL-BLAS library is also shown in the following code:

   // File: CBLAS_DGEMM_usage.c
   // Example code to demonstrate CBLAS DGEMM usage
   
   #include <stdio.h>
   #include "cblas.h"
   #define DIM 2
   
   int main() {
   
      double a[DIM * DIM] = { 1.0, 2.0, 3.0, 4.0 };
      double b[DIM * DIM] = { 5.0, 6.0, 7.0, 8.0 };
      double c[DIM * DIM];
      int I, J, M, N, K, lda, ldb, ldc; double alpha, beta;
   
      M = DIM;
      N = M; K = M;
      lda = M; ldb = K; ldc = M;
      alpha = 1.0;
      beta = 0.0;
   
      printf("a = \\n");
      for ( I = 0; I < M; I ++ ) {
        for ( J = 0; J < K; J ++ ) {
          printf("%f\t", a[I * K + J]);
        }
        printf("\n");
      }
   
      printf("b = \\n");
      for ( I = 0; I < K; I ++ ) {
        for ( J = 0; J < N; J ++ ) {
          printf("%f\t", b[I * N + J]);
        }
        printf("\n");
      }
   
      cblas_dgemm(CblasRowMajor, CblasNoTrans, CblasNoTrans, M, N, K,
      alpha, a, lda, b, ldb, beta, c, ldc);
   
      printf("c = \\n");
      for ( I = 0; I < M; I ++ ) {
        for ( J = 0; J < N; J ++ ) {
          printf("%f\t", c[I * N + J]);
        }
        printf("\n");
      }
   
      return 0;
   }
   

A sample compilation command with a gcc compiler for the code above is as follows:

$ gcc CBLAS_DGEMM_usage.c -I/path/to/include/aocl-blas/ \
  /path/to/libblis.a -lpthread -lm

4.2.2.4. Returning Complex Numbers#

The GNU Fortran compiler (gfortran), AOCC (Flang), and Intel Fortran compiler (ifort) have different requirements for returning complex numbers from the C functions as follows:

• AOCC (Flang) and Intel® (ifort) compilers returns complex numbers using hidden first argument. The caller must pass the pointer to the return value as the first parameter.

• GNU (gfortran) compiler returns complex numbers using registers. Thus, the complex numbers are returned as the return value of the function itself.

gfortran Example:

• Configure Option:

  --complex-return=gnu
  

• API Call:

  ret_value = cdotc_(&n, x, &incx, y, &incy);
  

AOCC Example:

• Configure Option:

  --complex-return=intel CC=clang CXX=clang++
  

• API Call:

  cdotc_(&ret_value, &n, x, &incx, y, &incy);
  

This feature is currently enabled only for cdotc, cdotu, zdotc, and zdotu APIs.

4.4.1.1. Purpose#

Before Dynamic Dispatch, the user had to build different binaries for each CPU architecture, that is, AMD “Zen”, AMD “Zen2”, and AMD “Zen3” architectures. Furthermore, when building the application, users had to ensure that they used the correct AMD “Zen”-based library as needed for the platform. This becomes challenging when using AOCL-BLAS on a cluster having nodes of different architectures.

Dynamic Dispatch addresses this issue by building a single binary compatible with all the AMD “Zen” architectures. At the runtime, the Dynamic Dispatch feature enables optimizations specific to the detected AMD “Zen” architecture.

4.4.1.2. On Non-AMD “Zen” Architectures#

The Dynamic Dispatch feature supports AMD “Zen”, AMD “Zen2”, AMD “Zen3”, AMD “Zen4”, and AMD “Zen5” architectures in a single binary. However, it also includes a generic architecture to support older x86-64 processors. The generic architecture uses a pure C implementation of the APIs and does not use any architecture-specific features.

The specific compiler flags used for building the library with generic configuration are:

-O2 -funsafe-math-optimizations -ffp-contract=fast -Wall \
-Wno-unused-function -Wfatal-errors

Previous AOCL-BLAS releases identified the processor based on Family, Model, and other cpuid features, and selected the appropriate code path based on the preprogrammed choices. With Dynamic Dispatch, an unknown processor would fall through to the slow generic code path, although users could override this by setting an environment variable BLIS_ARCH_TYPE to a suitable value.

From AOCL-BLAS 4.2, additional cpuid tests based on AVX2 and AVX512 instruction support are used to enable AMD “Zen3”, AMD “Zen4” or AMD “Zen5” code paths to be selected by default on suitable x86-64 processors (i.e. future AMD processors and current or future Intel processors). These AMD Zen code paths are not (re-)optimized specifically for these different architectures but should perform better than the slow generic code path.

To be more specific:

• AVX2 support requires AVX2 and FMA3.

• AVX512 support requires AVX512 F, DQ, CD, BW, and VL.

4.4.1.3. Using Dynamic Dispatch#

Building AOCL-BLAS

Dynamic Dispatch must be enabled while building the AOCL-BLAS library. This is done by building the library for amdzen configuration as explained in Build AOCL-BLAS from Source on Linux.

Code Path Information

Dynamic Dispatch can print debugging information on the selected code path. This is enabled by setting the environment variable BLIS_ARCH_DEBUG=1.

Architecture Selection at Runtime

For most use cases, Dynamic Dispatch will detect the underlying architecture and enable appropriate code paths and optimizations. However, AOCL-BLAS can be forced to use a specific architecture by setting either the environment variable AOCL_ENABLE_INSTRUCTIONS or BLIS_ARCH_TYPE as follows:

AOCL_ENABLE_INSTRUCTIONS=value <AOCL-BLAS linked application>

or

BLIS_ARCH_TYPE=value <AOCL-BLAS linked application>

Where, value = {avx512, avx2, zen5, zen4, zen3, zen2, zen, generic} You must note the following:

• The code path names are not case sensitive but the environment variable names are.

• Options for older x86-64 vector ISAs (e.g. avx, sse2) are also supported but in general will correspond to the generic code path.

• In AOCL-BLAS builds with configuration amdzen, avx512 is an alias for zen4 and avx2 is an alias for zen3.

• AOCL_ENABLE_INSTRUCTIONS is intended to become the standard option for controlling dynamic dispatch (where supported) across all the AOCL components.

• BLIS_ARCH_TYPE is specific to the BLIS code used in AOCL-BLAS.

• If both are specified, BLIS_ARCH_TYPE takes precedence and AOCL_ENABLE_INSTRUCTIONS is ignored by AOCL-BLAS. This provides and options of using AOCL_ENABLE_INSTRUCTIONS to control other AOCL libraries but specify different options for AOCL-BLAS using BLIS_ARCH_TYPE.

• The operation of AOCL_ENABLE_INSTRUCTIONS and BLIS_ARCH_TYPE are slightly different:

  • If AOCL_ENABLE_INSTRUCTIONS is in operation, AOCL-BLAS will check if the instruction set required by the code path selected is enabled within the library and supported by the processor. If not, it will use the default choice for that architecture. In other words, AOCL_ENABLE_INSTRUCTIONS should be used to restrict a processor to an earlier instruction set, rather than try to force a later one on an older processor.

  • By contrast, if BLIS_ARCH_TYPE is in operation, that code path will be used irrespective of the compatibility with the processor.

• Specifying a particular code path will completely override the automatic selection and thus, the following scenarios are possible:

  • A code path unavailable in the AOCL-BLAS build is being used. This will result in an error message from the AOCL-BLAS library which will then abort. This only applies when using BLIS_ARCH_TYPE (at AOCL 4.2 it also applied to AOCL_ENABLE_INSTRUCTIONS).

  • A code path executes instructions unavailable on the processor being used, for example, trying to run the AMD “Zen4” code path (which may use AVX512 instructions) on a AMD “Zen3” or older system. If this happens, the program may stop with an “illegal instruction” error. This applies only when BLIS_ARCH_TYPE is used; executing the illegal instruction may be routine and problem size dependent.

In some circumstances, AOCL-BLAS aborting on an error from BLIS_ARCH_TYPE being set incorrectly may not be acceptable. If you are building AOCL-BLAS from source, there are two options to mitigate this issue. One is to change the environment variable used from BLIS_ARCH_TYPE to another name, for example:

$ ./configure --enable-cblas --prefix=<your-install-dir> \
  --rename-blis-arch-type=MY_BLIS_ARCH_TYPE amdzen
... make aocl-blas library
... compile program linking with aocl-blas
$ export BLIS_ARCH_TYPE=zen3
$ export MY_BLIS_ARCH_TYPE=zen2
$ ./program.exe

This will cause program.exe (which uses AOCL-BLAS) to ignore the setting of BLIS_ARCH_TYPE to zen3. Instead, it will take the value of MY_BLIS_ARCH_TYPE and use the zen2 code path. When --rename-blis-arch-type is used, AOCL_ENABLE_INSTRUCTIONS remains enabled in the build, but MY_BLIS_ARCH_TYPE (in this example) would take precedence if both are set.

Alternatively, the mechanism to allow manual selection of code path can be disabled:

$ ./configure --enable-cblas --prefix=<your-install-dir> \
  --disable-blis-arch-type amdzen

In this case, Dynamic Dispatch will still occur among the included code paths, but only by automatic selection based on the processor architecture. Manual selection of code path by both AOCL_ENABLE_INSTRUCTIONS and BLIS_ARCH_TYPE is disabled.

Model Selection at Runtime

Recent AMD “Zen” generations have added more diverse choices of core designs and cache characteristics. For example, Milan and Milan-X variants at AMD “Zen3”; Genoa, Bergamo, and Genoa-X variants at AMD “Zen4”. Some AOCL-BLAS APIs may be tuned differently for these different models. The appropriate model will be selected automatically by Dynamic Dispatch.

However, AOCL can be forced to use a specific model by setting the environment variable BLIS_MODEL_TYPE as follows:

BLIS_MODEL_TYPE=value <AOCL-BLAS linked application>

where value = {Milan, Milan-X, Genoa, Bergamo, Genoa-X, Turin, Turin-Dense} Note the following:

• Different model values correspond to specific BLIS_ARCH_TYPE values (either set automatically or explicitly by the user). Thus, Milan and Milan-X correspond to AMD “Zen3”; Genoa, Bergamo, and Genoa-X correspond to AMD “Zen4”, and Turin and Turin-Dense correspond to AMD “Zen5”.

• Incorrect values of BLIS_MODEL_TYPE do not cause an error, the default model type for the selected architecture will be used.

• The number of APIs that have different optimizations by model type is currently very small. Setting this environment variable may provide consistent results across different models if consistency is a higher priority than best performance.

As with BLIS_ARCH_TYPE, when building BLAS from source, the name of the environment variable used to set the model type can be changed, for example:

$ ./configure --enable-cblas --prefix=<your-install-dir> \
  --rename-blis-model-type=MY_BLIS_MODEL_TYPE amdzen

Disabling the mechanism to allow the manual section of BLAS architecture will also disable the mechanism to allow the manual section of the model.

$ ./configure --enable-cblas --prefix=<your-install-dir> \
  --disable-blis-arch-type amdzen

Setting either of these environment variables makes sense only when using a build of AOCL-BLAS that includes multiple code paths.

Thus, AOCL_ENABLE_INSTRUCTIONS and BLIS_ARCH_TYPE are disabled by default in all the builds containing only a single code path.

4.4.2.1. Multi-Thread Test Suite Performance#

Starting from AOCL-BLAS 3.1, if the number of threads is not specified, AOCL-BLAS uses the maximum number of threads equal to the number of cores available on the system. A higher number of threads result in better performance for medium to large size matrices found in practical use cases.

However, the higher number of threads results in poor performance for very small sizes used by the test and check features. Hence, you must specify the number of threads while running the test/test suite.

The recommended number of threads to run the test suite is 1 or 2.

Running Test Suite

Execute the following command to invoke the test suite:

$ OMP_NUM_THREADS=2 make test

The sample output from the execution of the command is as follows:

$ OMP_NUM_THREADS=2 make test

Compiling obj/zen3/testsuite/test_addm.o
Compiling obj/zen3/testsuite/test_addv.o

<<< More compilation output >>>

Compiling obj/zen3/testsuite/test_xpbym.o
Compiling obj/zen3/testsuite/test_xpbyv.o

Linking test_libblis-mt.x against 'lib/zen3/libblis-mt.a -lm
-lpthread -fopenmp -lrt' Running test_libblis-mt.x with output
redirected to 'output.testsuite'

check-blistest.sh: All BLIS tests passed! Compiling
obj/zen3/blastest/cblat1.o Compiling obj/zen3/blastest/abs.o

<<< More compilation output >>>

Compiling obj/zen3/blastest/wsfe.o
Compiling obj/zen3/blastest/wsle.o
Archiving obj/zen3/blastest/libf2c.a

Linking cblat1.x against 'libf2c.a lib/zen3/libblis-mt.a -lm
-lpthread -fopenmp -lrt' Running cblat1.x > 'out.cblat1'

<<< More compilation output >>>

Linking zblat3.x against 'libf2c.a lib/zen3/libblis-mt.a -lm
-lpthread -fopenmp -lrt' Running zblat3.x <
'./blastest/input/zblat3.in' (output to 'out.zblat3')

check-blastest.sh: All BLAS tests passed!

4.5.3.1. Function Call Tracing#

The function call tracing is implemented using hard instrumentation of the AOCL-BLAS code. Here, the functions are grouped as per their position in the call stack. You can configure the level up to which the traces must be generated.

Complete the following steps to enable and view the traces:

1. Enable the trace support as follows:

   1. Modify the source code to enable tracing.

      Open file <aocl-blas folder>/aocl_dtl/aocldtlcf.h
      

   2. Change the following macro from 0 to 1:

      #define AOCL_DTL_TRACE_ENABLE 0
      

2. Configure the trace depth level.

   1. Modify the source code to specify the trace depth level.

      Open file <aocl-blas folder>/aocl_dtl/aocldtlcf.h
      

   2. Change the following macro as required. Beginning with Level 5 should be a good compromise in terms of details and resource requirement. The higher the level, the deeper is the call stack. A lower level reduces the depth of the call stack used for a trace generation.

      #define AOCL_DTL_TRACE_LEVEL AOCL_DTL_LEVEL_TRACE_5
      

3. Build the library as explained in Build AOCL-BLAS from Source on Linux.

4. Run the application to generate the trace data.

   The trace output file for each thread is generated in the current folder.

   The following figure shows a sample running the call tracing function using the test_gemm application:

   

   The trace data for each thread is saved in the file with appropriate naming conventions. The .txt extension is used to signify the readable file:

   P<process id>_T<thread id>_aocldtl_trace.txt

5. View the trace data.

   The output of the call trace is in a readable format, you can open the file in any of the text editors. The first column shows the level in call stack for the given function.

4.5.3.2. Debug Logging#

The debug logging works very similar to the function call tracing and uses the same infrastructure. However, it can be enabled independent of the trace feature to avoid cluttering of the overall debugging information. This feature is primarily used to print the input values of the AOCL-BLAS APIs. Additionally, it can also be used to print any arbitrary debugging data (buffers, matrices, arrays, or text).

Complete the following steps to enable and view the debug logs:

1. Enable the debug log support as follows:

   1. Modify the source code to enable debug logging.

      Open file <aocl-blas folder>/aocl_dtl/aocldtlcf.h
      

   2. Change the following macro from 0 to 1:

      #define AOCL_DTL_LOG_ENABLE 0
      

2. Configure the trace depth level.

   1. Modify the source code to specify the debug log depth level.

      Open file <aocl-blas folder>/aocl_dtl/aocldtlcf.h
      

   2. Change the following macro as required. Beginning with Level 5 should be a good compromise in terms of details and resource requirement. The higher the level (maximum is 10), the deeper is the call stack. A lower level reduces the depth of the call stack used for a trace generation.

      #define AOCL_DTL_TRACE_LEVEL AOCL_DTL_LEVEL_TRACE_5
      

3. Build the library as explained in Build AOCL-BLAS from Source on Linux.

4. Run the application to generate the trace data.

   The trace output files for each thread is generated in the current folder.

   The following figure shows a sample running of AOCL-BLAS with the debug logs enabled using the test_gemm application:

   

   The debug logs for each thread are saved in the file with appropriate naming conventions. The .txt extension is used to signify the readable file:

   P<process id>_T<thread id>_aocldtl_log.txt

5. View the debug logs.

   The output of the debug logs is in a readable format, you can open the file in any text editor. The following figure shows the sample output for one of the threads of the test_gemm application:

   $ cat P3386555_T0_aocldtl_log.txt
   dgemm_blis_impl D N N 4000 4000 4000 1.300000 0.000000 4000 4000 0.700000 0.000000 4000 nt=1 911.148 ms 70.482 GFLOPS
   dgemm_blis_impl D N N 4000 4000 4000 1.300000 0.000000 4000 4000 0.700000 0.000000 4000 nt=8 121.024 ms 557.641 GFLOPS
   

4.5.3.3. Usage and Limitations#

The debug and trace logs have the following usage and limitations:

• When tracing is enabled, there could be a significant drop in the performance.

• Only a function that has the trace feature in the code can be traced. To get the trace information for any other function, the source code must be updated to add the trace/log macros in them.

• The call trace and debug logging is a resource-dependent process and can generate a large size of data. Based on the hardware configuration (the disk space, number of cores and threads) required for the execution, logging may result in a sluggish or non-responsive system.

4.6.4.1. Architecture Features and APIs#

4.6.4.2. Utility APIs in aocl_gemm Add-on#

LPGEMM APIs supports reordering the entire input matrix before calling GEMM and on the go packing, where GEMM API takes care of packing of matrix internally. The following utility APIs are used to reorder input weight matrix before calling GEMM: