MPI Code Profiling - MPI Code Profiling - 57368

The MPI programs launched through mpirun or mpiexec launcher programs can be profiled by AMD uProf. To profile the MPI applications and analyze the data, complete the following the steps:

1. Collect the profile data using CLI collect command.
2. Process the profile data using CLI translate command which will generate the profile database.
3. Import the profile database in the GUI or generate the CSV report using CLI report command.
4. Multiple ranks profiling requires higher limit to be set for memory locking using one of the following methods:
   • Increase the memory lock limit using the command ulimit -l, depending on the number of ranks to be profiled on the target node.
   • Set proc/sys/kernel/perf_event_paranoid to -1 or higher value based on the profile config and scope.
   • Profile MPI applications with root privilege.
5. Multiple ranks profiling might require a high number of file descriptors. If the file descriptor limit is reached during profile data collection, an error message will be displayed. You can increase this limit in the file /etc/security/limits.conf.
6. For Multiple ranks profiling, if the /proc/sys/kernel/perf_event_paranoid value is greater than -1, you must increase the /proc/sys/kernel/perf_event_mlockb value depending on the number of ranks to profile. Alternatively, you can also use the -m option to decrease the number of memory data buffer pages used by each instance of AMDuProfCLI.

Support Matrix

The profiling of MPI applications supports components and their corresponding versions provided in this Table 1.