Timeline Analysis GUI in Linux - Timeline Analysis GUI in Linux - 57368

To configure threading analysis from the GUI:

1. Navigate to the Select Profile Configuration screen.
2. Select Predefined Configs from the tab.
3. Select Threading Analysis from the left vertical pane.

Profile data collected from CLI or GUI can be visualized in GUI by importing the session. On importing, the following section (Thread Timeline) is displayed on the ANALYZE page.

Time-series data is plotted in timelines per entity (thread, rank, device, and so on). Trace data (if collected) will only be plotted when you zoom into the timeline to address data size related scalability issues (trace data can have millions of records which will not be visually legible if plotted together). The entire view is broadly separated in three vertical parts, top data selectors, middle timelines, and bottom filters. You can use the timeline as follows:

• Hover the cursor over a timeline to view a vertical line containing the tool-tip for a specific entity, showing relevant details, and the current timestamp.
• If callstack data is collected, click at any point in the timeline to bring up the callstack of the corresponding entity in the bottom pane.

• If CPU profile data is collected, click and drag the mouse over the timeline to select a region across all timelines and brings up the Function Hotspot within the selected time range.
• Zoom-in/out horizontally into the timelines using one of the following
  • The mouse wheel
  • Pressing CTRL and +/- keys on the keyboard to zoom-in or zoom-out, respectively

The timeline section consists of:

1. Name of each thread in timeline with Thread ID.
2. Click to Load More button which loads more threads. By default, only a small number of thread timelines are loaded to limit the resource consumption. This button enables loading the next set of thread timelines. The next set is determined by the entries in the table below the timeline.
3. Select the Data Source drop-down to enable selection of data to display on the timeline. Different types of data source are as follows:
   • CPU Utilization: Plots the timeline for the CPU utilization (in %) per thread at a per second interval. To collect sufficient such data points, the total profile duration should be greater than or equal to 5 seconds. This is enabled only for the Threading Analysis configuration.
   • Memory Consumption: Plots the timeline for the memory consumption (in MB) categorized as physical and virtual memory consumed. This is enabled only for the Threading Analysis configuration.
   • Context Switches: Plots the timeline for both voluntary context switches count (sleep, yield, and so on) or involuntary context switches count (OS scheduler triggered context switch). This is enabled only for the Threading Analysis configuration.
   • CPU Profile Samples: Plots the timeline for the CPU sample collected for the CPU events. The CPU samples are plotted as a heatmap (with colors ranging from dark-green (highest samples in a window) to yellow-green (lowest samples in a window) with gray regions depicting where there are no samples collected). The following events are supported:

     Table 1. Supported CPU Events

     Events	Availability
     Retired Instructions	PMC event RETIRED_INSTRUCTIONS is collected.
     Cycles not in Halt	PMC event CYCLES_NOT_IN_HALT is collected.
     Op Cycles	IBS op event is collected with ‘count cycles’ unit mask.
     CPU Time	Time-based profiling is performed.

   • GPU Related Counts: GPU Kernel Time, Copy Time and HIP API Time are also plotted to provide time spent in GPU kernels, GPU memory copy operations, and time spent inside HIP APIs, respectively.
   • Thread Trace: Plots the timeline based on OS trace data which can either originate from eBPF Tracing or User-mode Tracing. The trace data is categorized and aggregated at certain intervals to generate time-series plotted in timelines. The following categories are created:

     Table 2. Thread Time Categories

     Category	Description
     Event Wait Time	Total time spent on IO Multiplexing (poll, select etc.), wait for process / thread to finish (wait, pthread_join etc.).
     Resource Wait Time	Total time spent on synchronization objects, reader and writer locks, thread barrier etc.
     Sleep Time	Total time spent on sleep and waiting for signal delivery system calls.
     I/O Sync Time	Total time spent on IO sync APIs (sync, fsync etc.).
     Running Time	Total active processor time, which includes the time spent in IO operations and spin lock operations.
     Spin Time	Total time spent on spin lock.
     I/O Time	Total Time spent in I/O syscalls, that is, read, write, pread, pwrite, and so on.
     Pause Time	Total time spent on profile paused state. User can pause the profile collection with profile control APIs and GUI pause after starting the profile collection.

4. The Select Trace Overlay drop-down enables selection of the type of trace data to display.
   • Thread State: Shows the current state of thread from eBPF or User-mode tracing. In the former, thread state is inferred from BPF data. In the latter, thread state is treated as Running if Running Time > 0, otherwise, Sleeping.
   • Thread Trace: Displays traces for the traced libpthread functions, such as pthread_mutex_lock, pthread_mutex_trylock, and so on.
   • Syscalls: Displays traces for traced syscall in the specific region of the timeline.
5. Trace Cutoff can be used to specify a duration in nanoseconds, which acts as a cutoff to load the trace data, that is, any traced function which takes less than the specified nanoseconds will not be displayed.
6. Click the Reset Zoom button to reset any zoom performed earlier.
7. Hover over any timeline to view the tool-tip containing the relevant data along with timestamp. If trace data is also present, the relevant traced functions with start time and duration.
8. Filter Threads/Ranks enables you to filter which thread's (or rank's) timelines must be displayed. By default, the timelines are sorted internally and the first 6 are loaded. However, from the table, you can select the required threads and clicking Apply Filter to apply the changes. If CPU profile data is collected, highlighting functions or modules is also possible. Each function is assigned a random color, which can be modified and highlighted in the timeline (implies there are samples from the function/module).
9. Each entry in the filter table has the necessary data, that is, name, parent object, and samples/trace times aggregated across the profile.
10. Click the Apply Filter button to apply a custom selection of entities or highlight entities in timeline. (If GPU acceleration is available, there is no need to click Apply as the changes are reflected instantaneously)
11. Click Deselect selected Items to deselect all the entries in the filtering table except the first one. This is useful when a custom selection is required but all timelines are already loaded.
12. At the bottom of the filtering pane, timeline legend is displayed, which helps in identifying how each type of ‘data source’ or ‘trace’ is mapped to which color.
13. The Show Core Transition button is disabled by default and works only when the CPU profiling data is collected. When enabled, a red line is displayed in each timeline to signify when a thread changes the core.
14. If any configuration is profiled with CSS enabled, select Threading Analysis > Select Data Source > CPU Profile Samples. The callstack section will be enabled only if you select a valid samples region.