Accuracy Level
Any
Enabled Analyses
Survey with GPU profiling + FLOP (Characterization) with GPU profiling
Note
Other analyses and properties control a CPU Roofline part of the report, which shows metrics for loops executed on CPU. You can add the CPU Roofline panes to the main view using the
button on the top pane. For details about CPU Roofline data, see
CPU / Memory Roofline Insights Perspective.
Result Interpretation
The farther a dot is from the topmost roofs, the more room for improvement there is. In accordance with Amdahl's Law, optimizing the loops that take the largest portion of the program's total run time will lead to greater speedups than optimizing the loops that take a smaller portion of the run time.
To read the GPU Roofline chart:
- The dots on the chart correspond to kernels running on GPU.
- By dot size and color, identify loops that take most of total program time and/or located very low in the chart. For example:
- Small, green dots take up relatively little time, so are likely not worth optimizing.
- Large, red dots take up the most time, so the best candidates for optimization are the large, red dots with a large amount of space between them and the topmost roofs.
- Diagonal lines indicate memory bandwidth limitations preventing kernels from achieving better performance without some form of optimization:
- L3 cache roof: Represents the maximal bandwidth of the L3 cache for your current graphics hardware. Measured using an optimized sequence of load operations, iterating over an array that fits entirely into L3 cache.
- SLM cache roof: Represents the maximal bandwidth of the Shared Local Memory for your current graphics hardware. Measured using an optimized sequence of load and store operations that work only with SLM.
- GTI roof: Represents the maximum bandwidth between the GPU and the rest of the SoC. This estimate is calculated via analytical formula based on the maximum frequency of your current graphics hardware.
- DRAM roof: Represents the maximal bandwidth of the DRAM memory available to your current graphics hardware. Measured using an optimized sequence of load operations, iterating over an array that does not fit in GPU caches.
- Horizontal lines indicate compute capacity limitations preventing kernels from achieving better performance without some form of optimization.
- The roofs above a dot represent the restrictions preventing it from achieving a higher performance, although the roofs below can contribute somewhat.
- A dot cannot exceed the topmost rooflines, as they represent the maximum capabilities of the hardware. However, not all kernels can use maximum hardware capabilities.
- The greater distance between a dot and the highest achievable roofline, the more opportunity exists for performance improvement.
Memory-Level GPU Roofline
By default, GPU Roofline reports data for all memory levels by default allowing you to examine each loop at different cache levels and arithmetic intensities and provides precise insights into which cache level causes the performance bottlenecks.
To configure the Memory-Level GPU Roofline chart:
- Expand the filter pane in the GPU Roofline chart toolbar.
- In the
Memory Level section, select the memory levels you want to see metrics for.
- Click Apply.
- In the GPU Roofline chart, double-click a loop to examine how the relationships between displayed memory levels and roofs. Labeled dots are displayed, representing memory levels with arithmetic intensity for the selected loop/function; lines connect the dots to indicate that they correspond to the selected loop/function.
Memory-Level GPU Roofline Data
Review the changes in the traffic from one memory level to another and compare it to respective to identify the memory hierarchy bottleneck for the kernel and determine optimization steps based on this information.
- When you double-click a loop, it is expanded to several dots and/or X marks representing different memory levels:
- CARM: Memory traffic generated by all execution units (EUs). Includes traffic between EUs and corresponding GPU cache or direct traffic to main memory. For each retired instruction with memory arguments, the size of each memory operand in bytes is added to this metric.
- L3: Data transferred directly between execution units and L3 cache.
- SLM: Memory access to/from Shared Local Memory (SLM), a dedicated structure within the L3 cache.
- GTI: Represents GTI traffic/GPU memory read bandwidth, the accesses between the GPU, chip uncore (LLC), and main memory. Use this to get a sense of external memory traffic.
- The vertical distance between memory dots and their respective roofline shows how much you are limited by a given memory subsystem. If a dot is close to its roof line, it means that the kernel is limited by the performance of this memory level.
- The horizontal distance between memory dots indicates how efficiently the loop/function uses cache. For example, if L3 and DRAM dots are very close on the horizontal axis for a single loop, the loop/function uses L3 and DRAM similarly. This means that it does not use L3 and DRAM efficiently. Improve re-usage of data in the code to improve application performance.
- Arithmetic intensity on the x axis determines the order in which dots are plotted, which can provide some insight into your code's performance. For example, the CARM dot is typically far to the right of the L3 dot, as read/write access is by cache lines and CARM traffic is the sum of actual bytes used in operations.
Kernel Details
Select a dot on the chart and switch to the Details tab in the right-side pane to examine code analytics for a specific kernel in more details.
In the Roofline, examine the Roofline chart for the selected kernel with the following data:
Amount of data transferred for each cache memory level
Pointer arrow that shows the exact roof that limits the kernel performance. The arrow points to what you should optimize the kernel for and shows the potential speedup after the optimization in the callout.
If the arrow points to a diagonal line, the kernel is mostly memory bound. If the arrow points to a horizontal line, the kernel is mostly compute bound.
For example, on the screenshot below, the kernel is bounded by the Int32 Vector Add peak. If you optimize the kernel for integer operations, the kernel gets up to 1.4x speedup.
Note
The chart is plotted for a dominant type of operations in a code (FLOAT or INT) and shows only roofs with cache memory levels, data types, and instructions mix used in the kernel. If there is no FLOP or INTOP in the kernel, the single-kernel Roofline chart is not shown.
In the OP/S and Bandwidth, review how well the kernel uses the compute and memory resources of the hardware. It indicates the total number of floating-points and integer operations transferred per second compared to the maximum hardware capability and the amount of data transferred at each cache memory (in gigabytes per second) compared to the memory level bandwidth. The red bar highlights the dominant operation type used in the kernel.
In the Memory Metrics:
Review how much time the kernel spends processing requests for each memory level in relation to the total time, in perspective, reported in the Impacts histogram.
A big value indicates a memory level that bounds the selected loop. Examine the difference between the two largest bars to see how much throughput you can gain if you reduce the impact on your main bottleneck. It also gives you a long-time plan to reduce your memory bound limitations as once you will solve the problems coming from the widest bar, your next issue will come from the second biggest bar and so on.
Ideally, you should see the L3 as the most impactful memory.
Review an amount of data that passes through each memory level reported in the Shares histogram.
Note
Data in the Memory Metrics pane is based on a dominant type of operations in a code (FLOAT or INT). The dominant type is indicated as a red bar in the OP/S and Bandwidth.In the Instruction Mix, examine types of instructions that the kernel executes. Intel Advisor automatically determines the data type used in operations and groups the instructions collected during Characterization analysis by the following categories:
Category |
Instruction Types |
|---|---|
| Compute (FLOP and INTOP) |
|
| Memory | LOAD, STORE, SLM_LOAD, SLM_STORE types depending on the argument: send, sendc, sends, sendsc |
| Other |
|
In the Performance Characteristics, review how effectively the kernel uses the GPU resources: activity of all execution units, percentage of time when both FPUs are used, percentage of cycles with a thread scheduled. Ideally, you should see a higher percentage of active execution units and other effectiveness metrics to use more GPU resources.