NVIDIA CUDA Visual Profiler Version 2.2

Published by
NVIDIA Corporation
2701 San Tomas Expressway
Santa Clara, CA 95050


Notice

BY DOWNLOADING THIS FILE, USER AGREES TO THE FOLLOWING:

ALL NVIDIA SOFTWARE, DESIGN SPECIFICATIONS, REFERENCE BOARDS, FILES, DRAWINGS, DIAGNOSTICS, LISTS, AND OTHER DOCUMENTS (TOGETHER AND SEPARATELY, "MATERIALS") ARE BEING PROVIDED "AS IS". NVIDIA MAKES NO WARRANTIES, EXPRESSED, IMPLIED, STATUTORY, OR OTHERWISE WITH RESPECT TO THE MATERIALS, AND EXPRESSLY DISCLAIMS ALL IMPLIED WARRANTIES OF NONINFRINGEMENT, MERCHANTABILITY, AND FITNESS FOR A PARTICULAR PURPOSE.

Information furnished is believed to be accurate and reliable. However, NVIDIA Corporation assumes no responsibility for the consequences of use of such information or for any infringement of patents or other rights of third parties that may result from its use. No license is granted by implication or otherwise under any patent or patent rights of NVIDIA Corporation. Specifications mentioned in this publication are subject to change without notice. These materials supersedes and replaces all information previously supplied. NVIDIA Corporation products are not authorized for use as critical components in life support devices or systems without express written approval of NVIDIA Corporation.

Trademarks
NVIDIA, CUDA, and the NVIDIA logo are trademarks or registered trademarks of NVIDIA Corporation in the United States and other countries. Other company and product names may be trademarks of the respective companies with which they are associated.

Copyright (C) 2007-2009 by NVIDIA Corporation. All rights reserved.

PLEASE REFER EULA.txt FOR THE LICENSE AGREEMENT FOR USING NVIDIA SOFTWARE.

List of supported features:

Execute a CUDA program with profiling enabled and view the profiler output as a table. The table has the following columns for each GPU method:

Please refer the "Interpreting Profiler Counters" section below for more information on profiler counters. Note that profiler counters are also referred to as profiler signals.

Display the summary profiler table. It has the following columns for each GPU method:
Display various kinds of plots:
Analysis of profiler output lists out method with high number of:
Compare profiler output for multiple program runs of the same program or for different programs.

Each program run is referred to as a session.

Save profiling data for multiple sessions. A group of sessions is referred to as a project.

Import/Export CUDA Profiler CSV format data.

Description of different plots:

Summary profiling data bar plot :
GPU time height plot:
It is a bar diagram in which the height of each bar is proportional to the GPU time for a method and a different bar color is assigned for each method. A legend is displayed which shows the color assignment for different methods. The width of each bar is fixed and the bars are displayed in the order in which the methods are executed.When the "fit in window" option is enabled the display is adjusted so as to fit all the bars in the displayed window width. In this case bars for multiple methods can overlap. The overlapped bars are displayed in decreasing order of height so that all the different bars are visible. When the "Show CPU Time" option is enabled the CPU time is shown as a bar in a different color on top of the GPU time bar. The height of this bar is proportional to the difference of CPU time and GPU time for the method.
GPU time width plot:
It is a bar diagram in which the width of each bar is proportional to the GPU time for a method and a different bar color is assigned for each method. A legend is displayed which shows the color assignment for different methods. The bars are displayed in the order in which the methods are executed. When time stamps are enabled the bars are positioned based on the time stamp. The height of each bar is based on the option chosen:
  1. Fixed height : height is fixed.
  2. Height proportional to instruction issue rate: the instruction issue rate for a method is equal to profiler "instructions" counter value divided by the gpu time for the method.
  3. Height proportional to incoherent load + store rate: the incoherent load + store rate for a method is equal to the sum of profiler "gld_incoherent" and "gst_incoherent" counter values divided by the gpu time for the method.
  4. Occupancy: Occupancy is proportional to height.
In case of multiple streams or multiple devices the "Split Options" can be used.
  1. No Split : A single horizontal group of bars is displayed. Even in case of multiple streams or multiple devices the data is displayed in a single group.
  2. Split on Device: In case of multiple devices one seperate horizontal group of bars is displayed for each device.
  3. Split on Stream: In case of multiple devices one seperate horizontal group of bars is displayed for each stream.
Profiler counter bar plot :
It is a bar plot for profiler counter values for a method from the profiler output table or the summary table. . One bar for each profiler counter. Bars sorted in decreasing profiler counter value .Bar length is proportional to profiler counter value.
Profiler output table column bar plot:
It is a bar plot for any column of values from the profiler output table or summary table . One bar for each row inthe table. Bars sorted in decreasing column value . Bar length is proportional to column value.
Comparison summary plot:
This plot can be used to compare GPU Time summary data for two sessions. The Base Session is the session with respect to which comparison is done and the other session which is selected for comparison is called Compare Session. GPU Times for matching kernels from the two sessions are shown in a group. For each matched kernel from Compare Session, percentage increment or decrement with respect to Base Session is displayed at the right end of the bar. After showing all the matched pairs, the unmatched kernels GPU Times are shown. At the bottom two bars with total GPU Times for the two sessions are shown.

Steps for sample cudaprof usage:


Sample1:



Sample2:

Brief description of some cudaprof GUI components:

Top line shows the main menu options: File, Profile, Session, Options, Window and Help. See the description below for details on the menu options.

Second line has 4 groups of tool bar icons.

Left vertical window lists all the sessions and profiler data for each device in a session as a child of the session in the current project. The child of a session is named as "Device_< device_number >" e.g Device_0. Right clicking on a session and its child brings up the context sensitive menus. See the description below for details on the menu options.

Session context menu.

Session->Device context menu.
Right workspace area contains windows which include Tabbed window for each session and each device in a session.
The only window added to the Tab for each session is:
The different windows for each device in a session are shown as different tabs:
Output window - Appears, when asked to display, at the bottom. It displays standard output & standard error for the CUDA program which is run. Also some additional status messages are displayed in this window.

Main menu

Tool bars

Dialogs

Session list context menu :

Session->Device context menu :

Profiler table context menu :

cudaprof project files saved to disk

cudaprof settings which are saved

Following is the list of cudaprof settings which are saved and remembered across different cudaprof sessions. On Windows these settings are saved in the system registry at the location "HKEY_CURRENT_USER\Software\NVIDIA\cudaprof".
On Linux these settings are saved to the file "$HOME/.config/NVIDIA/cudaprof.conf".

Interpreting profiler counters

The performance counter values do not correspond to individual thread activity. Instead, these values represent events within a thread warp. For example, a divergent branch within a thread warp will increment the divergent_branch counter by one. So the final counter value stores information for all divergent branches in all warps. In addition, the profiler can only target one of the multiprocessors in the GPU,so the counter values will not correspond to the total number of warps launched for a particular kernel. For this reason, when using the performance counter options in the profiler the user should always launch enough threads blocks to ensure that the target multiprocessor is given a consistent percentage of the total work. In practice, it is best to launch at least around 100 blocks for consistent results. For the reasons listed above, users should not expect the counter values to match the numbers one would get by inspecting kernel code. The values are best used to identify relative performance differences between unoptimized and optimized code. For example, if for the initial version of the program the profiler reports N non-coalesced global loads, it is easy to see if the optimized code produces less than N non-coalesced loads. In most cases, the goal is to make N go to 0, so the counter value is useful for tracking progress toward this goal.