3. Outputting annotated source (opannotate)

The opannotate utility generates annotated source files or assembly listings, optionally mixed with source. If you want to see the source file, the profiled application needs to have debug information, and the source must be available through this debug information. For GCC, you must use the -g option when you are compiling. If the binary doesn't contain sufficient debug information, you can still use opannotate --assembly to get annotated assembly as long as the binary has (at least) symbol information.

Note that for the reason explained in Section 3.1, “Hardware performance counters” the results can be inaccurate. The debug information itself can add other problems; for example, the line number for a symbol can be incorrect. Assembly instructions can be re-ordered and moved by the compiler, and this can lead to crediting source lines with samples not really "owned" by this line. Also see Chapter 5, Interpreting profiling results.

You can output the annotation to one single file, containing all the source found using the --source. You can use this in conjunction with --assembly to get combined source/assembly output.

You can also output a directory of annotated source files that maintains the structure of the original sources. Each line in the annotated source is prepended with the samples for that line. Additionally, each symbol is annotated giving details for the symbol as a whole. An example:

$ opannotate --source --output-dir=annotated /usr/local/oprofile-pp/bin/oprofiled
$ ls annotated/home/moz/src/oprofile-pp/daemon/
opd_cookie.h  opd_image.c  opd_kernel.c  opd_sample_files.c  oprofiled.c

Line numbers are maintained in the source files, but each file has a footer appended describing the profiling details. The actual annotation looks something like this :

...
               :static uint64_t pop_buffer_value(struct transient * trans)
 11510  1.9661 :{ /* pop_buffer_value total:  89901 15.3566 */
               :        uint64_t val;
               :
 10227  1.7469 :        if (!trans->remaining) {
               :                fprintf(stderr, "BUG: popping empty buffer !\n");
               :                exit(EXIT_FAILURE);
               :        }
               :
               :        val = get_buffer_value(trans->buffer, 0);
  2281  0.3896 :        trans->remaining--;
  2296  0.3922 :        trans->buffer += kernel_pointer_size;
               :        return val;
 10454  1.7857 :}
...

The first number on each line is the number of samples, whilst the second is the relative percentage of total samples.

3.1. Locating source files

Of course, opannotate needs to be able to locate the source files for the binary image(s) in order to produce output. Some binary images have debug information where the given source file paths are relative, not absolute. You can specify search paths to look for these files (similar to gdb's dir command) with the --search-dirs option.

Sometimes you may have a binary image which gives absolute paths for the source files, but you have the actual sources elsewhere (commonly, you've installed an SRPM for a binary on your system and you want annotation from an existing profile). You can use the --base-dirs option to redirect OProfile to look somewhere else for source files. For example, imagine we have a binary generated from a source file that is given in the debug information as /tmp/build/libfoo/foo.c, and you have the source tree matching that binary installed in /home/user/libfoo/. You can redirect OProfile to find foo.c correctly like this :

$ opannotate --source --base-dirs=/tmp/build/libfoo/ --search-dirs=/home/user/libfoo/ --output-dir=annotated/ /lib/libfoo.so

You can specify multiple (comma-separated) paths to both options.

3.2. Usage of opannotate

--assembly / -a

Output annotated assembly. If this is combined with --source, then mixed source / assembly annotations are output.

--base-dirs / -b [paths]/

Comma-separated list of path prefixes. This can be used to point OProfile to a different location for source files when the debug information specifies an absolute path on your system for the source that does not exist. The prefix is stripped from the debug source file paths, then searched in the search dirs specified by --search-dirs.

--demangle / -D none|normal|smart

none: no demangling. normal: use default demangler (default) smart: use pattern-matching to make C++ symbol demangling more readable.

--exclude-dependent / -x

Do not include application-specific images for libraries, kernel modules and the kernel.

--exclude-file [files]

Exclude all files in the given comma-separated list of glob patterns. This option is supported solely with the --source option. It can be used to filter out source files in the output using the following types of specifications:

  • filenames (basename -- i.e., no path)
  • filename glob specifications (all files whose base filename matches the given pattern)
  • directory segments (all source files located in the specified directory; e.g. "libio")
  • directory segment glob specifications (e.g., "libi*")

--exclude-symbols / -e [symbols]

Exclude all the symbols in the given comma-separated list.

--help / -? / --usage

Show help message.

--image-path / -p [paths]

Comma-separated list of additional paths to search for binaries. This is needed to find kernel modules.

--root / -R [path]

A path to a filesystem to search for additional binaries.

--include-file [files]

Only include files in the given comma-separated list of glob patterns. The same rules apply for this option as for the --exclude-file option.

--include-symbols / -i [symbols]

Only include symbols in the given comma-separated list.

--objdump-params [params]

Pass the given parameters as extra values when calling objdump. If more than one option is to be passed to objdump, the parameters must be enclosed in a quoted string.

An example of where this option is useful is when your toolchain does not automatically recognize instructions that are specific to your processor. For example, on IBM POWER7/RHEL 6, objdump must be told that a binary file may have POWER7-specific instructions. The opannotate option to show the POWER7-specific instructions is:

   --objdump-params=-Mpower7

The opannotate option to show the POWER7-specific instructions, the source code (--source) and the line numbers (-l) would be:

   --objdump-params="-Mpower7 -l --source"

--output-dir / -o [dir]

Output directory. This makes opannotate output one annotated file for each source file. This option can't be used in conjunction with --assembly.

--search-dirs / -d [paths]

Comma-separated list of paths to search for source files. This is useful to find source files when the debug information only contains relative paths.

--source / -s

Output annotated source. This requires debugging information to be available for the binaries.

--session-dir=dir_path

Use sample database from the specified directory dir_path instead of the default location. If this option is not specified, then opannotate will search for samples in <cur_dir>/oprofile_data first. If that directory does not exist, the standard session-dir of /var/lib/oprofile is used as the session directory.

--threshold / -t [percentage]

For annotated assembly, only output data for symbols that have more than the given percentage of total samples. For profiles using multiple events, if the threshold is reached for any event, then all sample data for the symbol is shown.

For annotated source, only output data for source files that have more than the given percentage of total samples. For profiles using multiple events, if the threshold is reached for any event, then all sample data for the source file is shown.

--verbose / -V [options]

Give verbose debugging output.

--version / -v

Show version.