1 <?xml version="1.0" encoding="ISO-8859-1"?>
2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
3 <html xmlns="http://www.w3.org/1999/xhtml">
5 <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1" />
6 <title>OProfile Internals</title>
7 <meta name="generator" content="DocBook XSL Stylesheets V1.75.2" />
10 <div class="book" title="OProfile Internals">
11 <div class="titlepage">
14 <h1 class="title"><a id="oprofile-internals"></a>OProfile Internals</h1>
17 <div class="authorgroup">
19 <h3 class="author"><span class="firstname">John</span> <span class="surname">Levon</span></h3>
20 <div class="affiliation">
23 <code class="email"><<a class="email" href="mailto:levon@movementarian.org">levon@movementarian.org</a>></code>
31 <p class="copyright">Copyright © 2003 John Levon</p>
38 <b>Table of Contents</b>
42 <span class="chapter">
43 <a href="#introduction">1. Introduction</a>
50 <a href="#overview">1. Overview</a>
55 <a href="#components">2. Components of the OProfile system</a>
62 <a href="#arch-specific-components">2.1. Architecture-specific components</a>
67 <a href="#filesystem">2.2. oprofilefs</a>
72 <a href="#driver">2.3. Generic kernel driver</a>
77 <a href="#daemon">2.4. The OProfile daemon</a>
82 <a href="#post-profiling">2.5. Post-profiling tools</a>
90 <span class="chapter">
91 <a href="#performance-counters">2. Performance counter management</a>
98 <a href="#performance-counters-ui">1. Providing a user interface</a>
103 <a href="#performance-counters-programming">2. Programming the performance counter registers</a>
110 <a href="#performance-counters-start">2.1. Starting and stopping the counters</a>
115 <a href="#idp4832768">2.2. IA64 and perfmon</a>
123 <span class="chapter">
124 <a href="#collecting-samples">3. Collecting and processing samples</a>
131 <a href="#receiving-interrupts">1. Receiving interrupts</a>
136 <a href="#core-structure">2. Core data structures</a>
141 <a href="#logging-sample">3. Logging a sample</a>
146 <a href="#logging-stack">4. Logging stack traces</a>
151 <a href="#synchronising-buffers">5. Synchronising the CPU buffers to the event buffer</a>
156 <a href="#dentry-cookies">6. Identifying binary images</a>
161 <a href="#finding-dentry">7. Finding a sample's binary image and offset</a>
167 <span class="chapter">
168 <a href="#sample-files">4. Generating sample files</a>
175 <a href="#processing-buffer">1. Processing the buffer</a>
182 <a href="#handling-kernel-samples">1.1. Handling kernel samples</a>
189 <a href="#sample-file-generation">2. Locating and creating sample files</a>
194 <a href="#sample-file-writing">3. Writing data to a sample file</a>
200 <span class="chapter">
201 <a href="#output">5. Generating useful output</a>
208 <a href="#profile-specification">1. Handling the profile specification</a>
213 <a href="#sample-file-collating">2. Collating the candidate sample files</a>
220 <a href="#sample-file-classifying">2.1. Classifying sample files</a>
225 <a href="#sample-file-inverting">2.2. Creating inverted profile lists</a>
232 <a href="#generating-profile-data">3. Generating profile data</a>
239 <a href="#bfd">3.1. Processing the binary image</a>
244 <a href="#processing-sample-files">3.2. Processing the sample files</a>
251 <a href="#generating-output">4. Generating output</a>
257 <span class="chapter">
258 <a href="#ext">6. Extended Feature Interface</a>
265 <a href="#ext-intro">1. Introduction</a>
270 <a href="#ext-name-and-handlers">2. Feature Name and Handlers</a>
275 <a href="#ext-enable">3. Enabling Features</a>
280 <a href="#ext-types-of-handlers">4. Type of Handlers</a>
287 <a href="#ext_init">4.1. ext_init Handler</a>
292 <a href="#ext_print_stats">4.2. ext_print_stats Handler</a>
297 <a href="#ext_sfile_handlers">4.3. ext_sfile Handler</a>
304 <a href="#ext-implementation">5. Extended Feature Reference Implementation</a>
311 <a href="#ext-ibs">5.1. Instruction-Based Sampling (IBS)</a>
319 <span class="glossary">
320 <a href="#glossary">Glossary of OProfile source concepts and types</a>
325 <div class="list-of-figures">
327 <b>List of Figures</b>
330 <dt>3.1. <a href="#idp4848096">The OProfile buffers</a></dt>
333 <div class="chapter" title="Chapter 1. Introduction">
334 <div class="titlepage">
337 <h2 class="title"><a id="introduction"></a>Chapter 1. Introduction</h2>
343 <b>Table of Contents</b>
348 <a href="#overview">1. Overview</a>
353 <a href="#components">2. Components of the OProfile system</a>
360 <a href="#arch-specific-components">2.1. Architecture-specific components</a>
365 <a href="#filesystem">2.2. oprofilefs</a>
370 <a href="#driver">2.3. Generic kernel driver</a>
375 <a href="#daemon">2.4. The OProfile daemon</a>
380 <a href="#post-profiling">2.5. Post-profiling tools</a>
388 This document is current for OProfile version 1.0.0.
389 This document provides some details on the internal workings of OProfile for the
390 interested hacker. This document assumes strong C, working C++, plus some knowledge of
391 kernel internals and CPU hardware.
393 <div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;">
394 <h3 class="title">Note</h3>
396 Only the "new" implementation associated with kernel 2.6 and above is covered here. 2.4
397 uses a very different kernel module implementation and daemon to produce the sample files.
400 <div class="sect1" title="1. Overview">
401 <div class="titlepage">
404 <h2 class="title" style="clear: both"><a id="overview"></a>1. Overview</h2>
409 OProfile is a statistical continuous profiler. In other words, profiles are generated by
410 regularly sampling the current registers on each CPU (from an interrupt handler, the
411 saved PC value at the time of interrupt is stored), and converting that runtime PC
412 value into something meaningful to the programmer.
415 OProfile achieves this by taking the stream of sampled PC values, along with the detail
416 of which task was running at the time of the interrupt, and converting into a file offset
417 against a particular binary file. Because applications <code class="function">mmap()</code>
418 the code they run (be it <code class="filename">/bin/bash</code>, <code class="filename">/lib/libfoo.so</code>
419 or whatever), it's possible to find the relevant binary file and offset by walking
420 the task's list of mapped memory areas. Each PC value is thus converted into a tuple
421 of binary-image,offset. This is something that the userspace tools can use directly
422 to reconstruct where the code came from, including the particular assembly instructions,
423 symbol, and source line (via the binary's debug information if present).
426 Regularly sampling the PC value like this approximates what actually was executed and
427 how often - more often than not, this statistical approximation is good enough to
428 reflect reality. In common operation, the time between each sample interrupt is regulated
429 by a fixed number of clock cycles. This implies that the results will reflect where
430 the CPU is spending the most time; this is obviously a very useful information source
431 for performance analysis.
434 Sometimes though, an application programmer needs different kinds of information: for example,
435 "which of the source routines cause the most cache misses ?". The rise in importance of
436 such metrics in recent years has led many CPU manufacturers to provide hardware performance
437 counters capable of measuring these events on the hardware level. Typically, these counters
438 increment once per each event, and generate an interrupt on reaching some pre-defined
439 number of events. OProfile can use these interrupts to generate samples: then, the
440 profile results are a statistical approximation of which code caused how many of the
444 Consider a simplified system that only executes two functions A and B. A
445 takes one cycle to execute, whereas B takes 99 cycles. Imagine we run at
446 100 cycles a second, and we've set the performance counter to create an
447 interrupt after a set number of "events" (in this case an event is one
448 clock cycle). It should be clear that the chances of the interrupt
449 occurring in function A is 1/100, and 99/100 for function B. Thus, we
450 statistically approximate the actual relative performance features of
451 the two functions over time. This same analysis works for other types of
452 events, providing that the interrupt is tied to the number of events
453 occurring (that is, after N events, an interrupt is generated).
456 There are typically more than one of these counters, so it's possible to set up profiling
457 for several different event types. Using these counters gives us a powerful, low-overhead
458 way of gaining performance metrics. If OProfile, or the CPU, does not support performance
459 counters, then a simpler method is used: the kernel timer interrupt feeds samples
460 into OProfile itself.
463 The rest of this document concerns itself with how we get from receiving samples at
464 interrupt time to producing user-readable profile information.
467 <div class="sect1" title="2. Components of the OProfile system">
468 <div class="titlepage">
471 <h2 class="title" style="clear: both"><a id="components"></a>2. Components of the OProfile system</h2>
475 <div class="sect2" title="2.1. Architecture-specific components">
476 <div class="titlepage">
479 <h3 class="title"><a id="arch-specific-components"></a>2.1. Architecture-specific components</h3>
484 If OProfile supports the hardware performance counters found on
485 a particular architecture, code for managing the details of setting
486 up and managing these counters can be found in the kernel source
487 tree in the relevant <code class="filename">arch/<span class="emphasis"><em>arch</em></span>/oprofile/</code>
488 directory. The architecture-specific implementation works via
489 filling in the oprofile_operations structure at init time. This
490 provides a set of operations such as <code class="function">setup()</code>,
491 <code class="function">start()</code>, <code class="function">stop()</code>, etc.
492 that manage the hardware-specific details of fiddling with the
493 performance counter registers.
496 The other important facility available to the architecture code is
497 <code class="function">oprofile_add_sample()</code>. This is where a particular sample
498 taken at interrupt time is fed into the generic OProfile driver code.
501 <div class="sect2" title="2.2. oprofilefs">
502 <div class="titlepage">
505 <h3 class="title"><a id="filesystem"></a>2.2. oprofilefs</h3>
510 OProfile implements a pseudo-filesystem known as "oprofilefs", mounted from
511 userspace at <code class="filename">/dev/oprofile</code>. This consists of small
512 files for reporting and receiving configuration from userspace, as well
513 as the actual character device that the OProfile userspace receives samples
514 from. At <code class="function">setup()</code> time, the architecture-specific may
515 add further configuration files related to the details of the performance
516 counters. For example, on x86, one numbered directory for each hardware
517 performance counter is added, with files in each for the event type,
521 The filesystem also contains a <code class="filename">stats</code> directory with
522 a number of useful counters for various OProfile events.
525 <div class="sect2" title="2.3. Generic kernel driver">
526 <div class="titlepage">
529 <h3 class="title"><a id="driver"></a>2.3. Generic kernel driver</h3>
534 This lives in <code class="filename">drivers/oprofile/</code>, and forms the core of
535 how OProfile works in the kernel. Its job is to take samples delivered
536 from the architecture-specific code (via <code class="function">oprofile_add_sample()</code>),
537 and buffer this data, in a transformed form as described later, until releasing
538 the data to the userspace daemon via the <code class="filename">/dev/oprofile/buffer</code>
542 <div class="sect2" title="2.4. The OProfile daemon">
543 <div class="titlepage">
546 <h3 class="title"><a id="daemon"></a>2.4. The OProfile daemon</h3>
551 The OProfile userspace daemon's job is to take the raw data provided by the
552 kernel and write it to the disk. It takes the single data stream from the
553 kernel and logs sample data against a number of sample files (found in
554 <code class="filename">$SESSION_DIR/samples/current/</code>, by default located at
555 <code class="filename">/var/lib/oprofile/samples/current/</code>. For the benefit
556 of the "separate" functionality, the names/paths of these sample files
557 are mangled to reflect where the samples were from: this can include
558 thread IDs, the binary file path, the event type used, and more.
561 After this final step from interrupt to disk file, the data is now
562 persistent (that is, changes in the running of the system do not invalidate
563 stored data). So the post-profiling tools can run on this data at any
564 time (assuming the original binary files are still available and unchanged,
568 <div class="sect2" title="2.5. Post-profiling tools"><div class="titlepage"><div><div><h3 class="title"><a id="post-profiling"></a>2.5. Post-profiling tools</h3></div></div></div>
569 So far, we've collected data, but we've yet to present it in a useful form
570 to the user. This is the job of the post-profiling tools. In general form,
571 they collate a subset of the available sample files, load and process each one
572 correlated against the relevant binary file, and finally produce user-readable
577 <div class="chapter" title="Chapter 2. Performance counter management">
578 <div class="titlepage">
581 <h2 class="title"><a id="performance-counters"></a>Chapter 2. Performance counter management</h2>
587 <b>Table of Contents</b>
592 <a href="#performance-counters-ui">1. Providing a user interface</a>
597 <a href="#performance-counters-programming">2. Programming the performance counter registers</a>
604 <a href="#performance-counters-start">2.1. Starting and stopping the counters</a>
609 <a href="#idp4832768">2.2. IA64 and perfmon</a>
616 <div class="sect1" title="1. Providing a user interface">
617 <div class="titlepage">
620 <h2 class="title" style="clear: both"><a id="performance-counters-ui"></a>1. Providing a user interface</h2>
625 The performance counter registers need programming in order to set the
626 type of event to count, etc. OProfile uses a standard model across all
627 CPUs for defining these events as follows :
629 <div class="informaltable">
638 <code class="option">event</code>
640 <td>The event type e.g. DATA_MEM_REFS</td>
644 <code class="option">unit mask</code>
646 <td>The sub-events to count (more detailed specification)</td>
650 <code class="option">counter</code>
652 <td>The hardware counter(s) that can count this event</td>
656 <code class="option">count</code>
658 <td>The reset value (how many events before an interrupt)</td>
662 <code class="option">kernel</code>
664 <td>Whether the counter should increment when in kernel space</td>
668 <code class="option">user</code>
670 <td>Whether the counter should increment when in user space</td>
676 The term "unit mask" is borrowed from the Intel architectures, and can
677 further specify exactly when a counter is incremented (for example,
678 cache-related events can be restricted to particular state transitions
682 All of the available hardware events and their details are specified in
683 the textual files in the <code class="filename">events</code> directory. The
684 syntax of these files should be fairly obvious. The user specifies the
685 names and configuration details of the chosen counters via
686 <span class="command"><strong>opcontrol</strong></span>. These are then written to the kernel
687 module (in numerical form) via <code class="filename">/dev/oprofile/N/</code>
688 where N is the physical hardware counter (some events can only be used
689 on specific counters; OProfile hides these details from the user when
690 possible). On IA64, the perfmon-based interface behaves somewhat
691 differently, as described later.
694 <div class="sect1" title="2. Programming the performance counter registers">
695 <div class="titlepage">
698 <h2 class="title" style="clear: both"><a id="performance-counters-programming"></a>2. Programming the performance counter registers</h2>
703 We have described how the user interface fills in the desired
704 configuration of the counters and transmits the information to the
705 kernel. It is the job of the <code class="function">->setup()</code> method
706 to actually program the performance counter registers. Clearly, the
707 details of how this is done is architecture-specific; it is also
708 model-specific on many architectures. For example, i386 provides methods
709 for each model type that programs the counter registers correctly
710 (see the <code class="filename">op_model_*</code> files in
711 <code class="filename">arch/i386/oprofile</code> for the details). The method
712 reads the values stored in the virtual oprofilefs files and programs
713 the registers appropriately, ready for starting the actual profiling
717 The architecture-specific drivers make sure to save the old register
718 settings before doing OProfile setup. They are restored when OProfile
719 shuts down. This is useful, for example, on i386, where the NMI watchdog
720 uses the same performance counter registers as OProfile; they cannot
721 run concurrently, but OProfile makes sure to restore the setup it found
722 before it was running.
725 In addition to programming the counter registers themselves, other setup
726 is often necessary. For example, on i386, the local APIC needs
727 programming in order to make the counter's overflow interrupt appear as
728 an NMI (non-maskable interrupt). This allows sampling (and therefore
729 profiling) of regions where "normal" interrupts are masked, enabling
730 more reliable profiles.
732 <div class="sect2" title="2.1. Starting and stopping the counters">
733 <div class="titlepage">
736 <h3 class="title"><a id="performance-counters-start"></a>2.1. Starting and stopping the counters</h3>
741 Initiating a profiling session is done via writing an ASCII '1'
742 to the file <code class="filename">/dev/oprofile/enable</code>. This sets up the
743 core, and calls into the architecture-specific driver to actually
744 enable each configured counter. Again, the details of how this is
745 done is model-specific (for example, the Athlon models can disable
746 or enable on a per-counter basis, unlike the PPro models).
749 <div class="sect2" title="2.2. IA64 and perfmon">
750 <div class="titlepage">
753 <h3 class="title"><a id="idp4832768"></a>2.2. IA64 and perfmon</h3>
758 The IA64 architecture provides a different interface from the other
759 architectures, using the existing perfmon driver. Register programming
760 is handled entirely in user-space (see
761 <code class="filename">daemon/opd_perfmon.c</code> for the details). A process
762 is forked for each CPU, which creates a perfmon context and sets the
763 counter registers appropriately via the
764 <code class="function">sys_perfmonctl</code> interface. In addition, the actual
765 initiation and termination of the profiling session is handled via the
766 same interface using <code class="constant">PFM_START</code> and
767 <code class="constant">PFM_STOP</code>. On IA64, then, there are no oprofilefs
768 files for the performance counters, as the kernel driver does not
769 program the registers itself.
772 Instead, the perfmon driver for OProfile simply registers with the
773 OProfile core with an OProfile-specific UUID. During a profiling
774 session, the perfmon core calls into the OProfile perfmon driver and
775 samples are registered with the OProfile core itself as usual (with
776 <code class="function">oprofile_add_sample()</code>).
781 <div class="chapter" title="Chapter 3. Collecting and processing samples">
782 <div class="titlepage">
785 <h2 class="title"><a id="collecting-samples"></a>Chapter 3. Collecting and processing samples</h2>
791 <b>Table of Contents</b>
796 <a href="#receiving-interrupts">1. Receiving interrupts</a>
801 <a href="#core-structure">2. Core data structures</a>
806 <a href="#logging-sample">3. Logging a sample</a>
811 <a href="#logging-stack">4. Logging stack traces</a>
816 <a href="#synchronising-buffers">5. Synchronising the CPU buffers to the event buffer</a>
821 <a href="#dentry-cookies">6. Identifying binary images</a>
826 <a href="#finding-dentry">7. Finding a sample's binary image and offset</a>
831 <div class="sect1" title="1. Receiving interrupts">
832 <div class="titlepage">
835 <h2 class="title" style="clear: both"><a id="receiving-interrupts"></a>1. Receiving interrupts</h2>
840 Naturally, how the overflow interrupts are received is specific
841 to the hardware architecture, unless we are in "timer" mode, where the
842 logging routine is called directly from the standard kernel timer
846 On the i386 architecture, the local APIC is programmed such that when a
847 counter overflows (that is, it receives an event that causes an integer
848 overflow of the register value to zero), an NMI is generated. This calls
849 into the general handler <code class="function">do_nmi()</code>; because OProfile
850 has registered itself as capable of handling NMI interrupts, this will
851 call into the OProfile driver code in
852 <code class="filename">arch/i386/oprofile</code>. Here, the saved PC value (the
853 CPU saves the register set at the time of interrupt on the stack
854 available for inspection) is extracted, and the counters are examined to
855 find out which one generated the interrupt. Also determined is whether
856 the system was inside kernel or user space at the time of the interrupt.
857 These three pieces of information are then forwarded onto the OProfile
858 core via <code class="function">oprofile_add_sample()</code>. Finally, the
859 counter values are reset to the chosen count value, to ensure another
860 interrupt happens after another N events have occurred. Other
861 architectures behave in a similar manner.
864 <div class="sect1" title="2. Core data structures">
865 <div class="titlepage">
868 <h2 class="title" style="clear: both"><a id="core-structure"></a>2. Core data structures</h2>
873 Before considering what happens when we log a sample, we shall digress
874 for a moment and look at the general structure of the data collection
878 OProfile maintains a small buffer for storing the logged samples for
879 each CPU on the system. Only this buffer is altered when we actually log
880 a sample (remember, we may still be in an NMI context, so no locking is
881 possible). The buffer is managed by a two-handed system; the "head"
882 iterator dictates where the next sample data should be placed in the
883 buffer. Of course, overflow of the buffer is possible, in which case
884 the sample is discarded.
887 It is critical to remember that at this point, the PC value is an
888 absolute value, and is therefore only meaningful in the context of which
889 task it was logged against. Thus, these per-CPU buffers also maintain
890 details of which task each logged sample is for, as described in the
891 next section. In addition, we store whether the sample was in kernel
892 space or user space (on some architectures and configurations, the address
893 space is not sub-divided neatly at a specific PC value, so we must store
897 As well as these small per-CPU buffers, we have a considerably larger
898 single buffer. This holds the data that is eventually copied out into
899 the OProfile daemon. On certain system events, the per-CPU buffers are
900 processed and entered (in mutated form) into the main buffer, known in
901 the source as the "event buffer". The "tail" iterator indicates the
902 point from which the CPU may be read, up to the position of the "head"
903 iterator. This provides an entirely lock-free method for extracting data
904 from the CPU buffers. This process is described in detail later in this chapter.
907 <a id="idp4848096"></a>
909 <b>Figure 3.1. The OProfile buffers</b>
911 <div class="figure-contents">
913 <img src="buffers.png" alt="The OProfile buffers" />
917 <br class="figure-break" />
919 <div class="sect1" title="3. Logging a sample">
920 <div class="titlepage">
923 <h2 class="title" style="clear: both"><a id="logging-sample"></a>3. Logging a sample</h2>
928 As mentioned, the sample is logged into the buffer specific to the
929 current CPU. The CPU buffer is a simple array of pairs of unsigned long
930 values; for a sample, they hold the PC value and the counter for the
931 sample. (The counter value is later used to translate back into the relevant
932 event type the counter was programmed to).
935 In addition to logging the sample itself, we also log task switches.
936 This is simply done by storing the address of the last task to log a
937 sample on that CPU in a data structure, and writing a task switch entry
938 into the buffer if the new value of <code class="function">current()</code> has
939 changed. Note that later we will directly de-reference this pointer;
940 this imposes certain restrictions on when and how the CPU buffers need
944 Finally, as mentioned, we log whether we have changed between kernel and
945 userspace using a similar method. Both of these variables
946 (<code class="varname">last_task</code> and <code class="varname">last_is_kernel</code>) are
947 reset when the CPU buffer is read.
950 <div class="sect1" title="4. Logging stack traces">
951 <div class="titlepage">
954 <h2 class="title" style="clear: both"><a id="logging-stack"></a>4. Logging stack traces</h2>
959 OProfile can also provide statistical samples of call chains (on x86). To
960 do this, at sample time, the frame pointer chain is traversed, recording
961 the return address for each stack frame. This will only work if the code
962 was compiled with frame pointers, but we're careful to abort the
963 traversal if the frame pointer appears bad. We store the set of return
964 addresses straight into the CPU buffer. Note that, since this traversal
965 is keyed off the standard sample interrupt, the number of times a
966 function appears in a stack trace is not an indicator of how many times
967 the call site was executed: rather, it's related to the number of
968 samples we took where that call site was involved. Thus, the results for
969 stack traces are not necessarily proportional to the call counts:
970 typical programs will have many <code class="function">main()</code> samples.
973 <div class="sect1" title="5. Synchronising the CPU buffers to the event buffer">
974 <div class="titlepage">
977 <h2 class="title" style="clear: both"><a id="synchronising-buffers"></a>5. Synchronising the CPU buffers to the event buffer</h2>
982 At some point, we have to process the data in each CPU buffer and enter
983 it into the main (event) buffer. The file
984 <code class="filename">buffer_sync.c</code> contains the relevant code. We
985 periodically (currently every <code class="constant">HZ</code>/4 jiffies) start
986 the synchronisation process. In addition, we process the buffers on
987 certain events, such as an application calling
988 <code class="function">munmap()</code>. This is particularly important for
989 <code class="function">exit()</code> - because the CPU buffers contain pointers
990 to the task structure, if we don't process all the buffers before the
991 task is actually destroyed and the task structure freed, then we could
992 end up trying to dereference a bogus pointer in one of the CPU buffers.
995 We also add a notification when a kernel module is loaded; this is so
996 that user-space can re-read <code class="filename">/proc/modules</code> to
997 determine the load addresses of kernel module text sections. Without
998 this notification, samples for a newly-loaded module could get lost or
999 be attributed to the wrong module.
1002 The synchronisation itself works in the following manner: first, mutual
1003 exclusion on the event buffer is taken. Remember, we do not need to do
1004 that for each CPU buffer, as we only read from the tail iterator (whilst
1005 interrupts might be arriving at the same buffer, but they will write to
1006 the position of the head iterator, leaving previously written entries
1007 intact). Then, we process each CPU buffer in turn. A CPU switch
1008 notification is added to the buffer first (for
1009 <code class="option">--separate=cpu</code> support). Then the processing of the
1013 As mentioned, the CPU buffer consists of task switch entries and the
1014 actual samples. When the routine <code class="function">sync_buffer()</code> sees
1015 a task switch, the process ID and process group ID are recorded into the
1016 event buffer, along with a dcookie (see below) identifying the
1017 application binary (e.g. <code class="filename">/bin/bash</code>). The
1018 <code class="varname">mmap_sem</code> for the task is then taken, to allow safe
1019 iteration across the tasks' list of mapped areas. Each sample is then
1020 processed as described in the next section.
1023 After a buffer has been read, the tail iterator is updated to reflect
1024 how much of the buffer was processed. Note that when we determined how
1025 much data there was to read in the CPU buffer, we also called
1026 <code class="function">cpu_buffer_reset()</code> to reset
1027 <code class="varname">last_task</code> and <code class="varname">last_is_kernel</code>, as
1028 we've already mentioned. During the processing, more samples may have
1029 been arriving in the CPU buffer; this is OK because we are careful to
1030 only update the tail iterator to how much we actually read - on the next
1031 buffer synchronisation, we will start again from that point.
1034 <div class="sect1" title="6. Identifying binary images">
1035 <div class="titlepage">
1038 <h2 class="title" style="clear: both"><a id="dentry-cookies"></a>6. Identifying binary images</h2>
1043 In order to produce useful profiles, we need to be able to associate a
1044 particular PC value sample with an actual ELF binary on the disk. This
1045 leaves us with the problem of how to export this information to
1046 user-space. We create unique IDs that identify a particular directory
1047 entry (dentry), and write those IDs into the event buffer. Later on,
1048 the user-space daemon can call the <code class="function">lookup_dcookie</code>
1049 system call, which looks up the ID and fills in the full path of
1050 the binary image in the buffer user-space passes in. These IDs are
1051 maintained by the code in <code class="filename">fs/dcookies.c</code>; the
1052 cache lasts for as long as the daemon has the event buffer open.
1055 <div class="sect1" title="7. Finding a sample's binary image and offset">
1056 <div class="titlepage">
1059 <h2 class="title" style="clear: both"><a id="finding-dentry"></a>7. Finding a sample's binary image and offset</h2>
1064 We haven't yet described how we process the absolute PC value into
1065 something usable by the user-space daemon. When we find a sample entered
1066 into the CPU buffer, we traverse the list of mappings for the task
1067 (remember, we will have seen a task switch earlier, so we know which
1068 task's lists to look at). When a mapping is found that contains the PC
1069 value, we look up the mapped file's dentry in the dcookie cache. This
1070 gives the dcookie ID that will uniquely identify the mapped file. Then
1071 we alter the absolute value such that it is an offset from the start of
1072 the file being mapped (the mapping need not start at the start of the
1073 actual file, so we have to consider the offset value of the mapping). We
1074 store this dcookie ID into the event buffer; this identifies which
1075 binary the samples following it are against.
1076 In this manner, we have converted a PC value, which has transitory
1077 meaning only, into a static offset value for later processing by the
1081 We also attempt to avoid the relatively expensive lookup of the dentry
1082 cookie value by storing the cookie value directly into the dentry
1083 itself; then we can simply derive the cookie value immediately when we
1084 find the correct mapping.
1088 <div class="chapter" title="Chapter 4. Generating sample files">
1089 <div class="titlepage">
1092 <h2 class="title"><a id="sample-files"></a>Chapter 4. Generating sample files</h2>
1098 <b>Table of Contents</b>
1102 <span class="sect1">
1103 <a href="#processing-buffer">1. Processing the buffer</a>
1109 <span class="sect2">
1110 <a href="#handling-kernel-samples">1.1. Handling kernel samples</a>
1116 <span class="sect1">
1117 <a href="#sample-file-generation">2. Locating and creating sample files</a>
1121 <span class="sect1">
1122 <a href="#sample-file-writing">3. Writing data to a sample file</a>
1127 <div class="sect1" title="1. Processing the buffer">
1128 <div class="titlepage">
1131 <h2 class="title" style="clear: both"><a id="processing-buffer"></a>1. Processing the buffer</h2>
1136 Now we can move onto user-space in our description of how raw interrupt
1137 samples are processed into useful information. As we described in
1138 previous sections, the kernel OProfile driver creates a large buffer of
1139 sample data consisting of offset values, interspersed with
1140 notification of changes in context. These context changes indicate how
1141 following samples should be attributed, and include task switches, CPU
1142 changes, and which dcookie the sample value is against. By processing
1143 this buffer entry-by-entry, we can determine where the samples should
1144 be accredited to. This is particularly important when using the
1145 <code class="option">--separate</code>.
1148 The file <code class="filename">daemon/opd_trans.c</code> contains the basic routine
1149 for the buffer processing. The <code class="varname">struct transient</code>
1150 structure is used to hold changes in context. Its members are modified
1151 as we process each entry; it is passed into the routines in
1152 <code class="filename">daemon/opd_sfile.c</code> for actually logging the sample
1153 to a particular sample file (which will be held in
1154 <code class="filename">$SESSION_DIR/samples/current</code>).
1157 The buffer format is designed for conciseness, as high sampling rates
1158 can easily generate a lot of data. Thus, context changes are prefixed
1159 by an escape code, identified by <code class="function">is_escape_code()</code>.
1160 If an escape code is found, the next entry in the buffer identifies
1161 what type of context change is being read. These are handed off to
1162 various handlers (see the <code class="varname">handlers</code> array), which
1163 modify the transient structure as appropriate. If it's not an escape
1164 code, then it must be a PC offset value, and the very next entry will
1165 be the numeric hardware counter. These values are read and recorded
1166 in the transient structure; we then do a lookup to find the correct
1167 sample file, and log the sample, as described in the next section.
1169 <div class="sect2" title="1.1. Handling kernel samples">
1170 <div class="titlepage">
1173 <h3 class="title"><a id="handling-kernel-samples"></a>1.1. Handling kernel samples</h3>
1178 Samples from kernel code require a little special handling. Because
1179 the binary text which the sample is against does not correspond to
1180 any file that the kernel directly knows about, the OProfile driver
1181 stores the absolute PC value in the buffer, instead of the file offset.
1182 Of course, we need an offset against some particular binary. To handle
1183 this, we keep a list of loaded modules by parsing
1184 <code class="filename">/proc/modules</code> as needed. When a module is loaded,
1185 a notification is placed in the OProfile buffer, and this triggers a
1186 re-read. We store the module name, and the loading address and size.
1187 This is also done for the main kernel image, as specified by the user.
1188 The absolute PC value is matched against each address range, and
1189 modified into an offset when the matching module is found. See
1190 <code class="filename">daemon/opd_kernel.c</code> for the details.
1194 <div class="sect1" title="2. Locating and creating sample files">
1195 <div class="titlepage">
1198 <h2 class="title" style="clear: both"><a id="sample-file-generation"></a>2. Locating and creating sample files</h2>
1203 We have a sample value and its satellite data stored in a
1204 <code class="varname">struct transient</code>, and we must locate an
1205 actual sample file to store the sample in, using the context
1206 information in the transient structure as a key. The transient data to
1207 sample file lookup is handled in
1208 <code class="filename">daemon/opd_sfile.c</code>. A hash is taken of the
1209 transient values that are relevant (depending upon the setting of
1210 <code class="option">--separate</code>, some values might be irrelevant), and the
1211 hash value is used to lookup the list of currently open sample files.
1212 Of course, the sample file might not be found, in which case we need
1213 to create and open it.
1216 OProfile uses a rather complex scheme for naming sample files, in order
1217 to make selecting relevant sample files easier for the post-profiling
1218 utilities. The exact details of the scheme are given in
1219 <code class="filename">oprofile-tests/pp_interface</code>, but for now it will
1220 suffice to remember that the filename will include only relevant
1221 information for the current settings, taken from the transient data. A
1222 fully-specified filename looks something like :
1224 <code class="computeroutput">
1225 /var/lib/oprofile/samples/current/{root}/usr/bin/xmms/{dep}/{root}/lib/tls/libc-2.3.2.so/CPU_CLK_UNHALTED.100000.0.28082.28089.0
1228 It should be clear that this identifies such information as the
1229 application binary, the dependent (library) binary, the hardware event,
1230 and the process and thread ID. Typically, not all this information is
1231 needed, in which cases some values may be replaced with the token
1232 <code class="filename">all</code>.
1235 The code that generates this filename and opens the file is found in
1236 <code class="filename">daemon/opd_mangling.c</code>. You may have realised that
1237 at this point, we do not have the binary image file names, only the
1238 dcookie values. In order to determine a file name, a dcookie value is
1239 looked up in the dcookie cache. This is to be found in
1240 <code class="filename">daemon/opd_cookie.c</code>. Since dcookies are both
1241 persistent and unique during a sampling session, we can cache the
1242 values. If the value is not found in the cache, then we ask the kernel
1243 to do the lookup from value to file name for us by calling
1244 <code class="function">lookup_dcookie()</code>. This looks up the value in a
1245 kernel-side cache (see <code class="filename">fs/dcookies.c</code>) and returns
1246 the fully-qualified file name to userspace.
1249 <div class="sect1" title="3. Writing data to a sample file">
1250 <div class="titlepage">
1253 <h2 class="title" style="clear: both"><a id="sample-file-writing"></a>3. Writing data to a sample file</h2>
1258 Each specific sample file is a hashed collection, where the key is
1259 the PC offset from the transient data, and the value is the number of
1260 samples recorded against that offset. The files are
1261 <code class="function">mmap()</code>ed into the daemon's memory space. The code
1262 to actually log the write against the sample file can be found in
1263 <code class="filename">libdb/</code>.
1266 For recording stack traces, we have a more complicated sample filename
1267 mangling scheme that allows us to identify cross-binary calls. We use
1268 the same sample file format, where the key is a 64-bit value composed
1269 from the from,to pair of offsets.
1273 <div class="chapter" title="Chapter 5. Generating useful output">
1274 <div class="titlepage">
1277 <h2 class="title"><a id="output"></a>Chapter 5. Generating useful output</h2>
1283 <b>Table of Contents</b>
1287 <span class="sect1">
1288 <a href="#profile-specification">1. Handling the profile specification</a>
1292 <span class="sect1">
1293 <a href="#sample-file-collating">2. Collating the candidate sample files</a>
1299 <span class="sect2">
1300 <a href="#sample-file-classifying">2.1. Classifying sample files</a>
1304 <span class="sect2">
1305 <a href="#sample-file-inverting">2.2. Creating inverted profile lists</a>
1311 <span class="sect1">
1312 <a href="#generating-profile-data">3. Generating profile data</a>
1318 <span class="sect2">
1319 <a href="#bfd">3.1. Processing the binary image</a>
1323 <span class="sect2">
1324 <a href="#processing-sample-files">3.2. Processing the sample files</a>
1330 <span class="sect1">
1331 <a href="#generating-output">4. Generating output</a>
1337 All of the tools used to generate human-readable output have to take
1338 roughly the same steps to collect the data for processing. First, the
1339 profile specification given by the user has to be parsed. Next, a list
1340 of sample files matching the specification has to obtained. Using this
1341 list, we need to locate the binary file for each sample file, and then
1342 use them to extract meaningful data, before a final collation and
1343 presentation to the user.
1345 <div class="sect1" title="1. Handling the profile specification">
1346 <div class="titlepage">
1349 <h2 class="title" style="clear: both"><a id="profile-specification"></a>1. Handling the profile specification</h2>
1354 The profile specification presented by the user is parsed in
1355 the function <code class="function">profile_spec::create()</code>. This
1356 creates an object representing the specification. Then we
1357 use <code class="function">profile_spec::generate_file_list()</code>
1358 to search for all sample files and match them against the
1359 <code class="varname">profile_spec</code>.
1362 To enable this matching process to work, the attributes of
1363 each sample file is encoded in its filename. This is a low-tech
1364 approach to matching specifications against candidate sample
1365 files, but it works reasonably well. A typical sample file
1366 might look like these:
1368 <table xmlns="" border="0" style="background: #E0E0E0;" width="90%">
1371 <pre class="screen">
1372 /var/lib/oprofile/samples/current/{root}/bin/ls/{dep}/{root}/bin/ls/{cg}/{root}/bin/ls/CPU_CLK_UNHALTED.100000.0.all.all.all
1373 /var/lib/oprofile/samples/current/{root}/bin/ls/{dep}/{root}/bin/ls/CPU_CLK_UNHALTED.100000.0.all.all.all
1374 /var/lib/oprofile/samples/current/{root}/bin/ls/{dep}/{root}/bin/ls/CPU_CLK_UNHALTED.100000.0.7423.7424.0
1375 /var/lib/oprofile/samples/current/{kern}/r128/{dep}/{kern}/r128/CPU_CLK_UNHALTED.100000.0.all.all.all
1381 This looks unnecessarily complex, but it's actually fairly simple. First
1382 we have the session of the sample, by default located here
1383 <code class="filename">/var/lib/oprofile/samples/current</code>. This location
1384 can be changed by specifying the --session-dir option at command-line.
1385 This session could equally well be inside an archive from <span class="command"><strong>oparchive</strong></span>.
1386 Next we have one of the tokens <code class="filename">{root}</code> or
1387 <code class="filename">{kern}</code>. <code class="filename">{root}</code> indicates
1388 that the binary is found on a file system, and we will encode its path
1389 in the next section (e.g. <code class="filename">/bin/ls</code>).
1390 <code class="filename">{kern}</code> indicates a kernel module - on 2.6 kernels
1391 the path information is not available from the kernel, so we have to
1392 special-case kernel modules like this; we encode merely the name of the
1396 Next there is a <code class="filename">{dep}</code> token, indicating another
1397 token/path which identifies the dependent binary image. This is used even for
1398 the "primary" binary (i.e. the one that was
1399 <code class="function">execve()</code>d), as it simplifies processing. Finally,
1400 if this sample file is a normal flat profile, the actual file is next in
1401 the path. If it's a call-graph sample file, we need one further
1402 specification, to allow us to identify cross-binary arcs in the call
1406 The actual sample file name is dot-separated, where the fields are, in
1407 order: event name, event count, unit mask, task group ID, task ID, and
1411 This sample file can be reliably parsed (with
1412 <code class="function">parse_filename()</code>) into a
1413 <code class="varname">filename_spec</code>. Finally, we can check whether to
1414 include the sample file in the final results by comparing this
1415 <code class="varname">filename_spec</code> against the
1416 <code class="varname">profile_spec</code> the user specified (for the interested,
1417 see <code class="function">valid_candidate()</code> and
1418 <code class="function">profile_spec::match</code>). Then comes the really
1422 <div class="sect1" title="2. Collating the candidate sample files">
1423 <div class="titlepage">
1426 <h2 class="title" style="clear: both"><a id="sample-file-collating"></a>2. Collating the candidate sample files</h2>
1431 At this point we have a duplicate-free list of sample files we need
1432 to process. But first we need to do some further arrangement: we
1433 need to classify each sample file, and we may also need to "invert"
1436 <div class="sect2" title="2.1. Classifying sample files">
1437 <div class="titlepage">
1440 <h3 class="title"><a id="sample-file-classifying"></a>2.1. Classifying sample files</h3>
1445 It's possible for utilities like <span class="command"><strong>opreport</strong></span> to show
1446 data in columnar format: for example, we might want to show the results
1447 of two threads within a process side-by-side. To do this, we need
1448 to classify each sample file into classes - the classes correspond
1449 with each <span class="command"><strong>opreport</strong></span> column. The function that handles
1450 this is <code class="function">arrange_profiles()</code>. Each sample file
1451 is added to a particular class. If the sample file is the first in
1452 its class, a template is generated from the sample file. Each template
1453 describes a particular class (thus, in our example above, each template
1454 will have a different thread ID, and this uniquely identifies each
1458 Each class has a list of "profile sets" matching that class's template.
1459 A profile set is either a profile of the primary binary image, or any of
1460 its dependent images. After all sample files have been listed in one of
1461 the profile sets belonging to the classes, we have to name each class and
1462 perform error-checking. This is done by
1463 <code class="function">identify_classes()</code>; each class is checked to ensure
1464 that its "axis" is the same as all the others. This is needed because
1465 <span class="command"><strong>opreport</strong></span> can't produce results in 3D format: we can
1466 only differ in one aspect, such as thread ID or event name.
1469 <div class="sect2" title="2.2. Creating inverted profile lists">
1470 <div class="titlepage">
1473 <h3 class="title"><a id="sample-file-inverting"></a>2.2. Creating inverted profile lists</h3>
1478 Remember that if we're using certain profile separation options, such as
1479 "--separate=lib", a single binary could be a dependent image to many
1480 different binaries. For example, the C library image would be a
1481 dependent image for most programs that have been profiled. As it
1482 happens, this can cause severe performance problems: without some
1483 re-arrangement, these dependent binary images would be opened each
1484 time we need to process sample files for each program.
1487 The solution is to "invert" the profiles via
1488 <code class="function">invert_profiles()</code>. We create a new data structure
1489 where the dependent binary is first, and the primary binary images using
1490 that dependent binary are listed as sub-images. This helps our
1491 performance problem, as now we only need to open each dependent image
1492 once, when we process the list of inverted profiles.
1496 <div class="sect1" title="3. Generating profile data">
1497 <div class="titlepage">
1500 <h2 class="title" style="clear: both"><a id="generating-profile-data"></a>3. Generating profile data</h2>
1505 Things don't get any simpler at this point, unfortunately. At this point
1506 we've collected and classified the sample files into the set of inverted
1507 profiles, as described in the previous section. Now we need to process
1508 each inverted profile and make something of the data. The entry point
1509 for this is <code class="function">populate_for_image()</code>.
1511 <div class="sect2" title="3.1. Processing the binary image">
1512 <div class="titlepage">
1515 <h3 class="title"><a id="bfd"></a>3.1. Processing the binary image</h3>
1520 The first thing we do with an inverted profile is attempt to open the
1521 binary image (remember each inverted profile set is only for one binary
1522 image, but may have many sample files to process). The
1523 <code class="varname">op_bfd</code> class provides an abstracted interface to
1524 this; internally it uses <code class="filename">libbfd</code>. The main purpose
1525 of this class is to process the symbols for the binary image; this is
1526 also where symbol filtering happens. This is actually quite tricky, but
1527 should be clear from the source.
1530 <div class="sect2" title="3.2. Processing the sample files">
1531 <div class="titlepage">
1534 <h3 class="title"><a id="processing-sample-files"></a>3.2. Processing the sample files</h3>
1539 The class <code class="varname">profile_container</code> is a hold-all that
1540 contains all the processed results. It is a container of
1541 <code class="varname">profile_t</code> objects. The
1542 <code class="function">add_sample_files()</code> method uses
1543 <code class="filename">libdb</code> to open the given sample file and add the
1544 key/value types to the <code class="varname">profile_t</code>. Once this has been
1545 done, <code class="function">profile_container::add()</code> is passed the
1546 <code class="varname">profile_t</code> plus the <code class="varname">op_bfd</code> for
1550 <code class="function">profile_container::add()</code> walks through the symbols
1551 collected in the <code class="varname">op_bfd</code>.
1552 <code class="function">op_bfd::get_symbol_range()</code> gives us the start and
1553 end of the symbol as an offset from the start of the binary image,
1554 then we interrogate the <code class="varname">profile_t</code> for the relevant samples
1555 for that offset range. We create a <code class="varname">symbol_entry</code>
1556 object for this symbol and fill it in. If needed, here we also collect
1557 debug information from the <code class="varname">op_bfd</code>, and possibly
1558 record the detailed sample information (as used by <span class="command"><strong>opreport
1559 -d</strong></span> and <span class="command"><strong>opannotate</strong></span>).
1560 Finally the <code class="varname">symbol_entry</code> is added to
1561 a private container of <code class="varname">profile_container</code> - this
1562 <code class="varname">symbol_container</code> holds all such processed symbols.
1566 <div class="sect1" title="4. Generating output">
1567 <div class="titlepage">
1570 <h2 class="title" style="clear: both"><a id="generating-output"></a>4. Generating output</h2>
1575 After the processing described in the previous section, we've now got
1576 full details of what we need to output stored in the
1577 <code class="varname">profile_container</code> on a symbol-by-symbol basis. To
1578 produce output, we need to replay that data and format it suitably.
1581 <span class="command"><strong>opreport</strong></span> first asks the
1582 <code class="varname">profile_container</code> for a
1583 <code class="varname">symbol_collection</code> (this is also where thresholding
1585 This is sorted, then a
1586 <code class="varname">opreport_formatter</code> is initialised.
1587 This object initialises a set of field formatters as requested. Then
1588 <code class="function">opreport_formatter::output()</code> is called. This
1589 iterates through the (sorted) <code class="varname">symbol_collection</code>;
1590 for each entry, the selected fields (as set by the
1591 <code class="varname">format_flags</code> options) are output by calling the
1592 field formatters, with the <code class="varname">symbol_entry</code> passed in.
1596 <div class="chapter" title="Chapter 6. Extended Feature Interface">
1597 <div class="titlepage">
1600 <h2 class="title"><a id="ext"></a>Chapter 6. Extended Feature Interface</h2>
1606 <b>Table of Contents</b>
1610 <span class="sect1">
1611 <a href="#ext-intro">1. Introduction</a>
1615 <span class="sect1">
1616 <a href="#ext-name-and-handlers">2. Feature Name and Handlers</a>
1620 <span class="sect1">
1621 <a href="#ext-enable">3. Enabling Features</a>
1625 <span class="sect1">
1626 <a href="#ext-types-of-handlers">4. Type of Handlers</a>
1632 <span class="sect2">
1633 <a href="#ext_init">4.1. ext_init Handler</a>
1637 <span class="sect2">
1638 <a href="#ext_print_stats">4.2. ext_print_stats Handler</a>
1642 <span class="sect2">
1643 <a href="#ext_sfile_handlers">4.3. ext_sfile Handler</a>
1649 <span class="sect1">
1650 <a href="#ext-implementation">5. Extended Feature Reference Implementation</a>
1656 <span class="sect2">
1657 <a href="#ext-ibs">5.1. Instruction-Based Sampling (IBS)</a>
1664 <div class="sect1" title="1. Introduction">
1665 <div class="titlepage">
1668 <h2 class="title" style="clear: both"><a id="ext-intro"></a>1. Introduction</h2>
1673 The Extended Feature Interface is a standard callback interface
1674 designed to allow extension to the OProfile daemon's sample processing.
1675 Each feature defines a set of callback handlers which can be enabled or
1676 disabled through the OProfile daemon's command-line option.
1677 This interface can be used to implement support for architecture-specific
1678 features or features not commonly used by general OProfile users.
1681 <div class="sect1" title="2. Feature Name and Handlers">
1682 <div class="titlepage">
1685 <h2 class="title" style="clear: both"><a id="ext-name-and-handlers"></a>2. Feature Name and Handlers</h2>
1690 Each extended feature has an entry in the <code class="varname">ext_feature_table</code>
1691 in <code class="filename">opd_extended.cpp</code>. Each entry contains a feature name,
1692 and a corresponding set of handlers. Feature name is a unique string, which is
1693 used to identify a feature in the table. Each feature provides a set
1694 of handlers, which will be executed by the OProfile daemon from pre-determined
1695 locations to perform certain tasks. At runtime, the OProfile daemon calls a feature
1696 handler wrapper from one of the predetermined locations to check whether
1697 an extended feature is enabled, and whether a particular handler exists.
1698 Only the handlers of the enabled feature will be executed.
1701 <div class="sect1" title="3. Enabling Features">
1702 <div class="titlepage">
1705 <h2 class="title" style="clear: both"><a id="ext-enable"></a>3. Enabling Features</h2>
1710 Each feature is enabled using the OProfile daemon (oprofiled) command-line
1711 option "--ext-feature=<extended-feature-name>:[args]". The
1712 "extended-feature-name" is used to determine the feature to be enabled.
1713 The optional "args" is passed into the feature-specific initialization handler
1714 (<code class="function">ext_init</code>). Currently, only one extended feature can be
1718 <div class="sect1" title="4. Type of Handlers">
1719 <div class="titlepage">
1722 <h2 class="title" style="clear: both"><a id="ext-types-of-handlers"></a>4. Type of Handlers</h2>
1727 Each feature is responsible for providing its own set of handlers.
1728 Types of handler are:
1730 <div class="sect2" title="4.1. ext_init Handler">
1731 <div class="titlepage">
1734 <h3 class="title"><a id="ext_init"></a>4.1. ext_init Handler</h3>
1739 "ext_init" handles initialization of an extended feature. It takes
1740 "args" parameter which is passed in through the "oprofiled --ext-feature=<
1741 extended-feature-name>:[args]". This handler is executed in the function
1742 <code class="function">opd_options()</code> in the file <code class="filename">daemon/oprofiled.c
1745 <div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;">
1746 <h3 class="title">Note</h3>
1748 The ext_init handler is required for all features.
1752 <div class="sect2" title="4.2. ext_print_stats Handler">
1753 <div class="titlepage">
1756 <h3 class="title"><a id="ext_print_stats"></a>4.2. ext_print_stats Handler</h3>
1761 "ext_print_stats" handles the extended feature statistics report. It adds
1762 a new section in the OProfile daemon statistics report, which is normally
1763 outputed to the file
1764 <code class="filename">/var/lib/oprofile/samples/oprofiled.log</code>.
1765 This handler is executed in the function <code class="function">opd_print_stats()</code>
1766 in the file <code class="filename">daemon/opd_stats.c</code>.
1769 <div class="sect2" title="4.3. ext_sfile Handler">
1770 <div class="titlepage">
1773 <h3 class="title"><a id="ext_sfile_handlers"></a>4.3. ext_sfile Handler</h3>
1778 "ext_sfile" contains a set of handlers related to operations on the extended
1779 sample files (sample files for events related to extended feature).
1780 These operations include <code class="function">create_sfile()</code>,
1781 <code class="function">sfile_dup()</code>, <code class="function">close_sfile()</code>,
1782 <code class="function">sync_sfile()</code>, and <code class="function">get_file()</code>
1783 as defined in <code class="filename">daemon/opd_sfile.c</code>.
1784 An additional field, <code class="varname">odb_t * ext_file</code>, is added to the
1785 <code class="varname">struct sfile</code> for storing extended sample files
1791 <div class="sect1" title="5. Extended Feature Reference Implementation">
1792 <div class="titlepage">
1795 <h2 class="title" style="clear: both"><a id="ext-implementation"></a>5. Extended Feature Reference Implementation</h2>
1799 <div class="sect2" title="5.1. Instruction-Based Sampling (IBS)">
1800 <div class="titlepage">
1803 <h3 class="title"><a id="ext-ibs"></a>5.1. Instruction-Based Sampling (IBS)</h3>
1808 An example of extended feature implementation can be seen by
1809 examining the AMD Instruction-Based Sampling support.
1811 <div class="sect3" title="5.1.1. IBS Initialization">
1812 <div class="titlepage">
1815 <h4 class="title"><a id="ibs-init"></a>5.1.1. IBS Initialization</h4>
1820 Instruction-Based Sampling (IBS) is a new performance measurement technique
1821 available on AMD Family 10h processors. Enabling IBS profiling is done simply
1822 by specifying IBS performance events through the "--event=" options.
1824 <table xmlns="" border="0" style="background: #E0E0E0;" width="90%">
1827 <pre class="screen">
1828 opcontrol --event=IBS_FETCH_XXX:<count>:<um>:<kernel>:<user>
1829 opcontrol --event=IBS_OP_XXX:<count>:<um>:<kernel>:<user>
1831 Note: * Count and unitmask for all IBS fetch events must be the same,
1832 as do those for IBS op.
1838 IBS performance events are listed in <code class="function">opcontrol --list-events</code>.
1839 When users specify these events, opcontrol verifies them using ophelp, which
1840 checks for the <code class="varname">ext:ibs_fetch</code> or <code class="varname">ext:ibs_op</code>
1841 tag in <code class="filename">events/x86-64/family10/events</code> file.
1842 Then, it configures the driver interface (/dev/oprofile/ibs_fetch/... and
1843 /dev/oprofile/ibs_op/...) and starts the OProfile daemon as follows.
1845 <table xmlns="" border="0" style="background: #E0E0E0;" width="90%">
1848 <pre class="screen">
1851 fetch:<IBS_FETCH_EVENT1>,<IBS_FETCH_EVENT2>,...,:<IBS fetch count>:<IBS Fetch um>|\
1852 op:<IBS_OP_EVENT1>,<IBS_OP_EVENT2>,...,:<IBS op count>:<IBS op um>
1858 Here, the OProfile daemon parses the <code class="varname">--ext-feature</code>
1859 option and checks the feature name ("ibs") before calling the
1860 the initialization function to handle the string
1861 containing IBS events, counts, and unitmasks.
1862 Then, it stores each event in the IBS virtual-counter table
1863 (<code class="varname">struct opd_event ibs_vc[OP_MAX_IBS_COUNTERS]</code>) and
1864 stores the event index in the IBS Virtual Counter Index (VCI) map
1865 (<code class="varname">ibs_vci_map[OP_MAX_IBS_COUNTERS]</code>) with IBS event value
1869 <div class="sect3" title="5.1.2. IBS Data Processing">
1870 <div class="titlepage">
1873 <h4 class="title"><a id="ibs-data-processing"></a>5.1.2. IBS Data Processing</h4>
1878 During a profile session, the OProfile daemon identifies IBS samples in the
1879 event buffer using the <code class="varname">"IBS_FETCH_CODE"</code> or
1880 <code class="varname">"IBS_OP_CODE"</code>. These codes trigger the handlers
1881 <code class="function">code_ibs_fetch_sample()</code> or
1882 <code class="function">code_ibs_op_sample()</code> listed in the
1883 <code class="varname">handler_t handlers[]</code> vector in
1884 <code class="filename">daemon/opd_trans.c </code>. These handlers are responsible for
1885 processing IBS samples and translate them into IBS performance events.
1888 Unlike traditional performance events, each IBS sample can be derived into
1889 multiple IBS performance events. For each event that the user specifies,
1890 a combination of bits from Model-Specific Registers (MSR) are checked
1891 against the bitmask defining the event. If the condition is met, the event
1892 will then be recorded. The derivation logic is in the files
1893 <code class="filename">daemon/opd_ibs_macro.h</code> and
1894 <code class="filename">daemon/opd_ibs_trans.[h,c]</code>.
1897 <div class="sect3" title="5.1.3. IBS Sample File">
1898 <div class="titlepage">
1901 <h4 class="title"><a id="ibs-sample-file"></a>5.1.3. IBS Sample File</h4>
1906 Traditionally, sample file information <code class="varname">(odb_t)</code> is stored
1907 in the <code class="varname">struct sfile::odb_t file[OP_MAX_COUNTER]</code>.
1908 Currently, <code class="varname">OP_MAX_COUNTER</code> is 8 on non-alpha, and 20 on
1909 alpha-based system. Event index (the counter number on which the event
1910 is configured) is used to access the corresponding entry in the array.
1911 Unlike the traditional performance event, IBS does not use the actual
1912 counter registers (i.e. <code class="filename">/dev/oprofile/0,1,2,3</code>).
1913 Also, the number of performance events generated by IBS could be larger than
1914 <code class="varname">OP_MAX_COUNTER</code> (currently upto 13 IBS-fetch and 46 IBS-op
1915 events). Therefore IBS requires a special data structure and sfile
1916 handlers (<code class="varname">struct opd_ext_sfile_handlers</code>) for managing
1917 IBS sample files. IBS-sample-file information is stored in a memory
1918 allocated by handler <code class="function">ibs_sfile_create()</code>, which can
1919 be accessed through <code class="varname">struct sfile::odb_t * ext_files</code>.
1925 <div class="glossary" title="Glossary of OProfile source concepts and types">
1926 <div class="titlepage">
1929 <h2 class="title"><a id="glossary"></a>Glossary of OProfile source concepts and types</h2>
1934 <dt>application image</dt>
1937 The primary binary image used by an application. This is derived
1938 from the kernel and corresponds to the binary started upon running
1939 an application: for example, <code class="filename">/bin/bash</code>.
1942 <dt>binary image</dt>
1945 An ELF file containing executable code: this includes kernel modules,
1946 the kernel itself (a.k.a. <code class="filename">vmlinux</code>), shared libraries,
1947 and application binaries.
1953 Short for "dentry cookie". A unique ID that can be looked up to provide
1954 the full path name of a binary image.
1957 <dt>dependent image</dt>
1960 A binary image that is dependent upon an application, used with
1961 per-application separation. Most commonly, shared libraries. For example,
1962 if <code class="filename">/bin/bash</code> is running and we take
1963 some samples inside the C library itself due to <span class="command"><strong>bash</strong></span>
1964 calling library code, then the image <code class="filename">/lib/libc.so</code>
1965 would be dependent upon <code class="filename">/bin/bash</code>.
1971 This refers to the ability to merge several distinct sample files
1972 into one set of data at runtime, in the post-profiling tools. For example,
1973 per-thread sample files can be merged into one set of data, because
1974 they are compatible (i.e. the aggregation of the data is meaningful),
1975 but it's not possible to merge sample files for two different events,
1976 because there would be no useful meaning to the results.
1979 <dt>profile class</dt>
1982 A collection of profile data that has been collected under the same
1983 class template. For example, if we're using <span class="command"><strong>opreport</strong></span>
1984 to show results after profiling with two performance counters enabled
1985 profiling <code class="constant">DATA_MEM_REFS</code> and <code class="constant">CPU_CLK_UNHALTED</code>,
1986 there would be two profile classes, one for each event. Or if we're on
1987 an SMP system and doing per-cpu profiling, and we request
1988 <span class="command"><strong>opreport</strong></span> to show results for each CPU side-by-side,
1989 there would be a profile class for each CPU.
1992 <dt>profile specification</dt>
1995 The parameters the user passes to the post-profiling tools that limit
1996 what sample files are used. This specification is matched against
1997 the available sample files to generate a selection of profile data.
2000 <dt>profile template</dt>
2003 The parameters that define what goes in a particular profile class.
2004 This includes a symbolic name (e.g. "cpu:1") and the code-usable
projects / platform / upstream / oprofile.git / blob