1 llvm-mca - LLVM Machine Code Analyzer
2 =====================================
9 :program:`llvm-mca` [*options*] [input]
14 :program:`llvm-mca` is a performance analysis tool that uses information
15 available in LLVM (e.g. scheduling models) to statically measure the performance
16 of machine code in a specific CPU.
18 Performance is measured in terms of throughput as well as processor resource
19 consumption. The tool currently works for processors with a backend for which
20 there is a scheduling model available in LLVM.
22 The main goal of this tool is not just to predict the performance of the code
23 when run on the target, but also help with diagnosing potential performance
26 Given an assembly code sequence, :program:`llvm-mca` estimates the Instructions
27 Per Cycle (IPC), as well as hardware resource pressure. The analysis and
28 reporting style were inspired by the IACA tool from Intel.
30 For example, you can compile code with clang, output assembly, and pipe it
31 directly into :program:`llvm-mca` for analysis:
35 $ clang foo.c -O2 -target x86_64-unknown-unknown -S -o - | llvm-mca -mcpu=btver2
41 $ clang foo.c -O2 -target x86_64-unknown-unknown -mllvm -x86-asm-syntax=intel -S -o - | llvm-mca -mcpu=btver2
43 (:program:`llvm-mca` detects Intel syntax by the presence of an `.intel_syntax`
44 directive at the beginning of the input. By default its output syntax matches
47 Scheduling models are not just used to compute instruction latencies and
48 throughput, but also to understand what processor resources are available
49 and how to simulate them.
51 By design, the quality of the analysis conducted by :program:`llvm-mca` is
52 inevitably affected by the quality of the scheduling models in LLVM.
54 If you see that the performance report is not accurate for a processor,
55 please `file a bug <https://bugs.llvm.org/enter_bug.cgi?product=libraries>`_
56 against the appropriate backend.
61 If ``input`` is "``-``" or omitted, :program:`llvm-mca` reads from standard
62 input. Otherwise, it will read from the specified filename.
64 If the :option:`-o` option is omitted, then :program:`llvm-mca` will send its output
65 to standard output if the input is from standard input. If the :option:`-o`
66 option specifies "``-``", then the output will also be sent to standard output.
71 Print a summary of command line options.
73 .. option:: -o <filename>
75 Use ``<filename>`` as the output filename. See the summary above for more
78 .. option:: -mtriple=<target triple>
80 Specify a target triple string.
82 .. option:: -march=<arch>
84 Specify the architecture for which to analyze the code. It defaults to the
87 .. option:: -mcpu=<cpuname>
89 Specify the processor for which to analyze the code. By default, the cpu name
90 is autodetected from the host.
92 .. option:: -output-asm-variant=<variant id>
94 Specify the output assembly variant for the report generated by the tool.
95 On x86, possible values are [0, 1]. A value of 0 (vic. 1) for this flag enables
96 the AT&T (vic. Intel) assembly format for the code printed out by the tool in
99 .. option:: -print-imm-hex
101 Prefer hex format for numeric literals in the output assembly printed as part
104 .. option:: -dispatch=<width>
106 Specify a different dispatch width for the processor. The dispatch width
107 defaults to field 'IssueWidth' in the processor scheduling model. If width is
108 zero, then the default dispatch width is used.
110 .. option:: -register-file-size=<size>
112 Specify the size of the register file. When specified, this flag limits how
113 many physical registers are available for register renaming purposes. A value
114 of zero for this flag means "unlimited number of physical registers".
116 .. option:: -iterations=<number of iterations>
118 Specify the number of iterations to run. If this flag is set to 0, then the
119 tool sets the number of iterations to a default value (i.e. 100).
121 .. option:: -noalias=<bool>
123 If set, the tool assumes that loads and stores don't alias. This is the
126 .. option:: -lqueue=<load queue size>
128 Specify the size of the load queue in the load/store unit emulated by the tool.
129 By default, the tool assumes an unbound number of entries in the load queue.
130 A value of zero for this flag is ignored, and the default load queue size is
133 .. option:: -squeue=<store queue size>
135 Specify the size of the store queue in the load/store unit emulated by the
136 tool. By default, the tool assumes an unbound number of entries in the store
137 queue. A value of zero for this flag is ignored, and the default store queue
138 size is used instead.
140 .. option:: -timeline
142 Enable the timeline view.
144 .. option:: -timeline-max-iterations=<iterations>
146 Limit the number of iterations to print in the timeline view. By default, the
147 timeline view prints information for up to 10 iterations.
149 .. option:: -timeline-max-cycles=<cycles>
151 Limit the number of cycles in the timeline view, or use 0 for no limit. By
152 default, the number of cycles is set to 80.
154 .. option:: -resource-pressure
156 Enable the resource pressure view. This is enabled by default.
158 .. option:: -register-file-stats
160 Enable register file usage statistics.
162 .. option:: -dispatch-stats
164 Enable extra dispatch statistics. This view collects and analyzes instruction
165 dispatch events, as well as static/dynamic dispatch stall events. This view
166 is disabled by default.
168 .. option:: -scheduler-stats
170 Enable extra scheduler statistics. This view collects and analyzes instruction
171 issue events. This view is disabled by default.
173 .. option:: -retire-stats
175 Enable extra retire control unit statistics. This view is disabled by default.
177 .. option:: -instruction-info
179 Enable the instruction info view. This is enabled by default.
181 .. option:: -show-encoding
183 Enable the printing of instruction encodings within the instruction info view.
185 .. option:: -all-stats
187 Print all hardware statistics. This enables extra statistics related to the
188 dispatch logic, the hardware schedulers, the register file(s), and the retire
189 control unit. This option is disabled by default.
191 .. option:: -all-views
195 .. option:: -instruction-tables
197 Prints resource pressure information based on the static information
198 available from the processor model. This differs from the resource pressure
199 view because it doesn't require that the code is simulated. It instead prints
200 the theoretical uniform distribution of resource pressure for every
201 instruction in sequence.
203 .. option:: -bottleneck-analysis
205 Print information about bottlenecks that affect the throughput. This analysis
206 can be expensive, and it is disabled by default. Bottlenecks are highlighted
207 in the summary view. Bottleneck analysis is currently not supported for
208 processors with an in-order backend.
212 Print the requested views in valid JSON format. The instructions and the
213 processor resources are printed as members of special top level JSON objects.
214 The individual views refer to them by index. However, not all views are
215 currently supported. For example, the report from the bottleneck analysis is
216 not printed out in JSON. All the default views are currently supported.
218 .. option:: -disable-cb
220 Force usage of the generic CustomBehaviour and InstrPostProcess classes rather
221 than using the target specific implementation. The generic classes never
222 detect any custom hazards or make any post processing modifications to
229 :program:`llvm-mca` returns 0 on success. Otherwise, an error message is printed
230 to standard error, and the tool returns 1.
232 USING MARKERS TO ANALYZE SPECIFIC CODE BLOCKS
233 ---------------------------------------------
234 :program:`llvm-mca` allows for the optional usage of special code comments to
235 mark regions of the assembly code to be analyzed. A comment starting with
236 substring ``LLVM-MCA-BEGIN`` marks the beginning of a code region. A comment
237 starting with substring ``LLVM-MCA-END`` marks the end of a code region. For
246 If no user-defined region is specified, then :program:`llvm-mca` assumes a
247 default region which contains every instruction in the input file. Every region
248 is analyzed in isolation, and the final performance report is the union of all
249 the reports generated for every code region.
251 Code regions can have names. For example:
255 # LLVM-MCA-BEGIN A simple example
259 The code from the example above defines a region named "A simple example" with a
260 single instruction in it. Note how the region name doesn't have to be repeated
261 in the ``LLVM-MCA-END`` directive. In the absence of overlapping regions,
262 an anonymous ``LLVM-MCA-END`` directive always ends the currently active user
265 Example of nesting regions:
276 Example of overlapping regions:
288 Note that multiple anonymous regions cannot overlap. Also, overlapping regions
289 cannot have the same name.
291 There is no support for marking regions from high-level source code, like C or
292 C++. As a workaround, inline assembly directives may be used:
296 int foo(int a, int b) {
297 __asm volatile("# LLVM-MCA-BEGIN foo");
299 __asm volatile("# LLVM-MCA-END");
304 However, this interferes with optimizations like loop vectorization and may have
305 an impact on the code generated. This is because the ``__asm`` statements are
306 seen as real code having important side effects, which limits how the code
307 around them can be transformed. If users want to make use of inline assembly
308 to emit markers, then the recommendation is to always verify that the output
309 assembly is equivalent to the assembly generated in the absence of markers.
310 The `Clang options to emit optimization reports <https://clang.llvm.org/docs/UsersManual.html#options-to-emit-optimization-reports>`_
311 can also help in detecting missed optimizations.
316 :program:`llvm-mca` takes assembly code as input. The assembly code is parsed
317 into a sequence of MCInst with the help of the existing LLVM target assembly
318 parsers. The parsed sequence of MCInst is then analyzed by a ``Pipeline`` module
319 to generate a performance report.
321 The Pipeline module simulates the execution of the machine code sequence in a
322 loop of iterations (default is 100). During this process, the pipeline collects
323 a number of execution related statistics. At the end of this process, the
324 pipeline generates and prints a report from the collected statistics.
326 Here is an example of a performance report generated by the tool for a
327 dot-product of two packed float vectors of four elements. The analysis is
328 conducted for target x86, cpu btver2. The following result can be produced via
329 the following command using the example located at
330 ``test/tools/llvm-mca/X86/BtVer2/dot-product.s``:
334 $ llvm-mca -mtriple=x86_64-unknown-unknown -mcpu=btver2 -iterations=300 dot-product.s
346 Block RThroughput: 2.0
355 [6]: HasSideEffects (U)
357 [1] [2] [3] [4] [5] [6] Instructions:
358 1 2 1.00 vmulps %xmm0, %xmm1, %xmm2
359 1 3 1.00 vhaddps %xmm2, %xmm2, %xmm3
360 1 3 1.00 vhaddps %xmm3, %xmm3, %xmm4
380 Resource pressure per iteration:
381 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13]
382 - - - 2.00 1.00 2.00 1.00 - - - - - - -
384 Resource pressure by instruction:
385 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] Instructions:
386 - - - - 1.00 - 1.00 - - - - - - - vmulps %xmm0, %xmm1, %xmm2
387 - - - 1.00 - 1.00 - - - - - - - - vhaddps %xmm2, %xmm2, %xmm3
388 - - - 1.00 - 1.00 - - - - - - - - vhaddps %xmm3, %xmm3, %xmm4
390 According to this report, the dot-product kernel has been executed 300 times,
391 for a total of 900 simulated instructions. The total number of simulated micro
392 opcodes (uOps) is also 900.
394 The report is structured in three main sections. The first section collects a
395 few performance numbers; the goal of this section is to give a very quick
396 overview of the performance throughput. Important performance indicators are
397 **IPC**, **uOps Per Cycle**, and **Block RThroughput** (Block Reciprocal
400 Field *DispatchWidth* is the maximum number of micro opcodes that are dispatched
401 to the out-of-order backend every simulated cycle. For processors with an
402 in-order backend, *DispatchWidth* is the maximum number of micro opcodes issued
403 to the backend every simulated cycle.
405 IPC is computed dividing the total number of simulated instructions by the total
408 Field *Block RThroughput* is the reciprocal of the block throughput. Block
409 throughput is a theoretical quantity computed as the maximum number of blocks
410 (i.e. iterations) that can be executed per simulated clock cycle in the absence
411 of loop carried dependencies. Block throughput is superiorly limited by the
412 dispatch rate, and the availability of hardware resources.
414 In the absence of loop-carried data dependencies, the observed IPC tends to a
415 theoretical maximum which can be computed by dividing the number of instructions
416 of a single iteration by the `Block RThroughput`.
418 Field 'uOps Per Cycle' is computed dividing the total number of simulated micro
419 opcodes by the total number of cycles. A delta between Dispatch Width and this
420 field is an indicator of a performance issue. In the absence of loop-carried
421 data dependencies, the observed 'uOps Per Cycle' should tend to a theoretical
422 maximum throughput which can be computed by dividing the number of uOps of a
423 single iteration by the `Block RThroughput`.
425 Field *uOps Per Cycle* is bounded from above by the dispatch width. That is
426 because the dispatch width limits the maximum size of a dispatch group. Both IPC
427 and 'uOps Per Cycle' are limited by the amount of hardware parallelism. The
428 availability of hardware resources affects the resource pressure distribution,
429 and it limits the number of instructions that can be executed in parallel every
430 cycle. A delta between Dispatch Width and the theoretical maximum uOps per
431 Cycle (computed by dividing the number of uOps of a single iteration by the
432 `Block RThroughput`) is an indicator of a performance bottleneck caused by the
433 lack of hardware resources.
434 In general, the lower the Block RThroughput, the better.
436 In this example, ``uOps per iteration/Block RThroughput`` is 1.50. Since there
437 are no loop-carried dependencies, the observed `uOps Per Cycle` is expected to
438 approach 1.50 when the number of iterations tends to infinity. The delta between
439 the Dispatch Width (2.00), and the theoretical maximum uOp throughput (1.50) is
440 an indicator of a performance bottleneck caused by the lack of hardware
441 resources, and the *Resource pressure view* can help to identify the problematic
444 The second section of the report is the `instruction info view`. It shows the
445 latency and reciprocal throughput of every instruction in the sequence. It also
446 reports extra information related to the number of micro opcodes, and opcode
447 properties (i.e., 'MayLoad', 'MayStore', and 'HasSideEffects').
449 Field *RThroughput* is the reciprocal of the instruction throughput. Throughput
450 is computed as the maximum number of instructions of a same type that can be
451 executed per clock cycle in the absence of operand dependencies. In this
452 example, the reciprocal throughput of a vector float multiply is 1
453 cycles/instruction. That is because the FP multiplier JFPM is only available
456 Instruction encodings are displayed within the instruction info view when flag
457 `-show-encoding` is specified.
459 Below is an example of `-show-encoding` output for the dot-product kernel:
469 [6]: HasSideEffects (U)
472 [1] [2] [3] [4] [5] [6] [7] Encodings: Instructions:
473 1 2 1.00 4 c5 f0 59 d0 vmulps %xmm0, %xmm1, %xmm2
474 1 4 1.00 4 c5 eb 7c da vhaddps %xmm2, %xmm2, %xmm3
475 1 4 1.00 4 c5 e3 7c e3 vhaddps %xmm3, %xmm3, %xmm4
477 The `Encoding Size` column shows the size in bytes of instructions. The
478 `Encodings` column shows the actual instruction encodings (byte sequences in
481 The third section is the *Resource pressure view*. This view reports
482 the average number of resource cycles consumed every iteration by instructions
483 for every processor resource unit available on the target. Information is
484 structured in two tables. The first table reports the number of resource cycles
485 spent on average every iteration. The second table correlates the resource
486 cycles to the machine instruction in the sequence. For example, every iteration
487 of the instruction vmulps always executes on resource unit [6]
488 (JFPU1 - floating point pipeline #1), consuming an average of 1 resource cycle
489 per iteration. Note that on AMD Jaguar, vector floating-point multiply can
490 only be issued to pipeline JFPU1, while horizontal floating-point additions can
491 only be issued to pipeline JFPU0.
493 The resource pressure view helps with identifying bottlenecks caused by high
494 usage of specific hardware resources. Situations with resource pressure mainly
495 concentrated on a few resources should, in general, be avoided. Ideally,
496 pressure should be uniformly distributed between multiple resources.
500 The timeline view produces a detailed report of each instruction's state
501 transitions through an instruction pipeline. This view is enabled by the
502 command line option ``-timeline``. As instructions transition through the
503 various stages of the pipeline, their states are depicted in the view report.
504 These states are represented by the following characters:
506 * D : Instruction dispatched.
507 * e : Instruction executing.
508 * E : Instruction executed.
509 * R : Instruction retired.
510 * = : Instruction already dispatched, waiting to be executed.
511 * \- : Instruction executed, waiting to be retired.
513 Below is the timeline view for a subset of the dot-product example located in
514 ``test/tools/llvm-mca/X86/BtVer2/dot-product.s`` and processed by
515 :program:`llvm-mca` using the following command:
519 $ llvm-mca -mtriple=x86_64-unknown-unknown -mcpu=btver2 -iterations=3 -timeline dot-product.s
527 [0,0] DeeER. . . vmulps %xmm0, %xmm1, %xmm2
528 [0,1] D==eeeER . . vhaddps %xmm2, %xmm2, %xmm3
529 [0,2] .D====eeeER . vhaddps %xmm3, %xmm3, %xmm4
530 [1,0] .DeeE-----R . vmulps %xmm0, %xmm1, %xmm2
531 [1,1] . D=eeeE---R . vhaddps %xmm2, %xmm2, %xmm3
532 [1,2] . D====eeeER . vhaddps %xmm3, %xmm3, %xmm4
533 [2,0] . DeeE-----R . vmulps %xmm0, %xmm1, %xmm2
534 [2,1] . D====eeeER . vhaddps %xmm2, %xmm2, %xmm3
535 [2,2] . D======eeeER vhaddps %xmm3, %xmm3, %xmm4
538 Average Wait times (based on the timeline view):
540 [1]: Average time spent waiting in a scheduler's queue
541 [2]: Average time spent waiting in a scheduler's queue while ready
542 [3]: Average time elapsed from WB until retire stage
545 0. 3 1.0 1.0 3.3 vmulps %xmm0, %xmm1, %xmm2
546 1. 3 3.3 0.7 1.0 vhaddps %xmm2, %xmm2, %xmm3
547 2. 3 5.7 0.0 0.0 vhaddps %xmm3, %xmm3, %xmm4
548 3 3.3 0.5 1.4 <total>
550 The timeline view is interesting because it shows instruction state changes
551 during execution. It also gives an idea of how the tool processes instructions
552 executed on the target, and how their timing information might be calculated.
554 The timeline view is structured in two tables. The first table shows
555 instructions changing state over time (measured in cycles); the second table
556 (named *Average Wait times*) reports useful timing statistics, which should
557 help diagnose performance bottlenecks caused by long data dependencies and
558 sub-optimal usage of hardware resources.
560 An instruction in the timeline view is identified by a pair of indices, where
561 the first index identifies an iteration, and the second index is the
562 instruction index (i.e., where it appears in the code sequence). Since this
563 example was generated using 3 iterations: ``-iterations=3``, the iteration
564 indices range from 0-2 inclusively.
566 Excluding the first and last column, the remaining columns are in cycles.
567 Cycles are numbered sequentially starting from 0.
569 From the example output above, we know the following:
571 * Instruction [1,0] was dispatched at cycle 1.
572 * Instruction [1,0] started executing at cycle 2.
573 * Instruction [1,0] reached the write back stage at cycle 4.
574 * Instruction [1,0] was retired at cycle 10.
576 Instruction [1,0] (i.e., vmulps from iteration #1) does not have to wait in the
577 scheduler's queue for the operands to become available. By the time vmulps is
578 dispatched, operands are already available, and pipeline JFPU1 is ready to
579 serve another instruction. So the instruction can be immediately issued on the
580 JFPU1 pipeline. That is demonstrated by the fact that the instruction only
581 spent 1cy in the scheduler's queue.
583 There is a gap of 5 cycles between the write-back stage and the retire event.
584 That is because instructions must retire in program order, so [1,0] has to wait
585 for [0,2] to be retired first (i.e., it has to wait until cycle 10).
587 In the example, all instructions are in a RAW (Read After Write) dependency
588 chain. Register %xmm2 written by vmulps is immediately used by the first
589 vhaddps, and register %xmm3 written by the first vhaddps is used by the second
590 vhaddps. Long data dependencies negatively impact the ILP (Instruction Level
593 In the dot-product example, there are anti-dependencies introduced by
594 instructions from different iterations. However, those dependencies can be
595 removed at register renaming stage (at the cost of allocating register aliases,
596 and therefore consuming physical registers).
598 Table *Average Wait times* helps diagnose performance issues that are caused by
599 the presence of long latency instructions and potentially long data dependencies
600 which may limit the ILP. Last row, ``<total>``, shows a global average over all
601 instructions measured. Note that :program:`llvm-mca`, by default, assumes at
602 least 1cy between the dispatch event and the issue event.
604 When the performance is limited by data dependencies and/or long latency
605 instructions, the number of cycles spent while in the *ready* state is expected
606 to be very small when compared with the total number of cycles spent in the
607 scheduler's queue. The difference between the two counters is a good indicator
608 of how large of an impact data dependencies had on the execution of the
609 instructions. When performance is mostly limited by the lack of hardware
610 resources, the delta between the two counters is small. However, the number of
611 cycles spent in the queue tends to be larger (i.e., more than 1-3cy),
612 especially when compared to other low latency instructions.
616 The ``-bottleneck-analysis`` command line option enables the analysis of
617 performance bottlenecks.
619 This analysis is potentially expensive. It attempts to correlate increases in
620 backend pressure (caused by pipeline resource pressure and data dependencies) to
621 dynamic dispatch stalls.
623 Below is an example of ``-bottleneck-analysis`` output generated by
624 :program:`llvm-mca` for 500 iterations of the dot-product example on btver2.
629 Cycles with backend pressure increase [ 48.07% ]
630 Throughput Bottlenecks:
631 Resource Pressure [ 47.77% ]
634 Data Dependencies: [ 0.30% ]
635 - Register Dependencies [ 0.30% ]
636 - Memory Dependencies [ 0.00% ]
638 Critical sequence based on the simulation:
640 Instruction Dependency Information
641 +----< 2. vhaddps %xmm3, %xmm3, %xmm4
645 | 0. vmulps %xmm0, %xmm1, %xmm2
646 +----> 1. vhaddps %xmm2, %xmm2, %xmm3 ## RESOURCE interference: JFPA [ probability: 74% ]
647 +----> 2. vhaddps %xmm3, %xmm3, %xmm4 ## REGISTER dependency: %xmm3
651 +----> 1. vhaddps %xmm2, %xmm2, %xmm3 ## RESOURCE interference: JFPA [ probability: 74% ]
654 According to the analysis, throughput is limited by resource pressure and not by
655 data dependencies. The analysis observed increases in backend pressure during
656 48.07% of the simulated run. Almost all those pressure increase events were
657 caused by contention on processor resources JFPA/JFPU0.
659 The `critical sequence` is the most expensive sequence of instructions according
660 to the simulation. It is annotated to provide extra information about critical
661 register dependencies and resource interferences between instructions.
663 Instructions from the critical sequence are expected to significantly impact
664 performance. By construction, the accuracy of this analysis is strongly
665 dependent on the simulation and (as always) by the quality of the processor
668 Bottleneck analysis is currently not supported for processors with an in-order
671 Extra Statistics to Further Diagnose Performance Issues
672 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
673 The ``-all-stats`` command line option enables extra statistics and performance
674 counters for the dispatch logic, the reorder buffer, the retire control unit,
675 and the register file.
677 Below is an example of ``-all-stats`` output generated by :program:`llvm-mca`
678 for 300 iterations of the dot-product example discussed in the previous
683 Dynamic Dispatch Stall Cycles:
684 RAT - Register unavailable: 0
685 RCU - Retire tokens unavailable: 0
686 SCHEDQ - Scheduler full: 272 (44.6%)
687 LQ - Load queue full: 0
688 SQ - Store queue full: 0
689 GROUP - Static restrictions on the dispatch group: 0
692 Dispatch Logic - number of cycles where we saw N micro opcodes dispatched:
693 [# dispatched], [# cycles]
699 Schedulers - number of cycles where we saw N micro opcodes issued:
700 [# issued], [# cycles]
705 Scheduler's queue usage:
707 [2] Average number of used buffer entries.
708 [3] Maximum number of used buffer entries.
709 [4] Total number of buffer entries.
717 Retire Control Unit - number of cycles where we saw N instructions retired:
718 [# retired], [# cycles]
723 Total ROB Entries: 64
724 Max Used ROB Entries: 35 ( 54.7% )
725 Average Used ROB Entries per cy: 32 ( 50.0% )
728 Register File statistics:
729 Total number of mappings created: 900
730 Max number of mappings used: 35
732 * Register File #1 -- JFpuPRF:
733 Number of physical registers: 72
734 Total number of mappings created: 900
735 Max number of mappings used: 35
737 * Register File #2 -- JIntegerPRF:
738 Number of physical registers: 64
739 Total number of mappings created: 0
740 Max number of mappings used: 0
742 If we look at the *Dynamic Dispatch Stall Cycles* table, we see the counter for
743 SCHEDQ reports 272 cycles. This counter is incremented every time the dispatch
744 logic is unable to dispatch a full group because the scheduler's queue is full.
746 Looking at the *Dispatch Logic* table, we see that the pipeline was only able to
747 dispatch two micro opcodes 51.5% of the time. The dispatch group was limited to
748 one micro opcode 44.6% of the cycles, which corresponds to 272 cycles. The
749 dispatch statistics are displayed by either using the command option
750 ``-all-stats`` or ``-dispatch-stats``.
752 The next table, *Schedulers*, presents a histogram displaying a count,
753 representing the number of micro opcodes issued on some number of cycles. In
754 this case, of the 610 simulated cycles, single opcodes were issued 306 times
755 (50.2%) and there were 7 cycles where no opcodes were issued.
757 The *Scheduler's queue usage* table shows that the average and maximum number of
758 buffer entries (i.e., scheduler queue entries) used at runtime. Resource JFPU01
759 reached its maximum (18 of 18 queue entries). Note that AMD Jaguar implements
762 * JALU01 - A scheduler for ALU instructions.
763 * JFPU01 - A scheduler floating point operations.
764 * JLSAGU - A scheduler for address generation.
766 The dot-product is a kernel of three floating point instructions (a vector
767 multiply followed by two horizontal adds). That explains why only the floating
768 point scheduler appears to be used.
770 A full scheduler queue is either caused by data dependency chains or by a
771 sub-optimal usage of hardware resources. Sometimes, resource pressure can be
772 mitigated by rewriting the kernel using different instructions that consume
773 different scheduler resources. Schedulers with a small queue are less resilient
774 to bottlenecks caused by the presence of long data dependencies. The scheduler
775 statistics are displayed by using the command option ``-all-stats`` or
776 ``-scheduler-stats``.
778 The next table, *Retire Control Unit*, presents a histogram displaying a count,
779 representing the number of instructions retired on some number of cycles. In
780 this case, of the 610 simulated cycles, two instructions were retired during the
781 same cycle 399 times (65.4%) and there were 109 cycles where no instructions
782 were retired. The retire statistics are displayed by using the command option
783 ``-all-stats`` or ``-retire-stats``.
785 The last table presented is *Register File statistics*. Each physical register
786 file (PRF) used by the pipeline is presented in this table. In the case of AMD
787 Jaguar, there are two register files, one for floating-point registers (JFpuPRF)
788 and one for integer registers (JIntegerPRF). The table shows that of the 900
789 instructions processed, there were 900 mappings created. Since this dot-product
790 example utilized only floating point registers, the JFPuPRF was responsible for
791 creating the 900 mappings. However, we see that the pipeline only used a
792 maximum of 35 of 72 available register slots at any given time. We can conclude
793 that the floating point PRF was the only register file used for the example, and
794 that it was never resource constrained. The register file statistics are
795 displayed by using the command option ``-all-stats`` or
796 ``-register-file-stats``.
798 In this example, we can conclude that the IPC is mostly limited by data
799 dependencies, and not by resource pressure.
803 This section describes the instruction flow through the default pipeline of
804 :program:`llvm-mca`, as well as the functional units involved in the process.
806 The default pipeline implements the following sequence of stages used to
807 process instructions.
809 * Dispatch (Instruction is dispatched to the schedulers).
810 * Issue (Instruction is issued to the processor pipelines).
811 * Write Back (Instruction is executed, and results are written back).
812 * Retire (Instruction is retired; writes are architecturally committed).
814 The in-order pipeline implements the following sequence of stages:
815 * InOrderIssue (Instruction is issued to the processor pipelines).
816 * Retire (Instruction is retired; writes are architecturally committed).
818 :program:`llvm-mca` assumes that instructions have all been decoded and placed
819 into a queue before the simulation start. Therefore, the instruction fetch and
820 decode stages are not modeled. Performance bottlenecks in the frontend are not
821 diagnosed. Also, :program:`llvm-mca` does not model branch prediction.
825 During the dispatch stage, instructions are picked in program order from a
826 queue of already decoded instructions, and dispatched in groups to the
827 simulated hardware schedulers.
829 The size of a dispatch group depends on the availability of the simulated
830 hardware resources. The processor dispatch width defaults to the value
831 of the ``IssueWidth`` in LLVM's scheduling model.
833 An instruction can be dispatched if:
835 * The size of the dispatch group is smaller than processor's dispatch width.
836 * There are enough entries in the reorder buffer.
837 * There are enough physical registers to do register renaming.
838 * The schedulers are not full.
840 Scheduling models can optionally specify which register files are available on
841 the processor. :program:`llvm-mca` uses that information to initialize register
842 file descriptors. Users can limit the number of physical registers that are
843 globally available for register renaming by using the command option
844 ``-register-file-size``. A value of zero for this option means *unbounded*. By
845 knowing how many registers are available for renaming, the tool can predict
846 dispatch stalls caused by the lack of physical registers.
848 The number of reorder buffer entries consumed by an instruction depends on the
849 number of micro-opcodes specified for that instruction by the target scheduling
850 model. The reorder buffer is responsible for tracking the progress of
851 instructions that are "in-flight", and retiring them in program order. The
852 number of entries in the reorder buffer defaults to the value specified by field
853 `MicroOpBufferSize` in the target scheduling model.
855 Instructions that are dispatched to the schedulers consume scheduler buffer
856 entries. :program:`llvm-mca` queries the scheduling model to determine the set
857 of buffered resources consumed by an instruction. Buffered resources are
858 treated like scheduler resources.
862 Each processor scheduler implements a buffer of instructions. An instruction
863 has to wait in the scheduler's buffer until input register operands become
864 available. Only at that point, does the instruction becomes eligible for
865 execution and may be issued (potentially out-of-order) for execution.
866 Instruction latencies are computed by :program:`llvm-mca` with the help of the
869 :program:`llvm-mca`'s scheduler is designed to simulate multiple processor
870 schedulers. The scheduler is responsible for tracking data dependencies, and
871 dynamically selecting which processor resources are consumed by instructions.
872 It delegates the management of processor resource units and resource groups to a
873 resource manager. The resource manager is responsible for selecting resource
874 units that are consumed by instructions. For example, if an instruction
875 consumes 1cy of a resource group, the resource manager selects one of the
876 available units from the group; by default, the resource manager uses a
877 round-robin selector to guarantee that resource usage is uniformly distributed
878 between all units of a group.
880 :program:`llvm-mca`'s scheduler internally groups instructions into three sets:
882 * WaitSet: a set of instructions whose operands are not ready.
883 * ReadySet: a set of instructions ready to execute.
884 * IssuedSet: a set of instructions executing.
886 Depending on the operands availability, instructions that are dispatched to the
887 scheduler are either placed into the WaitSet or into the ReadySet.
889 Every cycle, the scheduler checks if instructions can be moved from the WaitSet
890 to the ReadySet, and if instructions from the ReadySet can be issued to the
891 underlying pipelines. The algorithm prioritizes older instructions over younger
894 Write-Back and Retire Stage
895 """""""""""""""""""""""""""
896 Issued instructions are moved from the ReadySet to the IssuedSet. There,
897 instructions wait until they reach the write-back stage. At that point, they
898 get removed from the queue and the retire control unit is notified.
900 When instructions are executed, the retire control unit flags the instruction as
903 Instructions are retired in program order. The register file is notified of the
904 retirement so that it can free the physical registers that were allocated for
905 the instruction during the register renaming stage.
907 Load/Store Unit and Memory Consistency Model
908 """"""""""""""""""""""""""""""""""""""""""""
909 To simulate an out-of-order execution of memory operations, :program:`llvm-mca`
910 utilizes a simulated load/store unit (LSUnit) to simulate the speculative
911 execution of loads and stores.
913 Each load (or store) consumes an entry in the load (or store) queue. Users can
914 specify flags ``-lqueue`` and ``-squeue`` to limit the number of entries in the
915 load and store queues respectively. The queues are unbounded by default.
917 The LSUnit implements a relaxed consistency model for memory loads and stores.
920 1. A younger load is allowed to pass an older load only if there are no
921 intervening stores or barriers between the two loads.
922 2. A younger load is allowed to pass an older store provided that the load does
923 not alias with the store.
924 3. A younger store is not allowed to pass an older store.
925 4. A younger store is not allowed to pass an older load.
927 By default, the LSUnit optimistically assumes that loads do not alias
928 (`-noalias=true`) store operations. Under this assumption, younger loads are
929 always allowed to pass older stores. Essentially, the LSUnit does not attempt
930 to run any alias analysis to predict when loads and stores do not alias with
933 Note that, in the case of write-combining memory, rule 3 could be relaxed to
934 allow reordering of non-aliasing store operations. That being said, at the
935 moment, there is no way to further relax the memory model (``-noalias`` is the
936 only option). Essentially, there is no option to specify a different memory
937 type (e.g., write-back, write-combining, write-through; etc.) and consequently
938 to weaken, or strengthen, the memory model.
940 Other limitations are:
942 * The LSUnit does not know when store-to-load forwarding may occur.
943 * The LSUnit does not know anything about cache hierarchy and memory types.
944 * The LSUnit does not know how to identify serializing operations and memory
947 The LSUnit does not attempt to predict if a load or store hits or misses the L1
948 cache. It only knows if an instruction "MayLoad" and/or "MayStore." For
949 loads, the scheduling model provides an "optimistic" load-to-use latency (which
950 usually matches the load-to-use latency for when there is a hit in the L1D).
952 :program:`llvm-mca` does not know about serializing operations or memory-barrier
953 like instructions. The LSUnit conservatively assumes that an instruction which
954 has both "MayLoad" and unmodeled side effects behaves like a "soft"
955 load-barrier. That means, it serializes loads without forcing a flush of the
956 load queue. Similarly, instructions that "MayStore" and have unmodeled side
957 effects are treated like store barriers. A full memory barrier is a "MayLoad"
958 and "MayStore" instruction with unmodeled side effects. This is inaccurate, but
959 it is the best that we can do at the moment with the current information
962 A load/store barrier consumes one entry of the load/store queue. A load/store
963 barrier enforces ordering of loads/stores. A younger load cannot pass a load
964 barrier. Also, a younger store cannot pass a store barrier. A younger load
965 has to wait for the memory/load barrier to execute. A load/store barrier is
966 "executed" when it becomes the oldest entry in the load/store queue(s). That
967 also means, by construction, all of the older loads/stores have been executed.
969 In conclusion, the full set of load/store consistency rules are:
971 #. A store may not pass a previous store.
972 #. A store may not pass a previous load (regardless of ``-noalias``).
973 #. A store has to wait until an older store barrier is fully executed.
974 #. A load may pass a previous load.
975 #. A load may not pass a previous store unless ``-noalias`` is set.
976 #. A load has to wait until an older load barrier is fully executed.
978 In-order Issue and Execute
979 """"""""""""""""""""""""""""""""""""
980 In-order processors are modelled as a single ``InOrderIssueStage`` stage. It
981 bypasses Dispatch, Scheduler and Load/Store unit. Instructions are issued as
982 soon as their operand registers are available and resource requirements are
983 met. Multiple instructions can be issued in one cycle according to the value of
984 the ``IssueWidth`` parameter in LLVM's scheduling model.
986 Once issued, an instruction is moved to ``IssuedInst`` set until it is ready to
987 retire. :program:`llvm-mca` ensures that writes are committed in-order. However,
988 an instruction is allowed to commit writes and retire out-of-order if
989 ``RetireOOO`` property is true for at least one of its writes.
992 """"""""""""""""""""""""""""""""""""
993 Due to certain instructions not being expressed perfectly within their
994 scheduling model, :program:`llvm-mca` isn't always able to simulate them
995 perfectly. Modifying the scheduling model isn't always a viable
996 option though (maybe because the instruction is modeled incorrectly on
997 purpose or the instruction's behaviour is quite complex). The
998 CustomBehaviour class can be used in these cases to enforce proper
999 instruction modeling (often by customizing data dependencies and detecting
1000 hazards that :program:`llvm-mca` has no way of knowing about).
1002 :program:`llvm-mca` comes with one generic and multiple target specific
1003 CustomBehaviour classes. The generic class will be used if the ``-disable-cb``
1004 flag is used or if a target specific CustomBehaviour class doesn't exist for
1005 that target. (The generic class does nothing.) Currently, the CustomBehaviour
1006 class is only a part of the in-order pipeline, but there are plans to add it
1007 to the out-of-order pipeline in the future.
1009 CustomBehaviour's main method is `checkCustomHazard()` which uses the
1010 current instruction and a list of all instructions still executing within
1011 the pipeline to determine if the current instruction should be dispatched.
1012 As output, the method returns an integer representing the number of cycles
1013 that the current instruction must stall for (this can be an underestimate
1014 if you don't know the exact number and a value of 0 represents no stall).
1016 If you'd like to add a CustomBehaviour class for a target that doesn't
1017 already have one, refer to an existing implementation to see how to set it
1018 up. The classes are implemented within the target specific backend (for
1019 example `/llvm/lib/Target/AMDGPU/MCA/`) so that they can access backend symbols.
1022 """"""""""""""""""""""""""""""""""""
1023 :program:`llvm-mca` comes with several Views such as the Timeline View and
1024 Summary View. These Views are generic and can work with most (if not all)
1025 targets. If you wish to add a new View to :program:`llvm-mca` and it does not
1026 require any backend functionality that is not already exposed through MC layer
1027 classes (MCSubtargetInfo, MCInstrInfo, etc.), please add it to the
1028 `/tools/llvm-mca/View/` directory. However, if your new View is target specific
1029 AND requires unexposed backend symbols or functionality, you can define it in
1030 the `/lib/Target/<TargetName>/MCA/` directory.
1032 To enable this target specific View, you will have to use this target's
1033 CustomBehaviour class to override the `CustomBehaviour::getViews()` methods.
1034 There are 3 variations of these methods based on where you want your View to
1035 appear in the output: `getStartViews()`, `getPostInstrInfoViews()`, and
1036 `getEndViews()`. These methods returns a vector of Views so you will want to
1037 return a vector containing all of the target specific Views for the target in
1040 Because these target specific (and backend dependent) Views require the
1041 `CustomBehaviour::getViews()` variants, these Views will not be enabled if
1042 the `-disable-cb` flag is used.
1044 Enabling these custom Views does not affect the non-custom (generic) Views.
1045 Continue to use the usual command line arguments to enable / disable those