--- /dev/null
+JIT Optimizer Planning Guide
+============================
+
+The goal of this document is to capture some thinking about the process used to
+prioritize and validate optimizer investments. The overriding goal of such
+investments is to help ensure that the dotnet platform satisfies developers'
+performance needs.
+
+
+Benchmarking
+------------
+
+There are a number of public benchmarks which evaluate different platforms'
+relative performance, so naturally dotnet's scores on such benchmarks give
+some indication of how well it satisfies developers' performance needs. The JIT
+team has used some of these benchmarks, particularly [TechEmpower](https://www.techempower.com/benchmarks/)
+and [Benchmarks Game](http://benchmarksgame.alioth.debian.org/), for scouting
+out optimization opportunities and prioritizing optimization improvements.
+While it is important to track scores on such benchmarks to validate performance
+changes in the dotnet platform as a whole, when it comes to planning and
+prioritizing JIT optimization improvements specifically, they aren't sufficient,
+due to a few well-known issues:
+
+ - For macro-benchmarks, such as TechEmpower, compiler optimization is often not
+ the dominant factor in performance. The effects of individual optimizer
+ changes are most often in the sub-percent range, well below the noise level
+ of the measurements, which will usually be at least 3% or so even for the
+ most well-behaved macro-benchmarks.
+ - Source-level changes can be made much more rapidly than compiler optimization
+ changes. This means that for anything we're trying to track where the whole
+ team is effecting changes in source, runtime, etc., any particular code
+ sequence we may target with optimization improvements may well be targeted
+ with source changes in the interim, nullifying the measured benefit of the
+ optimization change when it is eventually merged. Source/library/runtime
+ changes are in play for TechEmpower and Benchmarks Game both.
+
+Compiler micro-benchmarks (like those in our [test tree](https://github.com/dotnet/coreclr/tree/master/tests/src/JIT/Performance/CodeQuality))
+don't share these issues, and adding them as optimizations are implemented is
+critical for validation and regression prevention; however, micro-benchmarks
+often aren't as representative of real-world code, and therefore not as
+reflective of developers' performance needs, so aren't well suited for scouting
+out and prioritizing opportunities.
+
+
+Benefits of JIT Optimization
+----------------------------
+
+While source changes can more rapidly and dramatically effect changes to
+targeted hot code sequences in macro-benchmarks, compiler changes have the
+advantage that they apply broadly to all compiled code. One of the best reasons
+to invest in compiler optimization improvements is to capitalize on this. A few
+specific benefits:
+
+ - Optimizer changes can effect "peanut-butter" improvements; by making an
+ improvement which is small in any particular instance to a code sequence that
+ is repeated thousands of times across a codebase, they can produce substantial
+ cumulative wins. These should accrue toward the standard metrics (benchmark
+ scores and code size), but identifying the most profitable "peanut-butter"
+ opportunities is difficult. Improving our methodology for identifying such
+ opportunities would be helpful; some ideas are below.
+ - Optimizer changes can unblock coding patterns that performance-sensitive
+ developers want to employ but consider prohibitively expensive. They may
+ have inelegant works-around in their code, such as gotos for loop-exiting
+ returns to work around poor block layout, manually scalarized structs to work
+ around poor struct promotion, manually unrolled loops to work around lack of
+ loop unrolling, limited use of lambdas to work around inefficient access to
+ heap-allocated closures, etc. The more the optimizer can improve such
+ situations, the better, as it both increases developer productivity and
+ increases the usefulness of abstractions provided by the language and
+ libraries. Finding a measurable metric to track this type of improvement
+ poses a challenge, but would be a big help toward prioritizing and validating
+ optimization improvements; again, some ideas are below.
+
+
+Brainstorm
+----------
+
+Listed here are several ideas for undertakings we might pursue to improve our
+ability to identify opportunities and validate/track improvements that mesh
+with the benefits discussed above. Thinking here is in the early stages, but
+the hope is that with some thought/discussion some of these will surface as
+worth investing in.
+
+ - Is there telemetry we can implement/analyze to identify "peanut-butter"
+ opportunities, or target "coding pattern"s? Probably easier to use this
+ to evaluate/prioritize patterns we're considering targeting than to identify
+ the patterns in the first place.
+ - Can we construct some sort of "peanut-butter profiler"? The idea would
+ roughly be to aggregate samples/counters under particular input constructs
+ rather than aggregate them under callstack. Might it be interesting to
+ group by MSIL opcode, or opcode pair, or opcode triplet... ?
+ - It might behoove us to build up some SPMI traces that could be data-mined
+ for any of these experiments.
+ - We should make it easy to view machine code emitted by the jit, and to
+ collect profiles and correlate them with that machine code. This could
+ benefit any developers doing performance analysis of their own code.
+ The JIT team has discussed this, options include building something on top of
+ the profiler APIs, enabling COMPlus_JitDisasm in release builds, and shipping
+ with or making easily available an alt jit that supports JitDisasm.
+ - Hardware companies maintain optimization/performance guides for their ISAs.
+ Should we maintain one for MSIL and/or C# (and/or F#)? If we hosted such a
+ thing somewhere publicly votable, we could track which anti-patterns people
+ find most frustrating to avoid, and subsequent removal of them. Does such
+ a guide already exist somewhere, that we could use as a starting point?
+ Should we collate GitHub issues or Stack Overflow issues to create such a thing?
+ - Maybe we should expand our labels on GitHub so that there are sub-areas
+ within "optimization"? It could help prioritize by letting us compare the
+ relative sizes of those buckets.
+ - Can we more effectively leverage the legacy JIT codebases for comparative
+ analysis? We've compared micro-benchmark performance against Jit64 and
+ manually compared disassembly of hot code, what else can we do? One concrete
+ idea: run over some large corpus of code (SPMI?), and do a path-length
+ comparison e.g. by looking at each sequence of k MSIL instructions (for some
+ small k), and for each combination of k opcodes collect statistics on the
+ size of generated machine code (maybe using debug line number info to do the
+ correlation?), then look for common sequences which are much longer with
+ RyuJIT.
+ - Maybe hook RyuJIT up to some sort of superoptimizer to identify opportunities?
+ - Microsoft Research has done some experimenting that involved converting RyuJIT
+ IR to LLVM IR; perhaps we could use this to identify common expressions that
+ could be much better optimized.
+ - What's a practical way to establish a metric of "unblocked coding patterns"?
+ - How developers give feedback about patterns/performance could use some thought;
+ the GitHub issue list is open, but does it need to be publicized somehow? We
+ perhaps should have some regular process where we pull issues over from other
+ places where people report/discuss dotnet performance issues, like
+ [Stack Overflow](https://stackoverflow.com/questions/tagged/performance+.net).
--- /dev/null
+Optimizer Codebase Status/Investments
+=====================================
+
+There are a number of areas in the optimizer that we know we would invest in
+improving if resources were unlimited. This document lists them and some
+thoughts about their current state and prioritization, in an effort to capture
+the thinking about them that comes up in planning discussions.
+
+
+Improved Struct Handling
+------------------------
+
+This is an area that has received recent attention, with the [first-class structs](https://github.com/dotnet/coreclr/blob/master/Documentation/design-docs/first-class-structs.md)
+work and the struct promotion improvements that went in for `Span<T>`. Work here
+is expected to continue and can happen incrementally. Possible next steps:
+
+ - Struct promotion stress mode (test mode to improve robustness/reliability)
+ - Promotion of more structs; relax limits on e.g. field count (should generally
+ help performance-sensitive code where structs are increasingly used to avoid
+ heap allocations)
+ - Improve handling of System V struct passing (I think we currently insert
+ some unnecessary round-trips through memory at call boundaries due to
+ internal representation issues)
+ - Implicit byref parameter promotion w/o shadow copy
+
+We don't have specific benchmarks that we know would jump in response to any of
+these. May well be able to find some with some looking, though this may be an
+area where current performance-sensitive code avoids structs.
+
+
+Exception handling
+------------------
+
+This is increasingly important as C# language constructs like async/await and
+certain `foreach` incantations are implemented with EH constructs, making them
+difficult to avoid at source level. The recent work on finally cloning, empty
+finally removal, and empty try removal targeted this. [Writethrough](https://github.com/dotnet/coreclr/blob/master/Documentation/design-docs/eh-writethru.md)
+is another key optimization enabler here, and we are actively pursuing it. Other
+things we've discussed include inlining methods with EH and computing funclet
+callee-save register usage independently of main function callee-save register
+usage, but I don't think we have any particular data pointing to either as a
+high priority.
+
+
+Loop Optimizations
+------------------
+
+We haven't been targeting benchmarks that spend a lot of time doing compuations
+in an inner loop. Pursuing loop optimizations for the peanut butter effect
+would seem odd. So this simply hasn't bubbled up in priority yet, though it's
+bound to eventually.
+
+
+More Expression Optimizations
+-----------------------------
+
+We again don't have particular benchmarks pointing to key missing cases, and
+balancing the CQ vs TP will be delicate here, so it would really help to have
+an appropriate benchmark suite to evaluate this work against.
+
+
+Forward Substitution
+--------------------
+
+This too needs an appropriate benchmark suite that I don't think we have at
+this time. The tradeoffs against register pressure increase and throughput
+need to be evaluated. This also might make more sense to do if/when we can
+handle SSA renames.
+
+
+Value Number Conservativism
+---------------------------
+
+We have some frustrating phase-ordering issues resulting from this, but the
+opt-repeat experiment indicated that they're not prevalent enough to merit
+pursuing changing this right now. Also, using SSA def as the proxy for value
+number would require handling SSA renaming, so there's a big dependency chained
+to this.
+Maybe it's worth reconsidering the priority based on throughput?
+
+
+High Tier Optimizations
+-----------------------
+
+We don't have that many knobs we can "crank up" (though we do have the tracked
+assertion count and could switch inliner policies), nor do we have any sort of
+benchmarking story set up to validate whether tiering changes are helping or
+hurting. We should get that benchmarking story sorted out and at least hook
+up those two knobs.
+
+
+Low Tier Back-Off
+-----------------
+
+We have some changes we know we want to make here: morph does more than it needs
+to in minopts, and tier 0 should be doing throughput-improving inlines, as
+opposed to minopts which does no inlining. It would be nice to have the
+benchmarking story set up to measure the effect of such changes when they go in,
+we should do that.
+
+
+Async
+-----
+
+We've made note of the prevalence of async/await in modern code (and particularly
+in web server code such as TechEmpower), and have some opportunities listed in
+[#7914](https://github.com/dotnet/coreclr/issues/7914). Some sort of study of
+async peanut butter to find more opportunities is probably in order, but what
+would that look like?
+
+
+Address Mode Building
+---------------------
+
+One opportunity that's frequently visible in asm dumps is that more address
+expressions could be folded into memory operands' address expressions. This
+would likely give a measurable codesize win. Needs some thought about where
+to run in phase list and how aggressive to be about e.g. analyzing across
+statements.
+
+
+If-Conversion (cmov formation)
+------------------------------
+
+This hits big in microbenchmarks where it hits. There's some work in flight
+on this (see #7447 and #10861).
+
+
+Mulshift
+--------
+
+Replacing multiplication by constants with shift/add/lea sequences is a
+classic optimization that keeps coming up in planning. An [analysis](https://gist.github.com/JosephTremoulet/c1246b17ea2803e93e203b9969ee5a25#file-mulshift-md)
+indicates that RyuJIT is already capitalizing on most of the opportunity here.
projects / platform / upstream / coreclr.git / commitdiff