From c6a5534ea4f8d0245880ae6444a973758e46826d Mon Sep 17 00:00:00 2001 From: Mehdi Amini Date: Tue, 24 Dec 2019 00:53:10 -0800 Subject: [PATCH] Remove static MLIR doc ; they are already on the website --- mlir/docs/DeveloperGuide.md | 107 --------------------------- mlir/docs/Glossary.md | 174 -------------------------------------------- mlir/docs/TestingGuide.md | 171 ------------------------------------------- 3 files changed, 452 deletions(-) delete mode 100644 mlir/docs/DeveloperGuide.md delete mode 100644 mlir/docs/Glossary.md delete mode 100644 mlir/docs/TestingGuide.md diff --git a/mlir/docs/DeveloperGuide.md b/mlir/docs/DeveloperGuide.md deleted file mode 100644 index 7450099..0000000 --- a/mlir/docs/DeveloperGuide.md +++ /dev/null @@ -1,107 +0,0 @@ -# Developer Guide - -This document attempts to describe a few developer policies used in MLIR (such -as coding standards used) as well as development approach (such as, testing -methods). - -## Style guide - -MLIR follows the [LLVM style](https://llvm.org/docs/CodingStandards.html) guide. -We also adhere to the following (which deviate from or are not specified in the -LLVM style guide): - -* Adopts [camelBack](https://llvm.org/docs/Proposals/VariableNames.html); -* Except for IR units (Region, Block, and Operation), non-nullable output - arguments are passed by non-const reference in general. -* IR constructs are not designed for [const correctness](UsageOfConst.md). -* Do *not* use recursive algorithms if the recursion can't be bounded - statically: that is avoid recursion if there is a possible IR input that can - trigger a stack overflow (for example traversing use-def chains in a - recursive way). At the moment, we tolerate it for the two following cases: - * The nesting of the IR: we use recursion when traversing nested regions. - * Type nesting: recursion may be used for the nesting of composite types. -* Follow the `git` conventions for writing a commit message, in particular the - first line is the "title", it should be followed by an empty line and an - optional description. This [post](https://chris.beams.io/posts/git-commit/) - give examples and more details. - -Please run clang-format on the files you modified with the `.clang-format` -configuration file available in the root directory. Check the clang-format -[documentation](https://clang.llvm.org/docs/ClangFormat.html) for more details -on integrating it with your development environment. In particular, if clang is -installed system-wide, running `git clang-format origin/master` will update the -files in the working directory with the relevant formatting changes; don't -forget to include those to the commit. - -## Pass name and other command line options - -To avoid collision between options provided by different dialects, the naming -convention is to prepend the dialect name to every dialect-specific passes and -options in general. Options that are specific to a pass should also be prefixed -with the pass name. For example, the affine dialect provides a loop tiling pass -that is registered on the command line as `-affine-tile`, and with a tile size -option that can be set with `-affine-tile-size`. - -We also avoid `cl::opt` to provide pass options in favor of the -[pass options](WritingAPass.md#instance-specific-pass-options) mechanism. This -allows for these options to be serialized in a pass pipeline description, as -well as passing different options to multiple instances of a pass in the same -pipeline. - -## Testing guidelines - -See here for the [testing guide](TestingGuide.md). - -## Guidelines on contributing a new dialect (or important components) - -To contribute a dialect (or a major component in MLIR), it is usual to write an -overview "RFC" (it can be just a few informal paragraphs) and send it to the -MLIR mailing list. When accepting a new component to MLIR, the community is also -accepting the burden of maintaining it. The following points should be -considered when evaluating whether a dialect is a good fit for the core MLIR -repository: - -* What is the overall goal of the dialect? What is the first implementation - milestone? -* How does it fit into the MLIR dialect ecosystem? - * Connection: how does it connect to the existing dialects in a - compilation pipeline(s)? - * Consolidation: is there already a dialect with a similar goal or - matching abstractions; if so, can it be improved instead of adding a new - one? - * Reuse: how does it generalize to similar but slightly different - use-cases? -* What is the community of users that it is serving? -* Who are the future contributors/maintainers beyond those who propose the - dialect? - -On a practical aspect, we will expect the code to follow the other sections of -this document, with an emphasis on the documentation alongside the source code. - -It is prefered to upstream your dialects/components in small incremental patches -that can be individually reviewed. That is, after the initial RFC has been -agreed on, we encourage dialects to be built progressively by faster iterations -in-tree; as long as it is clear they evolve towards their milestones and goals. - -We have seen the following broad categories of dialects: - -* Edge dialects that model a representation external to MLIR. Examples include - LLVM, SPIR-V dialects, TensorFlow, XLA/HLO, ... Such dialects may be a - better fit for the project that contains the original representation instead - of being added to the MLIR repository. In particular, because MLIR will not - take an external dependency on another project. -* Structured Abstraction dialects that generalize common features of several - other dialects or introduce a programming model. Generalization is sometimes - demonstrated by having several dialects lower to or originate from a new - dialect. While additional abstractions may be useful, they should be traded - off against the additional complexity of the dialect ecosystem. Examples of - abstraction dialects include the GPU and Loop dialects. -* Transformation dialects that serve as input/output for program - transformations. These dialects are commonly introduced to materialize - transformation pre- and post-conditions in the IR, while conditions can be - obtained through analysis or through operation semantics. Examples include - Affine and Linalg dialects. - -While it can be useful to frame the goals of a proposal, this categorization is -not exhaustive or absolute, and the community is open to discussing any new -dialect beyond this taxonomy. diff --git a/mlir/docs/Glossary.md b/mlir/docs/Glossary.md deleted file mode 100644 index 542d375..0000000 --- a/mlir/docs/Glossary.md +++ /dev/null @@ -1,174 +0,0 @@ -# MLIR Glossary - -This glossary contains definitions of MLIR-specific terminology. It is intended -to be a quick reference document. For terms which are well-documented elsewhere, -definitions are kept brief and the header links to the more in-depth -documentation. - - - -#### [Block](LangRef.md#blocks) - -A sequential list of operations without control flow. - -Also called a [basic block](https://en.wikipedia.org/wiki/Basic_block). - -#### Conversion - -The transformation of code represented in one dialect into a semantically -equivalent representation in another dialect (i.e. inter-dialect conversion) or -the same dialect (i.e. intra-dialect conversion). - -In the context of MLIR, conversion is distinct from [translation](#translation). -Conversion refers to a transformation between (or within) dialects, but all -still within MLIR, whereas translation refers to a transformation between MLIR -and an external representation. - -#### [Declarative Rewrite Rule](DeclarativeRewrites.md) (DRR) - -A [rewrite rule](https://en.wikipedia.org/wiki/Graph_rewriting) which can be -defined declaratively (e.g. through specification in a -[TableGen](https://llvm.org/docs/TableGen/) record). At compiler build time, -these rules are expanded into an equivalent `mlir::RewritePattern` subclass. - -#### [Dialect](LangRef.md#dialects) - -A dialect is a grouping of functionality which can be used to extend the MLIR -system. - -A dialect creates a unique `namespace` within which new -[operations](#operation-op), [attributes](LangRef.md#attributes), and -[types](LangRef.md#type-system) are defined. This is the fundamental method by -which to extend MLIR. - -In this way, MLIR is a meta-IR: its extensible framework allows it to be -leveraged in many different ways (e.g. at different levels of the compilation -process). Dialects provide an abstraction for the different uses of MLIR while -recognizing that they are all a part of the meta-IR that is MLIR. - -The tutorial provides an example of -[interfacing with MLIR](Tutorials/Toy/Ch-2.md#interfacing-with-mlir) in this -way. - -(Note that we have intentionally selected the term "dialect" instead of -"language", as the latter would wrongly suggest that these different namespaces -define entirely distinct IRs.) - -#### Export - -To transform code represented in MLIR into a semantically equivalent -representation which is external to MLIR. - -The tool that performs such a transformation is called an exporter. - -See also: [translation](#translation). - -#### [Function](LangRef.md#functions) - -An [operation](#operation-op) with a name containing one [region](#region). - -The region of a function is not allowed to implicitly capture values defined -outside of the function, and all external references must use function arguments -or attributes that establish a symbolic connection. - -#### Import - -To transform code represented in an external representation into a semantically -equivalent representation in MLIR. - -The tool that performs such a transformation is called an importer. - -See also: [translation](#translation). - -#### Legalization - -The process of transforming operations into a semantically equivalent -representation which adheres to the requirements set by the -[conversion target](DialectConversion.md#conversion-target). - -That is, legalization is accomplished if and only if the new representation -contains only operations which are legal, as specified in the conversion target. - -#### Lowering - -The process of transforming a higher-level representation of an operation into a -lower-level, but semantically equivalent, representation. - -In MLIR, this is typically accomplished through -[dialect conversion](DialectConversion.md). This provides a framework by which -to define the requirements of the lower-level representation, called the -[conversion target](DialectConversion.md#conversion-target), by specifying which -operations are legal versus illegal after lowering. - -See also: [legalization](#legalization). - -#### [Module](LangRef.md#module) - -An [operation](#operation-op) which contains a single region containing a single -block that is comprised of operations. - -This provides an organizational structure for MLIR operations, and is the -expected top-level operation in the IR: the textual parser returns a Module. - -#### [Operation](LangRef.md#operations) (op) - -A unit of code in MLIR. Operations are the building blocks for all code and -computations represented by MLIR. They are fully extensible (there is no fixed -list of operations) and have application-specific semantics. - -An operation can have zero or more [regions](#region). Note that this creates a -nested IR structure, as regions consist of blocks, which in turn, consist of a -list of operations. - -In MLIR, there are two main classes related to operations: `Operation` and `Op`. -Operation is the actual opaque instance of the operation, and represents the -general API into an operation instance. An `Op` is the base class of a derived -operation, like `ConstantOp`, and acts as smart pointer wrapper around a -`Operation*` - -#### [Region](LangRef.md#regions) - -A [CFG](https://en.wikipedia.org/wiki/Control-flow_graph) of MLIR -[blocks](#block). - -#### Round-trip - -The process of converting from a source format to a target format and then back -to the source format. - -This is a good way of gaining confidence that the target format richly models -the source format. This is particularly relevant in the MLIR context, since -MLIR's multi-level nature allows for easily writing target dialects that model a -source format (such as TensorFlow GraphDef or another non-MLIR format) -faithfully and have a simple conversion procedure. Further cleanup/lowering can -be done entirely within the MLIR representation. This separation - making the -[importer](#import) as simple as possible and performing all further -cleanups/lowering in MLIR - has proven to be a useful design pattern. - -#### [Terminator operation](LangRef.md#terminator-operations) - -An [operation](#operation-op) which *must* terminate a [block](#block). -Terminator operations are a special category of operations. - -#### Transitive lowering - -An A->B->C [lowering](#lowering); that is, a lowering in which multiple patterns -may be applied in order to fully transform an illegal operation into a set of -legal ones. - -This provides the flexibility that the [conversion](#conversion) framework may -perform the lowering in multiple stages of applying patterns (which may utilize -intermediate patterns not in the conversion target) in order to fully legalize -an operation. This is accomplished through -[partial conversion](DialectConversion.md#modes-of-conversion). - -#### Translation - -The transformation of code represented in an external (non-MLIR) representation -into a semantically equivalent representation in MLIR (i.e. -[importing](#import)), or the inverse (i.e. [exporting](#export)). - -In the context of MLIR, translation is distinct from [conversion](#conversion). -Translation refers to a transformation between MLIR and an external -representation, whereas conversion refers to a transformation within MLIR -(between or within dialects). diff --git a/mlir/docs/TestingGuide.md b/mlir/docs/TestingGuide.md deleted file mode 100644 index 723b78b..0000000 --- a/mlir/docs/TestingGuide.md +++ /dev/null @@ -1,171 +0,0 @@ -# Testing Guide - -Testing is an integral part of any software infrastructure. In general, all -commits to the MLIR repository should include an accompanying test of some form. -Commits that include no functional changes, such as API changes like symbol -renaming, should be tagged with NFC(no functional changes). This signals to the -reviewer why the change doesn't/shouldn't include a test. - -MLIR generally separates testing into two main categories, [Check](#check-tests) -tests and [Unit](#unit-tests) tests. - -## Check tests - -Check tests are tests that verify that some set of string tags appear in the -output of some program. These tests generally encompass anything related to the -state of the IR (and more); analysis, parsing, transformation, verification, -etc. They are written utilizing several different tools: - -### FileCheck tests - -[FileCheck](https://llvm.org/docs/CommandGuide/FileCheck.html) is a utility tool -that "reads two files (one from standard input, and one specified on the command -line) and uses one to verify the other." Essentially, one file contains a set of -tags that are expected to appear in the output file. MLIR utilizes FileCheck, in -combination with [lit](https://llvm.org/docs/CommandGuide/lit.html), to verify -different aspects of the IR - such as the output of a transformation pass. - -An example FileCheck test is shown below: - -```mlir -// RUN: mlir-opt %s -cse | FileCheck %s - -// CHECK-LABEL: func @simple_constant -func @simple_constant() -> (i32, i32) { - // CHECK-NEXT: %[[RESULT:.*]] = constant 1 - // CHECK-NEXT: return %[[RESULT]], %[[RESULT]] - - %0 = constant 1 : i32 - %1 = constant 1 : i32 - return %0, %1 : i32, i32 -} -``` - -The above test performs a check that after running Common Sub-Expression -elimination, only one constant remains in the IR. - -#### FileCheck best practices - -FileCheck is an extremely useful utility, it allows for easily matching various -parts of the output. This ease of use means that it becomes easy to write -brittle tests that are essentially `diff` tests. FileCheck tests should be as -self-contained as possible and focus on testing the minimal set of -functionalities needed. Let's see an example: - -```mlir -// RUN: mlir-opt %s -cse | FileCheck %s - -// CHECK-LABEL: func @simple_constant() -> (i32, i32) -func @simple_constant() -> (i32, i32) { - // CHECK-NEXT: %result = constant 1 : i32 - // CHECK-NEXT: return %result, %result : i32, i32 - // CHECK-NEXT: } - - %0 = constant 1 : i32 - %1 = constant 1 : i32 - return %0, %1 : i32, i32 -} -``` - -The above example is another way to write the original example shown in the main -[FileCheck tests](#filecheck-tests) section. There are a few problems with this -test; below is a breakdown of the no-nos of this test to specifically highlight -best practices. - -* Tests should be self-contained. - -This means that tests should not test lines or sections outside of what is -intended. In the above example, we see lines such as `CHECK-NEXT: }`. This line -in particular is testing pieces of the Parser/Printer of FuncOp, which is -outside of the realm of concern for the CSE pass. This line should be removed. - -* Tests should be minimal, and only check what is absolutely necessary. - -This means that anything in the output that is not core to the functionality -that you are testing should *not* be present in a CHECK line. This is a separate -bullet just to highlight the importance of it, especially when checking against -IR output. - -If we naively remove the unrelated `CHECK` lines in our source file, we may end -up with: - -```mlir -// CHECK-LABEL: func @simple_constant -func @simple_constant() -> (i32, i32) { - // CHECK-NEXT: %result = constant 1 : i32 - // CHECK-NEXT: return %result, %result : i32, i32 - - %0 = constant 1 : i32 - %1 = constant 1 : i32 - return %0, %1 : i32, i32 -} -``` - -It may seem like this is a minimal test case, but it still checks several -aspects of the output that are unrelated to the CSE transformation. Namely the -result types of the `constant` and `return` operations, as well the actual SSA -value names that are produced. FileCheck `CHECK` lines may contain -[regex statements](https://llvm.org/docs/CommandGuide/FileCheck.html#filecheck-regex-matching-syntax) -as well as named -[string substitution blocks](https://llvm.org/docs/CommandGuide/FileCheck.html#filecheck-string-substitution-blocks). -Utilizing the above, we end up with the example shown in the main -[FileCheck tests](#filecheck-tests) section. - -```mlir -// CHECK-LABEL: func @simple_constant -func @simple_constant() -> (i32, i32) { - /// Here we use a substitution variable as the output of the constant is - /// useful for the test, but we omit as much as possible of everything else. - // CHECK-NEXT: %[[RESULT:.*]] = constant 1 - // CHECK-NEXT: return %[[RESULT]], %[[RESULT]] - - %0 = constant 1 : i32 - %1 = constant 1 : i32 - return %0, %1 : i32, i32 -} -``` - -### Diagnostic verification tests - -MLIR provides rich source location tracking that can be used to emit errors, -warnings, etc. easily from anywhere throughout the codebase. Certain classes of -tests are written to check that certain diagnostics are emitted for a given -input program, such as an MLIR file. These tests are useful in that they allow -checking specific invariants of the IR without transforming or changing -anything. Some examples of tests in this category are: those that verify -invariants of operations, or check the expected results of an analysis. -Diagnostic verification tests are written utilizing the -[source manager verifier handler](Diagnostics.md#sourcemgr-diagnostic-verifier-handler), -accessible via the `verify-diagnostics` flag in mlir-opt. - -An example .mlir test running under `mlir-opt` is shown below: - -```mlir -// RUN: mlir-opt %s -split-input-file -verify-diagnostics - -// Expect an error on the same line. -func @bad_branch() { - br ^missing // expected-error {{reference to an undefined block}} -} - -// ----- - -// Expect an error on an adjacent line. -func @foo(%a : f32) { - // expected-error@+1 {{unknown comparison predicate "foo"}} - %result = cmpf "foo", %a, %a : f32 - return -} -``` - -## Unit tests - -Unit tests are written using -[Google Test](https://github.com/google/googletest/blob/master/googletest/docs/primer.md) -and are located in the unittests/ directory. Tests of these form *should* be -limited to API tests that cannot be reasonably written as [Check](#check-tests) -tests, e.g. those for data structures. It is important to keep in mind that the -C++ APIs are not stable, and evolve over time. As such, directly testing the C++ -IR interfaces makes the tests more fragile as those C++ APIs evolve over time. -This makes future API refactorings, which may happen frequently, much more -cumbersome as the number of tests scale. -- 2.7.4