From: Aaron Robinson Date: Fri, 22 Jun 2018 18:24:22 +0000 (-0700) Subject: Start moving testing documentation into a single location (dotnet/coreclr#18598) X-Git-Tag: submit/tizen/20210909.063632~11030^2~4538 X-Git-Url: http://review.tizen.org/git/?a=commitdiff_plain;h=ddcc1649cd01480c4f102c4ca6ef52e382a87db7;p=platform%2Fupstream%2Fdotnet%2Fruntime.git Start moving testing documentation into a single location (dotnet/coreclr#18598) Start moving testing documentation into a single location Update `test-configuration.md` with more details on creating a test and requirements Commit migrated from https://github.com/dotnet/coreclr/commit/8aad994743a1332a21a56728799c293a21115050 --- diff --git a/docs/coreclr/building/test-configuration.md b/docs/coreclr/building/test-configuration.md index 931a540..1274959 100644 --- a/docs/coreclr/building/test-configuration.md +++ b/docs/coreclr/building/test-configuration.md @@ -1,41 +1,54 @@ -## General Test Infrastructure Notes ## +# General Test Infrastructure -### Kinds of Build Properties ### -* Build Only -> `BuildOnly` - - * Builds an executable. - * Will not execute it. +## Test "Kind" +* Build Only + * Builds an executable. + * Will not execute. + * e.g. `BuildOnly` * Run Only -> `RunOnly` + * Can use output of `BuildOnly` or `BuildAndRun` projects with different command line arguments. + * e.g. `RunOnly` +* Build And Run + * Builds an executable. + * Will execute said executable. + * e.g. `BuildAndRun` +* Shared Libraries + * For building libraries common to zero or more tests. + * e.g. `SharedLibrary` - * Can use Ouput of Build and Run Project with different command line arguments. -* Build and Run -> `BuildAndRun` +By default (i.e. if not specified explicitly), test "Kind" is `BuildAndRun`. - * Builds an executable. - * Will execute said executable. -* Shared Libraries -> `SharedLibrary` +## Priority + +Test cases are categorized by priority level. The most important subset should be and is the smallest subset. This subset is called priority 0. - * For building libraries common to zero or more tests. +* By default, a test case is priority 0. Tests must be explicitly de-prioritized. +* Set the priority of a test by setting the property `` in the test's project file. + * e.g. `2` +* Lower priority values are always run in conjunction when running higher priority value tests. + * i.e. if a developer elects to do a priority 2 test run, then all priority 0, 1 and 2 tests are run. +## Adding Test Guidelines -By default (i.e. if not specified explicitly) a project file is BuildAndRun. +* All test source files should include the following banner: +``` + // Licensed to the .NET Foundation under one or more agreements. + // The .NET Foundation licenses this file to you under the MIT license. + // See the LICENSE file in the project root for more information. +``` +* Disable building of a test by conditionally setting the `` property. + * e.g. `true` +* Add NuGet/MyGet references by updating the following [project file](https://github.com/dotnet/coreclr/blob/master/tests/src/Common/test_dependencies/test_dependencies.csproj). +* Build against the `mscorlib` facade by adding `true` to the test project. -### Priority ### -Testcases are categorized by their priority levels. The most important subset should be and is the smallest subset. This subset is called priority 0. - * By default, a testcase is priority 0. You must elect to de-prioritize a test. - * To de-prioritize a test, add a property _CLRTestPriority_ to the test's project file. -> `2` - * Lower priority values are always run in conjunction when running higher priority value tests. I.e. if a developer elects to do a priority 2 test run, then all priority 0, 1 and 2 tests are run. +### Creating a new C# test project -### Adding Tests ### -#### Converting an existing C# project #### - * Remove AssemblyName - * Swap in dir.props - * Swap in dir.targets - * Assign a CLRTestKind - * (optional) Assign a priority value +**TODO** +### Converting an existing C# project + * Remove the `` property + * Import `dir.props` + * Import `dir.targets` + * Assign a `` + * (optional) Assign a priority value \ No newline at end of file diff --git a/docs/coreclr/workflow/RunningTests.md b/docs/coreclr/workflow/RunningTests.md index 65957f4..b7d4d5d 100644 --- a/docs/coreclr/workflow/RunningTests.md +++ b/docs/coreclr/workflow/RunningTests.md @@ -1,10 +1,42 @@ - # Running .NET Core Tests -TODO - Incomplete. +Details on test metadata can be found in [test-configuration.md](https://github.com/dotnet/coreclr/blob/master/Documentation/building/test-configuration.md). + +## Build All Tests + +1) Build the CoreCLR product + * [Unix](https://github.com/dotnet/coreclr/blob/master/Documentation/building/linux-instructions.md) + * [OSX](https://github.com/dotnet/coreclr/blob/master/Documentation/building/osx-instructions.md) + * [Windows](https://github.com/dotnet/coreclr/blob/master/Documentation/building/windows-instructions.md) +1) From the root directory run the following command: + * Non-Windows - `./build-test.sh` + * Windows - `build-test.cmd` + * Supply `-h` for usage flags + +### Examples + +* Build all tests priority `2` and higher + * `build-test.cmd -priority=2` + +## Build Individual Test + +Note: The CoreCLR must be built prior to building an individual test. See first step for building all tests. + +* Native Test: Build the generated CMake projects + * Projects are auto-generated when the `build-test.sh`/`build-test.cmd` script is run +* Managed Test: Invoke MSBuild on the project directly + * Non-Windows - All of the necessary tools to build are under `coreclr/Tools`. It is possible to use `coreclr/Tools/MSBuild.dll` as you would normally use MSBuild with a few caveats. The `coreclr/Tools/msbuild.sh` script exists to make the call shorter. + * **Note:** Passing `/p:__BuildOs=`[`OSX`|`Linux`] is required. Otherwise the following error will occur: `error MSB4801: The task factory "CodeTaskFactory" could not be loaded because this version of MSBuild does not support it.` + * Windows - Use Visual Studio Developer command prompt -See [Windows Instructions](../building/windows-test-instructions.md) -See [Unix Instructions](../building/unix-test-instructions.md) +### Examples +* Using the `msbuild.sh` script + * `coreclr/Tools/msbuild.sh /maxcpucount coreclr/tests/src/JIT/CodeGenBringUpTests/Array1.csproj /p:__BuildType=Release /p:__BuildOS=OSX` +* Calling `MSBuild.dll` directly + * `coreclr/Tools/dotnetcli/dotnet coreclr/Tools/MSBuild.dll /maxcpucount coreclr/tests/src/JIT/CodeGenBringUpTests/Array1.csproj /p:__BuildType=Release /p:__BuildOS=OSX` +## Additional Documents +* [Windows](https://github.com/dotnet/coreclr/blob/master/Documentation/building/windows-test-instructions.md) +* [Non-Windows](https://github.com/dotnet/coreclr/blob/master/Documentation/building/unix-test-instructions.md) \ No newline at end of file