From: Aaron Robinson Date: Wed, 8 Apr 2020 16:07:47 +0000 (-0700) Subject: Update documentation for CoreCLR testing. (#34674) X-Git-Tag: submit/tizen/20210909.063632~8695 X-Git-Url: http://review.tizen.org/git/?a=commitdiff_plain;h=174f13637c1f5b422fa01f5cf8ff3911bca2f3e0;p=platform%2Fupstream%2Fdotnet%2Fruntime.git Update documentation for CoreCLR testing. (#34674) * Update documentation for CoreCLR testing. --- diff --git a/docs/workflow/testing/coreclr/test-configuration.md b/docs/workflow/testing/coreclr/test-configuration.md index 66c5c75..490fc4a 100644 --- a/docs/workflow/testing/coreclr/test-configuration.md +++ b/docs/workflow/testing/coreclr/test-configuration.md @@ -60,7 +60,7 @@ Test cases are categorized by priority level. The most important subset should b 1. Indicate the success of the test by returning `100`. Failure can be indicated by any non-`100` value. Example: - ``` + ```CSharp static public int Main(string[] notUsed) { try diff --git a/docs/workflow/testing/coreclr/windows-test-instructions.md b/docs/workflow/testing/coreclr/windows-test-instructions.md index c1a423a..1dc8551 100644 --- a/docs/workflow/testing/coreclr/windows-test-instructions.md +++ b/docs/workflow/testing/coreclr/windows-test-instructions.md @@ -3,90 +3,92 @@ Building and running tests on Windows ## Building Tests -To build the tests simply navigate to the tests directory above the repo and run, -``` - C:\git\runtime>src\coreclr\build-test.cmd -``` - -## Cleaning Tests +Building coreclr tests must be done using a specific script as follows: -**Note:** Cleaning should be done before all tests to be sure that the test assets are initialized correctly. To do a clean build of the tests, in a clean command prompt, issue the following command: ``` - C:\git\runtime>src\coreclr\build-test.cmd -rebuild +> src\coreclr\build-test.cmd ``` -## Building Tests that will be precompiled +## Building Precompiled Tests ``` - C:\git\runtime>src\coreclr\build-test.cmd crossgen +> src\coreclr\build-test.cmd crossgen ``` -This will use `crossgen.exe` to precompile the test executables before they are executed. +This will use `crossgen.exe` to precompile test executables before they are executed. -## Building Other Priority Tests +## Building Specific Priority Tests ``` - C:\git\runtime>src\coreclr\build-test.cmd -priority=1 +> src\coreclr\build-test.cmd -priority=1 ``` -The number '1' is just an example. The default value (if no priority is specified) is 0. To clarify, if '1' is specified, all tests with CLRTestPriorty 0 **and** 1 will be built and consequently run. +The above is an example of requesting that priority '1' and below be built. The default priority value is '0'. If '1' is specified, all tests with `CLRTestPriorty` `0` **and** `1` will be built and run. ## Examples -To run a clean, priority 1, crossgen test pass: +To run a priority '0' and '1' and crossgen'd test pass: ``` - C:\git\runtime>src\coreclr\build-test.cmd -rebuild crossgen -priority=1 +> src\coreclr\build-test.cmd crossgen -priority=1 ``` -`src\coreclr\build-test.cmd -?` - will list additional supported parameters. +For additional supported parameters use the following: + +``` +> src\coreclr\build-test.cmd -? +``` ### Building Individual Tests -Note: `build-test.cmd` or `build.cmd skipnative` needs to be run at least once +**Note:** `build-test.cmd skipmanaged [Any additional flags]` needs to be run at least once if the individual test has native assets. * Native Test: Build the generated Visual Studio solution or makefile corresponding to test cmake file. -* Managed Test: You can invoke msbuild on the project directly from Visual Studio Command Prompt. +* Managed Test: Use `dotnet.cmd` from the root of the repo on the test project directly. ### Running Tests -`src\coreclr\tests\runtest.cmd /?` - will list supported parameters. +Will list supported parameters. + +``` +> src\coreclr\tests\runtest.cmd /? +``` -For example to run all of the tests using your checked build: +In order to run all of the tests using your checked build: ``` - \src\coreclr\tests\runtest.cmd checked +> src\coreclr\tests\runtest.cmd checked ``` -This will generate a report named as `TestRun__.html` (e.g. `TestRun_Windows_NT_x64_Checked.html`) in the subdirectory `\artifacts\log`. Any tests that failed will be listed in `TestRunResults_Windows_NT_x64_Checked.err`. +This will generate a report named `TestRun__.html` (e.g. `TestRun_Windows_NT_x64_Checked.html`) in the subdirectory `\artifacts\log`. Any tests that failed will be listed in `TestRunResults_Windows_NT_x64_Checked.err`. ### Investigating Test Failures Upon completing a test run, you may find one or more tests have failed. -The output of the Test will be available in Test reports directory, but by default the directory would be something like `\artifacts\tests\coreclr\Windows_NT.x64.Checked\Reports\Exceptions\Finalization`. +The output of the test will be available in `Test` reports directory, but by default the directory will be something like `\artifacts\tests\coreclr\Windows_NT.x64.Checked\Reports\Exceptions\Finalization`. There are 2 files of interest: - `Finalizer.output.txt` - Contains all the information logged by the test. -- `Finalizer.error.txt` - Contains the information reported by CoreRun.exe (which executed the test) when the test process crashes. +- `Finalizer.error.txt` - Contains the information reported by CoreRun.exe (which executed the test) when the test process crashes. -### Rerunning a failed test +### Re-run a failed test -If you wish to re-run a failed test, please follow the following steps: +If you wish to re-run a failed test, follow the following steps: -1. Set an environment variable, `CORE_ROOT`, pointing to the path to product binaries that was passed to runtest.cmd. +1) Set an environment variable, `CORE_ROOT`, pointing to the path to product binaries that was passed to runtest.cmd. For example using a checked build the location would be: `\artifacts\tests\coreclr\Windows_NT.x64.Checked\Tests\Core_Root` -2. Next, run the failed test, the command to which is also present in the test report for a failed test. It will be something like `\artifacts\tests\coreclr\Windows_NT.x64.Checked\Exceptions\Finalization\Finalizer.cmd`. +2) Run the failed test, the command to which is also present in the test report for a failed test. It will be something like `\artifacts\tests\coreclr\Windows_NT.x64.Checked\Exceptions\Finalization\Finalizer.cmd`. -If you wish to run the test under a debugger (e.g. [WinDbg](http://msdn.microsoft.com/en-us/library/windows/hardware/ff551063(v=vs.85).aspx)), append `-debug ` to the test command. For example: +If you wish to run the test under a debugger (e.g. [WinDbg](http://msdn.microsoft.com/library/windows/hardware/ff551063(v=vs.85).aspx)), append `-debug ` to the test command. For example: ``` - \artifacts\tests\coreclr\Windows_NT.x64.Checked\Exceptions\Finalization\Finalizer.cmd -debug +> artifacts\tests\coreclr\Windows_NT.x64.Checked\Exceptions\Finalization\Finalizer.cmd -debug ``` ### Modifying a test -If test changes are needed, make the change and build the test project. This will binplace the binaries in test binaries folder (e.g. `\artifacts\tests\coreclr\Windows_NT.x64.Checked\Exceptions\Finalization`). At this point, follow the steps to re-run a failed test to re-run the modified test. +If test changes are needed, make the change, and build the test project. This will binplace the binaries in the test binaries folder (e.g. `\artifacts\tests\coreclr\Windows_NT.x64.Checked\Exceptions\Finalization`). Then re-run the test following the instructions above.