Add some instructions (tests, new APIs)

- Instructions for running CoreCLR and PAL tests outside Windows
- Brief instructions for running CoreFX tests with a private build of CoreCLR outside Windows
- Adding new public APIs to mscorlib

diff --git a/Documentation/README.md b/Documentation/README.md
index 97b3b0f..225a4c2 100644 (file)
--- a/Documentation/README.md
+++ b/Documentation/README.md
@@ -30,6 +30,7 @@ Project Docs
 - [Contributing Workflow](project-docs/contributing-workflow.md)
 - [Performance Guidelines](project-docs/performance-guidelines.md)
 - [Garbage Collector Guidelines](project-docs/garbage-collector-guidelines.md)
+- [Adding new public APIs to mscorlib](project-docs/adding_new_public_apis.md)
 
 Coding Guidelines
 =================
@@ -47,6 +48,7 @@ Build CoreCLR from Source
 - [Building CoreCLR on Windows](building/windows-instructions.md)
 - [Debugging CoreCLR on Windows](building/windows-debugging-instructions.md)
 - [Testing Changes on Windows](building/windows-test-instructions.md)
+- [Testing Changes on Linux, OS X, and FreeBSD](building/unix-test-instructions.md)
 - [Testing with CoreFX](building/testing-with-corefx.md)
 - [Creating a Custom DNX](building/custom-dnx-instructions.md)
 

diff --git a/Documentation/building/testing-with-corefx.md b/Documentation/building/testing-with-corefx.md
index 97c8120..c16de63 100644 (file)
--- a/Documentation/building/testing-with-corefx.md
+++ b/Documentation/building/testing-with-corefx.md
@@ -3,6 +3,8 @@ Testing with CoreFX
 
 It may be valuable to use CoreFX tests to validate your changes to CoreCLR or mscorlib.
 
+**Windows**
+
 In order to do this you need to create a file called `localpublish.props` under the `<repo root>\packages` folder.
 The contents of the file should look like this (make sure to update the version to the current version of the CoreCLR package used by CoreFx):
 
@@ -21,4 +23,10 @@ The contents of the file should look like this (make sure to update the version
 
 Once this file is there, subsequent builds of the CoreCLR repo will install the CoreCLR package into the location specified by `InstallLocation`.
 
-To run tests, follow the procedure for running tests in CoreFX.
+To run tests, follow the procedure for [running tests in CoreFX](https://github.com/dotnet/corefx/blob/master/Documentation/building/windows-instructions.md).
+
+**Linux, OS X, FreeBSD**
+
+Refer to the procedure for [running tests in CoreFX](https://github.com/dotnet/corefx/blob/master/Documentation/building/cross-platform-testing.md)
+- Note the --coreclr-bins and --mscorlib-bins arguments to [run-test.sh](https://github.com/dotnet/corefx/blob/master/run-test.sh)
+- Pass in paths to your private build of CoreCLR

diff --git a/Documentation/building/unix-test-instructions.md b/Documentation/building/unix-test-instructions.md
new file mode 100644 (file)
index 0000000..d891331
--- /dev/null
+++ b/Documentation/building/unix-test-instructions.md
@@ -0,0 +1,68 @@
+Building and running tests on Linux, OS X, and FreeBSD
+======================================================
+
+CoreCLR tests
+-------------
+
+**Building**
+
+Build CoreCLR and CoreFX. Refer to building instructions in the respective repository.
+
+To build only the tests, on the Windows machine:
+
+> `C:\coreclr>tests\buildtest.cmd clean`
+
+**Running tests**
+
+The following instructions assume that on the Unix machine:
+- The CoreCLR repo is cloned at `~/coreclr`
+- The CoreFX repo is cloned at `~/corefx`
+- The Windows clone of the CoreCLR repo is mounted at `/media/coreclr`
+- The Windows clone of the CoreFX repo is mounted at `/media/corefx`
+
+Tests currently need to be built on Windows and copied over to the Unix machine for testing. Copy the test build over to the Unix machine:
+
+> `cp --recursive /media/coreclr/bin/tests/Windows_NT.x64.Debug ~/test/`
+
+See runtest.sh usage information:
+
+> `~/coreclr$ tests/runtest.sh --help`
+
+Run tests:
+
+> ```bash
+> ~/coreclr$ tests/runtest.sh
+>     --testRootDir="~/test/Windows_NT.x64.Debug"
+>     --testNativeBinDir="~/coreclr/bin/obj/Linux.x64.Debug/tests"
+>     --coreClrBinDir="~/coreclr/bin/Product/Linux.x64.Debug"
+>     --mscorlibDir="/media/coreclr/bin/Product/Linux.x64.Debug"
+>     --coreFxBinDir="~/corefx/bin/Linux.AnyCPU.Debug"
+>     --coreFxNativeBinDir="~/corefx/bin/Linux.x64.Debug"
+> ```
+
+Test results will go into:
+
+> `~/test/Windows_NT.x64.Debug/coreclrtests.xml`
+
+**Unsupported and temporarily disabled tests**
+
+These tests are skipped by default:
+- Tests that are not supported outside Windows, are listed in:
+>> `~/coreclr/tests/testsUnsupportedOutsideWindows.txt`
+- Tests that are temporarily disabled outside Windows due to unexpected failures (pending investigation), are listed in:
+>> `~/coreclr/tests/testsFailingOutsideWindows.txt`
+
+To run only the set of temporarily disabled tests, pass in the `--runFailingTestsOnly` argument to `runtest.sh`.
+
+PAL tests
+---------
+
+Build CoreCLR on the Unix machine.
+
+Run tests:
+
+> `~/coreclr$ src/pal/tests/palsuite/runpaltests.sh ~/coreclr/bin/obj/Linux.x64.Debug`
+
+Test results will go into:
+
+> `/tmp/PalTestOutput/default/pal_tests.xml`

diff --git a/Documentation/project-docs/adding_new_public_apis.md b/Documentation/project-docs/adding_new_public_apis.md
new file mode 100644 (file)
index 0000000..dbf305d
--- /dev/null
+++ b/Documentation/project-docs/adding_new_public_apis.md
@@ -0,0 +1,31 @@
+Adding new public APIs to mscorlib
+==================================
+
+Many of the CoreFX libraries type-forward their public APIs to the implementations in mscorlib.
+- The CoreFX build uses published contracts for mscorlib, and the CoreFX test build uses published contracts for some of the CoreFX libraries.
+- Some of the CoreFX libraries are not built in the CoreFX repository. For example, System.Runtime.Loader.dll is purely a facade and type-forwards everything to mscorlib. These libraries are built and published through a separate process.
+- Hence, when adding a new public API to mscorlib, changes must be staged to ensure that new prerequisites are published before they are used.
+
+**Staging the changes**
+
+Make the changes to CoreCLR, including mscorlib
+- Update `coreclr/src/mscorlib/model.xml` with the new APIs. APIs that are not listed in this file will be stripped out prior to publishing.
+- Note that at the moment, merging changes with new public APIs will cause an internal build failure. Please work with your PR reviewer to have these build breaks be fixed soon after merging, otherwise it will block the publishing process.
+- Merge the changes
+- Wait for a new mscorlib contract to be published. Check the latest published version [here](http://myget.org/gallery/dotnet-core).
+
+Make the changes to CoreFX consuming the new APIs in mscorlib
+- If the changes are to libraries that are built out of the CoreFX repository:
+  - You will likely see a build failure until a new mscorlib contract is published
+- If the changes are to libraries that are **not** built out of the CoreFX repository:
+  - For example, pure facades such as System.Runtime.Loader.dll
+  - There will likely not be a build failure
+  - But you will still need to wait for the new mscorlib contract to be published before merging the change, otherwise, facade generation will fail
+- Merge the changes
+- Wait for new contracts to be published for libraries with new APIs. Check the latest published versions [here](http://myget.org/gallery/dotnet-core).
+
+Add tests
+- You should now be able to consume the new APIs and add tests to the CoreFX test suite
+  - Until new contracts are published, you will likely see a build failure indicating that the new APIs don't exist.
+- Note that on Windows, CoreFX tests currently use a potentially old published build of CoreCLR
+  - You may need to disable the new tests on Windows until CoreFX tests are updated to use a newer build of CoreCLR.

diff --git a/tests/runtest.sh b/tests/runtest.sh
index 4f5970b..feedc24 100755 (executable)
--- a/tests/runtest.sh
+++ b/tests/runtest.sh
@@ -7,7 +7,7 @@ function print_usage {
     echo 'Typical command line:'
     echo ''
     echo 'coreclr/tests/runtest.sh'
-    echo '    --testRootDir="temp/Windows.x64.Debug"'
+    echo '    --testRootDir="temp/Windows_NT.x64.Debug"'
     echo '    --testNativeBinDir="coreclr/bin/obj/Linux.x64.Debug/tests"'
     echo '    --coreClrBinDir="coreclr/bin/Product/Linux.x64.Debug"'
     echo '    --mscorlibDir="windows/coreclr/bin/Product/Linux.x64.Debug"'