* Quality Week 2022: The First Step to a Better Documentation.
* Removed the Markdown Extension's additional development tags.
* Removed redundant stuff.
* Removed redundant stuff.
* Removed guide testing versions to avoid confusion that they would be the only supported versions.
* Addressed some Powershell comments, added a pointer to a yaml with links to other .NET sibling repos, and did a couple mini-corrections. Still got a few comments to address...
* Fixed a ';:' with Powershell's syntax.
* Addressed review comments: Redaction improvements, localization url's, improved code snippets, added info on individual subsets building configurations, changed COMPlus for DOTNET, and other misc improvements.
* Addressed remaining comments.
* Addressed some last comments regarding corerun, and the subset flags in the build scripts.
Co-authored-by: Ivan Diaz <bluehorizon186@gmail.com>
# .NET Runtime
+
[](https://dnceng.visualstudio.com/public/_build/latest?definitionId=686&branchName=main)
[](https://github.com/dotnet/runtime/labels/help%20wanted)
[](https://gitter.im/dotnet/runtime)
[](https://aka.ms/dotnet-discord)
+* [What is .NET?](#what-is-net)
+* [How can I contribute?](#how-can-i-contribute)
+* [Reporting security issues and security bugs](#reporting-security-issues-and-security-bugs)
+* [Filing issues](#filing-issues)
+* [Useful Links](#useful-links)
+* [.NET Foundation](#net-foundation)
+* [License](#license)
+
This repo contains the code to build the .NET runtime, libraries and shared host (`dotnet`) installers for
all supported platforms, as well as the sources to .NET runtime and libraries.
## What is .NET?
-Official Starting Page: https://dotnet.microsoft.com
+Official Starting Page: <https://dotnet.microsoft.com>
* [How to use .NET](https://docs.microsoft.com/dotnet/core/get-started) (with VS, VS Code, command-line CLI)
* [Install official releases](https://dotnet.microsoft.com/download)
We welcome contributions! Many people all over the world have helped make this project better.
* [Contributing](CONTRIBUTING.md) explains what kinds of contributions we welcome
-- [Workflow Instructions](docs/workflow/README.md) explains how to build and test
+* [Workflow Instructions](docs/workflow/README.md) explains how to build and test
* [Get Up and Running on .NET Core](docs/project/dogfooding.md) explains how to get nightly builds of the runtime and its libraries to test them in your own projects.
## Reporting security issues and security bugs
-Security issues and bugs should be reported privately, via email, to the Microsoft Security Response Center (MSRC) <secure@microsoft.com>. You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Further information, including the MSRC PGP key, can be found in the [Security TechCenter](https://www.microsoft.com/msrc/faqs-report-an-issue).
+Security issues and bugs should be reported privately, via email, to the Microsoft Security Response Center (MSRC) <secure@microsoft.com>. You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Further information, including the MSRC PGP key, can be found in the [Security TechCenter](https://www.microsoft.com/msrc/faqs-report-an-issue). You can also find these instructions in this repo's [Security doc](SECURITY.md).
Also see info about related [Microsoft .NET Core and ASP.NET Core Bug Bounty Program](https://www.microsoft.com/msrc/bounty-dot-net-core).
This repo should contain issues that are tied to the runtime, the class libraries and frameworks, the installation of the `dotnet` binary (sometimes known as the `muxer`) and installation of the .NET runtime and libraries.
-For other issues, please use the following repos:
-
-- For overall .NET SDK issues, file in the [dotnet/sdk](https://github.com/dotnet/sdk) repo
-- For ASP.NET issues, file in the [dotnet/aspnetcore](https://github.com/dotnet/aspnetcore) repo.
+For other issues, please file them to their appropriate sibling repos. Their respective links are described in the [config issue template doc](/.github/ISSUE_TEMPLATE/config.yml).
## Useful Links
There are many .NET related projects on GitHub.
-- [.NET home repo](https://github.com/Microsoft/dotnet) - links to 100s of .NET projects, from Microsoft and the community.
-- [ASP.NET Core home](https://docs.microsoft.com/aspnet/core) - the best place to start learning about ASP.NET Core.
+* [.NET home repo](https://github.com/Microsoft/dotnet) - links to 100s of .NET projects, from Microsoft and the community.
+* [ASP.NET Core home](https://docs.microsoft.com/aspnet/core) - the best place to start learning about ASP.NET Core.
This project has adopted the code of conduct defined by the [Contributor Covenant](https://contributor-covenant.org) to clarify expected behavior in our community. For more information, see the [.NET Foundation Code of Conduct](https://www.dotnetfoundation.org/code-of-conduct).
# Using Codespaces
-Codespaces allows you to develop in a Docker container running in the cloud. You can use an in-browser version of VS Code or the full VS Code application with the [GitHub Codespaces VS Code Extension](https://marketplace.visualstudio.com/items?itemName=GitHub.codespaces). This means you don't need to install dotnet/runtime's prerequisites on your current machine in order to develop in dotnet/runtime.
+
+* [Create a Codespace](#create-a-codespace)
+* [Updating dotnet/runtime's Codespaces Configuration](#updating-dotnetruntimes-codespaces-configuration)
+* [Testing out your Changes](#testing-out-your-changes)
+
+Codespaces allows you to develop in a Docker container running in the cloud. You can use an in-browser version of VS Code or the full VS Code application with the [GitHub Codespaces VS Code Extension](https://marketplace.visualstudio.com/items?itemName=GitHub.codespaces). This means you don't need to install any prerequisites on your current machine in order to develop in _dotnet/runtime_.
## Create a Codespace
-dotnet/runtime runs a nightly GitHub Action to build the latest code in the repo. This allows you to immediately start developing and testing after creating a codespace without having to build the whole repo. When the machine is created, it will have built the repo using the code as of 6 AM UTC that morning.
+The _dotnet/runtime_ repo runs a nightly GitHub Action to build the latest code in it. This allows you to immediately start developing and testing after creating a Codespace without having to build the whole repo. When the machine is created, it will have built the repo using the code as of 6 AM UTC of that morning.
-1. From https://github.com/dotnet/runtime, drop-down the `Code` button and select the `Codespaces` tab.
+1. From this [repository's root](https://github.com/dotnet/runtime), drop-down the _Code_ button and select the _Codespaces_ tab.
- - For `libraries` work, pick `.devcontainer/libraries/devcontainer.json`.
- - For `WASM` work, pick `.devcontainer/wasm/devcontainer.json`.
+* For `libraries` work, pick `.devcontainer/libraries/devcontainer.json`.
+* For `WASM` work, pick `.devcontainer/wasm/devcontainer.json`.
4. Select the Machine type. For `dotnet/runtime`, it is recommended to select at least a `4-core` machine. You can also verify that a `Prebuild` is ready.
-
+
-
-*If these instructions are out of date, see https://docs.github.com/codespaces/developing-in-codespaces/creating-a-codespace#creating-a-codespace for instructions on how to create a new codespace.*
+_If these instructions are out of date, see <https://docs.github.com/codespaces/developing-in-codespaces/creating-a-codespace#creating-a-codespace> for instructions on how to create a new Codespace._
## Updating dotnet/runtime's Codespaces Configuration
The Codespaces configuration is spread across the following places:
-1. The [.devcontainer](../../.devcontainer) folder contains folders for each "development scenario":
- - `libraries` - Used by developers working in `src/libraries`
- - `wasm` - Used by developers working on the browser-wasm workload
- - The `scripts` folder contains any scripts that are executed during the creation of the codespace. This has the build command that builds the entire repo for prebuilds.
-2. Each development scenario folder contains:
- - `devcontainer.json` file configures the codespace and has VS Code / Environment settings
- - The Dockerfile used to create the Docker image
-3. The GitHub Action can be configured by following the instructions at https://docs.github.com/codespaces/prebuilding-your-codespaces/configuring-prebuilds.
+1. The [.devcontainer](/.devcontainer) folder contains subfolders for each development scenario:
+ * _Libraries_: Used by developers working in `src/libraries`.
+ * _Wasm_: Used by developers working on the _browser-wasm_ workload.
+ * _Scripts_: Contains any scripts that are executed during the creation of the codespace. This has the build command that builds the entire repo for prebuilds.
+2. Each development scenario folder contains the following files:
+ * The `devcontainer.json` file that configures the codespace and has VS Code / Environment settings.
+ * The _Dockerfile_ used to create the Docker image
+3. The GitHub Action can be configured by following the instructions at <https://docs.github.com/codespaces/prebuilding-your-codespaces/configuring-prebuilds>.
+
+To test out changes to the `.devcontainer` files, you can follow the process in the [Applying Changes to your Configuration](https://docs.github.com/codespaces/customizing-your-codespace/configuring-codespaces-for-your-project#applying-changes-to-your-configuration) docs. This allows you to rebuild the Codespace privately before creating a PR.
-To test out changes to the `.devcontainer` files, you can follow the process in [Applying changes to your configuration](https://docs.github.com/codespaces/customizing-your-codespace/configuring-codespaces-for-your-project#applying-changes-to-your-configuration) docs. This allows you to rebuild the Codespace privately before creating a PR.
+## Testing out your Changes
-To test out your changes you can run the [Codespaces Prebuilds Action](https://github.com/dotnet/runtime/actions/workflows/codespaces/create_codespaces_prebuilds) in your fork against a branch with your changes.
\ No newline at end of file
+To test out your changes you can run the [Codespaces Prebuilds Action](https://github.com/dotnet/runtime/actions/workflows/codespaces/create_codespaces_prebuilds) in your fork against a branch with your changes.
# Workflow Guide
-The repo can be built for the following platforms, using the provided setup and the following instructions. Before attempting to clone or build, please check these requirements.
+* [Build Requirements](#build-requirements)
+* [Getting Yourself Started](#getting-yourself-started)
+* [Configurations and Subsets](#configurations-and-subsets)
+ * [What does this mean for me?](#what-does-this-mean-for-me)
+* [Full Instructions on Building and Testing the Runtime Repo](#full-instructions-on-building-and-testing-the-runtime-repo)
+
+The repo can be built for the following platforms, using the provided setup and the following instructions. Before attempting to clone or build, please check the requirements that match your machine, and ensure you install and prepare all as necessary.
## Build Requirements
| ARM64 | ✔ | ✔ | ✔ | |
| | [Requirements](requirements/windows-requirements.md) | [Requirements](requirements/linux-requirements.md) | [Requirements](requirements/macos-requirements.md) | [Requirements](requirements/freebsd-requirements.md)
-Before proceeding further, please click on the link above that matches your machine and ensure you have installed all the prerequisites for the build to work.
-
Additionally, keep in mind that cloning the full history of this repo takes roughly 400-500 MB of network transfer, inflating to a repository that can consume somewhere between 1 to 1.5 GB. A build of the repo can take somewhere between 10 and 20 GB of space for a single OS and Platform configuration depending on the portions of the product built. This might increase over time, so consider this to be a minimum bar for working with this codebase.
-## Concepts
-
-The runtime repo can be built from a regular, non-administrator command prompt, from the root of the repo, as follows:
-
-For Linux and macOS
-```bash
-./build.sh
-```
+## Getting Yourself Started
-For Windows:
-```cmd
-build.cmd
-```
+The runtime repo can be built from a regular, non-administrator command prompt, from the root of the repo.
-This builds the product (in the default debug configuration), but not the tests.
+The repository currently consists of three different major parts:
-For information about the different options available, supply the argument `--help|-h` when invoking the build script:
-```
-build -h
-```
+* The Runtimes
+* The Libraries
+* The Installer
-On Unix like systems, arguments can be passed in with a single `-` or double hyphen `--`.
+More info on this, as well as the different build configurations in the [Configurations and Subsets section](#configurations-and-subsets).
-The repository currently consists of different major parts: the runtimes, the libraries, and the installer.
-To build just one part you use the root build script (build.cmd/sh), and you add the `-subset` flag.
+This was a concise introduction and now it's time to show the specifics of building specific subsets in any given supported platform, since most likely you will want to customize your builds according to what component(s) you're working on, as well as how you configured your build environment. We have links to instructions depending on your needs [in this section](#full-instructions-on-building-and-testing-the-runtime-repo).
-## Editing and Debugging
+* For instructions on how to edit code and make changes, see [Editing and Debugging](editing-and-debugging.md).
+* For instructions on how to debug CoreCLR, see [Debugging CoreCLR](/docs/workflow/debugging/coreclr/debugging-coreclr.md).
+* For instructions on using GitHub Codespaces, see [Codespaces](/docs/workflow/Codespaces.md).
-For instructions on how to edit code and debug your changes, see [Editing and Debugging](editing-and-debugging.md).
-
-For instructions on using GitHub Codespaces, see [Codespaces](Codespaces.md).
-
-## Configurations
+## Configurations and Subsets
You may need to build the tree in a combination of configurations. This section explains why.
-A quick reminder of some concepts -- see the [glossary](../project/glossary.md) for more on these:
+<!-- LINK-UPDATES -->
+A quick reminder of some concepts -- see the [glossary](/docs/project/glossary.md) for more on these:
* **Debug Configuration** -- Non-optimized code. Asserts are enabled.
-* **Checked Configuration** -- Optimized code. Asserts are enabled. Only relevant to CoreCLR runtime.
-* **Release Configuration** -- Optimized code. Asserts are disabled. Runs at the best speed, and suitable for performance profiling. You will have limited debugging experience.
+* **Checked Configuration** -- Optimized code. Asserts are enabled. _Only relevant to CoreCLR runtime._
+* **Release Configuration** -- Optimized code. Asserts are disabled. Runs at the best speed, and suitable for performance profiling. This will impact the debugging experience due to compiler optimizations that make understanding what the debugging is showing difficult to reason about, relative to the source code.
When we talk about mixing configurations, we're discussing the following sub-components:
+<!-- LINK-UPDATES -->
* **Runtime** is the execution engine for managed code and there are two different implementations available. Both are written in C/C++, therefore, easier to debug when built in a Debug configuration.
- * CoreCLR is the comprehensive execution engine which, if built in Debug Configuration, executes managed code very slowly. For example, it will take a long time to run the managed code unit tests. The code lives under [src/coreclr](../../src/coreclr).
- * Mono is portable and also slimmer runtime and it's not that sensitive to Debug Configuration for running managed code. You will still need to build it without optimizations to have good runtime debugging experience though. The code lives under [src/mono](../../src/mono).
-* **CoreLib** (also known as System.Private.CoreLib) is the lowest level managed library. It has a special relationship to the runtimes and therefore it must be built in the matching configuration, e.g., if the runtime you are using was built in a Debug configuration, this must be in a Debug configuration. The runtime agnostic code for this library can be found at [src/libraries/System.Private.CoreLib/src](../../src/libraries/System.Private.CoreLib/src/README.md).
-* **Libraries** is the bulk of the dlls that are oblivious to the configuration that runtimes and CoreLib were built in. They are most debuggable when built in a Debug configuration, and, happily, they still run sufficiently fast in that configuration that it's acceptable for development work. The code lives under [src/libraries](../../src/libraries).
+ * CoreCLR is the comprehensive execution engine which, if built in Debug Configuration, executes managed code very slowly. For example, it will take a long time to run the managed code unit tests. The code lives under [src/coreclr](/src/coreclr).
+ * Mono is a portable and also slimmer runtime and it's not that sensitive to Debug Configuration for running managed code. You will still need to build it without optimizations to have good runtime debugging experience though. The code lives under [src/mono](/src/mono).
+* **CoreLib** (also known as System.Private.CoreLib) is the lowest level managed library. It has a special relationship to the runtimes and therefore it must be built in the matching configuration, e.g., if the runtime you are using was built in a Debug configuration, this must be in a Debug configuration. The runtime agnostic code for this library can be found at [src/libraries/System.Private.CoreLib/src](/src/libraries/System.Private.CoreLib/src/README.md).
+* **Libraries** is the bulk of the dlls that are oblivious to the configuration that runtimes and CoreLib were built in. They are most debuggable when built in a Debug configuration, and happily, they still run sufficiently fast in that configuration that it's acceptable for development work. The code lives under [src/libraries](/src/libraries).
+
+<!-- TODO: Provide a list of the possible subsets, since right now it's all up to one's own knowledge and guessing. -->
+To build just one part of the repo, you add the `-subset` flag with the subset you wish to build to the root build script _(build.cmd/sh)_. You can specify more than one by linking them with the `+` operator (e.g. `-subset clr+libs` would build CoreCLR and the libraries). Note that if the subset is the first argument you pass to the script, you can omit the `--subset` flag altogether.
### What does this mean for me?
-At this point you probably know what you are planning to work on primarily: the runtimes or libraries.
+At this point you probably know what you are planning to work on primarily: the runtimes or libraries. As general suggestions on how to proceed, here are some ideas:
+
+* If you're working in runtimes, you may want to build everything in the Debug configuration, depending on how comfortable you are debugging optimized native code.
+* If you're working in libraries, you will want to use debug libraries with a release version of runtime and CoreLib, because the tests will run faster.
+* If you're working in CoreLib - you probably want to try to get the job done with release runtime and CoreLib, and fall back to debug if you need to. The [Building Libraries](/docs/workflow/building/libraries/README.md) document explains how you'll do this.
-* if you're working in runtimes, you may want to build everything in the Debug configuration, depending on how comfortable you are debugging optimized native code.
-* if you're working in libraries, you will want to use debug libraries with a release version of runtime and CoreLib, because the tests will run faster.
-* if you're working in CoreLib - you probably want to try to get the job done with release runtime and CoreLib, and fall back to debug if you need to. The [Building Libraries](building/libraries/README.md) document explains how you'll do this.
+## Full Instructions on Building and Testing the Runtime Repo
-Now you know about configurations and how we use them, you will want to read how to build what you plan to work on. Pick one of these:
+Now you know about configurations and how we use them, so now you will want to read how to build what you plan to work on. Each of these will have further specific instructions or links for whichever platform you are developing on.
-- [Building CoreCLR runtime](building/coreclr/README.md)
-- [Building Mono runtime](building/mono/README.md)
-- [Building Libraries](building/libraries/README.md)
+* [Building CoreCLR runtime](/docs/workflow/building/coreclr/README.md)
+* [Building Mono runtime](/docs/workflow/building/mono/README.md)
+* [Building Libraries](/docs/workflow/building/libraries/README.md)
After that, here's information about how to run tests:
-- [Testing CoreCLR runtime](testing/coreclr/testing.md)
-- [Testing Mono runtime](testing/mono/testing.md)
-- [Testing Libraries](testing/libraries/testing.md)
+* [Testing CoreCLR runtime](/docs/workflow/testing/coreclr/testing.md)
+* [Testing Mono runtime](/docs/workflow/testing/mono/testing.md)
+* [Testing Libraries](/docs/workflow/testing/libraries/testing.md)
And how to measure performance:
-- [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
-- [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
+* [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
+* [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
-# Building
+# Building CoreCLR
-These are instructions for building the common form of CoreCLR on common platforms. For instructions on particular form factors, see also [Build NativeAOT](nativeaot.md), [Build CoreCLR on Linux](linux-instructions.md), [Build CoreCLR on OS X](osx-instructions.md), [Build CoreCLR on FreeBSD](freebsd-instructions.md),
-[Cross Compilation for ARM on Windows](cross-building.md), [Cross Compilation for Android on Linux](android.md).
+* [Introduction](#introduction)
+* [Common Building Options](#common-building-options)
+ * [Build Drivers](#build-drivers)
+ * [Extra Flags](#extra-flags)
+ * [Build Results Layout](#build-results-layout)
+* [Platform-Specific Instructions](#platform-specific-instructions)
+* [Testing CoreCLR](#testing-coreclr)
-To build just CoreCLR, use the `-subset` flag to the `build.sh` (or `build.cmd`) script at the repo root:
+## Introduction
-For Linux:
-```
-./build.sh -subset clr
+Here is a brief overview on how to build the common form of CoreCLR in general. For further specific instructions on each platform, we have links to instructions later on in [Platform-Specific Instructions](#platform-specific-instructions).
+
+To build just CoreCLR, use the `subset` flag to the `build.sh` or `build.cmd` script at the repo root. Note that specifying `-subset` explicitly is not necessary if it is the first argument (i.e. `./build.sh --subset clr` and `./build.sh clr` are equivalent). However, if you specify any other argument beforehand, then you must specify the `-subset` flag.
+
+For Linux and MacOS:
+
+```bash
+./build.sh --subset clr
```
For Windows:
-```
-build.cmd -subset clr
-```
-Specifying `-subset` explicitly is not necessary if it is the first argument: `./build -subset clr` and `./build clr` are equivalent.
-By default, build generates a 'debug' build type, that includes asserts and is easier for some people to debug. If you want to make performance measurements, or just want tests to execute more quickly, you can also build the 'release' version (which does not have these checks) by adding the flag `-configuration release` (or `-c release`), for example:
-```
-./build.sh -subset clr -configuration release
+```cmd
+.\build.cmd -subset clr
```
-CoreCLR also supports a 'checked' build type which has asserts enabled like 'debug', but is built with the native compiler optimizer enabled, so it runs much faster. This is the usual mode used for running tests in the CI system. You can build that using, for example:
+## Common Building Options
+
+By default, the script generates a _Debug_ build type, which is not optimized code and includes asserts. As its name suggests, this makes it easier and friendlier to debug the code. If you want to make performance measurements, you ought to build the _Release_ version instead, which doesn't have any asserts and has all code optimizations enabled. Likewise, if you plan on running tests, the _Release_ configuration is more suitable since it's considerably faster than the _Debug_ one. For this, you add the flag `-configuration release` (or `-c release`). For example:
+
+```bash
+./build.sh --subset clr --configuration release
```
-./build.sh -subset clr -configuration checked
+
+As mentioned before in the [general building document](/docs/workflow/README.md#configurations-and-subsets), CoreCLR also supports a _Checked_ build type which has asserts enabled like _Debug_, but is built with the native compiler optimizer enabled, so it runs much faster. This is the usual mode used for running tests in the CI system.
+
+Now, it is also possible to select a different configuration for each subset when building them together. The `--configuration` flag applies universally to all subsets, but it can be overridden with any one or more of the following ones:
+
+* `--runtimeConfiguration (-rc)`: Flag for the CLR build configuration.
+* `--librariesConfiguration (-lc)`: Flag for the libraries build configuration.
+* `--hostConfiguration (-hc)`: Flag for the host build configuration.
+
+For example, a very common scenario used by developers and the repo's test scripts with default options, is to build the _clr_ in _Debug_ mode, and the _libraries_ in _Release_ mode. To achieve this, the command-line would look like the following:
+
+```bash
+./build.sh --subset clr+libs --configuration Release --runtimeConfiguration Debug
```
-If you want to use Ninja to drive the native build instead of Make on non-Windows platforms, you can pass the `-ninja` flag to the build script as follows:
+Or alternatively:
+
+```bash
+./build.sh --subset clr+libs --librariesConfiguration Release --runtimeConfiguration Debug
```
-./build.cmd -subset clr -ninja
+
+For more information about all the different options available, supply the argument `-help|-h` when invoking the build script. On Unix-like systems, non-abbreviated arguments can be passed in with a single `-` or double hyphen `--`.
+
+### Build Drivers
+
+If you want to use _Ninja_ to drive the native build instead of _Make_ on non-Windows platforms, you can pass the `-ninja` flag to the build script as follows:
+
+```bash
+./build.sh --subset clr --ninja
```
-If you want to use Visual Studio's MSBuild to drive the native build on Windows, you can pass the `-msbuild` flag to the build script similarly to the `-ninja` flag.
+If you want to use Visual Studio's _MSBuild_ to drive the native build on Windows, you can pass the `-msbuild` flag to the build script similarly to the `-ninja` flag.
+
+We recommend using _Ninja_ for building the project on Windows since it more efficiently uses the build machine's resources for the native runtime build in comparison to Visual Studio's _MSBuild_.
-We recommend using Ninja for building the project on Windows since it more efficiently uses the build machine's resources for the native runtime build in comparison to Visual Studio's MSBuild.
+### Extra Flags
To pass extra compiler/linker flags to the coreclr build, set the environment variables `EXTRA_CFLAGS`, `EXTRA_CXXFLAGS` and `EXTRA_LDFLAGS` as needed. Don't set `CFLAGS`/`CXXFLAGS`/`LDFLAGS` directly as that might lead to configure-time tests failing.
-This will produce outputs as follows:
+### Build Results Layout
+
+Once the build has concluded, it will have produced its output artifacts in the following structure:
+
+* Product binaries will be dropped in `artifacts\bin\coreclr\<OS>.<arch>.<configuration>` folder.
+* A NuGet package, _Microsoft.Dotnet.CoreCLR_, will be created under `artifacts\bin\coreclr\<OS>.<arch>.<configuration>\.nuget` folder.
+* Test binaries (if built) will be dropped under `artifacts\tests\coreclr\<OS>.<arch>.<configuration>` folder. However, remember the root build script will not build the tests. The instructions for working with tests (building and running) are [in the testing doc](/docs/workflow/testing/coreclr/testing.md).
+* The build places logs in `artifacts\log` and these are useful when the build fails.
+* The build places all of its intermediate output in the `artifacts\obj\coreclr` directory.
-- Product binaries will be dropped in `artifacts\bin\coreclr\<OS>.<arch>.<flavor>` folder.
-- A NuGet package, Microsoft.Dotnet.CoreCLR, will be created under `artifacts\bin\coreclr\<OS>.<arch>.<flavor>\.nuget` folder.
-- Test binaries will be dropped under `artifacts\tests\coreclr\<OS>.<arch>.<flavor>` folder. However, the root build script will not build the tests.
+If you want to force a full rebuild of the subsets you specified when calling the build script, pass the `-rebuild` flag to it, in addition to any other arguments you might require.
-The build places logs in `artifacts\log` and these are useful when the build fails.
+## Platform-Specific Instructions
-The build places all of its intermediate output in the `artifacts\obj\coreclr` directory, so if you remove that directory you can force a
-full rebuild.
+Now that you've got the general idea on how the _CoreCLR_ builds work, here are some further documentation links on platform-specific caveats and features.
-To build CoreCLR, the root build script invokes the `src\coreclr\build.cmd` (or build.sh) script. To build the CoreCLR tests, you must use this script.
-Use `build -?` to learn about the options to this script.
+* [Build CoreCLR on Windows](windows-instructions.md)
+* [Build CoreCLR on MacOS](macos-instructions.md)
+* [Build CoreCLR on Linux](linux-instructions.md)
+* [Build CoreCLR on FreeBSD](freebsd-instructions.md)
-See [Running Tests](../../testing/coreclr/testing.md) for instructions on running the tests.
+We also have specific instructions for building _NativeAOT_ [here](/docs/workflow/building/coreclr/nativeaot.md).
+## Testing CoreCLR
+For testing your build, the [testing docs](/docs/workflow/testing/coreclr/testing.md) have detailed instructions on how to do it.
-Cross Compilation for ARM on Windows
-==================================
+# Cross-Building for Different Architectures and Operating Systems
-Building ARM for Windows can be done using cross compilation.
+* [Windows Cross-Building](#windows-cross-building)
+ * [Cross-Compiling for ARM32 and ARM64 on Windows](#cross-compiling-for-arm32-and-arm64-on-windows)
+ * [Cross-Compiling for x86 on Windows](#cross-compiling-for-x86-on-windows)
+* [MacOS Cross-Building](#macos-cross-building)
+* [Linux Cross-Building](#linux-cross-building)
+ * [Generating the ROOTFS](#generating-the-rootfs)
+ * [ROOTFS for FreeBSD](#rootfs-for-freebsd)
+ * [Cross-Compiling CoreCLR](#cross-compiling-coreclr)
+ * [CoreCLR for FreeBSD](#coreclr-for-freebsd)
+ * [Cross-Compiling CoreCLR for other VFP Configurations](#cross-compiling-coreclr-for-other-vfp-configurations)
+ * [Building the Cross-Targeting Tools](#building-the-cross-targeting-tools)
+* [Cross-Building using Docker](#cross-building-using-docker)
+ * [Cross-Compiling for ARM32 and ARM64 with Docker](#cross-compiling-for-arm32-and-arm64-with-docker)
+ * [Cross-Compiling for FreeBSD with Docker](#cross-compiling-for-freebsd-with-docker)
-Requirements
-------------
+This guide will go more in-depth on how to do cross-building across multiple operating systems and architectures. It's worth mentioning this is not an any-to-any scenario. Only the combinations explained here are possible/supported. If/When any other combinations get supported/discovered, this document will get updated accordingly.
-Install the ARM tools and Windows SDK, as described [here](https://github.com/dotnet/runtime/blob/main/docs/workflow/requirements/windows-requirements.md).
+## Windows Cross-Building
-Cross compiling CoreCLR
------------------------
+This section will go over cross-compiling on Windows. Currently, Windows allows you to cross-compile from x64 to basically any other architecture.
-Build using "arm" as the architecture. For example:
+### Cross-Compiling for ARM32 and ARM64 on Windows
- C:\runtime> build.cmd -subset clr.runtime -arch arm -c debug
+To do cross-compilation for ARM32/ARM64 on Windows, first make sure you have the appropriate tools and Windows SDK installed. This is described in detail in the [Windows requirements doc](/docs/workflow/requirements/windows-requirements.md#visual-studio).
--or-
+Once you have all the required dependencies, it is a straightforward process. Windows knows how to cross-build behind curtains, so all you have to do is specify which architecture you want to build for:
- C:\runtime> src\coreclr\build-runtime.cmd -arm -debug
-
-Cross Compilation for ARM64 on macOS
-================================================
-
-The toolset required to do native compilation described in [macOS requirements](https://github.com/dotnet/runtime/blob/main/docs/workflow/requirements/macos-requirements.md) has cross compilation capabilities with no additional installs required. The only needed change is to use the `-cross` flag in the build scripts. For example, a release CoreCLR runtime that targets ARM64 can be built using the following commandline from the root of the repo:
-
- ./build.sh -subset clr.runtime -c release -cross -arch arm64 /p:CrossBuild=true
-
-Cross Compilation for ARM, ARM64 or x86 on Linux
-================================================
-
-Through cross compilation, on Linux it is possible to build CoreCLR for arm or arm64. Note that this documentation exists to explain using `runtime/eng/common/build-rootfs.sh`. This will build a rootfs and then use it to cross build. Newer documentation [linux-instructions.md](linux-instructions.md) exists which leverages docker to use a prebuilt environment to cross build.
-
-Requirements for targeting Debian based distros
-------------------------------------------------
-
-You need a Debian based host and the following packages need to be installed:
-
- ~/runtime/ $ sudo apt-get install qemu qemu-user-static binfmt-support debootstrap
+```cmd
+.\build.cmd -s clr -c Release -arch arm64
+```
-In addition, to cross compile CoreCLR the binutils for the target are required. So for arm you need:
+### Cross-Compiling for x86 on Windows
- ~/runtime/ $ sudo apt-get install binutils-arm-linux-gnueabihf
+Building for x86 doesn't require any additional software installed or configured, since all the x64 build tools also have the capability of building for x86. Just specify it when calling the build script:
-and conversely for arm64:
+```cmd
+.\build.cmd -s clr -c Release -arch x86
+```
- ~/runtime/ $ sudo apt-get install binutils-aarch64-linux-gnu
+## MacOS Cross-Building
-and for armel (ARM softfp):
+This section will go over cross-compiling on MacOS. Currently, MacOS allows you to cross-compile between x64 and ARM64.
- ~/runtime/ $ sudo apt-get install binutils-arm-linux-gnueabi
+Similarly to targeting Windows x86, the native tooling you installed back in the [MacOS requirements doc](/docs/workflow/requirements/macos-requirements.md) has the capabilities to effectuate the cross-compilation. You have simply to pass the `-cross` flag, along with the designated architecture. For example, for an arm64 build on an Intel x64 Mac:
+```bash
+./build.sh -s clr -c Release --cross -a arm64
+```
-Requirements for targeting ARM or ARM64 Alpine Linux
------------------------------------------------------
+## Linux Cross-Building
-You can use any Linux distro as a host. The qemu, qemu-user-static and binfmt-support packages need to be installed (the names may be different for some distros).
+This section will go over cross-compiling on Linux. Currently, Linux allows you to cross-compile from x64 to ARM32 and ARM64, as well as to other Unix-based operating systems, like FreeBSD and Alpine.
-In addition, to cross compile CoreCLR, the binutils for Alpine need to be built from the https://github.com/richfelker/musl-cross-make repo, since they are not available as packages.
+### Generating the ROOTFS
-To build them, use the following steps:
-* Clone the repo
-* Create a new config.mak file in the root directory of the repo and add the following lines into it:
- * `TARGET = armv6-alpine-linux-musleabihf` for ARM or `TARGET = aarch64-alpine-linux-musl` for ARM64
- * `OUTPUT = /usr`
- * `BINUTILS_CONFIG=--enable-gold=yes`
-* Run `make` with current directory set to the root of the repo
-* Run `sudo make install`
+Before you can attempt to do any Linux cross-building, you will need to generate the _ROOTFS_ corresponding to the platform you want to target. The script located in `eng/common/cross/build-rootfs.sh` is in charge of effectuating this task. Note that this script must be run with `sudo`, as it needs to make some symlinks to the system that would not be allowed otherwise.
-Generating the rootfs
----------------------
-The `eng/common/cross/build-rootfs.sh` script can be used to download the files needed for cross compilation. It will generate a rootfs as this is what CoreCLR targets.
+For example, let's try generating a _ROOTFS_ targeting Ubuntu 18 (Bionic) for ARM64:
- Usage: ./eng/common/cross/build-rootfs.sh [BuildArch] [LinuxCodeName] [lldbx.y] [--skipunmount]
- BuildArch can be: arm(default), armel, arm64, x86
- LinuxCodeName - optional, Code name for Linux, can be: trusty(default), vivid, wily, xenial or alpine. If BuildArch is armel, LinuxCodeName is jessie(default) or tizen.
- lldbx.y - optional, LLDB version, can be: lldb3.6(default), lldb3.8. This is ignored when building rootfs for Alpine Linux.
+```bash
+sudo ./eng/common/cross/build-rootfs.sh arm64 bionic
+```
-The `build-rootfs.sh` script must be run as root as it has to make some symlinks to the system, it will by default generate the rootfs in `eng/common/cross/rootfs/<BuildArch>` however this can be changed by setting the `ROOTFS_DIR` environment variable.
+The _rootfs_ binaries will be placed in `.tools/rootfs/<arch>`. So, for this example, it would be `.tools/rootfs/arm64`. Note that the Linux codename argument is optional, and if you omit it, the script will pick its default one.
-For example, to generate an arm rootfs:
+It is also possible to have `build-rootfs.sh` generate its output elsewhere. For that, you have to set the environment variable `ROOTFS_DIR` to the path where you want your _rootfs_ binaries to be placed in.
- ~/runtime/ $ sudo ./eng/common/cross/build-rootfs.sh arm
+#### ROOTFS for FreeBSD
-You can choose Linux code name to match your target, give `vivid` for `Ubuntu 15.04`, `wily` for `Ubuntu 15.10`. The default is `trusty`, version `Ubuntu 14.04`.
+Generating the _ROOTFS_ for FreeBSD cross-compiling is virtually the same as for other Linux distributions in other architectures. The only difference is you have to specify it so. For example, for an x64 cross-compilation for FreeBSD 13:
- ~/runtime/ $ sudo ./eng/common/cross/build-rootfs.sh arm wily
+```bash
+sudo ./eng/common/cross/build-rootfs.sh x64 freebsd13
+```
-and if you wanted to generate the rootfs elsewhere:
+### Cross-Compiling CoreCLR
- ~/runtime/ $ sudo ROOTFS_DIR=/home/cross/arm ./eng/common/cross/build-rootfs.sh arm
+Once you have your _ROOTFS_ generated, make sure to set the environment variable `ROOTFS_DIR` to where your binaries are located if you didn't do so in the previous step. Then, build normally and pass the `--cross` flag to the build script:
-For example, to generate an armel rootfs:
+```bash
+export ROOTFS_DIR=/path/to/runtime/.tools/rootfs/arm64
+./build.sh --subset clr --configuration Release --arch arm64 --cross
+```
- ~/runtime/ $ sudo ./eng/common/cross/build-rootfs.sh armel
+Like with any other build, you'll find the built binaries at `artifacts/bin/coreclr/Linux.<arch>.<configuration>`. For our example, it would be `artifacts/bin/coreclr/Linux.arm64.Release`.
-You can choose code name to match your target, give `jessie` for `Debian`, `tizen` for `Tizen`. The default is `jessie`.
+#### CoreCLR for FreeBSD
- ~/runtime/ $ sudo ./eng/common/cross/build-rootfs.sh armel tizen
+Very similarly to generating the _ROOTFS_, cross-building for FreeBSD follows the same process as for other architectures, which is described above. The only difference is that, in addition to the `--cross` flag, you also have to specify it is for FreeBSD by means of the `--os` flag:
-and if you wanted to generate the rootfs elsewhere:
+```bash
+export ROOTFS_DIR=/path/to/runtime/.tools/rootfs/x64
+./build.sh --subset clr --configuration Release --cross --os FreeBSD
+```
- ~/runtime/ $ sudo ROOTFS_DIR=/home/armel ./eng/common/cross/build-rootfs.sh armel tizen
+#### Cross-Compiling CoreCLR for other VFP Configurations
+The default ARM compilation configuration for CoreCLR is armv7-a with thumb-2 instruction set, and VFPv3 floating point with 32 64-bit FPU registers.
-Cross compiling CoreCLR
------------------------
-`ROOTFS_DIR` must be set when running `build-runtime.sh`.
+CoreCLR JIT requires 16 64-bit or 32 32-bit FPU registers.
- ~/runtime/ $ ROOTFS_DIR=/home/arm ./build.sh --subset clr.runtime --arch arm -c debug -v verbose --cross
+A set of FPU configuration options have been provided in the build scripts to accommodate different CPU types. These FPU configuration options are:
--or-
+* _CLR\_ARM\_FPU\_TYPE_: Translates to a value given to the `-mfpu` compiler option. Please refer to your compiler documentation for possible options.
+* _CLR\_ARM\_FPU\_CAPABILITY_: Used by the PAL code to decide which FPU registers should be saved and restored during context switches (the supported options are 0x3 and 0x7):
+ * Bit 0 unused always set to 1.
+ * Bit 1 corresponds to 16 64-bit FPU registers.
+ * Bit 2 corresponds to 32 64-bit FPU registers.
- ~/runtime/ $ ROOTFS_DIR=/home/arm ./src/coreclr/build-runtime.sh -arm -debug -verbose -cross
+For example, if you wanted to support armv7 CPU with VFPv3-d16, you'd use the following compile options:
-As usual, the resulting binaries will be found in `artifacts/bin/coreclr/TargetOS.BuildArch.BuildType/`
+```bash
+./build.sh --subset clr --configuration Release --cross --arch arm --cmakeargs "-DCLR_ARM_FPU_CAPABILITY=0x3" --cmakeargs "-DCLR_ARM_FPU_TYPE=vfpv3-d16"
+```
-Cross compiling CoreCLR for Other VFP configurations
-----------------------------------------------------------
-The default arm compilation configuration for CoreCLR is armv7-a with thumb-2 instruction set and
-VFPv3 floating point with 32 64-bit FPU registers.
+### Building the Cross-Targeting Tools
-CoreCLR JIT requires 16 64-bit or 32 32-bit FPU registers.
+Certain parts of the build process need some native components that are built for the current machine architecture, regardless of whichever you are targeting. These tools are referred to as cross-targeting tools or "cross tools". There are two categories of these tools today:
-A set of FPU configuration options have been provided via build-runtime.sh to accommodate different CPU types.
-These FPU configuration options are: CLR_ARM_FPU_CAPABILITY and CLR_ARM_FPU_TYPE.
+* Crossgen2 JIT Tools
+* Diagnostic Libraries
-CLR_ARM_FPU_TYPE translates to a value given to -mfpu compiler option. Please refer to
-your compiler documentation for possible options.
+The Crossgen2 JIT tools are used to run Crossgen2 on libraries built during the current build, such as during the `clr.nativecorelib` stage. Under normal circumstances, you should have no need to worry about this, since these tools are automatically built when using the `.\build.cmd` or `./build.sh` scripts at the root of the repo to build any of the CoreCLR native files.
-CLR_ARM_FPU_CAPABILITY is used by the PAL code to decide which FPU registers should be saved and
-restored during context switches.
+However, you might find yourself needing to (re)build them because either you made changes to them, or you built CoreCLR in a different way using `build-runtime.sh` instead of the usual default script at the root of the repo. To build these tools, you need to run the `src/coreclr/build-runtime.sh` script, and pass the `-hostarch` flag with the architecture of the host machine, alongside the `-component crosscomponents` flag to specify that you only want to build the cross-targeting tools. Retaking our previous example of building for ARM64 using an x64 Linux machine:
-Bit 0 unused always set to 1.
-Bit 1 corresponds to 16 64-bit FPU registers.
-Bit 2 corresponds to 32 64-bit FPU registers.
+```bash
+./src/coreclr/build-runtime.sh -arm64 -hostarch x64 -component crosscomponents -cmakeargs "-DCLR_CROSS_COMPONENTS_BUILD=1"
+```
-Supported options are 0x3 and 0x7.
+The output of running this command is placed in `artifacts/bin/coreclr/Linux.<target_arch>.<configuration>/<host_arch>`. For our example, it would be `artifacts/bin/coreclr/Linux.arm64.Release/x64`.
-If you wanted to support armv7 CPU with VFPv3-d16, you'd use the following compile options:
+On Windows, you can build these cross-targeting diagnostic libraries with the `linuxdac` and `alpinedac` subsets from the root `build.cmd` script. That said, you can also use the `build-runtime.cmd` script, like with Linux. These builds also require you to pass the `-os` flag to specify the target OS. For example:
-```
-./src/coreclr/build-runtime.sh -cross -arm -cmakeargs -DCLR_ARM_FPU_CAPABILITY=0x3 -cmakeargs -DCLR_ARM_FPU_TYPE=vfpv3-d16
+```cmd
+.\src\coreclr\build-runtime.cmd -arm64 -hostarch x64 -os Linux -component crosscomponents -cmakeargs "-DCLR_CROSS_COMPONENTS_BUILD=1"
```
-Building the Cross-Targeting Tools
-------------------------------------
+If you're building the cross-components in powershell, you'll need to wrap `"-DCLR_CROSS_COMPONENTS_BUILD=1"` with single quotes (`'`) to ensure things are escaped correctly for CMD.
-Some parts of our build process need some native components that are built for the current machine architecture, even when you are building for a different target architecture. These tools are referred to as cross-targeting tools or "cross tools". There are two categories of these tools today:
+## Cross-Building using Docker
-- Crossgen2 JIT tools
-- Diagnostic libraries
+When it comes to building, Docker offers the most flexibility when it comes to targeting different Linux platforms and other similar Unix-based ones, like FreeBSD. This is thanks to the multiple existing Docker images already configured for doing such cross-platform building, and Docker's ease of use of running out of the box on Windows machines with [WSL](https://docs.microsoft.com/windows/wsl/about) enabled, installed, and up and running, as well as Linux machines.
-The Crossgen2 JIT tools are used to run Crossgen2 on libraries built during the current build, such as during the clr.nativecorelib stage. These tools are automatically built when using the `./build.cmd` or `./build.sh` scripts at the root of the repo to build any of the CoreCLR native files, but they are not automatically built when using the `build-runtime.cmd/sh` scripts. To build these tools, you need to pass the `-hostarch` flag with the architecture of the host machine and the `-component crosscomponents` flag to specify that you only want to build the cross-targeting tools. For example:
+### Cross-Compiling for ARM32 and ARM64 with Docker
-```
-./src/coreclr/build-runtime.sh -arm -hostarch x64 -component crosscomponents -cmakeargs -DCLR_CROSS_COMPONENTS_BUILD=1
-```
+As mentioned in the [Linux Cross-Building section](#linux-cross-building), the _ROOTFS\_DIR_ environment variable has to be set to the _crossrootfs_ location. The prereqs Docker images already have _crossrootfs_ built, so you only need to specify it when creating the Docker container by means of the `-e` flag. These locations are specified in the [Docker Images table](/docs/workflow/building/coreclr/linux-instructions.md#docker-images).
-On Windows, the cross-targeting diagnostic libraries are built with the `linuxdac` and `alpinedac` subsets from the root `build.cmd` script, but they can also be built manually with the `build-runtime.cmd` scripts. These builds also require you to pass the `-os` flag to specify the target OS. For example:
+In addition, you also have to specify the `--cross` flag with the target architecture. For example, the following command would create a container to build CoreCLR for Linux ARM64:
-```
-src\coreclr\build-runtime.cmd -arm64 -hostarch x64 -os Linux -component crosscomponents -cmakeargs "-DCLR_CROSS_COMPONENTS_BUILD=1"
+```bash
+docker run --rm -v <RUNTIME_REPO_PATH>:/runtime -w /runtime -e ROOTFS_DIR=/crossrootfs/arm64 mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-cross-arm64-20220427171722-6e40d49 ./build.sh --subset clr --cross --arch arm64 --clang9
```
-If you're building the cross-components in powershell, you'll need to wrap `"-DCLR_CROSS_COMPONENTS_BUILD=1"` with single quotes (`'`) to ensure things are escaped correctly for CMD.
-
-Build System.Private.CoreLib on Ubuntu
---------------------------------------
-The following instructions assume you are on a Linux machine such as Ubuntu 14.04 x86 64bit.
+### Cross-Compiling for FreeBSD with Docker
-To build System.Private.CoreLib for Linux, run the following command:
+Using Docker to cross-build for FreeBSD is very similar to any other Docker Linux build. You only need to use the appropriate image and pass `--os` as well to specify this is not an architecture(-only) build. For example, to make a FreeBSD x64 build:
-```
- lgs@ubuntu ~/git/runtime/ $ ./build.sh --subset clr.corelib+clr.nativecorelib --arch arm -c debug -v verbose
-```
-
-The output is at `artifacts/bin/coreclr/<TargetOS>.arm.Debug/IL/System.Private.CoreLib.dll`.
-
-```
- lgs@ubuntu ~/git/runtime/ $ file ./artifacts/bin/coreclr/Linux.arm.Debug/IL/System.Private.CoreLib.dll
- ./artifacts/bin/coreclr/Linux.arm.Debug/IL/System.Private.CoreLib.dll: PE32 executable (DLL)
- (console) ARMv7 Thumb Mono/.NET assembly, for MS Windows
+```bash
+docker run --rm -v <RUNTIME_REPO_PATH>:/runtime -w /runtime -e ROOTFS_DIR=/crossrootfs/x64 mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-cross-freebsd-12-20220831130538-f13d79e ./build.sh --subset clr --cross --os FreeBSD
```
+++ /dev/null
-Using CrossGen to Create Native Images
-======================================
-
-Introduction
-------------
-
-When you create a .NET assembly using C# compiler, your assembly contains only MSIL code.
-When the app runs, the JIT compiler translates the MSIL code into native code, before the CPU can execute them.
-This execution model has some advantages. For example, your assembly code can be portable across all platforms and architectures that support .NET.
-However, this portability comes with a performance cost. Your app starts up more slowly because the JIT compiler has to spend time to translate the code.
-
-To help make your app start up faster, CoreCLR includes a tool called CrossGen, which can pre-compile the MSIL code into native code.
-
-Getting CrossGen
-----------------
-
-If you build CoreCLR yourself, the CrossGen tool (`crossgen.exe` on Windows, or `crossgen` on other platforms) is created as part of the build and stored in the same output directory as other CoreCLR binaries.
-If you install CoreCLR using a NuGet package, you can find CrossGen in the `tools` folder of the NuGet package.
-
-Regardless of how you obtain CrossGen, it is very important that it must match other CoreCLR binaries.
-- If you build CrossGen yourself, you should use it with runtime and `System.Private.CoreLib.dll` generated from the same build. Do not attempt to mix CrossGen from one build with binaries generated from another build.
-- If you install CrossGen from NuGet, make sure you use CrossGen from exactly the same NuGet package as the rest of your CoreCLR binaries. Do not attempt to mix binaries from multiple NuGet packages.
-
-If you do not follow the above rules, you are likely to encounter errors while running CrossGen.
-
-Using CrossGen
---------------
-
-In most cases, the build script automatically runs CrossGen to create the native image for `System.Private.CoreLib.dll`.
-When this happens, you will find `System.Private.CoreLib.ni.dll` in your output directory.
-`System.Private.CoreLib.dll` is the MSIL assemblies created by the C# compiler, while `System.Private.CoreLib.ni.dll` is the native images that contain CPU-specific code.
-Once the build is done, you only need `System.Private.CoreLib.ni.dll` to use CoreCLR.
-The original MSIL assemblies are no longer needed by the runtime.
-
-If you installed CoreCLR from a NuGet package, `System.Private.CoreLib.ni.dll` is included in the package.
-
-If for some reason you did not get `System.Private.CoreLib.dll` with the rest of your CoreCLR, you can easily create it yourself using CrossGen.
-
-If your `System.Private.CoreLib.dll` and JIT compiler (`clrjit.dll` on Windows or `libclrjit.*` on other platforms) are all in the same directory as CrossGen itself, you can compile `System.Private.CoreLib.dll` with the following commands (first two commands for Windows, next two commands for other platforms):
-
- .\crossgen.exe System.Private.CoreLib.dll
- ./crossgen System.Private.CoreLib.dll
-
-If your files are scattered in different directories, or if you want to create native images for other assemblies, the command line is slightly more complex:
-
- .\crossgen.exe /JITPath path\clrjit.dll /p "path1;path2;..." path\assemblyName.dll
- ./crossgen -JITPath path/libclrjit.so -p "path1:path2:..." path/assemblyName.dll
-
-The /p is used to specify the locations of all the dependencies of the input assembly, including the input assembly itself.
-You should use full paths for these locations. Relative paths do not always work.
-If there are multiple paths, separate them with semicolons (`;`) on Windows, or colons (`:`) on non-Windows platforms.
-
-Another way to reference assemblies is by using the /r switch, which takes file names instead of paths. This switch can be used multiple times, once for each reference file, since is not a semicolon/colon separated list of file names. Example:
-
- .\crossgen.exe /JITPath path\clrjit.dll /r "reference1.dll" /r "path\reference2.dll" /r ... path\assemblyName.dll
- ./crossgen -JITPath path/libclrjit.so -r "reference1.dll" -r "path/reference2.dll" /r ... path/assemblyName.dll
-
-It is generally a good idea to enclose the path list or file names in quotes to protect any special characters from the shell.
-Note that the /r and /p options cannot be used together.
-
-Using native images
--------------------
-
-Running CrossGen on an assembly creates a "native image" file, with the extension of `.ni.dll` or `.ni.exe`.
-You should include the native images in your app, either by replacing the original MSIL assemblies with the native images or by putting the native images next to the MSIL assemblies.
-When the native images are present, the CoreCLR runtime will automatically use it instead of the original MSIL assemblies.
-
-Common errors
--------------
-
-The following are some of the command errors while creating or using native images:
-- "Error: Could not load file or assembly '...' or one of its dependencies. The system cannot find the file specified. (Exception from HRESULT: 0x80070002)": CrossGen wasn't able to find a particular dependency that it needs. Verify that you have the assembly specified in the error message, and make sure its location is included in `/Platform_Assemblies_Paths`.
-- CoreCLR unable to initialize: While there are many possible causes of this error, one possibility is a mismatch between System.Private.CoreLib.ni.dll and coreclr.dll (or libcoreclr.so). Make sure they come from the same build or NuGet package.
-- "Unable to load Jit Compiler": Please get a copy of `clrjit.dll` (or `libclrjit.so` or `libclrjit.dylib`, depending on your platform), and place it in the same directory as CrossGen (or use /JITPath option). You can either build `clrjit.dll` yourself or get it from `Microsoft.NETCore.Jit` NuGet package. To avoid possible issues, please use `clrjit.dll` from the same build as `crossgen.exe` if possible.
-Build CoreCLR on FreeBSD
-======================
+# Build CoreCLR on FreeBSD
-This guide will walk you through building CoreCLR on FreeBSD and running Hello World. We'll start by showing how to set up your environment from scratch.
+* [Build using Docker](#build-using-docker)
+* [Build using cross-compilation on Linux](#build-using-cross-compilation-on-linux)
+* [Build directly on FreeBSD](#build-directly-on-freebsd)
+* [Old Documentation](#old-documentation)
+ * [Environment](#environment)
+ * [Toolchain Setup](#toolchain-setup)
+ * [Debugging CoreCLR (Optional)](#debugging-coreclr-optional)
+ * [Git Setup](#git-setup)
+ * [Build the Runtime](#build-the-runtime)
+ * [Build the Framework Native Components](#build-the-framework-native-components)
+ * [Build the Framework Managed Components](#build-the-framework-managed-components)
+ * [Download Dependencies](#download-dependencies)
+ * [Install Mono](#install-mono)
+ * [Download the NuGet Client](#download-the-nuget-client)
+ * [Download NuGet Packages](#download-nuget-packages)
+ * [Compile an App](#compile-an-app)
+ * [Run your App](#run-your-app)
+ * [Run the test suite](#run-the-test-suite)
+ * [Note on Clang/LLVM versions](#note-on-clangllvm-versions)
-Environment
-===========
+This guide will walk you through building CoreCLR on FreeBSD.
-These instructions are written assuming FreeBSD 10.1-RELEASE, since that's the release the team uses.
+As mentioned in the [FreeBSD requirements doc](/docs/workflow/requirements/freebsd-requirements.md), there are three ways to go on about to build CoreCLR for FreeBSD:
+
+* Build using Docker
+* Build using cross-compilation on your own Linux environment
+* Build directly on FreeBSD
+
+## Build using Docker
+
+Building for FreeBSD with Docker follows a very similar workflow to using Docker for Linux. Since this is also a cross-building scenario, the instructions are found in the [Docker section of the cross-building doc](/docs/workflow/building/coreclr/cross-building.md#cross-compiling-for-freebsd-with-docker).
+
+## Build using cross-compilation on Linux
+
+Ensure you have all of the prerequisites installed from the [Linux Requirements](/docs/workflow/requirements/linux-requirements.md), and the additional ones listed in the [FreeBSD Requirements](/docs/workflow/requirements/freebsd-requirements.md#linux-environment).
+
+Once that is done, refer to the [Linux section of the cross-building doc](/docs/workflow/building/coreclr/cross-building.md#linux-cross-building). There are detailed instructions on how to cross-compile using your Linux environment, including a section dedicated to FreeBSD building.
+
+## Build directly on FreeBSD
+
+Ensure you have all of the prerequisites installed from the [FreeBSD Requirements](/docs/workflow/requirements/freebsd-requirements.md).
+
+Instructions for building directly on FreeBSD coming soon!
+
+Meanwhile, here are the old instructions.
+
+## Old Documentation
+
+These instructions were written quite a while ago, and they may or may not work today. Updated instructions coming soon.
+
+### Environment
These instructions assume you use the binary package tool `pkg` (analog to `apt-get` or `yum` on Linux) to install the environment. Compiling the dependencies from source using the ports tree might work too, but is untested.
Minimum RAM required to build is 1GB. The build is known to fail on 512 MB VMs ([Issue 4069](https://github.com/dotnet/runtime/issues/4069)).
-Toolchain Setup
----------------
+#### Toolchain Setup
Install the following packages for the toolchain:
The command above will install Clang and LLVM 3.7. For information on building CoreCLR with other versions, see section on [Clang/LLVM versions](#note-on-clangllvm-versions).
-Debugging CoreCLR (Optional)
-----------------------------
+### Debugging CoreCLR (Optional)
Note: This step is not required to build CoreCLR itself. If you intend on hacking or debugging the CoreCLR source code, you need to follow these steps. You must follow these steps *before* starting the build itself.
./src/pal/tests/palsuite/runpaltests.sh $PWD/artifacts/obj/FreeBSD.x64.Debug $PWD/artifacts/paltestout
```
-Git Setup
----------
+### Git Setup
This guide assumes that you've cloned the corefx and coreclr repositories into `~/git/corefx` and `~/git/coreclr` on your FreeBSD machine and the corefx and coreclr repositories into `D:\git\corefx` and `D:\git\coreclr` on Windows. If your setup is different, you'll need to pay careful attention to the commands you run. In this guide, I'll always show what directory I'm in on both the FreeBSD and Windows machine.
-Build the Runtime
-=================
+### Build the Runtime
To build the runtime on FreeBSD, run build.sh from the root of the coreclr repository:
janhenke@freebsd-frankfurt:~/git/coreclr % cp artifacts/Product/FreeBSD.x64.Debug/libcoreclr*.so ~/coreclr-demo/runtime
```
-Build the Framework Native Components
-======================================
+### Build the Framework Native Components
```sh
janhenke@freebsd-frankfurt:~/git/corefx$ ./build-native.sh
janhenke@freebsd-frankfurt:~/git/corefx$ cp artifacts/FreeBSD.x64.Debug/Native/*.so ~/coreclr-demo/runtime
```
-Build the Framework Managed Components
-======================================
+### Build the Framework Managed Components
We don't _yet_ have support for building managed code on FreeBSD, so you'll need a Windows machine with clones of both the CoreCLR and CoreFX projects.
System.Console.dll System.Diagnostics.Debug.dll corerun libcoreclr.so libcoreclrpal.so System.Private.CoreLib.dll
```
-Download Dependencies
-=====================
+### Download Dependencies
The rest of the assemblies you need to run are presently just facades that point to System.Private.CoreLib. We can pull these dependencies down via NuGet (which currently requires Mono).
janhenke@freebsd-frankfurt:~/git/coreclr % cd ~/coreclr-demo/packages
```
-Install Mono
-------------
+### Install Mono
If you don't already have Mono installed on your system, use the pkg tool again:
janhenke@freebsd-frankfurt:~/coreclr-demo/packages % sudo pkg install mono
```
-Download the NuGet Client
--------------------------
+### Download the NuGet Client
Grab NuGet (if you don't have it already)
```sh
janhenke@freebsd-frankfurt:~/coreclr-demo/packages % curl -L -O https://nuget.org/nuget.exe
```
-Download NuGet Packages
------------------------
+### Download NuGet Packages
With Mono and NuGet in hand, you can use NuGet to get the required dependencies.
janhenke@freebsd-frankfurt:~/coreclr-demo/packages % find . -wholename '*/aspnetcore50/*.dll' -exec cp -n {} ~/coreclr-demo/runtime \;
```
-Compile an App
-==============
+### Compile an App
Now you need a Hello World application to run. You can write your own, if you'd like. Personally, I'm partial to the one on corefxlab which will draw Tux for us.
janhenke@freebsd-frankfurt:~/coreclr-demo/runtime % mcs /nostdlib /noconfig /r:../packages/System.Console.4.0.0-beta-22703/lib/contract/System.Console.dll /r:../packages/System.Runtime.4.0.20-beta-22703/lib/contract/System.Runtime.dll HelloWorld.cs
```
-Run your App
-============
+### Run your App
You're ready to run Hello World! To do that, run corerun, passing the path to the managed exe, plus any arguments. The HelloWorld from corefxlab will print a daemon if you pass "freebsd" as an argument, so:
A sample that builds Hello World on FreeBSD using the correct references but via XBuild or MonoDevelop would be great! Some of our processes (e.g. the System.Private.CoreLib build) rely on Windows-specific tools, but we want to figure out how to solve these problems for FreeBSD as well. There's still a lot of work ahead, so if you're interested in helping, we're ready for you!
-Run the test suite
-==================
+### Run the test suite
If you've made changes to the CoreCLR PAL code, you might want to run the PAL tests directly to validate your changes.
This can be done after a clean build, without any other dependencies.
This should run all the tests associated with the PAL.
-Note on Clang/LLVM versions
-===========================
+### Note on Clang/LLVM versions
The minimum version to build CoreCLR is Clang 3.5 or above.
-Build CoreCLR on Linux
-======================
+# Build CoreCLR on Linux
+
+* [Build using Docker](#build-using-docker)
+ * [Docker Images](#docker-images)
+* [Build using your own Environment](#build-using-your-own-environment)
+ * [Set the maximum number of file-handles](#set-the-maximum-number-of-file-handles)
+* [Build the Runtime](#build-the-runtime)
+ * [Cross-Compilation](#cross-compilation)
+* [Create the Core_Root](#create-the-core_root)
This guide will walk you through building CoreCLR on Linux.
-First, set up your environment to build using the instructions [here](../../requirements/linux-requirements.md).
+As mentioned in the [Linux requirements doc](/docs/workflow/requirements/linux-requirements.md), there are two options to build CoreCLR on Linux:
+
+* Build using Docker.
+* Build using your own environment.
-Choose one of the following options for building.
+## Build using Docker
-[Build using Docker](#Build-using-Docker)
+Building using Docker will require that you choose the correct image for your environment.
-[Build with own environment](#Environment)
+Note that the OS is strictly speaking not important. For example if you are on Ubuntu 20.04 and build using the Ubuntu 18.04 x64 image there should be no issues. You can even use Linux images on a Windows OS if you have [WSL](https://docs.microsoft.com/windows/wsl/about) enabled. However, note that you can't run multiple OS's on the same _Docker Daemon_, as it takes resources from the underlying kernel as needed. In other words, you can run either Linux on WSL, or Windows containers. You have to switch between them if you need both, and restart Docker.
-Build using Docker
-==================
+The target architecture is more important, as building arm32 using the x64 image will not work. There will be missing _rootfs_ components required by the build. See [Docker Images](#docker-images) below, for more information on choosing an image to build with.
-Building using Docker will require that you choose the correct image for your environment. Note that the OS is strictly speaking not extremely important, for example if you are on Ubuntu 18.04 and build using the Ubuntu 16.04 x64 image there should be no issues. The target architecture is more important, as building arm32 using the x64 image will not work: there will be missing rootfs components required by the build. See [Docker Images](#Docker-Images), below, for more information on choosing an image to build with.
+**NOTE**: The image's architecture has to match your machine's supported platforms. For example, you can't run arm32 images on an x64 machine. But you could run x64 and arm64 images on an M1 Mac, for example. This is thanks to the _Rosetta_ emulator that Apple Silicon provides. Same case applies to running x86 on an x64 Windows machine thanks to Windows' _SYSWOW64_. Likewise, you can run Linux arm32 images on a Linux arm64 host.
-Please note that when choosing an image choosing the same image as the host os you are running on you will allow you to run the product/tests outside of the docker container you built in.
+Please note that choosing the same image as the host OS you are running on will allow you to run the product/tests outside of the docker container you built in.
-Once you have chosen an image the build is one command run from the root of the runtime repository:
+Once you have chosen an image, the build is one command run from the root of the runtime repository:
-```sh
-docker run --rm -v <RUNTIME_REPO_PATH>:/runtime -w /runtime mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-20200508132555-78cbb55 ./build.sh -subset clr -clang9
+```bash
+docker run --rm -v <RUNTIME_REPO_PATH>:/runtime -w /runtime mcr.microsoft.com/dotnet-buildtools/prereqs:mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-20210714125435-9b5bbc2 ./build.sh --subset clr
```
Dissecting the command:
-- `--rm`: erase the created container after use.
-- `-v <RUNTIME_REPO_PATH>:/runtime`: mount the runtime repository under `/runtime`. Replace `<RUNTIME_REPO_PATH>` with the full path to your `runtime` repo clone, e.g., `-v /home/user/runtime:/runtime`.
-- `-w: /runtime`: set /runtime as working directory for the container.
-- `mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-20200508132555-78cbb55`: Docker image name.
-- `./build.sh`: command to be run in the container: run the root build command.
-- `-subset clr`: build the clr subset (excluding libraries and installers).
-- `-clang9`: argument to use clang 9 for the build (the only compiler in the build image).
-
-If you are attempting to cross build for arm or arm64 then use the crossrootfs location to set the ROOTFS_DIR. The command would add `-e ROOTFS_DIR=<crossrootfs location>`. See [Docker Images](#Docker-Images) for the crossrootfs location. In addition you will need to specify `-cross`. For example:
+* `--rm`: Erase the created container after use.
+* `-v <RUNTIME_REPO_PATH>:/runtime`: Mount the runtime repository under `/runtime`. Replace `<RUNTIME_REPO_PATH>` with the full path to your `runtime` repo clone, e.g., `-v /home/user/runtime:/runtime`.
+* `-w: /runtime`: Set /runtime as working directory for the container.
+* `mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-20210714125435-9b5bbc2`: Docker image name.
+* `./build.sh`: Command to be run in the container: run the root build command.
+* `-subset clr`: Build the clr subset (excluding libraries and installers).
-```sh
-docker run --rm -v <RUNTIME_REPO_PATH>:/runtime -w /runtime -e ROOTFS_DIR=/crossrootfs/arm64 mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-cross-arm64-20200508132638-b2c2436 ./build.sh -arch arm64 -subset clr -cross -clang9
-```
-
-Note that instructions on building the crossrootfs location can be found at [cross-building.md](cross-building.md). These instructions are suggested only if there are plans to change the rootfs, or the Docker images for arm/arm64 are insufficient for you build.
+To do cross-building for ARM32 or ARM64 using Docker, you need to use either specific images designated for this purpose, or configure your own. Detailed information on this can be found in the [cross-building doc](/docs/workflow/building/coreclr/cross-building.md#cross-building-using-docker).
-Docker Images
-=============
+### Docker Images
-This table of images might often become stale as we change our images as our requirements change. The images used for our our official builds can be found in [the pipeline resources](../../../../eng/pipelines/common/templates/pipeline-with-resources.yml) of our Azure DevOps builds under the `container` key of the platform you plan to build.
+This table of images might often become stale as we change our images as our requirements change. The images used for our official builds can be found in [the pipeline resources](/eng/pipelines/common/templates/pipeline-with-resources.yml) of our Azure DevOps builds under the `container` key of the platform you plan to build.
-| OS | Target Arch | Image location | crossrootfs location | Clang Version |
-| --------------------------- | --------------- | ---------------------------------------------------------------------------------------------------- | -------------------- | ------------- |
-| Alpine | x64 | `mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.13-WithNode-20210910135845-c401c85` | - | -clang5.0 |
-| CentOS 7 (build for RHEL 7) | x64 | `mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-20210714125435-9b5bbc2` | - | -clang9 |
-| Ubuntu 16.04 (x64, arm ROOTFS) | arm32 (armhf) | `mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-cross-20210719121212-8a8d3be` | `/crossrootfs/arm` | -clang9 |
-| Ubuntu 16.04 (x64, arm64 ROOTFS) | arm64 (arm64v8) | `mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-cross-arm64-20210719121212-8a8d3be` | `/crossrootfs/arm64` | -clang9 |
-| Alpine (x64, arm ROOTFS) | arm | `mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-cross-arm-alpine-20210923140502-78f7860` | `/crossrootfs/arm64` | -clang9 |
-| Alpine (x64, arm64 ROOTFS) | arm64 (arm64v8) | `mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-cross-arm64-alpine-20210719121212-b2c2436` | `/crossrootfs/arm64` | -clang9 |
+| OS | Target Arch | Image location | crossrootfs location | Clang Version |
+| --------------------------------- | --------------- | ---------------------------------------------------------------------------------------------------- | -------------------- | ------------- |
+| Alpine | x64 | `mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.13-WithNode-20210910135845-c401c85` | - | -clang5.0 |
+| CentOS 7 (build for RHEL 7) | x64 | `mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-20210714125435-9b5bbc2` | - | -clang9 |
+| Ubuntu 18.04 (x64, arm ROOTFS) | arm32 (armhf) | `mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-cross-20220426130400-6e40d49` | `/crossrootfs/arm` | -clang9 |
+| Ubuntu 18.04 (x64, arm64 ROOTFS) | arm64 (arm64v8) | `mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-cross-arm64-20220427171722-6e40d49` | `/crossrootfs/arm64` | -clang9 |
+| Alpine (x64, arm ROOTFS) | arm | `mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-cross-arm-alpine-20210923140502-78f7860` | `/crossrootfs/arm64` | -clang9 |
+| Alpine (x64, arm64 ROOTFS) | arm64 (arm64v8) | `mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-cross-arm64-alpine-20210923140502-78f7860` | `/crossrootfs/arm64` | -clang9 |
+| FreeBSD (x64 ROOTFS) | x64 | `mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-cross-freebsd-12-20210917001307-f13d79e` | `/crossrootfs/x64` | (default) |
-Environment
-===========
+These Docker images are built using the Dockerfiles maintained in the [dotnet-buildtools-prereqs-docker repo](https://github.com/dotnet/dotnet-buildtools-prereqs-docker).
-Git Setup
----------
+## Build using your own Environment
-This guide assumes that you've cloned the runtime repository.
+Ensure you have all of the prerequisites installed from the [Linux Requirements](/docs/workflow/requirements/linux-requirements.md).
-Set the maximum number of file-handles
---------------------------------------
+### Set the maximum number of file-handles
-To ensure that your system can allocate enough file-handles for the libraries build run `sysctl fs.file-max`. If it is less than 100000, add `fs.file-max = 100000` to `/etc/sysctl.conf`, and then run `sudo sysctl -p`.
+To ensure that your system can allocate enough file-handles for the libraries build, run the command in your terminal `sysctl fs.file-max`. If it is less than 100000, add `fs.file-max = 100000` to `/etc/sysctl.conf`, and then run `sudo sysctl -p`.
-Build the Runtime and System.Private.CoreLib
-=============================================
+## Build the Runtime
-To build the runtime on Linux, run build.sh to build the CoreCLR subset category of the runtime:
+To build CoreCLR on Linux, run `build.sh` while specifying the `clr` subset:
-```
-./build.sh -subset clr
+```bash
+./build.sh --subset clr <other args>
```
-After the build is completed, there should some files placed in `artifacts/bin/coreclr/Linux.x64.Debug`. The ones we are most interested in are:
+After the build is completed, there should be some files placed in `artifacts/bin/coreclr/Linux.<arch>.<configuration>` (for example `artifacts/bin/coreclr/Linux.x64.Release`). The most important binaries are the following:
-* `corerun`: The command line host. This program loads and starts the CoreCLR runtime and passes the managed program you want to run to it.
+* `corerun`: The command line host. This program loads and starts the CoreCLR runtime and passes the managed program (e.g. `program.dll`) you want to run with it.
* `libcoreclr.so`: The CoreCLR runtime itself.
* `System.Private.CoreLib.dll`: The core managed library, containing definitions of `Object` and base functionality.
-Create the Core_Root
-===================
+### Cross-Compilation
-The Core_Root folder will contain the built binaries, generated by `build.sh`, as well as the library packages required to run tests. It is required that you build
-the libraries subset (`-subset libs`) before this command can be run. Note that this by default searches the libraries in Release mode, regardless of the runtime
-configuration you specify. If you built your libs in another configuration, then you have to pass down the appropriate flag `/p:LibrariesConfiguration=<your_config>`.
+Just like you can use specialized Docker images, you can also do any of the supported cross-builds for ARM32 or ARM64 on your own Linux environment. Detailed instructions are found in the [cross-building doc](/docs/workflow/building/coreclr/cross-building.md#linux-cross-building).
-```
-./src/tests/build.sh generatelayoutonly
-```
-
-After the build is complete you will find the output in the `artifacts/tests/coreclr/Linux.x64.Debug/Tests/Core_Root` folder.
+## Create the Core_Root
-Running a single test
-===================
-
-After `src/tests/build.sh` is run, `corerun` from the Core_Root folder is ready to be run. This can be done by using the full absolute path to `corerun`, or by setting an environment variable to the Core_Root folder.
-
-```sh
-export CORE_ROOT=/runtime/artifacts/tests/coreclr/Linux.x64.Debug/Tests/Core_Root
-$CORE_ROOT/corerun hello_world.dll
-```
+The Core_Root provides one of the main ways to test your build. Full instructions on how to build it in the [CoreCLR testing doc](/docs/workflow/testing/coreclr/testing.md), and we also have a detailed guide on how to use it for your own testing in [its own dedicated doc](/docs/workflow/testing/using-corerun-and-coreroot.md).
--- /dev/null
+# Build CoreCLR on MacOS
+
+* [Environment](#environment)
+* [Build the Runtime](#build-the-runtime)
+ * [Cross-Compilation](#cross-compilation)
+* [Create the Core_Root](#create-the-core_root)
+
+This guide will walk you through building CoreCLR on macOS.
+
+## Environment
+
+Ensure you have all of the prerequisites installed from the [MacOS Requirements](/docs/workflow/requirements/macos-requirements.md).
+
+## Build the Runtime
+
+To build CoreCLR on MacOS, run `build.sh` while specifying the `clr` subset:
+
+```bash
+./build.sh --subset clr <other args>
+```
+
+After the build has completed, there should be some files placed in `artifacts/bin/coreclr/OSX.<arch>.<configuration>` (for example `artifacts/bin/coreclr/OSX.x64.Release`). The most important binaries are the following:
+
+* `corerun`: The command line host. This program loads and starts the CoreCLR runtime and passes the managed program (e.g. `program.dll`) you want to run with it.
+* `libcoreclr.dylib`: The CoreCLR runtime itself.
+* `System.Private.CoreLib.dll`: The core managed library, containing definitions of `Object` and base functionality.
+
+### Cross-Compilation
+
+It is possible to get a MacOS ARM64 build using an Intel x64 Mac and vice versa, an x64 one using an M1 Mac. Instructions on how to do this are in the [cross-building doc](/docs/workflow/building/coreclr/cross-building.md#macos-cross-building).
+
+## Create the Core_Root
+
+The Core_Root provides one of the main ways to test your build. Full instructions on how to build it in the [CoreCLR testing doc](/docs/workflow/testing/coreclr/testing.md), and we also have a detailed guide on how to use it for your own testing in [its own dedicated doc](/docs/workflow/testing/using-corerun-and-coreroot.md).
# Native AOT Developer Workflow
+* [Building](#building)
+ * [Using built binaries](#using-built-binaries)
+ * [Building packages](#building-packages)
+* [High Level Overview](#high-level-overview)
+* [Visual Studio Solutions](#visual-studio-solutions)
+* [Convenience Visual Studio "repro" project](#convenience-visual-studio-repro-project)
+* [Running tests](#running-tests)
+ * [Running library tests](#running-library-tests)
+* [Design Documentation](#design-documentation)
+* [Further Reading](#further-reading)
+
The Native AOT toolchain can be currently built for Linux (x64/arm64), macOS (x64) and Windows (x64/arm64).
## Building
-1. [Install pre-requisites](../../README.md#build-requirements)
+1. [Install pre-requisites](/docs/workflow/README.md#build-requirements)
1. Run `build[.cmd|.sh] clr.aot+libs -rc [Debug|Release]` from the repo root to build binaries for local development. This will build individual components, but not the NuGet packages and builds much faster.
### Using built binaries
### Building packages
Run `build[.cmd|.sh] -c Release` from the repo root to build the NativeAOT toolchain packages. The build will place the toolchain packages at `artifacts\packages\Release\Shipping`. To publish your project using these packages:
- - Add the package directory to your `nuget.config` file. For example, add `<add key="local" value="C:\runtime\artifacts\packages\Release\Shipping" />`
- - Run `dotnet publish --packages pkg -r [win-x64|linux-x64|osx-64] -c [Debug|Release]` to publish your project. `--packages pkg` option restores the package into a local directory that is easy to cleanup once you are done. It avoids polluting the global nuget cache with your locally built dev package.
+
+* Add the package directory to your `nuget.config` file. For example, add `<add key="local" value="C:\runtime\artifacts\packages\Release\Shipping" />`
+* Run `dotnet publish --packages pkg -r [win-x64|linux-x64|osx-64] -c [Debug|Release]` to publish your project. `--packages pkg` option restores the package into a local directory that is easy to cleanup once you are done. It avoids polluting the global nuget cache with your locally built dev package.
## High Level Overview
The repository has a number of Visual Studio Solutions files (`*.sln`) that are useful for editing parts of the repository. Build the repo from command line first before building using the solution files. Remember to select the appropriate configuration that you built. By default, `build.cmd` builds Debug x64 and so `Debug` and `x64` must be selected in the solution build configuration drop downs.
-- `src\coreclr\nativeaot\nativeaot.sln`. This solution is for the runtime libraries.
-- `src\coreclr\tools\aot\ilc.sln`. This solution is for the compiler.
+Solutions related to this:
+
+* `src\coreclr\nativeaot\nativeaot.sln`. This solution is for the runtime libraries.
+* `src\coreclr\tools\aot\ilc.sln`. This solution is for the compiler.
Typical workflow for working on the compiler:
-- Open `ilc.sln` in Visual Studio
-- Set "ILCompiler" project in solution explorer as your startup project
-- Set Working directory in the project Debug options to your test project directory, e.g. `C:\test`
-- Set Application arguments in the project Debug options to the response file that was generated by regular native aot publishing of your test project, e.g. `@obj\Release\net6.0\win-x64\native\HelloWorld.ilc.rsp`
-- Build & run using **F5**
+
+* Open `ilc.sln` in Visual Studio
+* Set "ILCompiler" project in solution explorer as your startup project
+* Set Working directory in the project Debug options to your test project directory, e.g. `C:\test`
+* Set Application arguments in the project Debug options to the response file that was generated by regular native aot publishing of your test project, e.g. `@obj\Release\net6.0\win-x64\native\HelloWorld.ilc.rsp`
+* Build & run using **F5**
## Convenience Visual Studio "repro" project
The workflow looks like this:
-- Build the repo using the Building instructions above
-- Open the ilc.sln solution described above. This solution contains the compiler, but also an unrelated project named "repro". This repro project is a small Hello World. You can place any piece of C# you would like to compile in it. Building the project will compile the source code into IL, but also generate a response file that is suitable to pass to the AOT compiler.
-- Make sure you set the solution configuration in VS to the configuration you just built (e.g. x64 Debug).
-- In the ILCompiler project properties, on the Debug tab, set the "Application arguments" to the generated response file. This will be a file such as "C:\runtime\artifacts\bin\repro\x64\Debug\compile-with-Release-libs.rsp". Prefix the path to the file with "@" to indicate this is a response file so that the "Application arguments" field looks like "@some\path\to\file.rsp".
-- Build & run ILCompiler using **F5**. This will compile the repro project into an `.obj` file. You can debug the compiler and set breakpoints in it at this point.
-- The last step is linking the object file into an executable so that we can launch the result of the AOT compilation.
-- Open the src\coreclr\tools\aot\ILCompiler\reproNative\reproNative.vcxproj project in Visual Studio. This project is configured to pick up the `.obj` file we just compiled and link it with the rest of the runtime.
-- Set the solution configuration to the tuple you've been using so far (e.g. x64 Debug)
-- Build & run using **F5**. This will run the platform linker to link the obj file with the runtime and launch it. At this point you can debug the runtime and the various System.Private libraries.
+* Build the repo using the Building instructions above
+* Open the ilc.sln solution described above. This solution contains the compiler, but also an unrelated project named "repro". This repro project is a small Hello World. You can place any piece of C# you would like to compile in it. Building the project will compile the source code into IL, but also generate a response file that is suitable to pass to the AOT compiler.
+* Make sure you set the solution configuration in VS to the configuration you just built (e.g. x64 Debug).
+* In the ILCompiler project properties, on the Debug tab, set the "Application arguments" to the generated response file. This will be a file such as "C:\runtime\artifacts\bin\repro\x64\Debug\compile-with-Release-libs.rsp". Prefix the path to the file with "@" to indicate this is a response file so that the "Application arguments" field looks like "@some\path\to\file.rsp".
+* Build & run ILCompiler using **F5**. This will compile the repro project into an `.obj` file. You can debug the compiler and set breakpoints in it at this point.
+* The last step is linking the object file into an executable so that we can launch the result of the AOT compilation.
+* Open the src\coreclr\tools\aot\ILCompiler\reproNative\reproNative.vcxproj project in Visual Studio. This project is configured to pick up the `.obj` file we just compiled and link it with the rest of the runtime.
+* Set the solution configuration to the tuple you've been using so far (e.g. x64 Debug)
+* Build & run using **F5**. This will run the platform linker to link the obj file with the runtime and launch it. At this point you can debug the runtime and the various System.Private libraries.
## Running tests
Sometimes it's handy to be able to rebuild the managed test manually or run the compilation under a debugger. A response file that was used to invoke the ahead of time compiler can be found in `$repo_root\artifacts\tests\coreclr\obj\[Windows|Linux|OSX].x64.[Debug|Release]\Managed`.
-For more advanced scenarios, look for at [Building Test Subsets](../../testing/coreclr/windows-test-instructions.md#building-test-subsets) and [Generating Core_Root](../../testing/coreclr/windows-test-instructions.md#generating-core_root)
+For more advanced scenarios, look for at [Building the Tests](/docs/workflow/testing/coreclr/testing.md#building-the-tests) and [Building the Core_Root](../../testing/coreclr/testing.md#building-the-coreroot)
### Running library tests
-Build library tests by passing the `libs.tests` subset together with the `/p:TestNativeAot=true` to build the libraries, i.e. `clr.aot+libs+libs.tests /p:TestNativeAot=true` together with the full arguments as specified [above](#building). Then, to run a specific library, go to the tests directory of the library and run the usual command to run tests for the library (see [Running tests for a single library](../../testing/libraries/testing.md#running-tests-for-a-single-library)) but add the `/p:TestNativeAot=true` and the build configuration that was used, i.e. `dotnet.cmd build /t:Test /p:TestNativeAot=true -c Release`.
+Build library tests by passing the `libs.tests` subset together with the `/p:TestNativeAot=true` to build the libraries, i.e. `clr.aot+libs+libs.tests /p:TestNativeAot=true` together with the full arguments as specified [above](#building). Then, to run a specific library, go to the tests directory of the library and run the usual command to run tests for the library (see [Running tests for a single library](/docs/workflow/testing/libraries/testing.md#running-tests-for-a-single-library)) but add the `/p:TestNativeAot=true` and the build configuration that was used, i.e. `dotnet.cmd build /t:Test /p:TestNativeAot=true -c Release`.
## Design Documentation
-- [ILC Compiler Architecture](../../../design/coreclr/botr/ilc-architecture.md)
-- [Managed Type System](../../../design/coreclr/botr/managed-type-system.md)
+* [ILC Compiler Architecture](/docs/design/coreclr/botr/ilc-architecture.md)
+* [Managed Type System](/docs/design/coreclr/botr/managed-type-system.md)
+
+## Further Reading
+
+If you want to know more about working with _NativeAOT_ in general, you can check out their [more in-depth docs](/src/coreclr/nativeaot/docs/README.md) in the `src/coreclr/nativeaot` subtree.
+++ /dev/null
-Build CoreCLR on macOS
-=====================
-
-This guide will walk you through building CoreCLR on macOS. We'll start by showing how to set up your environment from scratch.
-
-Environment
-===========
-
-Ensure you have all of the prerequisites installed from the [macOS Requirements](/docs/workflow/requirements/macos-requirements.md).
-
-Build the Runtime and System.Private.CoreLib
-============================================
-
-To Build CoreCLR, run build.sh to build the CoreCLR subset category of the runtime:
-
-```
-./build.sh -subset clr
-```
-
-After the build has completed, there should some files placed in `artifacts/bin/coreclr/OSX.x64.Debug`. The ones we are interested in are:
-
-- `corerun`: The command line host. This program loads and starts the CoreCLR runtime and passes the managed program you want to run to it.
-- `libcoreclr.dylib`: The CoreCLR runtime itself.
-- `System.Private.CoreLib.dll`: Microsoft Core Library.
-
-Create the Core_Root
-===================
-
-The Core_Root folder will contain the built binaries, generated by `build.sh`, as well as the library packages required to run tests. It is required that you build
-the libraries subset (`-subset libs`) before this command can be run. Note that this by default searches the libraries in Release mode, regardless of the runtime
-configuration you specify. If you built your libs in another configuration, then you have to pass down the appropriate flag `/p:LibrariesConfiguration=<your_config>`.
-
-```
-./src/tests/build.sh generatelayoutonly
-```
-
-After the build is complete you will be able to find the output in the `artifacts/tests/coreclr/OSX.x64.Debug/Tests/Core_Root` folder.
-
-Running a single test
-===================
-
-After `src/tests/build.sh` is run, corerun from the Core_Root folder is ready to be run. This can be done by using the full absolute path to corerun, or by setting
-an environment variable to the Core_Root folder.
-
-```sh
-export CORE_ROOT=/path/to/runtime/artifacts/tests/coreclr/OSX.x64.Debug/Tests/Core_Root
-$CORE_ROOT/corerun hello_world.dll
-```
--- /dev/null
+# Build CoreCLR on Windows
+
+* [Environment](#environment)
+* [Build the Runtime](#build-the-runtime)
+ * [Using the Command Line](#using-the-command-line)
+ * [CoreCLR](#coreclr)
+ * [Cross-Compilation](#cross-compilation)
+ * [Core_Root](#core_root)
+ * [Native ARM64 (Experimental)](#native-arm64-experimental)
+ * [Using Visual Studio](#using-visual-studio)
+
+This guide will walk you through building CoreCLR on Windows.
+
+## Environment
+
+Ensure you have all of the prerequisites installed from the [Windows Requirements](/docs/workflow/requirements/windows-requirements.md).
+
+## Build the Runtime
+
+As the main development platform for .NET, Windows has more than one way to build the _dotnet/runtime_ repo. You can do it entirely through the command-line as you'd do in Linux and MacOS, or you can leverage the integration with Visual Studio that the repository provides. This document will go on about both ways so you can choose whichever fits your needs best, or even use a combination of both.
+
+### Using the Command Line
+
+This section will cover how to build the product using the command-line.
+
+#### CoreCLR
+
+To build CoreCLR on Windows, run `build.cmd` while specifying the `clr` subset:
+
+```cmd
+.\build.cmd -subset clr <other args>
+```
+
+After the build has completed, there should be some files placed in `artifacts/bin/coreclr/windows.<arch>.<configuration>` (for example `artifacts/bin/coreclr/windows.x64.Release`). The most important binaries are the following:
+
+* `corerun.exe`: The command line host. This program loads and starts the CoreCLR runtime and passes the managed program (e.g. `program.dll`) you want to run with it.
+* `coreclr.dll`: The CoreCLR runtime itself.
+* `System.Private.CoreLib.dll`: The core managed library, containing definitions of `Object` and base functionality.
+
+##### Cross-Compilation
+
+It is possible to get Windows x86, ARM32, and ARM64 builds using an x64 machine. Instructions on how to do this are in the [cross-building doc](/docs/workflow/building/coreclr/cross-building.md#windows-cross-building).
+
+#### Core_Root
+
+The Core_Root provides one of the main ways to test your build. Full instructions on how to build it in the [CoreCLR testing doc](/docs/workflow/testing/coreclr/testing.md), and we also have a detailed guide on how to use it for your own testing in [its own dedicated doc](/docs/workflow/testing/using-corerun-and-coreroot.md).
+
+#### Native ARM64 (Experimental)
+
+Building natively on ARM64 requires you to have installed the appropriate ARM64 build tools and Windows SDK, as specified in the [Windows requirements doc](/docs/workflow/requirements/windows-requirements.md).
+
+Once those requirements are satisfied, you have to specify you are doing an Arm64 build, and explicitly tell the build script you want to use `MSBuild`. `Ninja` is not yet supported on Arm64 platforms.
+
+```cmd
+build.cmd -s clr -c Release -arch arm64 -msbuild
+```
+
+Since this is still in an experimental phase, the recommended way for building ARM64 is cross-compiling from an x64 machine. Instructions on how to do this can be found at the [cross-building doc](/docs/workflow/building/coreclr/cross-building.md#cross-compiling-for-arm32-and-arm64).
+
+### Using Visual Studio
+
+Visual Studio instructions coming soon!
-How to Debug CoreCLR AOT Compilers
-=================
+# Debugging CoreCLR AOT Compilers
+
+* [Important concerns to be aware of when debugging the managed compilers](#important-concerns-to-be-aware-of-when-debugging-the-managed-compilers)
+* [Built-in debugging aids in the managed compilers](#built-in-debugging-aids-in-the-managed-compilers)
+* [Example of debugging a test application in Crossgen2](#example-of-debugging-a-test-application-in-crossgen2)
+* [Debugging compilation graph](#debugging-compilation-graph)
CoreCLR comes with two AOT compilers that are built around a shared C# codebase - crossgen2 and ilc. Crossgen2 generates ReadyToRun images that are loadable into the JIT-based CoreCLR runtime. ILC generates platform-specific object files (COFF on Windows, ELF on Linux, Mach-O on macOS) that can be linked with the NativeAOT flavor of the CoreCLR runtime to form a self-contained executable or a shared library.
The managed AOT compilers bring with them a number of new challenges for debugging the compilation process. Fortunately, in addition to challenges, the compilers are designed to enhance various parts of the debugging experience.
-Important concerns to be aware of when debugging the managed compilers
----------------------------------
+## Important concerns to be aware of when debugging the managed compilers
+
+* Other than the JIT, the AOT compilers are managed applications
+* By default the AOT compilers use a multi-core compilation strategy
+* A compilation process will have 2 copies of the JIT in the process at the same time, the one used to compile the target, and the one used to compile compiler itself.
+* The compilers don't parse environment variables for controlling the JIT (or any other behavior), all behavior is controlled via the command line
+* The AOT compiler command line as generated by the project system is quite complex
-- Other than the JIT, the AOT compilers are managed applications
-- By default the AOT compilers use a multi-core compilation strategy
-- A compilation process will have 2 copies of the JIT in the process at the same time, the one used to compile the target, and the one used to compile compiler itself.
-- The compilers don't parse environment variables for controlling the JIT (or any other behavior), all behavior is controlled via the command line
-- The AOT compiler command line as generated by the project system is quite complex
+## Built-in debugging aids in the managed compilers
+When debugging a multi-threaded component of the compiler and not investigating a multi-threading issue itself, it is generally advisable to disable the use of multiple threads. To do this use the `--parallelism 1` switch to specify that the maximum parallelism of the process shall be 1.
-Built in debugging aids in the managed compilers
----------------------------------
+When debugging the behavior of compiling a single method, the compiler may be instructed to only compile a single method. This is done via the various --singlemethod options:
-- When debugging a multi-threaded component of the compiler and not investigating a multi-threading issue itself, it is generally advisable to disable the use of multiple threads.
-To do this use the `--parallelism 1` switch to specify that the maximum parallelism of the process shall be 1.
+* These options work by specifying a specific method by type, method name, generic method arguments, and if those are insufficiently descriptive to uniquely identify the method, by index. Types are described using the same format that the managed Type.GetType(string) function uses, which is documented in the normal .NET documentation. As this format can be quite verbose, the compiler provides a `--print-repro-instructions` switch which will print the arguments necessary to compile a function to the console.
+* `--singlemethodindex` is used in cases where the method signature is the only distinguishing factor about the method. An index is used instead of a series of descriptive arguments, as specifying a signature exactly is extraordinarily complicated.
+* Repro args will look like the following ``--singlemethodtypename "Internal.Runtime.CompilerServices.Unsafe" --singlemethodname As --singlemethodindex 2 --singlemethodgenericarg "System.Runtime.Intrinsics.Vector256`1[[System.SByte]]" --singlemethodgenericarg "System.Runtime.Intrinsics.Vector256`1[[System.Double]]"``
-- When debugging the behavior of compiling a single method, the compiler may be instructed to only compile a single method. This is done via the various --singlemethod options
+Since the compilers are by default multi-threaded, they produce results fairly quickly even when compiling using a _Debug_ variant of the JIT. In general, when debugging JIT issues, we recommend using the _Debug_ JIT regardless of which environment caused a problem.
- - These options work by specifying a specific method by type, method name, generic method arguments, and if those are insufficiently descriptive to uniquely identify the method, by index. Types are described using the same format that the managed Type.GetType(string) function uses, which is documented in the normal .NET documentation. As this format can be quite verbose, the compiler provides a `--print-repro-instructions` switch which will print the arguments necessary to compile a function to the console.
- - `--singlemethodindex` is used in cases where the method signature is the only distinguishing factor about the method. An index is used instead of a series of descriptive arguments, as specifying a signature exactly is extraordinarily complicated.
- - Repro args will look like the following ``--singlemethodtypename "Internal.Runtime.CompilerServices.Unsafe" --singlemethodname As --singlemethodindex 2 --singlemethodgenericarg "System.Runtime.Intrinsics.Vector256`1[[System.SByte]]" --singlemethodgenericarg "System.Runtime.Intrinsics.Vector256`1[[System.Double]]"``
+The compilers support nearly arbitrary cross-targeting, including OS and architecture cross targeting. The only restriction is that 32-bit architectures cannot compile targeting 64-bit architectures. This cross-targeting flexibility allows the use of the debugging environment most convenient to the developer. In particular, if there is an issue which crosses the managed/native boundary, it is often convenient to debug using the mixed mode debugger on Windows x64.
-- Since the compilers are by default multi-threaded, they produce results fairly quickly even when compiling using a Debug variant of the JIT. In general, when debugging JIT issues we recommend using the debug JIT regardless of which environment caused a problem.
+If the correct set of assemblies/command line arguments are passed to the compiler, it should produce binary identical output on all platforms.
-- The compilers support nearly arbitrary cross-targeting, including OS and architecture cross targeting. The only restriction is that 32bit architecture cannot compile targeting a 64bit architecture. This allows the use of the debugging environment most convenient to the developer. In particular, if there is an issue which crosses the managed/native boundary, it is often convenient to debug using the mixed mode debugger on Windows X64.
- - If the correct set of assemblies/command line arguments are passed to the compiler, it should produce binary identical output on all platforms.
- - The compiler does not check the OS/Architecture specified for input assemblies, which allows compiling using a non-architecture/OS matched version of the framework to target an arbitrary target. While this isn't useful for producing the diagnosing all issues, it can be cheaply used to identify the general behavior of a change on the full swath of supported architectures.
+Note that the compiler does not check the OS/Architecture specified for input assemblies, which allows compiling using a non-architecture/OS matched version of the framework to target an arbitrary target. While this might not be totally useful for diagnosing all issues, it can be cheaply used to identify the general behavior of a change on the full swath of supported architectures.
Control compilation behavior by using the `--targetos` and `--targetarch` switches. The default behavior is to target the compiler's own OS/Arch pair, but all 64bit versions of the compilers are capable of targeting arbitrary OS/Arch combinations.
+
At the time of writing the current supported sets of valid arguments are:
-| Command line arguments
+
+| Command line arguments |
| --- |
| `--targetos windows --targetarch x86` |
| `--targetos windows --targetarch x64` |
|`--targetos osx --targetarch x64` |
|`--targetos osx --targetarch arm64` |
-- Passing special jit behavior flags to the compiler is done via the `--codegenopt` switch. As an example to turn on tailcall loop optimizations and dump all code compiled use a pair of them like `--codegenopt JitDump=* --codegenopt TailCallLoopOpt=1`.
+Passing special jit behavior flags to the compiler is done via the `--codegenopt` switch. As an example to turn on tailcall loop optimizations and dump all code compiled, use a pair of them like `--codegenopt NgenDump=* --codegenopt TailCallLoopOpt=1`.
-- When using the JitDump feature of the JIT, disable parallelism as described above or specify a single method to be compiled. Otherwise, output from multiple functions will be interleaved and inscrutable.
+When using the NgenDump feature of the JIT, disable parallelism as described above or specify a single method to be compiled. Otherwise, output from multiple functions will be interleaved and inscrutable.
-- Since there are 2 jits in the process, when debugging in the JIT, if the source files match up, there is a decent chance that a native debugger will stop at unfortunate and unexpected locations. This is extremely annoying, and to combat this, we generally recommend making a point of using a runtime which doesn't exactly match that of the compiler in use. However, if that isn't feasible, it is also possible to disable symbol loading in most native debuggers. For instance, in Visual Studio, one would use the "Specify excluded modules" feature.
+Since there are 2 jits in the process, when debugging in the JIT, if the source files match up, there is a decent chance that a native debugger will stop at unfortunate and unexpected locations. This is extremely annoying, and to combat this, we generally recommend making a point of using a runtime which doesn't exactly match that of the compiler in use. However, if that isn't feasible, it is also possible to disable symbol loading in most native debuggers. For instance, in Visual Studio, one would use the "Specify excluded modules" feature.
-- The compiler identifies the JIT to use by the means of a naming convention. By default it will use a JIT located in the same directory as the crossgen2.dll file. In addition there is support for a `--jitpath` switch to use a specific JIT. This option is intended to support A/B testing by the JIT team. The `--jitpath` option should only be used if the jit interface has not been changed. The JIT specified by the `--jitpath` switch must be compatible with the current settings of the `--targetos` and `--targetarch` switches.
+The compiler identifies the JIT to use by the means of a naming convention. By default it will use a JIT located in the same directory as the crossgen2.dll file. In addition there is support for a `--jitpath` switch to use a specific JIT. This option is intended to support A/B testing by the JIT team. The `--jitpath` option should only be used if the jit interface has not been changed. The JIT specified by the `--jitpath` switch must be compatible with the current settings of the `--targetos` and `--targetarch` switches.
-- In parallel to the crossgen2 project, there is a tool known as r2rdump. This tool can be used to dump the contents of a produced ReadyToRun image to examine what was actually produced in the final binary. It has a large multitude of options to control exactly what is dumped, but in general it is able to dump any image produced by crossgen2, and display its contents in a human readable fashion. Specify `--disasm` to display disassembly.
+In parallel to the crossgen2 project, there is a tool known as r2rdump. This tool can be used to dump the contents of a produced ReadyToRun image to examine what was actually produced in the final binary. It has a large multitude of options to control exactly what is dumped, but in general it is able to dump any image produced by crossgen2, and display its contents in a human readable fashion. Specify `--disasm` to display disassembly.
-- If there is a need to debug the dependency graph of the compiler (which is a very rare need at this time), there is a visualization tool located in src\coreclr\tools\aot\DependencyGraphViewer. To use that tool, compile it, and run it on Windows before the compilation begins. It will present a live view of the graph as it is generated and allow for exploration to determine why some node is in the graph. Every node in the graph has a unique id that is visible to this tool, and it can be used in parallel with a debugger to understand what is happening in the compilation process. Changes to improve the fairly horrible UI are encouraged.
+If there is a need to debug the dependency graph of the compiler (which is a very rare need at this time), there is a visualization tool located in src\coreclr\tools\aot\DependencyGraphViewer. To use that tool, compile it, and run it on Windows before the compilation begins. It will present a live view of the graph as it is generated and allow for exploration to determine why some node is in the graph. Every node in the graph has a unique id that is visible to this tool, and it can be used in parallel with a debugger to understand what is happening in the compilation process. Changes to improve the fairly horrible UI are encouraged.
-- When used in the official build system, the set of arguments passed to the compiler are extremely complex, especially with regards to the set of reference paths (each assembly is specified individually). To make it easier to use crossgen2 from the command line manually the tool will accept wildcards in its parsing of references. Please note that on Unix that the shell will happily expand these arguments by itself, which will not work correctly. In those situations enclose the argument in quotes to prevent the shell expansion.
+When used in the official build system, the set of arguments passed to the compiler are extremely complex, especially with regards to the set of reference paths (each assembly is specified individually). To make it easier to use crossgen2 from the command line manually the tool will accept wildcards in its parsing of references. Please note that on Unix that the shell will happily expand these arguments by itself, which will not work correctly. In those situations enclose the argument in quotes to prevent the shell expansion.
-- Crossgen2 supports a `--map` and `--mapcsv` arguments to produce map files of the produced output. These are primarily used for diagnosing size issues, as they describe the generated file in fairly high detail, as well as providing a number of interesting statistics about the produced output.
+Crossgen2 supports a `--map` and `--mapcsv` arguments to produce map files of the produced output. These are primarily used for diagnosing size issues, as they describe the generated file in fairly high detail, as well as providing a number of interesting statistics about the produced output.
-- ILC also supports the `--map` argument but the format is different from crossgen2 because the output format is different too.
+ILC also supports the `--map` argument but the format is different from crossgen2 because the output format is different too.
-- Diagnosing why a specific method failed to compile in crossgen2 can be done by passing the `--verbose` switch to crossgen2. This will print many things, but in particular it will print the reason why a compilation was aborted due to an R2R format limitation.
+Diagnosing why a specific method failed to compile in crossgen2 can be done by passing the `--verbose` switch to crossgen2. This will print many things, but in particular it will print the reason why a compilation was aborted due to an R2R format limitation.
-- The compilers can use either the version of dotnet that is used to build the product (as found by the dotnet.cmd or dotnet.sh script found in the root of the runtime repo) or it can use a sufficiently recent corerun.exe produced by constructing a test build. It is strongly recommended if using corerun.exe to use a release build of corerun for this purpose, as crossgen2 runs a very large amount of managed code. The version of corerun used does not need to come from the same build as the crossgen2.dll/ilc.dll that is being debugging. In fact, I would recommnend using a different enlistment to build that corerun to avoid any confusion.
+The compilers can use either the version of dotnet that is used to build the product (as found by the dotnet.cmd or dotnet.sh script found in the root of the runtime repo) or it can use a sufficiently recent corerun.exe produced by constructing a test build. It is strongly recommended if using corerun.exe to use a release build of corerun for this purpose, as crossgen2 runs a very large amount of managed code. The version of corerun used does not need to come from the same build as the crossgen2.dll/ilc.dll that is being debugging. In fact, I would recommnend using a different enlistment to build that corerun to avoid any confusion.
-- In the runtime testbed, each test can be commanded to compile with crossgen2 by using environment variables. Just set the `RunCrossgen2` variable to 1, and optionally set the `CompositeBuildMode` variable to 1 if you wish to see the R2R behavior with composite image creation.
+In the runtime testbed, each test can be commanded to compile with crossgen2 by using environment variables. Just set the `RunCrossgen2` variable to 1, and optionally set the `CompositeBuildMode` variable to 1 if you wish to see the R2R behavior with composite image creation.
-- By default the runtime test bed will simply use `dotnet` to run the managed compiler. If you run the test batch script from the root of the enlistment on Windows this will just work; otherwise, you must set the `__TestDotNetCmd` environment variable to point at copy of `dotnet` or `corerun` that can run the compiler. This is often the easiest way to run a simple test with the AOT compiler for developers practiced in the CoreCLR testbed. See the example below of various techniques to use when diagnosing issues under crossgen2.
+By default the runtime test bed will simply use `dotnet` to run the managed compiler. If you run the test batch script from the root of the enlistment on Windows this will just work; otherwise, you must set the `__TestDotNetCmd` environment variable to point at copy of `dotnet` or `corerun` that can run the compiler. This is often the easiest way to run a simple test with the AOT compiler for developers practiced in the CoreCLR testbed. See the example below of various techniques to use when diagnosing issues under crossgen2.
-- When attempting to build crossgen2, you must build the clr.tools subset. If rebuilding a component of the JIT and wanting to use that in your inner loop, you must build as well with either the clr.jit or clr.alljits subsets. If the jit interface is changed, the clr.runtime subset must also be rebuilt.
+When attempting to build crossgen2, you must build the clr.tools subset. If rebuilding a component of the JIT and wanting to use that in your inner loop, you must build as well with either the clr.jit or clr.alljits subsets. If the jit interface is changed, the clr.runtime subset must also be rebuilt.
-- After completion of a product build, a functional copy of crossgen2.dll will be located in a bin directory in a path like `bin\coreclr\windows.x64.Debug\crossgen2`. After creating a test native layout via a command such as `src\tests\build generatelayoutonly` then there will be a copy of crossgen2 located in the `%CORE_ROOT%\crossgen2` directory. The version of crossgen2 in the test core_root directory will have the appropriate files for running under either an x64 dotnet.exe or under the target architecture. This was done to make it somewhat easier to do cross platform development, and assumes the primary development machine is x64,
+After completion of a product build, a functional copy of crossgen2.dll will be located in a bin directory in a path like `bin\coreclr\windows.x64.Debug\crossgen2`. After creating a test native layout via a command such as `src\tests\build generatelayoutonly` then there will be a copy of crossgen2 located in the `%CORE_ROOT%\crossgen2` directory. The version of crossgen2 in the test core_root directory will have the appropriate files for running under either an x64 dotnet.exe or under the target architecture. This was done to make it somewhat easier to do cross platform development, and assumes the primary development machine is x64,
-- The object files generated by the ILC compiler contain debug information for method bodies and types in the platform specific format (CodeView on Windows, DWARF elsewhere). They also contain unwinding information in the platform format. As a result of that, NativeAOT executables can be debugged with platform debuggers (VS, WinDbg, GDB, LLDB) without any SOS-like extensions. They can also be inspected using disassemblers and tools that deal with native code (dumpbin, Ghidra, IDA). Make sure to pass `-g` command line argument to enable debug info generation.
+The object files generated by the ILC compiler contain debug information for method bodies and types in the platform specific format (CodeView on Windows, DWARF elsewhere). They also contain unwinding information in the platform format. As a result of that, NativeAOT executables can be debugged with platform debuggers (VS, WinDbg, GDB, LLDB) without any SOS-like extensions. They can also be inspected using disassemblers and tools that deal with native code (dumpbin, Ghidra, IDA). Make sure to pass `-g` command line argument to enable debug info generation.
-- The ILC compiler typically compiles the whole program - it loosely corresponds to the composite mode of crossgen2. There is a multifile mode, where each managed assembly corresponds to a single object file, but this mode is not shipping.
+The ILC compiler typically compiles the whole program - it loosely corresponds to the composite mode of crossgen2. There is a multifile mode, where each managed assembly corresponds to a single object file, but this mode is not shipping.
-- The object files generated by the ILC compiler are written out using an LLVM-based object writer (consumed as a NuGet package built out of the dotnet/llvm-project repo, branch objwriter/12.x). The object writer uses the LLVM assembler APIs (APIs meant to be used by tools that convert textual assembly into machine code) to emit object files in PE/ELF/Mach-O formats.
+The object files generated by the ILC compiler are written out using an LLVM-based object writer (consumed as a NuGet package built out of the dotnet/llvm-project repo, branch objwriter/12.x). The object writer uses the LLVM assembler APIs (APIs meant to be used by tools that convert textual assembly into machine code) to emit object files in PE/ELF/Mach-O formats.
+
+## Example of debugging a test application in Crossgen2
-Example of debugging a test application in Crossgen2
-================================================
This example is to demonstrate debugging of a simple test in the CoreCLR testbed.
The example assumes that `CORE_ROOT` is set appropriately, and that `__TestDotNetCmd` is set appropriately. See comments above for details on `__TestDotNetCmd`
The test begins by setting `RunCrossgen2=1` This will instruct the test batch script to run crossgen2 on input binaries. It will also create a copy of the input binaries which needs to be deleted if you modify the test. To do so, delete the directory where the test binaries are, and rebuild the test.
-```
+```cmd
C:\git2\runtime>set RunCrossgen2=1
C:\git2\runtime>c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\jit\Directed\Arrays\Complex1\Complex1.cmd
END EXECUTION - PASSED
PASSED
```
+
From that invocation you can see that crossgen2 was launched with a response file containing a list of arguments including all of the details for references. Then you can manually run the actual crossgen2 command, which is prefixed with the value of the `__TestDotNetCmd` environment variable. For instance, once I saw the above output, I copied and pasted the last command, and ran it.
-```
+
+```cmd
C:\git2\runtime>"dotnet" "c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\Tests\Core_Root\crossgen2\crossgen2.dll" @"c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\jit\Directed\Arrays\Complex1\\Complex1.dll.rsp" -r:c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\jit\Directed\Arrays\Complex1\IL-CG2\*.dll
C:\git2\runtime\.dotnet
Emitting R2R PE file: c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\jit\Directed\Arrays\Complex1\\Complex1.dll
And then wanted to debug the individual method compilation, and ran it with the `--print-repro-instructions` switch
-```
+```cmd
C:\git2\runtime>"dotnet" "c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\Tests\Core_Root\crossgen2\crossgen2.dll" @"c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\jit\Directed\Arrays\Complex1\\Complex1.dll.rsp" -r:c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\jit\Directed\Arrays\Complex1\IL-CG2\*.dll --print-repro-instructions
C:\git2\runtime\.dotnet
Single method repro args:--singlemethodtypename "Complex,Complex1" --singlemethodname mul_em --singlemethodindex 1
Emitting R2R PE file: c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\jit\Directed\Arrays\Complex1\\Complex1.dll
```
-I then wanted to see some more detail from the jit. To keep the size of this example small, I'm just using the `JitOrder=1`switch, but jit developers would more likely use `JitDump=*` switch.
-```
-C:\git2\runtime>"dotnet" "c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\Tests\Core_Root\crossgen2\crossgen2.dll" @"c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\jit\Directed\Arrays\Complex1\\Complex1.dll.rsp" -r:c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\jit\Directed\Arrays\Complex1\IL-CG2\*.dll --print-repro-instructions --singlemethodtypename "Complex_Array_Test,Complex1" --singlemethodname Main --singlemethodindex 1 --codegenopt JitOrder=1
+I then wanted to see some more detail from the jit. To keep the size of this example small, I'm just using the `NgenOrder=1`switch, but jit developers would more likely use `NgenDump=*` switch.
+
+```cmd
+C:\git2\runtime>"dotnet" "c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\Tests\Core_Root\crossgen2\crossgen2.dll" @"c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\jit\Directed\Arrays\Complex1\\Complex1.dll.rsp" -r:c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\jit\Directed\Arrays\Complex1\IL-CG2\*.dll --print-repro-instructions --singlemethodtypename "Complex_Array_Test,Complex1" --singlemethodname Main --singlemethodindex 1 --codegenopt NgenOrder=1
C:\git2\runtime\.dotnet
Single method repro args:--singlemethodtypename "Complex_Array_Test,Complex1" --singlemethodname Main --singlemethodindex 1
| Profiled | Method | Method has | calls | Num |LclV |AProp| CSE | Perf |bytes | x64 codesize|
And finally, as the last `--targetarch` and `--targetos` switch is the meaningful one, it is simple to target a different architecture for ad hoc exploration...
-```
-C:\git2\runtime>"dotnet" "c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\Tests\Core_Root\crossgen2\crossgen2.dll" @"c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\jit\Directed\Arrays\Complex1\\Complex1.dll.rsp" -r:c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\jit\Directed\Arrays\Complex1\IL-CG2\*.dll --print-repro-instructions --singlemethodtypename "Complex_Array_Test,Complex1" --singlemethodname Main --singlemethodindex 1 --codegenopt JitOrder=1 --targetarch arm64
+```cmd
+C:\git2\runtime>"dotnet" "c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\Tests\Core_Root\crossgen2\crossgen2.dll" @"c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\jit\Directed\Arrays\Complex1\\Complex1.dll.rsp" -r:c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\jit\Directed\Arrays\Complex1\IL-CG2\*.dll --print-repro-instructions --singlemethodtypename "Complex_Array_Test,Complex1" --singlemethodname Main --singlemethodindex 1 --codegenopt NgenOrder=1 --targetarch arm64
C:\git2\runtime\.dotnet
Single method repro args:--singlemethodtypename "Complex_Array_Test,Complex1" --singlemethodname Main --singlemethodindex 1
| Profiled | Method | Method has | calls | Num |LclV |AProp| CSE | Perf |bytes | arm64 codesize|
06000002 | | | f656934b | | fp | LOOP | 3 | 0 | 35 | 59 | 48 | 10 | 63828 | 490 | 1048 | 0 | Complex_Array_Test:Main(System.String[]):int
Emitting R2R PE file: c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\jit\Directed\Arrays\Complex1\\Complex1.dll
```
+
Note that the only difference in the command line was to pass the `--targetarch arm64` switch, and the JIT now compiles the method as arm64.
Finally, attaching a debugger to crossgen2.
Since this example uses `dotnet` as the `__TestDotNetCmd` you will need to debug the `c:\git2\runtime\.dotnet\dotnet.exe` process.
-```
-devenv /debugexe C:\git2\runtime\.dotnet\dotnet.exe "c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\Tests\Core_Root\crossgen2\crossgen2.dll" @"c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\jit\Directed\Arrays\Complex1\\Complex1.dll.rsp" -r:c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\jit\Directed\Arrays\Complex1\IL-CG2\*.dll --print-repro-instructions --singlemethodtypename "Complex_Array_Test,Complex1" --singlemethodname Main --singlemethodindex 1 --codegenopt JitOrder=1 --targetarch arm64
+```cmd
+devenv /debugexe C:\git2\runtime\.dotnet\dotnet.exe "c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\Tests\Core_Root\crossgen2\crossgen2.dll" @"c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\jit\Directed\Arrays\Complex1\\Complex1.dll.rsp" -r:c:\git2\runtime\artifacts\tests\coreclr\windows.x64.Debug\jit\Directed\Arrays\Complex1\IL-CG2\*.dll --print-repro-instructions --singlemethodtypename "Complex_Array_Test,Complex1" --singlemethodname Main --singlemethodindex 1 --codegenopt NgenOrder=1 --targetarch arm64
```
This will launch the Visual Studio debugger, with a solution setup for debugging the dotnet.exe process. By default this solution will debug the native code of the process only. To debug the managed components, edit the properties on the solution and set the `Debugger Type` to `Managed (.NET Core, .NET 5+)` or `Mixed (.NET Core, .NET 5+)`.
-# Debugging compilation graph
+## Debugging compilation graph
The AOT compilation is driven by a dependency graph. If you need to troubleshoot the dependency graph (to figure out why something was or wasn't generated) you can follow [this guide](debugging-compiler-dependency-analysis.md)
-Debugging Compiler Dependency Analysis
-============================
+# Debugging Compiler Dependency Analysis
-The general technique is to identify what node is missing from the graph or is erroneously present in the graph, and change the dependency analysis logic to adjust the graph. This document describes the various ways of debugging to identify what's happening.
+* [Dependency Graph Viewer](#dependency-graph-viewer)
+ * [Graphs View](#graphs-view)
+ * [Dependency Graph View](#dependency-graph-view)
+ * [Single Node Exploration](#single-node-exploration)
+* [Why DGML](#why-dgml)
-Analysis techniques for the dependency graph.
-1. Use the DependencyGraphViewer tool (if running on Windows). This tool is located in `src/coreclr/tools/aot/DependencyGraphViewer`
- - This is the only convenient way to examine the graph while also simultaneously debugging the compiler.
- - While this is currently Windows only due to use of WinForms, it would be fairly straightforward to make a command line based tool.
- - Dependency graph does not support multiple simultaneous logging facilities, so make sure that you do not set IlcGenerateDgmlFile or invoke ILCompiler with the DGML generation turned on.
-2. Pass command line switches to the compiler to generate a dependency graph dgml file. This will produce the same data as is viewable in the viewer tool, but in a textual xml format.
- - Future efforts may make the xml file loadable by the viewer tool.
-3. Instrument the compiler dependency analysis. (This may be necessary in cases where the viewer is unable to provide sufficient information about why the graph is structured as it is.)
+The general technique is to identify what node is missing from the graph, or is erroneously present in the graph, and change the dependency analysis logic to adjust the graph. This document describes the various ways of debugging to identify what's happening.
-DependencyGraphViewer
-====================================
+Analysis techniques for the dependency graph:
+
+* Use the _DependencyGraphViewer_ tool (if running on Windows). This tool is located in `src/coreclr/tools/aot/DependencyGraphViewer`
+ * This is the only convenient way to examine the graph while also simultaneously debugging the compiler.
+ * While this is currently Windows only due to use of _WinForms_, it would be fairly straightforward to make a command line based tool.
+ * Dependency graph does not support multiple simultaneous logging facilities, so make sure that you do not set _IlcGenerateDgmlFile_ or invoke _ILCompiler_ with the _DGML_ generation turned on.
+* Pass command line switches to the compiler to generate a _dependency graph DGML_ file. This will produce the same data that is viewable in the viewer tool, but in a textual XML format.
+ * Future efforts may make the XML file loadable by the viewer tool.
+* Instrument the compiler dependency analysis. (This may be necessary in cases where the viewer is unable to provide sufficient information about why the graph is structured as it is.)
+
+## Dependency Graph Viewer
This application allows viewing the dependency graph produced by the AOT compilation.
+<!-- TODO: Add an example. -->
Usage instructions:
-1. Launch the process as an administrator
-2. Run the compiler
-- The compiler can be run to completion, or stopped.
-3. Explore through the graph
-
-# Graphs View #
-- Choose one of the graphs that appears in the Dependency Graphs view to explore. As compilers execute, new graphs will automatically appear here.
-- The set of graphs loaded into the process is limited by available memory space. To clear the used memory, close all windows of the application.
-
-# Graph View #
-- In the Dependency Graph view, enter a regular expression in the text box, and then press ""Filter"". This will display a list of the nodes in the graph which have names which match the regular expression.
-- Commonly, if there is a object file symbol associated with the node it should be used as part of the regular expression. See the various implementations of GetName in the compiler for naming behavior.
-- Additionally, the event source marking mode assigns an Id to each node, and that is found as the mark object on the node, so if a specific id is known, just type that in, and it will appear in the window. (This is for use when using this tool in parallel with debugging the compiler.
-
-# Single Node Exploration #
-Once the interesting node(s) have been identified in the dependency graph window, select one of them, and then press Explore.
- - In the Node Explorer window, the Dependent nodes (the ones which depend on the current node are the nodes displayed above, and the Dependee nodes (the nodes that this node depends on) are displayed below. Each node in the list is paired with a textual reason as to why that edge in the graph exists.
- - Select a node to explore further and press the corresponding button to make it happen.
-
-WhyDGML
-=======
+
+1. Launch the process as an administrator.
+2. Run the compiler. The compiler can be run to completion, or stopped.
+3. Explore through the graph.
+
+### Graphs View
+
+* Choose one of the graphs that appears in the Dependency Graphs view to explore. As compilers execute, new graphs will automatically appear here.
+* The set of graphs loaded into the process is limited by available memory space. To clear the used memory, close all windows of the application.
+
+### Dependency Graph View
+
+* In the Dependency Graph View, enter a regular expression in the text box, and then press `Filter`. This will display a list of the nodes in the graph which have names that match the regular expression.
+* Commonly, if there is an object file symbol associated with the node, it should be used as part of the regular expression. See the various implementations of `GetName` in the compiler for naming behavior.
+* Additionally, the event source marking mode assigns an _id_ to each node, and that is found as the mark object on the node. So, if a specific _id_ is known, just type that in, and it will appear in the window. This is for use when using this tool in parallel with debugging the compiler.
+
+### Single Node Exploration
+
+* Once the interesting node(s) have been identified in the dependency graph window, select one of them, and then press `Explore`.
+* In the _Node Explorer_ window, the Dependent nodes (the ones which depend on the current node) are the ones displayed above, and the Dependee nodes (the ones that this node depends on) are displayed below. Each node in the list is paired with a textual reason as to why that edge in the graph exists.
+* Select a node to explore further and press the corresponding button to make it happen.
+
+## Why DGML
+
This tool can be used to visualize paths from a node of interest to the roots. To use it, pass command line option to the compiler to generate the DGML file (`--dgmllog name_of_output_file`) and then use this tool to find the path to the root. If you're looking at an optimized NativeAOT compilation, `--scandgmllog` might be preferable since it will have more details.
+
The input to the tool is the DGML file and name of a node of interest. The output is the list of reasons why that node was included.
This tool located in folder `src/coreclr/tools/aot/WhyDgml`
-See https://github.com/dotnet/corert/pull/7962 for example of usage and output.
+See <https://github.com/dotnet/corert/pull/7962> for example of usage and output.
--- /dev/null
+# Debugging CoreCLR
+
+* [Debugging CoreCLR on Windows](#debugging-coreclr-on-windows)
+ * [Using Visual Studio](#using-visual-studio)
+ * [Using Visual Studio Open Folder with CMake](#using-visual-studio-open-folder-with-cmake)
+ * [Using Visual Studio Code](#using-visual-studio-code)
+ * [Using SOS with Windbg or Cdb on Windows](#using-sos-with-windbg-or-cdb-on-windows)
+* [Debugging CoreCLR on Linux and MacOS](#debugging-coreclr-on-linux-and-macos)
+ * [SOS on Unix](#sos-on-unix)
+ * [Debugging CoreCLR with lldb](#debugging-coreclr-with-lldb)
+ * [Disabling Managed Attach/Debugging](#disabling-managed-attachdebugging)
+ * [Debugging core dumps with lldb](#debugging-core-dumps-with-lldb)
+* [Debugging AOT compilers](#debugging-aot-compilers)
+* [Debugging Managed Code](#debugging-managed-code)
+ * [Using Visual Studio Code for Managed Code](#using-visual-studio-code-for-managed-code)
+ * [Using Visual Studio for Managed Code](#using-visual-studio-for-managed-code)
+
+These instructions will lead you through debugging CoreCLR.
+
+As an initial note, SOS (the main plugin to aid with CoreCLR debugging) no longer resides here. For more information on how to get it, installing it, and how to use it, check it out in its new home, the [diagnostics repo](https://github.com/dotnet/diagnostics).
+
+## Debugging CoreCLR on Windows
+
+In order to start debugging CoreCLR, it is highly recommended to at least build the _clr_ subset under the _Debug_ configuration, in order to generate the necessary artifact files for the best debugging experience:
+
+```cmd
+.\build.cmd -s clr -c Debug
+```
+
+Note that you can omit the `-c Debug` flag, since it's the default one when none other is specified. We are leaving it here in this doc for the sake of clarity.
+
+If for some reason `System.Private.CoreLib.dll` is missing, you can rebuild it with the following command, instead of having to go through the whole build again:
+
+```cmd
+.\build.cmd -s clr.corelib+clr.corelibnative -c Debug
+```
+
+**NOTE**: When debugging with _CORE\_LIBRARIES_, the `libs` subset must also be built prior to attempting any debugging.
+
+### Using Visual Studio
+
+Visual Studio's capabilities as a full IDE provide a lot of help making the runtime debugging more amiable.
+
+1. Open the CoreCLR solution _(coreclr.sln)_ in Visual Studio.
+ * _Method 1_: Use the build scripts to open the solution:
+ 1. Run `.\build.cmd -vs coreclr.sln -a <architecture> -c <configuration>`. This will create and launch the CoreCLR solution in VS for the specified architecture and configuration. By default, this will be `x64 Debug`.
+ * _Method 2_: Manually build and open the solution:
+ 1. Perform a build of the repo with the `-msbuild` flag.
+ 2. Open solution `path\to\runtime\artifacts\obj\coreclr\windows.<architecture>.<configuration>\ide\CoreCLR.sln` in Visual Studio. As in the previous method, the architecture and configuration by default are `x64` and `Debug`, unless explicitly stated otherwise.
+2. Right-click the **INSTALL** project and choose `Set as StartUp Project`.
+3. Bring up the properties page for the **INSTALL** project.
+4. Select _Configuration Properties -> Debugging_ from the left side tree control.
+5. Set `Command=$(SolutionDir)\..\..\..\..\bin\coreclr\windows.$(Platform).$(Configuration)\corerun.exe`. This points to the folder where the built runtime binaries are present.
+6. Set `Command Arguments=<managed app you wish to run>` (e.g. HelloWorld.dll).
+7. Set `Working Directory=$(SolutionDir)\..\..\..\..\bin\coreclr\windows.$(Platform).$(Configuration)`. This points to the folder containing CoreCLR binaries.
+8. Set `Environment=CORE_LIBRARIES=$(SolutionDir)\..\..\..\..\bin\runtime\<target-framework>-windows-$(Configuration)-$(Platform)`, where '\<target-framework\>' is the target framework of current branch; for example `netcoreapp3.1` `net5.0`, `net6.0`, or `net7.0`. A few notes on this step:
+
+* This points to the folder containing core libraries except `System.Private.CoreLib`.
+* This step can be skipped if you are debugging CLR tests that reference only `System.Private.CoreLib`. Otherwise, it's required to debug a real-world application that references anything else, including `System.Runtime`.
+
+9. Right-click the **INSTALL** project and choose `Build`. This will load necessary information from _CMake_ to Visual Studio.
+10. Press F11 to start debugging at `wmain` in _corerun_, or set a breakpoint in source and press F5 to run to it. As an example, set a breakpoint for the `EEStartup()` function in `ceemain.cpp` to break into CoreCLR startup.
+
+Steps 1-9 only need to be done once as long as there's been no changes to the CMake files in the repository. Afterwards, step 10 can be repeated whenever you want to start debugging. As of now, it is highly recommended to use Visual Studio 2022 or Visual Studio 2019.
+
+#### Using Visual Studio Open Folder with CMake
+
+1. Open the _dotnet/runtime_ repository in Visual Studio using the _open folder_ feature. When opening the repository root, Visual Studio will prompt about finding _CMake_ files. Select `src\coreclr\CMakeList.txt` as the CMake workspace.
+2. Set the `corerun` project as startup project. When using the folder view instead of the CMake targets view, right click `coreclr\hosts\corerun\CMakeLists.txt` to set as startup project, or select it from debug target dropdown of Visual Studio.
+3. Right click the `corerun` project and open the `Debug` configuration. You can also click on _Debug -> Debug and Launch Configuration_ from the Visual Studio main bar menu.
+4. In the opened `launch.vs.json`, set following properties to the configuration of `corerun`:
+
+```json
+ {
+ "type": "default",
+ "project": "CMakeLists.txt",
+ "projectTarget": "corerun.exe (hosts\\corerun\\corerun.exe)",
+ "name": "corerun.exe (hosts\\corerun\\corerun.exe)",
+ "environment": [
+ {
+ "name": "CORE_ROOT",
+ "value": "${cmake.installRoot}"
+ },
+ {
+ "name": "CORE_LIBRARIES",
+ // for example net7.0-windows-Debug-x64
+ "value": "${cmake.installRoot}\\..\\..\\runtime\\<tfm>-windows-<configuration>-<arch>\\"
+ }
+ ],
+ "args": [
+ // path to a managed application to debug
+ // remember to use double backslashes (\\)
+ "HelloWorld.dll"
+ ]
+ }
+```
+
+**NOTE**: For Visual Studio 17.3, changing the location of launched executable doesn't work, so the `CORE_ROOT` is necessary.
+
+5. Right click the CoreCLR project or `coreclr\CMakeLists.txt` in the folder view, and then invoke the _Install_ command.
+6. Press F10 or F11 to start debugging at main, or set a breakpoint and press F5.
+
+Whenever you make changes to the CoreCLR source code, don't forget to invoke the _Install_ command again to have them set in place.
+
+### Using Visual Studio Code
+
+It will be very nice to be able to achieve all this using Visual Studio Code as well, since it's the editor of choice for lots of developers.
+
+Visual Studio Code instructions coming soon!
+
+### Using SOS with Windbg or Cdb on Windows
+
+Under normal circumstances, SOS usually comes shipped with Windbg, so no additional installation is required. However, if this is not the case for you, you want to use another version, or any other circumstance that requires you to install it separately/additionally, here are two links with useful information on how to get it set up:
+
+* The official [Microsoft docs on SOS](https://docs.microsoft.com/dotnet/core/diagnostics/dotnet-sos).
+* The instructions at the [diagnostics repo](https://github.com/dotnet/diagnostics/blob/master/documentation/installing-sos-windows-instructions.md).
+
+For more information on SOS commands click [here](https://github.com/dotnet/diagnostics/blob/master/documentation/sos-debugging-extension-windows.md).
+
+## Debugging CoreCLR on Linux and MacOS
+
+Very similarly to Windows, Linux and MacOS also require to have at least the _clr_ subset built prior to attempting to debug, most preferably under the _Debug_ configuration:
+
+```bash
+./build.sh -s clr -c Debug
+```
+
+Note that you can omit the `-c Debug` flag, since it's the default one when none other is specified. We are leaving it here in this doc for the sake of clarity.
+
+If for some reason `System.Private.CoreLib.dll` is missing, you can rebuild it with the following command, instead of having to go through the whole build again:
+
+```bash
+./build.sh -s clr.corelib+clr.corelibnative -c Debug
+```
+
+**NOTE**: When debugging with _CORE\_LIBRARIES_, the `libs` subset must also be built prior to attempting any debugging.
+
+### SOS on Unix
+
+For Linux and MacOS, you have to install SOS by yourself, as opposed to Windows' Windbg. The instructions are very similar however, and you can find them on these two links:
+
+* The official [Microsoft docs on SOS](https://docs.microsoft.com/dotnet/core/diagnostics/dotnet-sos).
+* The instructions at the [diagnostics repo](https://github.com/dotnet/diagnostics/blob/master/documentation/installing-sos-instructions.md).
+
+It might also be the case that you would need the latest changes in SOS, or you're working with a not-officially-supported scenario that actually works. The most common occurrence of this scenario is when using MacOS Arm64. In this case, you have to build SOS from the diagnostics repo (linked above). Once you have it done, then simply load it to your `lldb`. More details in the following section.
+
+### Debugging CoreCLR with lldb
+
+**NOTE**: Only `lldb` is supported to use with SOS. You can also use `gdb`, `cgdb`, or other debuggers, but you might not have access to SOS.
+
+1. Perform a build of the _clr_ subset of the runtime repo.
+2. Start lldb passing `corerun`, the app to run (e.g. `HelloWorld.dll`), and any arguments this app might need: `lldb /path/to/corerun /path/to/app.dll <app args go here>`
+3. If you're using the installed version of SOS, you can skip this step. If you built SOS manually, you have to load it before starting the debugging session: `plugin load /path/to/built/sos/libsosplugin.so`. Note that `.so` is for Linux, and `.dylib` is for MacOS. You can find more information in the diagnostics repo [private sos build doc](https://github.com/dotnet/diagnostics/blob/main/documentation/using-sos-private-build.md).
+4. Launch program: `process launch -s`
+5. To stop breaks on _SIGUSR1_ signals used by the runtime run the following command: `process handle -s false SIGUSR1`
+6. Set a breakpoint where CoreCLR is initialized, as it's the most stable point to begin debugging: `breakpoint set -n coreclr_execute_assembly`.
+7. Get to that point by issuing `process continue` after setting the breakpoint.
+8. Now, you're ready to begin your debugging session. You can set breakpoints or run SOS commands like `clrstack` or `sos VerifyHeap`. Note that SOS command names are case sensitive.
+
+#### Disabling Managed Attach/Debugging
+
+The `DOTNET_EnableDiagnostics` _environment variable_ can be used to disable managed debugging. This prevents the various OS artifacts used for debugging, such as named pipes and semaphores on Linux and MacOS, from being created.
+
+```bash
+export DOTNET_EnableDiagnostics=0
+```
+
+### Debugging core dumps with lldb
+
+Our friends at the diagnostics repo have a very detailed guide on core dumps debugging [here in their repo](https://github.com/dotnet/diagnostics/blob/master/documentation/debugging-coredump.md).
+
+## Debugging AOT compilers
+
+Debugging AOT compilers is described in [its related document](debugging-aot-compilers.md).
+
+## Debugging Managed Code
+
+Native C++ code is not everything in our runtime. Nowadays, there are lots of stuff to debug that stay in the higher C# managed code level.
+
+### Using Visual Studio Code for Managed Code
+
+* Install the [C# Extension](https://marketplace.visualstudio.com/items?itemName=ms-dotnettools.csharp).
+* Open the folder containing the source you want to debug in VS Code.
+* Open the debug window: `ctrl-shift-D`/`cmd-shift-D` or click on the button on the left.
+* Click the gear button at the top to create a launch configuration, and select `.NET 5+ and .NET Core` from the selection dropdown.
+* It will create a `launch.json` file, where you can configure what and how you want to debug it. Here is a basic template on how to fill it:
+
+```json
+{
+ "version": "0.2.0",
+ "configurations": [
+ {
+ "name": "My Configuration", // Any identifiable name you might like.
+ "type": "coreclr", // We want to debug a CoreCLR app.
+ "request": "launch", // Start the app with the debugger attached.
+ "program": "/path/to/corerun", // Point to your 'corerun', in order to run the app using your build.
+ "args": ["app-to-debug.dll", "app arg1", "app arg2"], // First argument is your app, second and on are the app's arguments.
+ "cwd": "/path/to/app-to-debug", // Can be anywhere. For simplicity, choose where your app is stationed. Otherwise, you have to adjust paths in the other parameters.
+ "stopAtEntry": true, // This can be either. Keeping it to 'true' allows you to see when the debugger is ready.
+ "console": "internalConsole", // Use VSCode's internal console instead of launching more terminals.
+ "justMyCode": false, // Be able to debug into native assemblies.
+ "enableStepFiltering": false, // Be able to debug into class initializations, field accessors, etc.
+ }
+ ]
+}
+```
+
+* Set a breakpoint and launch the debugger, inspecting variables and call stacks will now work
+
+### Using Visual Studio for Managed Code
+
+* Use _File -> Open Project_ (not open file) and select the binary you want to use as your host (typically _dotnet.exe_ or _corerun.exe_).
+* Open the project properties for the new project that was just created and set the following:
+ * _Arguments_: Make this match whatever arguments you would have used at the command-line. For example if you would have run `dotnet.exe exec Foo.dll`, then set `arguments = "exec Foo.dll"` (**NOTE**: Make sure you use `dotnet exec` instead of `dotnet run` because the run verb command is implemented to launch the app in a child process, and the debugger won't be attached to that child process).
+ * _Working Directory_: Make this match whatever you would have used on the command-line.
+ * _Debugger Type_: Set this to `Managed (.NET Core, .NET 5+)`. If you're going to debug the native C++ code, then you would select `Native Only` instead.
+ * _Environment_: Add any environment variables you would have added at the command-line. You may also consider adding `DOTNET_ZapDisable=1` and `DOTNET_ReadyToRun=0`, which disable NGEN and R2R pre-compilation respectively, and allow the JIT to create debuggable code. This will give you a higher quality C# debugging experience inside the runtime framework assemblies, at the cost of somewhat lower app performance.
+* For managed debugging, there are some additional settings in _Debug -> Options_, _Debugging -> General_ that might be useful:
+ * Uncheck `Just My Code`. This will allow you debug into the framework libraries.
+ * Check `Enable .NET Framework Source Stepping`. This will configure the debugger to download symbols and source automatically for runtime framework binaries. If you built the framework yourself, then you can omit this step without any problems.
+ * Check `Suppress JIT optimzation on module load`. This tells the debugger to tell the .NET runtime JIT to generate debuggable code even for modules that may not have been compiled in a `Debug` configuration by the C# compiler. This code is slower, but it provides much higher fidelity breakpoints, stepping, and local variable access. It is the same difference you see when debugging .NET apps in the `Debug` project configuration vs the `Release` project configuration.
+++ /dev/null
-Debugging CoreCLR
-=================
-
-These instructions will lead you through debugging CoreCLR.
-
-SOS has moved to the diagnostics repo. For more information on SOS, installation and how to use it click [here](https://github.com/dotnet/diagnostics#net-core-diagnostics-repo).
-
-Debugging CoreCLR on Windows
-============================
-
-When working with IDE, it's suggested to run `./build.cmd -subset clr` in prior to generate necessary artifact files.
-
-If CoreLib is missing, using `./build.cmd -subset clr.corelib+clr.corelibnative` can generate it only.
-
-When debugging with `CORE_LIBRARIES`, `./build -subset libraries` is required.
-
-### Using Visual Studio Solution
-
-1. Open the CoreCLR solution in Visual Studio.
- - Method 1: Use the build scripts to open the solution.
- 1. Run `./build.cmd -vs coreclr.sln -a <platform> -c <configuration>`. This will create and launch the CoreCLR solution in VS for the specified architecture and configuration.
- - Method 2: Manually build and open the solution.
- 1. Perform a build of the repo with the `-msbuild` flag.
- 2. Open solution `\<reporoot>\artifacts\obj\coreclr\windows.\<platform>.\<configuration>\ide\CoreCLR.sln` in Visual Studio. `<platform>` and `<configuration>` are based
- on type of build you did. By default they are 'x64' and 'Debug'.
-2. Right-click the INSTALL project and choose ‘Set as StartUp Project’
-3. Bring up the properties page for the INSTALL project
-4. Select Configuration Properties->Debugging from the left side tree control
-5. Set Command=`$(SolutionDir)\..\..\..\..\bin\coreclr\windows.$(Platform).$(Configuration)\corerun.exe`
- 1. This points to the folder where the built runtime binaries are present.
-6. Set Command Arguments=`<managed app you wish to run>` (e.g. HelloWorld.dll)
-7. Set Working Directory=`$(SolutionDir)\..\..\..\..\bin\coreclr\windows.$(Platform).$(Configuration)`
- 1. This points to the folder containing CoreCLR binaries.
-8. Set Environment=`CORE_LIBRARIES=$(SolutionDir)\..\..\..\..\bin\runtime\<current tfm>-windows-$(Configuration)-$(Platform)`,
- where '\<current tfm\>' is the target framework of current branch, for example `netcoreapp3.1` `net5.0`.
- 1. This points to the folder containing core libraries except `System.Private.CoreLib`.
- 2. This step can be skipped if you are debugging CLR tests that references only `System.Private.CoreLib`.
- Otherwise, it's required to debug a realworld application that references anything else, including `System.Runtime`.
-9. Right-click the INSTALL project and choose 'Build'
- 1. This will load necessary information from cmake to Visual Studio.
-10. Press F11 to start debugging at wmain in corerun (or set a breakpoint in source and press F5 to run to it)
- 1. As an example, set a breakpoint for the EEStartup function in ceemain.cpp to break into CoreCLR startup.
-
-Steps 1-9 only need to be done once as long as there's been no changes to the CMake files in the repository. Afterwards, step 10 can be repeated whenever you want to start debugging. The above can be done with Visual Studio 2019 as writing.
-Keeping with latest version of Visual Studio is recommended.
-
-### Using Visual Studio Open Folder with CMake
-
-1. Open either runtime repository or coreclr directory in Visual Studio using open folder feature.
- - When opening repository root, Visual Studio will prompt about finding CMake files. Select `src\coreclr\CMakeList.txt` as the CMake workspace.
-2. Set the corerun project as startup project.
- - When using folder view instead of CMake targets view, right click `coreclr\hosts\corerun\CMakeLists.txt` to set as startup project, or select it from debug target dropdown of Visual Studio.
-3. Right click the corerun project and open debug configuration.
- - Can also use 'Debug->Debug and Launch Configuration' from Visual Studio menu.
-4. In the opened `launch.vs.json`, set following properties to the configuration of corerun:
-```json
- {
- "type": "default",
- "project": "CMakeLists.txt",
- "projectTarget": "corerun.exe (hosts\\corerun\\corerun.exe)",
- "name": "corerun.exe (hosts\\corerun\\corerun.exe)",
- "environment": [
- {
- "name": "CORE_ROOT",
- "value": "${cmake.installRoot}"
- },
- {
- "name": "CORE_LIBRARIES",
- // for example net8.0-windows-Debug-x64
- "value": "${cmake.installRoot}\\..\\..\\runtime\\<tfm>-windows-<configuration>-<arch>\\"
- }
- ],
- "args": [
- // path to a managed application to debug
- // remember to use double backslashes (\\)
- "HelloWorld.dll"
- ]
- }
-```
-*For VS 17.3, changing the location of launched executable doesn't work, so CORE_ROOT is required.*
-*Requirement of CORE_LIBRARIES is the same with solution experience.*
-
-5. Right click the root CoreCLR project or `coreclr\CMakeLists.txt` in folder view, invoke the 'Install' command.
-6. Press F10 or F11 to start debugging at main, or set a breakpoint and press F5.
-
-When there's source code change in coreclr, invoke the 'Install' command again to set them in place.
-
-### Using SOS with windbg or cdb on Windows ###
-
-See the SOS installation instructions [here](https://github.com/dotnet/diagnostics/blob/main/documentation/installing-sos-windows-instructions.md).
-
-For more information on SOS commands click [here](https://github.com/dotnet/diagnostics/blob/main/documentation/sos-debugging-extension-windows.md).
-
-Debugging CoreCLR on Linux and macOS
-====================================
-
-See the SOS installation instructions [here](https://github.com/dotnet/diagnostics/blob/main/documentation/installing-sos-instructions.md). After SOS is installed, it will automatically be loaded by lldb.
-
-Only lldb is supported by SOS. Gdb can be used to debug the coreclr code but with no SOS support.
-
-1. Perform a build of the coreclr repo.
-2. Install the corefx managed assemblies to the binaries directory.
-3. cd to build's binaries: `cd ./artifacts/bin/coreclr/Linux.x64.Debug`
-4. Start lldb: `lldb corerun HelloWorld.exe linux`
-6. Launch program: `process launch -s`
-7. To stop annoying breaks on SIGUSR1/SIGUSR2 signals used by the runtime run: `process handle -s false SIGUSR1 SIGUSR2`
-8. Get to a point where coreclr is initialized by setting a breakpoint (i.e. `breakpoint set -n LoadLibraryExW` and then `process continue`) or stepping into the runtime.
-9. Run a SOS command like `clrstack` or `sos VerifyHeap`. The command name is case sensitive.
-
-See https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-sos for information on how to install SOS.
-
-**Note:** _corerun_ is a simple host that does not support resolving NuGet dependencies. It relies on libraries being locatable via the `CORE_LIBRARIES` environment variable or present in the same directory as the corerun executable. The instructions above are equally applicable to the _dotnet_ host, however - e.g. for step 4 `lldb-3.9 dotnet bin/Debug/netcoreapp2.1/MvcApplication.dll` will let you debug _MvcApplication_ in the same manner.
-
-### Debugging core dumps with lldb
-
-See this [link](https://github.com/dotnet/diagnostics/blob/main/documentation/debugging-coredump.md) in the diagnostics repo.
-
-Disabling Managed Attach/Debugging
-==================================
-
-The "COMPlus_EnableDiagnostics" environment variable can be used to disable managed debugging. This prevents the various OS artifacts used for debugging like the named pipes and semaphores on Linux/MacOS and shared memory on Windows from being created.
-
- export COMPlus_EnableDiagnostics=0
-
-Debugging AOT compilers
-==================================
-
-Debugging AOT compilers is described in [this](debugging-aot-compilers.md) document.
-
-Using Visual Studio Code
-========================
-
-- Install [Visual Studio Code](https://code.visualstudio.com/)
-- Install the [C# Extension](https://marketplace.visualstudio.com/items?itemName=ms-dotnettools.csharp)
-- Open the folder containing the source you want to debug in VS Code
-- Open the debug window: `ctrl-shift-D` or click on the button on the left
-- Click the gear button at the top to create a launch configuration, select `.NET Core` from the selection dropdown
-- In the `.NET Core Launch (console)` configuration do the following
- - delete the `preLaunchTask` property
- - set `program` to the full path to corerun in the test directory
- - set `cwd` to the test directory
- - set `args` to the command line arguments to pass to the test
- - something like: `[ "xunit.console.netcore.exe", "<test>.dll", "-notrait", .... ]`
-- Set a breakpoint and launch the debugger, inspecting variables and call stacks will now work
-
-Using Visual Studio
-===================
-
-- Install [Visual Studio](https://visualstudio.microsoft.com/vs/)
-- Use File->Open Project (not open file) and select the binary you want to use as your host (typically dotnet.exe or corerun.exe)
-- Open the project properties for the new project that was just created and set:
- - Arguments: Make this match whatever arguments you would have used at the command-line. For example if you would have run "dotnet.exe exec Foo.dll", then set arguments = "exec Foo.dll"
- (Note: you probably want 'dotnet exec' rather than 'dotnet run' because the run verb is implemented to launch the app in a child-process and the debugger won't be attached to that child process)
- - Working Directory: Make this match whatever you would have used on the command-line
- - Debugger Type: Set this to either 'Managed (CoreCLR)' or 'Native Only' depending on whether you want to debug the C+ or native code respectively.
- - Environment: Add any environment variables you would have added at the command-line. You may also consider adding COMPlus_ZapDisable=1 and COMPlus_ReadyToRun=0 which disable NGEN and R2R pre-compilation respectively and allow the JIT to create debuggable code. This will give you a higher quality C# debugging experience inside the runtime framework assemblies, at the cost of somewhat lower app performance.
-- For managed debugging, there are some settings in Debug->Options, Debugging->General that might be useful:
- - Uncheck 'Just My Code'. This will allow you debug into the framework libraries.
- - Check 'Enable .NET Framework Source Stepping.' This will configure the debugger to download symbols and source automatically for runtime framework binaries. If you built the framework yourself this may be irrelevant because you already have all the source on your machine but it doesn't hurt.
- - Check 'Suppress JIT optimzation on module load'. This tells the debugger to tell the .NET runtime JIT to generate debuggable code even for modules that may not have been compiled in a 'Debug' configuration by the C# compiler. This code is slower, but it provides much higher fidelity breakpoints, stepping, and local variable access. It is the same difference you see when debugging .NET apps in the 'Debug' project configuration vs. the 'Release' project configuration.
Notice that the CoreCLR and corehost solutions are under the `artifacts` directory. This is because they are created as part of the build.
Thus you can only launch these solutions after you have built at least once with the `-msbuild` flag or run the `./build.cmd -vs CoreCLR.sln` or `./build.cmd -vs corehost.sln` command line with the specified architecture and configuration.
-* See [Debugging CoreCLR](debugging/coreclr/debugging.md)
+* See [Debugging CoreCLR](debugging/coreclr/debugging-runtime.md)
# See Also
-Requirements to build and run dotnet/runtime on FreeBSD
-=====================
+# Requirements to build dotnet/runtime on FreeBSD
+
+* [Docker](#docker)
+* [Linux Environment](#linux-environment)
+* [FreeBSD Environment](#freebsd-environment)
+* [Old Documentation](#old-documentation)
+ * [Toolchain Setup](#toolchain-setup)
+ * [Running on FreeBSD](#running-on-freebsd)
+
+This guide will walk you through the requirements needed to build and run _dotnet/runtime_ on FreeBSD. We'll start by showing how to set up your environment from scratch.
-This guide will walk you through the requirements needed to build and run dotnet/runtime on FreeBSD. We'll start by showing how to set up your environment from scratch.
Since there is no official build and FreeBSD package, native build on FreeBSD is not trivial. There are generally three options, sorted by ease of use:
-- cross-compile on Linux using Docker
-- cross-compile on Linux using Toolchain
-- build on FreeBSD
-Environment
-===========
+* Cross-compile using prebuilt Docker Images
+* Cross-compile on Linux using your environment
+* Build directly on FreeBSD
-These instructions were validated for and on FreeBSD 12.2.
+## Docker
-Build using Docker on Linux
----------------------------
+Install Docker. For further instructions on installation, see [here](https://docs.docker.com/install/). You can find the official .NET images in [their Docker hub](https://hub.docker.com/_/microsoft-dotnet).
-This is similar to [Linux](linux-requirements.md) instructions. https://github.com/dotnet/dotnet-buildtools-prereqs-docker repro provides images
-with all needed prerequisites to build. As the example bellow may become stale, https://github.com/dotnet/versions/blob/master/build-info/docker/image-info.dotnet-dotnet-buildtools-prereqs-docker-master.json offers list of latest Docker tags.
+All the required build tools are included in the Docker images used to do the build, so no additional setup is required.
-```sh
-TAG=mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-cross-freebsd-12-20210917001307-f13d79e
-docker run --rm --volume $(pwd):$(pwd) --workdir $(pwd) --env ROOTFS_DIR=/crossrootfs/x64 -ti $TAG ./build.sh -cross -os FreeBSD
-```
+## Linux Environment
-Build using Toolchain Setup
----------------------------
-To build FreeBSD images, prerequisites described in [Linux](linux-requirements.md) are needed. Additionally, crossrootfs for FreeBSD needs to be constructed.
-In order to successfully build FreeBSD crossrootfs, few more packages needs to be installed. Following example is for Ubuntu 18:
-```sh
-apt-get install -y libbz2-dev libz-dev liblzma-dev libarchive-dev libbsd-dev
-```
-With prerequisites for crossrootfs one can run:
-```sh
-./eng/common/cross/build-rootfs.sh freebsd11 $(pwd)/rootfs/freebsd
-```
-After that, FreeBSD build can be started by running
-```
-ROOTFS_DIR=$(pwd)/rootfs/freebsd ./build.sh -cross -os FreeBSD
-```
+To cross-build FreeBSD on your Linux environment, first make sure you have all the [normal Linux prerequisites](/docs/workflow/requirements/linux-requirements.md) fulfilled. Then, the _crossrootfs_ for FreeBSD has to be constructed, and this requires a few more packages to be installed:
+* libbz2-dev
+* liblzma-dev
+* libarchive-dev
+* libbsd-dev
-Building on FreeBSD
--------------------
+## FreeBSD Environment
-Building dotnet/runtime depends on several tools to be installed.
+These instructions assume you use FreeBSD's default binary package tool `pkg` (analog to `apt`, `apt-get`, or `yum` on Linux) to install the environment. Compiling the dependencies from source using the ports tree might work too, but is untested.
-Install the following packages:
+FreeBSD Prerequisites Coming Soon!
+
+Meanwhile here are the old instructions.
+
+## Old Documentation
-- cmake
-- icu
-- libunwind
-- lttng-ust
-- krb5
-- openssl (optional)
-- ninja (optional, enables building native code with ninja instead of make)
+These instructions were written quite a while ago, and they may or may not work today. Updated instructions coming soon.
+
+### Toolchain Setup
+
+Building the _dotnet/runtime_ repo requires several tools to be installed.
+
+Install the following packages:
-The lines to install all the packages above using package manager.
+* Bash
+* CMake
+* icu
+* libunwind
+* krb5
+* openssl (optional)
+* python39
+* libinotify
+* ninja (optional, enables building native code with ninja instead of make)
```sh
sudo pkg install --yes libunwind icu libinotify lttng-ust krb5 cmake openssl ninja
```
-Additionally, working dotnet cli with SDK is needed. On other platforms this would be downloaded automatically during build but it is not currently available for FreeBSD.
-It needs to be built once on supported platform or obtained via community resources.
+### Running on FreeBSD
-Running on FreeBSD
--------------------
Install the following packages:
-- icu
-- libunwind
-- lttng-ust (optional, debug support)
-- krb5
-- openssl (optional, SSL support)
-- libinotify
-- terminfo-db (optional, terminal colors)
-The lines to install all the packages above using package manager.
+* icu
+* libunwind
+* lttng-ust (optional, debug support)
+* krb5
+* openssl (optional, SSL support)
+* libinotify
+* terminfo-db (optional, terminal colors)
```sh
sudo pkg install --yes libunwind icu libinotify lttng-ust krb5 openssl terminfo-db
"VERSION" is the same version as the SDK from above.
-- Microsoft.NETCore.App.Host.freebsd-x64.VERSION.nupkg
-- Microsoft.NETCore.App.Runtime.freebsd-x64.VERSION.nupkg
-- Microsoft.AspNetCore.App.Runtime.freebsd-x64.VERSION.nupkg
+* Microsoft.NETCore.App.Host.freebsd-x64.VERSION.nupkg
+* Microsoft.NETCore.App.Runtime.freebsd-x64.VERSION.nupkg
+* Microsoft.AspNetCore.App.Runtime.freebsd-x64.VERSION.nupkg
Add the following line to any `NuGet.config` you are using under the `<packageSources>` section:
-Requirements to build dotnet/runtime on Linux
-======================
+# Requirements to build dotnet/runtime on Linux
-This guide will walk you through the requirements to build dotnet/runtime on Linux. Before building there is environment setup that needs to happen to pull in all the dependencies required by the build. There are two suggested ways to go about doing this. First you are able to use the Docker environments provided by https://github.com/dotnet/dotnet-buildtools-prereqs-docker, or you can set up the environment yourself. The documentation will go over both ways of building. Using Docker allows you to leverage our existing images which already have an environment set up.
+* [Docker](#docker)
+* [Environment](#environment)
+ * [Toolchain Setup](#toolchain-setup)
+ * [Additional Requirements for Cross-Building](#additional-requirements-for-cross-building)
+ * [Alpine ARM Cross-Building Requirements](#alpine-arm-cross-building-requirements)
+ * [Gentoo Special Case](#gentoo-special-case)
-General instructions for building are [here](../README.md).
-Instructions for building CoreCLR for Linux are [here](../building/coreclr/linux-instructions.md).
+This guide will walk you through the requirements to build _dotnet/runtime_ on Linux. Before building there is environment setup that needs to happen to pull in all the dependencies required by the build.
+There are two suggested ways to go about doing this. You can use the Docker images used in the official builds, or you can set up the environment yourself. The documentation will go over both ways. Using Docker allows you to leverage our existing images which already have an environment set up, while using your own environment grants you better flexibility on having other tools at hand you might need.
-Docker
-==================
+## Docker
-Install Docker; see https://docs.docker.com/install/.
+Install Docker. For further installation instructions, see [here](https://docs.docker.com/install/). Details on the images used by the official builds can be found in the [Linux building instructions doc](/docs/workflow/building/coreclr/linux-instructions.md#docker-images). All the required build tools are included in the Docker images used to do the build, so no additional setup is required.
-All the required build tools are included in the Docker images used to do the build, so no additional setup is required.
+## Environment
-
-Environment
-===========
-
-These instructions are written assuming the current Ubuntu LTS, since that's the distro the team uses. Pull Requests are welcome to address other environments as long as they don't break the ability to use Ubuntu LTS.
+These instructions are written assuming the current Ubuntu LTS, since that's the officially used distribution. Pull Requests are welcome to address other environments as long as they don't break the ability to use Ubuntu LTS.
Minimum RAM required to build is 1GB. The build is known to fail on 512 MB VMs ([dotnet/runtime#4069](https://github.com/dotnet/runtime/issues/4069)).
-Toolchain Setup
----------------
+### Toolchain Setup
-Building the repo requires CMake 3.14.5 or newer on Linux. Add Kitware's APT feed to your configuration for a newer version of CMake. See their instructions at <https://apt.kitware.com/>. You may need to add LLVM's APT feed to your configuration to obtain the required version of clang/LLVM. See their instructions at <https://apt.llvm.org/>.
+Building the repo requires CMake 3.14.5 or newer on Linux. Add Kitware's APT feed to your configuration for a newer version of CMake (optional). See their instructions at their [website](https://apt.kitware.com/).
Install the following packages for the toolchain:
-- cmake
-- llvm
-- clang
-- build-essential
-- python
-- curl
-- git
-- lldb
-- liblldb-dev
-- libunwind8
-- libunwind8-dev
-- gettext
-- libicu-dev
-- liblttng-ust-dev
-- libssl-dev
-- libkrb5-dev
-- libnuma-dev (optional, enables numa support)
-- zlib1g-dev
-- ninja-build (optional, enables building native code with ninja instead of make)
-
-```
-sudo apt-get install -y cmake llvm clang \
-build-essential python curl git lldb liblldb-dev \
-libunwind8 libunwind8-dev gettext libicu-dev liblttng-ust-dev \
+* CMake
+* llvm
+* lld
+* clang
+* build-essential
+* python-is-python3
+* curl
+* git
+* lldb
+* libicu-dev
+* liblttng-ust-dev
+* libssl-dev
+* libkrb5-dev
+* libnuma-dev (optional, enables numa support)
+* zlib1g-dev
+* ninja-build (optional, enables building native code with ninja instead of make)
+
+```bash
+sudo apt install -y cmake llvm lld clang build-essential
+python-is-python3 curl git lldb libicu-dev liblttng-ust-dev \
libssl-dev libnuma-dev libkrb5-dev zlib1g-dev ninja-build
```
You now have all the required components.
-*Unsupported OSes*:
-In case you have Gentoo you can run following commands:
-```
+#### Additional Requirements for Cross-Building
+
+If you are planning to use your Linux environment to do cross-building for other architectures (e.g. Arm32, Arm64) and/or other operating systems (e.g. Alpine, FreeBSD), you need to install these additional dependencies:
+
+* qemu
+* qemu-user-static
+* binfmt-support
+* debootstrap
+
+**NOTE**: These dependencies are used to build the `crossrootfs`, not the runtime itself.
+
+### Gentoo Special Case
+
+In case you have Gentoo you can run following command:
+
+```bash
emerge --ask clang dev-util/lttng-ust app-crypt/mit-krb5
```
-Requirements to build dotnet/runtime on macOS
-=====================
+# Requirements to build dotnet/runtime on MacOS
-This guide will walk you through the requirements needed to build dotnet/runtime on macOS. We'll start by showing how to set up your environment from scratch.
+* [Environment](#environment)
+ * [Xcode](#xcode)
+ * [Toolchain Setup](#toolchain-setup)
-Environment
-===========
+This guide will walk you through the requirements needed to build _dotnet/runtime_ on MacOS. We'll start by showing how to set up your environment from scratch.
-These instructions were validated on macOS 10.15 (Catalina).
+## Environment
-Xcode
------
+Here are the components you will need to install and setup to work with the repo.
-Install Apple Xcode developer tools from the Mac App Store ([link](https://apps.apple.com/us/app/xcode/id497799835)).
+### Xcode
-Configure the Xcode command line tools:
-Run Xcode, open Preferences, and on the Locations tab, change "Command Line Tools" to point to this installation of Xcode.app.
-Alternately, run `sudo xcode-select --switch /Applications/Xcode.app/Contents/Developer` in a terminal.
-(Adjust the path if you renamed Xcode.app)
+* Install Apple Xcode developer tools from the [Mac App Store](https://apps.apple.com/us/app/xcode/id497799835).
+* Configure the Xcode command line tools:
+ * Run Xcode, open Preferences, and on the Locations tab, change "Command Line Tools" to point to this installation of _Xcode.app_. This usually comes already done by default, but it's always good to ensure.
+ * Alternately, you can run `sudo xcode-select --switch /Applications/Xcode.app/Contents/Developer` in a terminal (Adjust the path if you renamed _Xcode.app_).
-Toolchain Setup
----------------
+### Toolchain Setup
-Building dotnet/runtime depends on several tools to be installed. You can download them individually or use [Homebrew](https://brew.sh) for easier toolchain setup.
+Building _dotnet/runtime_ depends on several tools to be installed. You can download them individually or use [Homebrew](https://brew.sh) for easier toolchain setup.
Install the following packages:
-- cmake 3.15.5 or newer
-- icu4c
-- openssl@1.1 or openssl@3
-- pkg-config
-- python3
-- ninja (optional, enables building native code with ninja instead of make)
+* CMake 3.15.5 or newer
+* icu4c
+* openssl@1.1 or openssl@3
+* pkg-config
+* python3
+* ninja (optional, enables building native code with ninja instead of make)
-You can install all the required packages above using Homebrew by running this command in the repository root:
+You can install all the required packages above using _Homebrew_ by running this command in the repository root:
-```
+```bash
brew bundle --no-lock --file eng/Brewfile
```
-Requirements to build dotnet/runtime on Windows.
-========================
+# Requirements to build dotnet/runtime on Windows
-These instructions will lead you through the requirements to build dotnet/runtime on Windows.
+* [Environment](#environment)
+ * [Enable Long Paths](#enable-long-paths)
+ * [Visual Studio](#visual-studio)
+ * [Build Tools](#build-tools)
+ * [CMake](#cmake)
+ * [Ninja](#ninja)
+ * [Python](#python)
+ * [Git](#git)
+ * [PowerShell](#powershell)
+ * [.NET SDK](#net-sdk)
+ * [Adding to the default PATH variable](#adding-to-the-default-path-variable)
+ * [ARM64 (Experimental)](#arm64-experimental)
-----------------
-# Environment
+These instructions will lead you through the requirements to build _dotnet/runtime_ on Windows.
-You must install several components to build the dotnet/runtime repository. These instructions were tested on Windows 10 Pro, version 1903.
+## Environment
-## Enable Long Paths
+Here are the components you will need to install and setup to work with the repo.
-The runtime repository requires long paths to be enabled. Follow [the instructions provided here](https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation#enable-long-paths-in-windows-10-version-1607-and-later) to enable that feature.
+### Enable Long Paths
+
+The runtime repository requires long paths to be enabled. Follow [the instructions provided here](https://docs.microsoft.com/windows/win32/fileio/maximum-file-path-limitation#enable-long-paths-in-windows-10-version-1607-and-later) to enable that feature.
+
+If using Git for Windows you might need to also configure long paths there. Using an administrator terminal simply type:
-If using Git for Windows you might need to also configure long paths there. Using an admin terminal simply type:
```cmd
git config --system core.longpaths true
```
-## Visual Studio
+### Visual Studio
-- Install [Visual Studio 2022](https://visualstudio.microsoft.com/downloads/). The Community edition is available free of charge. Visual Studio 2022 17.3 or later is required.
+Install [Visual Studio 2022](https://visualstudio.microsoft.com/downloads/). The Community edition is available free of charge. Visual Studio 2022 17.3 or later is required. Note that Visual Studio and the development tools described below are required, regardless of whether you plan to use the IDE or not. The installation process goes as follows:
-Visual Studio 2022 installation process:
-- It's recommended to use **Workloads** installation approach. The following are the minimum requirements:
- - **.NET Desktop Development** with all default components,
- - **Desktop Development with C++** with all default components.
-- The build tools (CMake, Ninja and Python) can be downloaded and installed separately (see detailed instructions in the [section below](#build-tools)) or by selecting the following **Individual Components**:
- - **C++ CMake tools for Windows** (includes Ninja, but might not work on ARM64 machines),
- - **Python 3 64-bit** (3.7.4 or newer).
-- To build for Arm32 or Arm64, make sure that you have the right architecture-specific compilers installed. In the **Individual components** window, in the **Compilers, build tools, and runtimes** section:
- - For Arm32, check the box for **MSVC v143 - VS 2022 C++ ARM build tools (Latest)** (v14.31 or newer),
- - For Arm64, check the box for **MSVC v143 - VS 2022 C++ ARM64 build tools (Latest)** (v14.31 or newer).
-- To build the tests, you will need some additional components:
- - **Windows 10 SDK (10.0.19041)** or newer. This component is installed by default as a part of **Desktop Development with C++** workload.
- - **C++/CLI support for v142 build tools (Latest)** (v14.23 or newer).
+* It's recommended to use **Workloads** installation approach. The following are the minimum requirements:
+ * **.NET Desktop Development** with all default components,
+ * **Desktop Development with C++** with all default components.
+* The build tools (CMake, Ninja and Python) can be downloaded and installed separately (see detailed instructions in the [section below](#build-tools)) or by selecting the following **Individual Components**:
+ * **C++ CMake tools for Windows** (includes Ninja, but might not work on ARM64 machines),
+ * **Python 3 64-bit** (3.7.4 or newer).
+* To build for Arm32 or Arm64, make sure that you have the right architecture-specific compilers installed. In the **Individual components** window, in the **Compilers, build tools, and runtimes** section:
+ * For Arm32, check the box for _MSVC v143* VS 2022 C++ ARM build tools (Latest)_, which must be _v14.31_ or newer.
+ * For Arm64, check the box for _MSVC v143* VS 2022 C++ ARM64 build tools (Latest)_, which must be _v14.31_ or newer.
+* To build the tests, you will need some additional components:
+ * **Windows 10 SDK (10.0.19041)** or newer. This component is installed by default as a part of **Desktop Development with C++** workload.
+ * **C++/CLI support for v142 build tools (Latest)**, which must be v14.23 or newer.
-A `.vsconfig` file is included in the root of the dotnet/runtime repository that includes all components needed to build the dotnet/runtime repository. You can [import `.vsconfig` in your Visual Studio installer](https://docs.microsoft.com/en-us/visualstudio/install/import-export-installation-configurations?view=vs-2022#import-a-configuration) to install all necessary components. You may get a message saying 'Microsoft.Net.Component.4.5.2.TargetingPack has no matching workload or component found'. This is not an issue as long as you have a newer targeting pack installed.
+A `.vsconfig` file is included in the root of the _dotnet/runtime_ repository that includes all components needed to build the _dotnet/runtime_ repository. You can [import `.vsconfig` in your Visual Studio installer](https://docs.microsoft.com/visualstudio/install/import-export-installation-configurations?view=vs-2022#import-a-configuration) to install all necessary components. You may get a message saying `Microsoft.Net.Component.4.5.2.TargetingPack has no matching workload or component found`. This is not an issue as long as you have a newer targeting pack installed.
-## Build Tools
+### Build Tools
These steps are required only in case the tools have not been installed as Visual Studio **Individual Components** (described above).
-### CMake
+#### CMake
-- Install [CMake](https://cmake.org/download) for Windows. For ARM64 machines please use the `windows-i386` installer (`windows-x86_64` is not going to work).
-- Add its location (e.g. C:\Program Files (x86)\CMake\bin) to the PATH environment variable.
- The installation script has a check box to do this, but you can do it yourself after the fact following the instructions at [Adding to the Default PATH variable](#adding-to-the-default-path-variable).
+* Install [CMake](https://cmake.org/download) for Windows. For ARM64 machines please use the `windows-i386` installer (`windows-x86_64` is not going to work).
+* Add its location (e.g. C:\Program Files (x86)\CMake\bin) to the PATH environment variable. The installation script has a check box to do this, but you can do it yourself after the fact following the instructions at [Adding to the Default PATH variable](#adding-to-the-default-path-variable).
-The dotnet/runtime repository recommends using CMake 3.16.4 or newer, but it may work with CMake 3.15.5.
+The _dotnet/runtime_ repository recommends using CMake 3.16.4 or newer, but it may work with CMake 3.15.5.
-### Ninja
+#### Ninja
-- Install Ninja in one of the two following ways
- - [Download the executable](https://github.com/ninja-build/ninja/releases) and add its location to [the Default PATH variable](#adding-to-the-default-path-variable).
- - [Install via a package manager](https://github.com/ninja-build/ninja/wiki/Pre-built-Ninja-packages), which should automatically add it to the PATH environment variable.
-- Since there is no official Ninja support for ARM64, you can use MSBuild instead by passing `-msbuild` to `build.cmd`.
+* Install Ninja in one of the two following ways
+ * [Download the executable](https://github.com/ninja-build/ninja/releases) and add its location to [the Default PATH variable](#adding-to-the-default-path-variable).
+ * [Install via a package manager](https://github.com/ninja-build/ninja/wiki/Pre-built-Ninja-packages), which should automatically add it to the PATH environment variable.
+* Since there is no official Ninja support for ARM64, you can use MSBuild instead by passing `-msbuild` to `build.cmd`.
-### Python
+#### Python
-- Install [Python](https://www.python.org/downloads/) for Windows.
-- Add its location (e.g. C:\Python*\) to the PATH environment variable.
+* Install [Python](https://www.python.org/downloads/) for Windows.
+* Add its location (e.g. C:\Python*\\) to the PATH environment variable.
The installation script has a check box to do this, but you can do it yourself after the fact following the instructions at [Adding to the Default PATH variable](#adding-to-the-default-path-variable).
-The dotnet/runtime repository requires at least Python 3.7.4.
+The _dotnet/runtime_ repository requires at least Python 3.7.4.
-## Git
+### Git
-- Install [Git](https://git-for-windows.github.io/) for Windows.
-- Add its location (e.g. C:\Program Files\Git\cmd) to the PATH environment variable.
+* Install [Git](https://git-for-windows.github.io/) for Windows.
+* Add its location (e.g. C:\Program Files\Git\cmd) to the PATH environment variable.
The installation script has a check box to do this, but you can do it yourself after the fact following the instructions at [Adding to the Default PATH variable](#adding-to-the-default-path-variable).
-The dotnet/runtime repository requires at least Git 2.22.0.
+The _dotnet/runtime_ repository requires at least Git 2.22.0.
-## PowerShell
+### PowerShell
-- Ensure that `powershell.exe` is accessible via the PATH environment variable. Typically this is `%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\` and its automatically set upon Windows installation.
-- Powershell version must be 3.0 or higher. Use `$PSVersionTable.PSVersion` to determine the engine version.
+* Ensure that `powershell.exe` is accessible via the PATH environment variable. Typically this is `%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\` and its automatically set upon Windows installation.
+* Powershell version must be 3.0 or higher. Use `$PSVersionTable.PSVersion` to determine the engine version.
-## .NET SDK
+### .NET SDK
-While not strictly needed to build or test this repository, having the .NET SDK installed lets you browse solution files in this repository with Visual Studio and use the dotnet.exe command to run .NET applications in the 'normal' way.
-We use this in the [Using Your Build](../testing/using-your-build.md) instructions.
-The minimum required version of the SDK is specified in the [global.json file](https://github.com/dotnet/runtime/blob/main/global.json#L3). [You can find the installers and binaries for latest development builds of .NET SDK here](https://github.com/dotnet/installer#installers-and-binaries).
+While not strictly needed to build or test this repository, having the .NET SDK installed lets you browse solution files in this repository with Visual Studio and use the `dotnet.exe` command to run .NET applications in the 'normal' way.
-Alternatively, to avoid modifying your machine state, you can use the repository's locally acquired SDK by passing in the solution to load via the `-vs` switch:
+We use this in the [build testing with the installed SDK](/docs/workflow/testing/using-your-build-with-installed-sdk.md), and [build testing with dev shipping packages](/docs/workflow/testing/using-dev-shipping-packages.md) instructions. The minimum required version of the SDK is specified in the [global.json file](https://github.com/dotnet/runtime/blob/main/global.json#L3). You can find the installers and binaries for latest development builds of .NET SDK in the [installer repo](https://github.com/dotnet/installer#installers-and-binaries).
-```
-build.cmd -vs System.Text.RegularExpressions
+Alternatively, to avoid modifying your machine state, you can use the repository's locally acquired SDK by passing in the solution to load via the `-vs` switch. For example:
+
+```cmd
+.\build.cmd -vs System.Text.RegularExpressions
```
-This will set the `DOTNET_ROOT` and `PATH` environment variables to point to the locally acquired SDK under `runtime\.dotnet\` and will launch the Visual Studio instance which is registered for the `sln` extension.
+This will set the `DOTNET_ROOT` and `PATH` environment variables to point to the locally acquired SDK under `runtime\.dotnet` and will launch the Visual Studio instance that is registered for the `sln` extension.
-## Adding to the default PATH variable
+### Adding to the default PATH variable
-The commands above need to be on your command lookup path. Some installers will automatically add them to the path as part of the installation, but if not here is how you can do it.
+The commands above need to be on your command lookup path. Some installers will automatically add them to the path as part of the installation, but if not, here is how you can do it.
-You can, of course, add a directory to the PATH environment variable with the syntax `set PATH=%PATH%;DIRECTORY_TO_ADD_TO_PATH`.
+You can also temporarily add a directory to the PATH environment variable with the command-prompt syntax `set PATH=%PATH%;DIRECTORY_TO_ADD_TO_PATH`. If you're working with Powershell, then the syntax would be `$Env:PATH += ";DIRECTORY_TO_ADD_TO_PATH"`. However, this change will only last until the command windows close.
-However, the change above will only last until the command windows close.
-You can make your change to the PATH variable persistent by going to Control Panel -> System And Security -> System -> Advanced system settings -> Environment Variables,
-and select the 'Path' variable in the 'System variables' (if you want to change it for all users) or 'User variables' (if you only want to change it for the current user).
-Simply edit the PATH variable's value and add the directory (with a semicolon separator).
+You can make your change to the PATH variable persistent by going to _Control Panel -> System And Security -> System -> Advanced system settings -> Environment Variables_, and select the `Path` variable under `System Variables` (if you want to change it for all users) or `User Variables` (if you only want to change it for the current user).
-## ARM64
+Simply edit the PATH variable's value and add the directory (with a semicolon separator).
-- Install [CMake](https://cmake.org/download) using the `windows-i386` installer (`windows-x86_64` is not going to work, `C++ CMake tools for Windows` from VS might also not work).
-- Use MSBuild instead of Ninja (no official support for ARM64 yet).
-- Specify the architecture in explicit way.
+### ARM64 (Experimental)
-```cmd
-build.cmd -c Release -subset clr+libs+libs.tests -arch arm64 -msbuild
-```
+* Install [CMake](https://cmake.org/download) using the `windows-i386` installer (`windows-x86_64` is not going to work, `C++ CMake tools for Windows` from VS might also not work).
+* Use MSBuild instead of Ninja (no official support for ARM64 yet).
-# Running Tests
+# Building and Running CoreCLR Tests
-Details on test metadata can be found in [test-configuration.md](test-configuration.md).
+* [Requirements](#requirements)
+* [Overview](#overview)
+* [Building the Core_Root](#building-the-core_root)
+* [Building the Tests](#building-the-tests)
+ * [Building an Individual Test](#building-an-individual-test)
+ * [Building a Test Directory](#building-a-test-directory)
+ * [Building a Test Subtree](#building-a-test-subtree)
+ * [Building C++/CLI Native Test Components Against the Live Ref Assemblies](#building-ccli-native-test-components-against-the-live-ref-assemblies)
+ * [Test Priorities](#test-priorities)
+* [Running the Tests](#running-the-tests)
+ * [Running Individual Tests](#running-individual-tests)
+ * [Tests Without a Main Method](#tests-without-a-main-method)
+ * [PAL Tests (MacOS and Linux only)](#pal-tests-macos-and-linux-only)
+ * [Building PAL Tests](#building-pal-tests)
+ * [Running PAL Tests](#running-pal-tests)
+* [Modifying Tests](#modifying-tests)
+* [Investigating Test Failures](#investigating-test-failures)
-## Build All Tests
+This guide will walk you through building and running the CoreCLR tests. These are located within the `src/tests` subtree of the runtime repo.
-1) Build the CoreCLR product
- * [Unix](../../building/coreclr/linux-instructions.md)
- * [macOS](../../building/coreclr/osx-instructions.md)
- * [Windows](../../building/coreclr/README.md)
-2) [Build the libraries](../../building/libraries/README.md) in Release configuration. Pass the configuration of CoreCLR you just built to the build script (e.g. `-runtimeconfiguration debug`).
-3) From the root directory run the following command:
- * Non-Windows - `src/tests/build.sh`
- * Windows - `src\tests\build.cmd`
- * Supply `-h` for usage flags
+## Requirements
-### Test priority
+In order to build CoreCLR tests, you will need to have built the runtime and the libraries (that is, _clr_ and _libs_ subsets). You can find more detailed instructions per platform in their dedicated docs:
-The CoreCLR tests have two priorities, 0 and 1. The priority 0 tests run by default on all pull requests (PRs), while the priority 1 tests run in outerloop CI runs.
+* [Windows](/docs/workflow/building/coreclr/windows-instructions.md)
+* [MacOS](/docs/workflow/building/coreclr/macos-instructions.md)
+* [Linux](/docs/workflow/building/coreclr/linux-instructions.md)
-### Examples
+For CoreCLR testing purposes, it is more than enough to simply build the _libs_ subset, as far as it concerns the libraries. If you want to know more in-depth about them, they have their own [libraries dedicated docs section](/docs/workflow/building/libraries/README.md).
-* Build all tests priority 1 and higher
- * `build.cmd -priority=1`
- * `build.sh -priority1`
+## Overview
-## Build Individual Test
+As mentioned in the introduction, all test-building work is done from the `src/tests` folder, so we will consider that our starting point for the rest of this guide.
-Note: CoreCLR must be built prior to building an individual test. See the first step, above, for building all tests.
+Building the tests can be as simple as calling the build script without any arguments. This will by default look for a _Debug_ built runtime and _Release_ built libraries. However, by passing the appropriate flags, you can define which configurations you want to use. For example, let's suppose you have a _Checked_ runtime with _Debug_ libraries:
-* Native Test: Build the generated CMake projects
- * Projects are auto-generated when the `build.sh`/`build.cmd` script is run
- * It is possible to explicitly run only the native test build with `build.sh/cmd skipmanaged`
-* Managed Test: Invoke `dotnet build` on the project directly. `dotnet` can be the `dotnet.sh` or `dotnet.cmd` script in the repo root.
- ```
- <runtime-repo-root>/dotnet.sh build <runtime-repo-root>/src/tests/<path-to-project> -c <configuration>
- ```
- * To build managed test projects with dependencies on native test projects, the native test project must first be built.
+```bash
+./src/tests/build.sh checked /p:LibrariesConfiguration=Debug
+```
-## Additional Documents
+Note that for the libraries configuration, we are passing the argument directly to MSBuild instead of the build script, hence the `/p:LibrariesConfiguration` flag. Also, make sure you use the correct syntax depending on our platform. The _cmd_ script takes the arguments by placing, while the _sh_ script requires them to be with a hyphen.
-* [Windows](windows-test-instructions.md)
-* [Non-Windows](unix-test-instructions.md)
+**NOTE**: Building the whole test suite is a very lengthy process, so it is highly recommended you build individual tests, and/or test subtrees as you need them, to make your workflow more efficient. This is explained in detail later on in this doc.
+
+## Building the Core_Root
+
+The Core_Root folder is some sort of "dev-easy-to-use full build" of the product. It contains the built runtime binaries, as well as the library packages required to run tests. It is required that you build the libraries subset (`--subset libs`) before this command can be run.
+
+Note that, as mentioned in the section above, running the tests build script by default searches the libraries in _Release_ mode, regardless of the runtime configuration you specify. If you built your libraries in another configuration, then you have to pass down the appropriate flag `/p:LibrariesConfiguration=<your_config>`.
+
+The simplest command to generate the _Core\_Root_ from the repository's root path is the following:
+
+```cmd
+.\src\tests\build.cmd generatelayoutonly
+```
+
+This example assumes you built CoreCLR on _Debug_ mode and the Libraries on _Release_ mode, hence no additional flags are needed. After the build is complete, you will be able to find the output in the `artifacts/tests/coreclr/<OS>.<arch>.<configuration>/Tests/Core_Root` folder.
+
+## Building the Tests
+
+The following subsections will explain how to segment the test suite according to your needs. There are three main scopes of building tests:
+
+* Individual Test
+* Full Directory
+* Entire Subtree
+
+When no set is specified, almost the whole test suite (`src/tests`) is built.
+
+One of the most important attributes tests have is their **priority**. By default, only those tests with _Priority 0_ are built, regardless of whether it's an individual one, a directory, or a subtree, unless otherwise specified with the `-priority` flag. More info on this in [its dedicated section](#test-priorities). Regardless of which subset you build, all the outputs will be placed in `artifacts/tests/coreclr/<OS>.<arch>.<configuration>`.
+
+**NOTE**: Some tests have native components to them. It is highly recommended you build all of those prior to attempting to build any managed test in the following sections, as it's not a very costly or lengthy process:
+
+```cmd
+.\src\tests\build.cmd skipmanaged
+```
+
+### Building an Individual Test
+
+To build an individual test, you have to pass the `-test` flag along with the path to the test's `csproj` file to the build script. You can select more than one by repeating the `-test` flag. For example, let's try building a couple JIT tests:
+
+On Windows:
+
+```cmd
+.\src\tests\build.cmd test JIT\Intrinsics\MathRoundDouble_ro.csproj test JIT\Intrinsics\MathFloorDouble_ro.csproj
+```
+
+On MacOS and Linux:
+
+```bash
+./src/tests/build.sh -test:JIT/Intrinsics/MathRoundDouble_ro.csproj -test:JIT/Intrinsics/MathFloorDouble_ro.csproj
+```
+
+Alternatively, you can call _build_ directly using the `dotnet.cmd/dotnet.sh` script at the root of the repo and pass all arguments directly yourself:
+
+```bash
+./dotnet.sh build -c <Your Configuration> src/tests/path/to/test/csproj
+```
+
+### Building a Test Directory
+
+To build all the tests contained in an individual directory, you have to pass the `-dir` flag along with the directory's path to the build script. Just like with individual tests, you can select more than one by repeating the `-dir` flag. For example, let's try a couple of folders in the JIT subtree:
+
+On Windows:
+
+```cmd
+.\src\tests\build.cmd dir JIT\Methodical\Arrays\lcs dir JIT\Methodical\cctor\misc\Desktop
+```
+
+On MacOS and Linux:
+
+```bash
+./src/tests/build.sh -dir:JIT/Methodical/Arrays/lcs -dir:JIT/Methodical/cctor/misc/Desktop
+```
+
+### Building a Test Subtree
+
+To build a whole subtree, you have to pass the path to the root of the subtree you want with the `-tree` flag. Just like with any other subset, you can select more than one by repeating the `-tree` flag. For example, let's try building all the base services exceptions, and methodical JIT tests:
+
+On Windows:
+
+```cmd
+.\src\tests\build.cmd tree baseservices\exceptions tree JIT\Methodical
+```
+
+On MacOS and Linux:
+
+```bash
+./src/tests/build.sh -tree:baseservices/exceptions -tree:JIT/Methodical
+```
+
+### Building C++/CLI Native Test Components Against the Live Ref Assemblies
+
+By default, the _C++/CLI_ native test components build against the _ref pack_ from the SDK specified in the `global.json` file in the root of the repository. To build these components against the _ref assemblies_ produced in the build, pass the `-cmakeargs -DCPP_CLI_LIVE_REF_ASSEMBLIES=1` parameters to the test build. For example:
+
+```bash
+./src/tests/build.sh skipmanaged -cmakeargs -DCPP_CLI_LIVE_REF_ASSEMBLIES=1
+```
+
+### Test Priorities
+
+As mentioned earlier in this guide, each test has a priority number assigned to them, and only tests with _Priority 0_ are built by default.
+
+Now, here is where things get a little complicated. Test priority filtering is orthogonal to specifying test subsets. This means that even if when specifying tests, directories, and/or subtrees, you have to explicitly provide the priority if the test(s) of interest are not priority 0. Otherwise, the build will skip them.
+
+Another very important thing to keep in mind, is that priorities are accumulative. This means that if for example, you pass `-priority=1` to the build script, all priority 0 _AND_ priority 1 tests get built.
+
+Let's take one of the examples used in the previous subsections. Assume you want to build all _JIT Methodical Div_ tests, including both _pri0_ and _pri1_. This is how the command-line would look:
+
+On Windows:
+
+```cmd
+.\src\tests\build.cmd dir JIT\Methodical\divrem\div -priority=1
+```
+
+On MacOS and Linux:
+
+```bash
+./src/tests/build.sh -dir:JIT/Methodical/divrem/div -priority1
+```
+
+**NOTE**: Yes, you're seeing it right. The `priority` flag is a bit different between the Windows and MacOS/Linux scripts.
+
+## Running the Tests
+
+The simplest way to run in-bundle the tests you've built is by using the `run.cmd/run.sh` script after you've worked with `build.cmd/build.sh`. The running script takes flags very similarly to the build one. Let's suppose you have a _Checked_ runtime you want to test on an _x64_ machine. You'd run all your built tests with the following command-line:
+
+```cmd
+.\src\tests\run.cmd x64 checked
+```
+
+The `run.cmd/run.sh` scripts also have a number of flags you can pass to set specific conditions and environment variables, such as _JIT Stress_, _GC Stress_, and so on. Run it with only any one of the help flags for more details.
+
+Once your tests are done running, a report will be generated with all the results under the `artifacts/log` folder, and will be named `TestRun_<Arch>_<Configuration>.html`. The tests that failed will be listed in a file called `TestRunResults_<OS>_<Arch>_<Configuration>.err`.
+
+For individual test results and outputs, those are written in a `Reports` folder within the test root's directory (`artifacts/tests/coreclr/<OS>.<Arch>.<Configuration>`). For example, the results for the _JIT Intrinsics Math Round Double_ test shown in [Building an Individual Test](#building-an-individual-test), would be placed in `artifacts/tests/coreclr/<OS>.<Arch>.<Configuration>/Reports/JIT/Intrinsics/MathRoundDouble_ro`.
+
+### Running Individual Tests
+
+After you've built one (or more) tests, the way to run them is by calling the generated `cmd/sh` script alongside them. These scripts take three optional arguments:
+
+* `-debug`: Receives the path of a debugger to run the test under in.
+* `-env`: Path to a _.env_ file to specify environment variables to be set for the test. More info about _dotenv_ can be found in [their repo](https://github.com/motdotla/dotenv).
+* -coreroot: The path to the Core_Root you wish to use. Note that this flag is mandatory unless you have the `CORE_ROOT` environment variable set. Then, you can omit it and the script will use that one.
+
+If this list of parameters feels familiar, it's because it's virtually the same as the arguments that `corerun` receives. More info on `corerun` in its [how-to-use doc](/docs/workflow/testing/using-corerun-and-coreroot.md).
+
+These scripts have a couple more hidden functionalities, which can be activated by setting their environment variables prior to running the script:
+
+* Run with Crossgen2: `RunCrossGen2=1`
+* Build and run as composite: `CompositeBuildMode=1`. Note that this one depends on `RunCrossGen2` being set.
+
+Let's run one of the Intrinsics tests as an example:
+
+On Windows Command Prompt:
+
+```cmd
+set CORE_ROOT=<repo_root>\artifacts\tests\coreclr\windows.<Arch>.<Configuration>\Tests\Core_Root
+cd path\to\JIT\Intrinsics\MathRoundDouble_ro
+.\MathRoundDouble_ro.cmd
+```
+
+On MacOS/Linux:
+
+```bash
+export CORE_ROOT=<repo_root>/artifacts/tests/coreclr/<OS>.<Arch>.<Configuration>/Tests/Core_Root
+cd path/to/JIT/Intrinsics/MathRoundDouble_ro
+./MathRoundDouble_ro.sh
+```
+
+On Powershell:
+
+```powershell
+$Env:CORE_ROOT = '<repo_root>\artifacts\tests\coreclr\windows.<Arch>.<Configuration>\Tests\Core_Root'
+cd path\to\JIT\Intrinsics\MathRoundDouble_ro
+.\MathRoundDouble_ro.cmd
+```
+
+Alternatively, instead of setting the _CORE\_ROOT_ environment variable, you can specify it directly to the test's script via the `-coreroot` flag, as mentioned at the beginning of this section:
+
+On Windows:
+
+```cmd
+cd path\to\JIT\Intrinsics\MathRoundDouble_ro
+.\MathRoundDouble_ro.cmd -coreroot <repo_root>\artifacts\tests\coreclr\windows.<Arch>.<Configuration>\Tests\Core_Root
+```
+
+On MacOS/Linux:
+
+```bash
+cd path/to/JIT/Intrinsics/MathRoundDouble_ro
+./MathRoundDouble_ro.sh -coreroot=<repo_root>/artifacts/tests/coreclr/<OS>.<Arch>.<Configuration>/Tests/Core_Root
+```
+
+#### Tests Without a Main Method
+
+Guide on how to run tests without a `Main()` Method coming soon!
+
+### PAL Tests (MacOS and Linux only)
+
+The PAL layer tests are exclusive to Unix-based operating systems. This section will go on how to work with them.
+
+#### Building PAL Tests
+
+Firstly, build them by passing the `paltests` subset to the main build script of the repo:
+
+```bash
+./build.sh -s clr.paltests
+```
+
+#### Running PAL Tests
+
+Once you're done building them, you can run them all either including or excluding the disabled tests:
+
+_Including Disabled Tests:_
+
+```bash
+./src/coreclr/pal/tests/palsuite/runpaltests.sh artifacts/bin/coreclr/<OS>.<Arch>.<Configuration>/paltests
+```
+
+_Excluding Disabled Tests:_
+
+```bash
+cd artifacts/bin/coreclr/<OS>.<Arch>.<Configuration>
+./paltests/runpaltests.sh paltests
+```
+
+In order to run only specific tests, edit `paltestlist.txt` in under `src/coreclr/pal/tests/palsuite`, and adjust it to your needs (don't check in those changes though). The test(s) results will be output to `/tmp/PalTestOutput/default/pal_tests.xml`.
+
+To disable tests in the CI, edit `src/coreclr/pal/tests/palsuite/issues.targets` accordingly.
+
+## Modifying Tests
+
+If you need to edit any given test's source code, simply make your changes and rebuild the test project. Then, you can re-run it as needed following the instructions detailed in the sections above.
+
+## Investigating Test Failures
+
+Upon completing a test run with `run.sh/run.cmd`, you may find one or more tests failing. If this is the case, there will be additional files detailing the failures in each test's `Reports` folder (see [Running the Tests](#running-the-tests) for more info regarding reports). There are 2 main files of interest:
+
+* `<Test>.output.txt`: Contains all the information logged by the test.
+* `<Test>.error.txt`: Contains all the information reported by _CoreRun_ when the test process crashed.
+
+The test's report will also contain the test command exactly as it run it, so you can investigate it further, either by running the app in the way you see fit and/or as the [Running Individual Tests](#running-individual-tests) details.
# Testing Workloads
-Workloads based on packages in `artifacts` can be installed, and used for testing.
-
-- This is done by installing a specified SDK version (`$(SdkVersionForWorkloadTesting)`) in `artifacts/bin/dotnet-workload`.
-- Then the manifest for the workload in `@(WorkloadIdForTesting)` is installed
- - Then workload packs are installed
- - packs, or manifests not generated by `runtime`, are restored from nuget
+* [Multiple runtime packs](#multiple-runtime-packs)
+* [Limitations](#limitations)
-- The SDK is installed by `ProvisionSdkForWorkloadTesting` target
-- and the workload is installed by `InstallWorkloadUsingArtifacts` target, using the `InstallWorkloadFromArtifacts` task
+Workloads based on packages in `artifacts` can be installed, and used for testing.
-- `@(WorkloadIdForTesting)`:
+* This is done by installing a specified SDK version (`$(SdkVersionForWorkloadTesting)`) in `artifacts/bin/dotnet-workload`.
+* Then the manifest for the workload in `@(WorkloadIdForTesting)` is installed
+* Then workload packs are installed
+* Packs or manifests not generated by `runtime`, are restored from nuget
+* The SDK is installed by `ProvisionSdkForWorkloadTesting` target and the workload is installed by `InstallWorkloadUsingArtifacts` target, using the `InstallWorkloadFromArtifacts` task `@(WorkloadIdForTesting)`:
Example for wasm:
+
```xml
- <WorkloadIdForTesting Include="wasm-tools"
- Name="microsoft.net.workload.mono.toolchain"
- ManifestName="Microsoft.NET.Workload.Mono.ToolChain"
- Version="$(PackageVersion)"
- VersionBand="$(SdkBandVersion)" />
+<WorkloadIdForTesting Include="wasm-tools"
+ Name="microsoft.net.workload.mono.toolchain"
+ ManifestName="Microsoft.NET.Workload.Mono.ToolChain"
+ Version="$(PackageVersion)"
+ VersionBand="$(SdkBandVersion)" />
```
-- Currently, this is used only by `src/mono/wasm/Wasm.Build.Tests`
+Currently, this is used only by `src/tests/BuildWasmApps/Wasm.Build.Tests`
## Multiple runtime packs
-The workload depends on three runtime packs - single threaded, multithreaded, and for perf tracing. If you have a local
-runtime build, for say multithreaded, then the workload install will fail because of missing runtime pack nugets for
-the other two variants.
+The workload depends on three runtime packs - single threaded, multithreaded, and for perf tracing. If you have a local runtime build, for say multithreaded, then the workload install will fail because of missing runtime pack nugets for the other two variants.
-For non-CI builds, we build the same runtime pack nuget but with the different expected names. So, essentially you get all
-the expected nugets, but they are all the same except for the name.
+For non-CI builds, we build the same runtime pack nuget but with the different expected names. So, essentially you get all the expected nugets, but they are all the same except for the name.
If you have all the nugets available, and want to avoid the above behavior then set `WasmSkipMissingRuntimeBuild=true`.
-## Limitations:
+## Limitations
-- The cross compiler package is built manually from the `InstallWorkloadUsingArtifacts` target
+The cross compiler package is built manually from the `InstallWorkloadUsingArtifacts` target.
--- /dev/null
+# Using Corerun To Run a .NET Application
+
+* [Introduction](#introduction)
+* [The CoreRun](#the-corerun)
+ * [Running Apps with CoreRun](#running-apps-with-corerun)
+ * [Using CoreRun with the class library from the shared system-wide .NET installation](#using-corerun-with-the-class-library-from-the-shared-system-wide-net-installation)
+ * [Using CoreRun to Execute a Published Self-Contained Application](#using-corerun-to-execute-a-published-self-contained-application)
+* [The Core_Root](#the-core_root)
+ * [Additional CoreRun Options](#additional-corerun-options)
+
+This guide will walk you through using the Corerun and Core_Root your own build from the runtime repo for testing, running apps, and so on. This doc assumes you've already built at least the _clr_ subset of the repo, and have the binaries under `artifacts/bin/coreclr/<OS>.<arch>.<configuration>`. If this is not your case, the [CoreCLR building docs](/docs/workflow/building/coreclr/README.md) have detailed instructions on how to get these artifacts.
+
+## Introduction
+
+To run a .NET app with the runtime you've built, you will need a _host_ program that will load the runtime, as well as all the other .NET libraries that your application might need. There are three main ways to go about this:
+
+* Use your machine's installed .NET SDK and replace the necessary binaries in a self-contained app.
+* Use your build's _Dev Shipping Packages_ to run your app.
+* Use the _CoreRun_ host generated as part of your build's artifacts.
+
+This guide focuses on the third of the bullet points described above. For the other two, we have docs dedicated to them:
+
+* [Using your build with your machine's installed SDK](using-your-build-with-installed-sdk.md)
+* [Using your build's dev shipping packages](using-dev-shipping-packages.md)
+
+## The CoreRun
+
+The `corerun` binary is designed to be a platform agnostic tool for quick testing of a locally built .NET runtime. It helps facilitate .NET runtime development and investigation of test failures. This method is the most recommended one when you are making lots of changes that you want to keep continually testing and debugging, since it's the fastest way to apply them.
+
+`Corerun` does not know about NuGet at all. It just needs to find the .NET runtime, `coreclr.dll`, `libcoreclr.dylib`, or `libcoreclr.so` depending on your platform, and any class library assemblies like for example, `System.Runtime.dll`, `System.IO.dll`, and so on.
+
+`Corerun` achieves these goals by using heuristics in the following order:
+
+1. Check if the user passed the `--clr-path` argument.
+2. Check if the `CORE_ROOT` environment variable is defined.
+3. Check if the .NET runtime binary is in the same directory as the `corerun` binary.
+
+Regardless of which method is used to discover the .NET runtime binary, its location is used to also find all of the base class library assemblies. Additional directories can be included in the set of class library assemblies by defining the `CORE_LIBRARIES` environment variable.
+
+The above heuristics can be used in a number of ways, providing you with multiple options to test using your `corerun`.
+
+### Running Apps with CoreRun
+
+In the following subsections, we will describe how to run any apps you might create, but using your built runtime instead of the one installed on your machine.
+
+#### Using CoreRun with the class library from the shared system-wide .NET installation
+
+For this example, let's create a simple _Hello World_ app:
+
+```cmd
+mkdir HelloWorld && cd HelloWorld
+dotnet new console
+dotnet build
+```
+
+Now, instead of running our app the usual way, we will use our newly built `corerun` to execute it using our build of the runtime. For this, we will follow the steps denoted below:
+
+* First we will add the `Core_Root` folder to the `PATH` environment variable for ease of use. Note that you can always skip this step and fully qualify the name instead.
+ * This example assumes you built on the _Debug_ configuration for the _x64_ architecture. Make sure you adjust the path accordingly to your kind of build.
+* Then, we also need the libraries. Since we only built the runtime, we will tell `corerun` to use the ones shipped with .NET's default installation on your machine.
+ * This example assumes your default .NET installation's version is called "_7.0.0_". Same deal as with your runtime build path, adjust to the version you have installed on your machine.
+* Afterwards, we can finally run our app.
+
+On Windows Command Prompt:
+
+```cmd
+set PATH=%PATH%;<repo_root>\artifacts\bin\coreclr\windows.x64.Debug
+set CORE_LIBRARIES=%ProgramFiles%\dotnet\shared\Microsoft.NETCore.App\7.0.0
+corerun HelloWorld.dll
+```
+
+On MacOS and Linux:
+
+```bash
+# Change osx to linux if you're on a Linux machine.
+export PATH="$PATH:<repo_root>/artifacts/bin/coreclr/osx.x64.Debug"
+export CORE_LIBRARIES="/usr/local/share/dotnet/shared/Microsoft.NETCore.App/7.0.0"
+corerun HelloWorld.dll
+```
+
+On Powershell:
+
+```powershell
+# Note the '+=' since we're appending to the already existing PATH variable.
+# Also, replace the ';' with ':' if on Linux or MacOS.
+$Env:PATH += ';<repo_root>\artifacts\bin\coreclr\windows.x64.Debug'
+$Env:CORE_LIBRARIES = %ProgramFiles%\dotnet\shared\Microsoft.NETCore.App\7.0.0
+corerun HelloWorld.dll
+```
+
+Once you set the `PATH` and `CORE_LIBRARIES` environment variables, when you issue `corerun HelloWorld.dll` following the snippets above, `corerun` now knows where to get the assemblies it needs. Note that this setup only has to be done once, as long as you stay in the same terminal instance. After a rebuild with more changes you might make, you can simply rerun `corerun` directly to run your application. The stage is set for it to work as expected.
+
+#### Using CoreRun to Execute a Published Self-Contained Application
+
+When an application is published as self-contained (`dotnet publish --self-contained`), it deploys all the class libraries needed as well. Thus if you simply change the `CORE_LIBRARIES` defined in the previous section to point at that publication directory, then the effect will be that your `corerun` will be getting all that libraries' code from your deployed application.
+
+## The Core_Root
+
+The test build script (`src/tests/build.cmd` or `src/tests/build.sh`) sets up a directory where it gathers the CoreCLR that has just been built with the pieces of the class libraries that the tests need. It places these binaries in the directory `artifacts/tests/coreclr/<OS>.<Arch>.<Configuration>/Tests/Core_Root`. Note that the test building process is a lengthy one, so it is recommended to only generate the Core_Root with the `-generatelayoutonly` flag to the tests build script, and build individual tests and/or test trees as you need them.
+
+**NOTE**: In order to generate the Core_Root, you must also have built the libraries beforehand with `-subset libs`. More details in the [testing CoreCLR doc](/docs/workflow/testing/coreclr/testing.md).
+
+Once you have your Core_Root, it's just a matter of calling it directly or adding it to your `PATH` environment variable, and you're ready to run your apps with it.
+
+On Windows Command Prompt:
+
+```cmd
+set PATH=%PATH%;<repo_root>\artifacts\tests\coreclr\windows.x64.Debug\Tests\Core_Root
+corerun HelloWorld.dll
+```
+
+On MacOS and Linux:
+
+```bash
+# Change linux to osx if you're on a MacOS machine.
+export PATH="$PATH:<repo_root>/artifacts/tests/coreclr/linux.x64.Debug/Tests/Core_Root"
+corerun HelloWorld.dll
+```
+
+On Powershell:
+
+```powershell
+# Note the '+=' since we're appending to the already existing PATH variable.
+# Also, replace the ';' with ':' if on Linux or MacOS.
+$Env:PATH += ';<repo_root>\artifacts\tests\coreclr\windows.x64.Debug\Tests\Core_Root'
+.\corerun HelloWorld.dll
+```
+
+The advantage of generating the Core_Root, instead of sticking to the _corerun_ from the _clr_ build, is that you can also test and debug libraries at the same time.
+
+### Additional CoreRun Options
+
+The `corerun` binary has a few optional command-line arguments described when issuing `corerun --help`:
+
+* `--clr-path <PATH>`: Pass the location of Core_Root on the command line. You can omit this flag if either the `corerun` you're using is within the Core_Root folder, or you have set the Core_Root's path by means of the `CORE_ROOT` environment variable.
+ * Example: `corerun --clr-path /path/to/core_root HelloWorld.dll`
+* `--property <PROPERTY>`: Supply a property to pass to the .NET runtime during initialization.
+ * Example: `corerun --property System.GC.Concurrent=true HelloWorld.dll`
+* `--debug`: Wait for a debugger to attach prior to loading the .NET runtime.
+ * Example: `corerun --debug HelloWorld.dll`
+* `--env`: Pass the path to a `.env` file to specify environment variables for the test run. More info about `dotenv` can be found in [their repo](https://github.com/motdotla/dotenv).
+ * For example, `corerun --env gcstress.env HelloWorld.dll`
+++ /dev/null
-
-# Using `corerun` To Run a .NET Application
-
-The page [Using your .NET Runtime Build with dotnet cli](../using-dotnet-cli.md) gives detailed instructions on using the standard
-command line host (that is, `dotnet.exe` or `dotnet`), and SDK to run an application with a local build of the
-.NET Runtime. This is the preferred mechanism for you to officially deploy
-your changes to other people since dotnet.exe and NuGet ensure that you end up with a consistent
-set of binaries that can work together.
-
-However, packing and unpacking the runtime binaries adds extra steps to the deployment process. When
-working in a tight edit-build-debug loop, these extra steps become cumbersome.
-
-For this tight edit-build-debug loop, there is a simplified alternative to `dotnet` called `corerun` which
-does not know about NuGet at all. It just needs to find the .NET runtime (for example, `coreclr.dll`)
-and any class library assemblies (for example, `System.Runtime.dll`, `System.IO.dll`, etc).
-
-It does this using heuristics in the following order:
-
-1. Check if the user passed the `--clr-path` argument.
-1. Check if the `CORE_ROOT` environment variable is defined.
-1. Check if the .NET runtime binary is in the same directory as the `corerun` binary.
-
-Regardless of which method is used to discover the .NET runtime binary, its location is used to discover
-both the .NET runtime binary and all base class library assemblies. Additional directories can be included
-in the set of class library assemblies by defining the `CORE_LIBRARIES` environment variable.
-
-The above heuristics can be used in a number of ways.
-
-## Getting the class library from the shared system-wide runtime
-
-Consider that you already have a .NET application assembly called `HelloWorld.dll` and wish to run it.
-You could make such an assembly by using an officially installed .NET runtime with `dotnet new` and `dotnet build` in a `HelloWorld` directory.
-
-If you execute the following on Windows, the `HelloWorld` assembly will be run.
-
-```cmd
-set PATH=%PATH%;<repo_root>\artifacts\tests\coreclr\windows.x64.Debug\Tests\Core_Root\
-set CORE_LIBRARIES=%ProgramFiles%\dotnet\shared\Microsoft.NETCore.App\1.0.0
-
-corerun HelloWorld.dll
-```
-
-On non-Windows platforms, setting environment variables is different but the logic is identical. For example, on macOS use `/usr/local/share` for `%ProgramFiles%`.
-
-The `<repo_root>` represents the base of your dotnet/runtime repository. The first line puts the build output directory
-(your OS, architecture, and buildType may be different) and thus the `corerun` binary on your path.
-The second line tells `corerun` where to find class library assemblies. In this case we tell it to find them where
-the installation of `dotnet` placed its copy. The version number in the path may be different depending on what
-is currently installed on your system.
-
-Thus when you run `corerun HelloWorld.dll`, `corerun` knows where to get the assemblies it needs.
-Once you set the path and `CORE_LIBRARIES` environment variable, after a rebuild you can simply use
-`corerun` to run your application – you don't have to move any binaries around.
-
-## Using `corerun` to Execute a Published Application
-
-When `dotnet publish` publishes an application, it deploys all the class libraries needed as well.
-Thus if you simply change the `CORE_LIBRARIES` definition in the previous instructions to point at
-that publication directory, but run the `corerun` from your build output, the effect will be that you
-run your new runtime getting all the other code needed from that deployed application. This is
-very convenient because you don't need to modify the deployed application in order to test
-your new runtime.
-
-## How CoreCLR Tests use `corerun`
-
-The test build script (`src/tests/build.cmd` or `src/tests/build.sh`) sets up a directory where it
-gathers the CoreCLR that has just been built with the pieces of the class library that tests need.
-It places this runtime in the directory
-`artifacts\tests\coreclr\<OS>.<Arch>.<BuildType>\Tests\Core_Root`
- starting at the repository root. The way the tests are expected to work is that you can set the environment
-variable `CORE_ROOT` to this directory – you don't have to set `CORE_LIBRARIES` since the test environment has copied all base class libraries assemblies to this `Core_Root` directory – and you can run any test. For example, after building the tests
-(running `src\tests\build` from the repository base), you can do the following on Windows to set up an environment where `corerun` can run any test.
-
-```cmd
-set PATH=%PATH%;<repo_root>\artifacts\Product\windows.x64.Debug
-set CORE_ROOT=<repo_root>\artifacts\tests\coreclr\windows.x64.Debug\Tests\Core_Root
-```
-For example, the following runs the finalizeio test on Windows.
-
-```cmd
-corerun artifacts\tests\coreclr\windows.x64.Debug\GC\Features\Finalizer\finalizeio\finalizeio\finalizeio.dll
-```
-
-## Additional `corerun` options
-
-The `corerun` binary is designed to be a platform agnostic tool for quick testing of a locally built .NET runtime.
-This means the `corerun` binary must be able to feasibly exercise any scenario the official `dotnet` binary is capable
-of. It must also be able to help facilitate .NET runtime development and investigation of test failures.
-See `corerun --help` for additional details.
-
-**Options**
-
-`--clr-path <PATH>` - Pass the location of Core Root on the command line.
-- For example, `corerun --clr-path /usr/scratch/private_build HelloWorld.dll`
-
-`--property <PROPERTY>` - Supply a property to pass to the .NET runtime during initialization.
-- For example, `corerun --property System.GC.Concurrent=true HelloWorld.dll`
-
-`--debug` - Wait for a debugger to attach prior to loading the .NET runtime.
-- For example, `corerun --debug HelloWorld.dll`
-
-`--env` - Pass the path to a file in the [`dotenv`](https://github.com/motdotla/dotenv) format to specify environment variables for the test run.
-- For example, `corerun --env gcstress.env HelloWorld.dll`.
--- /dev/null
+# Using your build's Shipping Packages with the .NET SDK
+
+* [Requirements](#requirements)
+ * [Build the Shipping Packages](#build-the-shipping-packages)
+ * [Acquire the latest development .NET SDK](#acquire-the-latest-development-net-sdk)
+* [Creating and running the app with your build](#creating-and-running-the-app-with-your-build)
+ * [Create and Configure the App](#create-and-configure-the-app)
+ * [Write a small test](#write-a-small-test)
+ * [Publish and Run the App](#publish-and-run-the-app)
+* [Making Changes and Consuming Updated Packages](#making-changes-and-consuming-updated-packages)
+
+This guide will walk you through using the shipping packages of your own build from the runtime repo for testing, running apps, and so on. This is the lengthiest but most end-user-like of all the CoreCLR testing methods.
+
+This guided focuses on this scenario, but we also have detailed docs on the other ways of testing:
+
+* [Using your build with your machine's installed SDK](using-your-build-with-installed-sdk.md)
+* [Using CoreRun and CoreRoot](using-corerun-and-coreroot.md)
+
+## Requirements
+
+The following subsections will describe the requirements you will need to have ready in advance, and how to prepare them.
+
+### Build the Shipping Packages
+
+The shipping packages are comprised of four subsets of the runtime repo: Clr, Libraries, Packs, and the Host. So, to build all of those, issue the following command from the root of the repo:
+
+For Windows:
+
+```cmd
+.\build.cmd -s clr+libs+packs+host
+```
+
+For MacOS and Linux:
+
+```bash
+./build.sh -s clr+libs+packs+host
+```
+
+This will place several installers, Nuget packages, compressed archives, and other files within `artifacts/packages/<configuration>/Shipping`. You could actually install your built runtime to your machine using the installers here, but that's not recommended.
+
+### Acquire the latest development .NET SDK
+
+The [installer repo](https://github.com/dotnet/installer) has downloads to all nightly builds for all the currently supported platforms. Find the one that matches your machine and download it.
+
+To setup the nightly SDK, you can either install it to your machine or use a portable build. If you downloaded the _installer_, then just follow the usual installation instructions, and you're done.
+
+To use a portable build (recommended way), first extract somewhere the _zip/tar.gz_ you downloaded at the beginning of this section. Then, you can either add the path where you extracted it to your `PATH` environment variable, or always fully qualify the path to the `dotnet` you extracted (e.g. `/path/to/nightly/build/dotnet`).
+
+After setting up the new dotnet you can verify you are using the newer version by issuing the `dotnet --version` command on it. At the time of writing, the version ought to be equal or greater than `8.0.100-*`.
+
+## Creating and running the app with your build
+
+Now that you have your environment set up, let's get the test app prepared.
+
+### Create and Configure the App
+
+First, create a simple console app like you usually do:
+
+```bash
+mkdir HelloWorld && cd HelloWorld
+dotnet new console
+```
+
+**NOTE**: Make sure you're using the nightly SDK you downloaded in the previous step.
+
+Next, we have to somehow tell the SDK that our built NuGet packages exist, and that we want to use them. For this purpose, we will create a `NuGet.Config` file within our app's folder with the `dotnet new nugetconfig` command.
+
+This config file will require a handful of modifications to work as we need it to. Firstly, create a folder where you'll want Nuget to cache its stuff locally for this experiment. We will call it `LocalNugetCache` in this example. Then, edit your config file with the new stuff:
+
+```xml
+<configuration>
+
+ <config>
+ <!-- Set the "value" here to the folder you will be using for your local Nuget cache. -->
+ <add key="globalPackagesFolder" value="Path/To/LocalNugetCache" />
+ </config>
+
+ <packageSources>
+ <!--To inherit the global NuGet package sources remove the <clear/> line below -->
+ <clear />
+
+ <!-- Any packages that might be required, but not present in your build, will have to be taken from the latest NuGet feed. -->
+ <!-- More info on: https://github.com/dotnet/installer#installers-and-binaries -->
+ <add key="dotnet8" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet8/nuget/v3/index.json" />
+
+ <!-- Set this path to where your Shipping Artifacts are located. Note that we are assuming a 'Debug' build in this example. -->
+ <add key="local runtime" value="Path/To/Runtime/artifacts/packages/Debug/Shipping" />
+ </packageSources>
+
+</configuration>
+```
+
+Once we have your `NuGet.Config` file ready, we have to make our project aware that we will be using a different runtime. Add the following to the `csproj` file of your app:
+
+```xml
+<ItemGroup>
+ <!-- At the time of writing, '8.0.0-dev' is the version of the runtime repo's shipping packages. -->
+ <FrameworkReference Update="Microsoft.NETCore.App" RuntimeFrameworkVersion="8.0.0-dev" />
+</ItemGroup>
+```
+
+If you're unsure of what version your packages are, it is included as part of their filenames. For example, pick the `nupkg` file that will be used with your app from your shipping folder (`artifacts/packages/<configuration>/Shipping`). It's name is something like `Microsoft.NETCore.App.Runtime.win-x64.8.0.0-dev.nupkg`, depending on the current version and your target platform.
+
+### Write a small test
+
+For illustration purposes, we will have our app print the version of the runtime in this example:
+
+```csharp
+using System.Diagnostics;
+
+var coreAssemblyInfo = FileVersionInfo.GetVersionInfo(typeof(object).Assembly.Location);
+Console.WriteLine($"Hello World from .NET {coreAssemblyInfo.ProductVersion}.");
+Console.WriteLine($"The location of System.Private.CoreLib.dll is '{typeof(object).Assembly.Location}'");
+```
+
+### Publish and Run the App
+
+The following command will build and publish your app:
+
+```cmd
+dotnet publish -r win-x64
+```
+
+Adjust the `win-x64` to match your machine's OS and architecture.
+
+Running this little app should yield an output like the following:
+
+```text
+Hello World from .NET 8.0.0-dev
+The location of System.Private.CoreLib.dll is '/path/to/your/app/bin/Debug/net7.0/win-x64/publish/System.Private.CoreLib.dll'
+```
+
+You might be wondering why it says `net7.0` if we are using an `8.0.0` SDK. At the time of writing, we are still preparing all that's needed to fully migrate development to .NET 8. In the not so distant future, the path should be `net8.0` as you might expect, and this doc will be updated accordingly.
+
+## Making Changes and Consuming Updated Packages
+
+You've now successfully tested your runtime build. However, more likely than not, you will be making further changes that you'll want to test. The issue here is you can't simply build the repo again and have it work. This is because of the _NuGet Cache_ mentioned earlier. Since the version number doesn't change locally, NuGet doesn't realize changes have been made and thus uses its cached version. To get around this, we have to get rid of such cache. That's why we set a local one using the `globalPackagesFolder` in the `nuget.config` file we created.
+
+So the steps to apply and test changes are the following:
+
+1. Build the runtime again like you did in the "[Build the Shipping Packages section](#build-the-shipping-packages)". Note that if you only make changes to CoreCLR _(clr)_ or the libraries _(libs)_, you can omit the other in the build command.
+2. Delete the local NuGet cache.
+3. Publish and run your app again.
+
+Now your app will be using the updated package.
--- /dev/null
+# Using your .NET Runtime Build with the Installed SDK
+
+* [Introduction](#introduction)
+ * [Acquire the latest development .NET SDK](#acquire-the-latest-development-net-sdk)
+* [Create a sample self-contained application](#create-a-sample-self-contained-application)
+ * [Publish your App](#publish-your-app)
+* [Update CoreCLR and System.Private.CoreLib.dll with your build](#update-coreclr-and-systemprivatecorelibdll-with-your-build)
+* [Confirm that the app used your new runtime (Optional)](#confirm-that-the-app-used-your-new-runtime-optional)
+* [Troubleshooting](#troubleshooting)
+ * [If it's not using your copied binaries](#if-its-not-using-your-copied-binaries)
+ * [If you get a consistency check assertion failure](#if-you-get-a-consistency-check-assertion-failure)
+ * [If you get a JIT load error](#if-you-get-a-jit-load-error)
+
+This guide will walk you through using your own build from the runtime repo for testing, running apps, and so on. This doc assumes you've already built at least the _clr_ subset of the repo, and have the binaries under `artifacts/bin/coreclr/<OS>.<arch>.<configuration>`. If this is not your case, the [CoreCLR building docs](/docs/workflow/building/coreclr/README.md) have detailed instructions on how to get these artifacts.
+
+## Introduction
+
+To run a .NET app with the runtime you've built, you will need a _host_ program that will load the runtime, as well as all the other .NET libraries that your application might need. There are three main ways to go about this:
+
+* Use your machine's installed .NET SDK and replace the necessary binaries in a self-contained app.
+* Use your build's _Dev Shipping Packages_ to run your app.
+* Use the _CoreRun_ host generated as part of your build's artifacts.
+
+This guide focuses on the first of the bullet points described above. For the other two, we have docs dedicated to them:
+
+* [Using your build's dev shipping packages](using-dev-shipping-packages.md)
+* [Using CoreRun and CoreRoot](using-corerun-and-coreroot.md)
+
+**NOTE**: It's unlikely, but it's possible that the officially released version of `dotnet` may not be compatible with the live repository. If this happens to you, then unfortunately, you will be limited to either installing a nightly build on your machine (not that recommended), or use another of the methods described above to test your build. This is because this method requires a _self-contained_ app, and the portable builds do not support this at the present time.
+
+### Acquire the latest development .NET SDK
+
+The [installer repo](https://github.com/dotnet/installer) has downloads to all nightly builds for all the currently supported platforms. Find the one that matches your machine and download it.
+
+To setup the nightly SDK, you can either install it to your machine or use a portable build. If you downloaded the _installer_, then just follow the usual installation instructions, and you're done.
+
+To use a portable build (check the note above though), first extract somewhere the _zip/tar.gz_ you downloaded at the beginning of this section. Then, you can either add the path where you extracted it to your `PATH` environment variable, or always fully qualify the path to the `dotnet` you extracted (e.g. `/path/to/nightly/build/dotnet`).
+
+After setting up dotnet you can verify you are using the newer version by issuing the `dotnet --version` command on it. At the time of writing, the version must be equal or greater than `8.0.100-*`.
+
+<!-- TODO: It feels like this link may or may not be more appropriate elsewhere. Need to dig deeper into the documentation, so leaving it here for the time being. -->
+For another small walkthrough see [Dogfooding .NET SDK](https://github.com/dotnet/runtime/blob/main/docs/project/dogfooding.md).
+
+## Create a sample self-contained application
+
+First things first. We need a sample app to test our runtime build on. Let's create a quick 'Hello World' app for this example.
+
+```cmd
+mkdir HelloWorld
+cd HelloWorld
+dotnet new console
+```
+
+In order to run with your local changes, the application needs to be self-contained, as opposed to running on the installed shared framework. In order to do that, you will need a `RuntimeIdentifier` for your project. You can specify it directly in the command-line later on, or you can write it in your app's `.csproj` file:
+
+```xml
+<PropertyGroup>
+ ...
+ <RuntimeIdentifier>win-x64</RuntimeIdentifier>
+</PropertyGroup>
+```
+
+We are using Windows x64 for this example. Make sure you set it to the platform and configuration you have your build in. The codenames for the most common OS's are:
+
+* _Windows_: `win`
+* _MacOS_: `osx`
+* _Linux_: `linux`
+
+For example, if we were testing a MacOS ARM64 build, our `RuntimeIdentifier` would be `osx-arm64`.
+
+### Publish your App
+
+Now is the time to build and publish. This step will trigger _restore_ and _build_.
+
+```cmd
+dotnet publish --self-contained
+```
+
+**NOTE:** If publish fails to restore runtime packages, then you'll need to configure custom NuGet feeds. This is a side-effect of using a dogfood .NET SDK: Its dependencies are not yet on the regular NuGet feed. To configure this, you have to:
+
+1. Run the command `dotnet new nugetconfig`
+2. Go to the newly created `NuGet.Config` file and replace the content with the following template:
+
+```xml
+<?xml version="1.0" encoding="utf-8"?>
+<configuration>
+ <packageSources>
+ <!--To inherit the global NuGet package sources remove the <clear/> line below -->
+ <clear />
+ <add key="nuget" value="https://api.nuget.org/v3/index.json" />
+ <add key="dotnet8" value="https://dnceng.pkgs.visualstudio.com/public/_packaging/dotnet8/nuget/v3/index.json" />
+ </packageSources>
+</configuration>
+```
+
+After you publish successfully, you will find all the binaries needed to run your application under `bin\Debug\net7.0\win-x64\publish`. You might be wondering why `net7.0` if we are using an `8.0.100` SDK. At the time of writing, we are still preparing all that's needed to fully migrate development to .NET 8. In the not so distant future, the path should be `net8.0` as you might expect, and this doc will be updated accordingly.
+
+**But we are not done yet, you need to replace the published runtime files with the files from your local build!**
+
+## Update CoreCLR and System.Private.CoreLib.dll with your build
+
+The publishing step described above creates a directory that has all the files necessary to run your app, including the CoreCLR runtime and the required libraries. Out of all these binaries, there are three notable ones that will contain any changes you make to the runtime:
+
+* `coreclr.dll (windows)/libcoreclr.dylib (macos)/libcoreclr.so (linux)`: Most modifications (with the exception of the JIT compiler and tools) that are C++ code update this binary.
+* `System.Private.CoreLib.dll`: If you modified managed C# code, it will end up here.
+* `clrjit.dll`: The JIT compiler. It is also required you copy this one to your published app.
+
+Now, here comes the main deal to test your build. Once you have your self-contained app published, and CoreCLR built, you will replace the binaries listed above with the generated artifacts. Copy them from `artifacts/bin/coreclr/<OS>.<arch>.<configuration>/` to your app's publication directory, which by default is `your-app-folder/bin/<configuration>/net7.0/<os-code>-<arch>/publish`.
+
+In our previous example this would be:
+
+* From: `artifacts/bin/coreclr/windows.x64.Debug/`
+* To: `HelloWorld/bin/Debug/net7.0/win-x64/publish/`
+
+## Confirm that the app used your new runtime (Optional)
+
+Congratulations, you have successfully used your newly built runtime.
+
+If you want to further ensure this is indeed the case before delving into more complex experiments and testing, you can run the following piece of code in your app:
+
+```csharp
+using System.Diagnostics;
+
+var coreAssemblyInfo = FileVersionInfo.GetVersionInfo(typeof(object).Assembly.Location);
+Console.WriteLine($"Core Runtime Info: {coreAssemblyInfo.ProductVersion}");
+Console.WriteLine($"System.Private.CoreLib.dll is located at: {typeof(object).Assembly.Location}");
+```
+
+That should tell you the version, and which user and machine built the assembly, as well as the _commit hash_ of the code at the time of building:
+
+```text
+Core Runtime Info: 8.0.0-dev
+System.Private.CoreLib.dll is located at: /path/to/your/app/bin/Debug/net7.0/win-x64/publish/System.Private.CoreLib.dll
+```
+
+What you are looking for here is that the core runtime used is labelled as `-dev`. This means it is indeed using the one you built in the runtime repo. Also, ensure that the picked _System.Private.CoreLib.dll_ is indeed the one in your `publish` folder.
+
+## Troubleshooting
+
+Here are a few very common errors you might encounter, and how to fix them.
+
+### If it's not using your copied binaries
+
+Make sure you are running the executable directly.
+
+```cmd
+.\bin\Debug\net7.0\win-x64\publish\HelloWorld.exe
+```
+
+If you use `dotnet run` it will overwrite your custom binaries before executing the app.
+
+### If you get a consistency check assertion failure
+
+This failure happens when you only copy `coreclr`, but not `System.Private.Corelib.dll` as well.
+
+```text
+Assert failure(PID 13452 [0x0000348c], Thread: 10784 [0x2a20]): Consistency check failed: AV in clr at this callstack:
+```
+
+### If you get a JIT load error
+
+If you forget to also copy `clrjit.dll`, you will get the following error message:
+
+```text
+Fatal error. Failed to load JIT compiler.
+```
+++ /dev/null
-
-# Using your .NET Runtime Build
-
-We assume that you have successfully built the repository and thus have files of the form
-```
- ~/runtime/artifacts/bin/coreclr/<OS>.<arch>.<flavor>/
-```
-
-To run your newly built .NET Runtime in addition to the application itself, you will need
-a 'host' program that will load the Runtime as well as all the other .NET libraries
-code that your application needs. The easiest way to get all this other stuff is to simply use the
-standard 'dotnet' host that installs with .NET SDK.
-
-The released version of 'dotnet' tool may not be compatible with the live repository. The following steps
-assume use of a dogfood build of the .NET SDK.
-
-## Acquire the latest development .NET SDK
-
-- [Win 64-bit Latest](https://aka.ms/dotnet/6.0/daily/dotnet-sdk-win-x64.zip)
-- [macOS 64-bit Latest](https://aka.ms/dotnet/6.0/daily/dotnet-sdk-osx-x64.tar.gz)
-- [Others](https://github.com/dotnet/installer#installers-and-binaries)
-
-To setup the SDK download the zip and extract it somewhere and add the root folder to your [path](../requirements/windows-requirements.md#adding-to-the-default-path-variable)
-or always fully qualify the path to dotnet in the root of this folder for all the instructions in this document.
-
-After setting up dotnet you can verify you are using the newer version by:
-
-`dotnet --info` -- the version should be greater than 3.0.0-*
-
-For another small walkthrough see [Dogfooding .NET SDK](https://github.com/dotnet/runtime/blob/main/docs/project/dogfooding.md).
-
-## Create sample self-contained application
-
-At this point, you can create a new 'Hello World' program in the standard way.
-
-```cmd
-mkdir HelloWorld
-cd HelloWorld
-dotnet new console
-```
-
-### Change project to be self-contained
-
-In order to update with your local changes, the application needs to be self-contained, as opposed to running on the
-shared framework. In order to do that you will need to add a `RuntimeIdentifier` to your project.
-
-```xml
-<PropertyGroup>
- ...
- <RuntimeIdentifier>win-x64</RuntimeIdentifier>
-</PropertyGroup>
-```
-For Windows you will want `win-x64`, for macOS `osx-x64` and `linux-x64` for Linux.
-
-### Publish
-
-Now is the time to publish. The publish step will trigger restore and build. You can iterate on build by calling `dotnet build` as
-needed.
-
-```cmd
-dotnet publish
-```
-
-**Note:** If publish fails to restore runtime packages you need to configure custom NuGet feeds. This is because you are using a dogfood .NET SDK: its dependencies are not yet on the regular NuGet feed. To do so you have to:
-
-1. run `dotnet new nugetconfig`
-2. go to the `NuGet.Config` file and replace the content with:
-
-```xml
-<?xml version="1.0" encoding="utf-8"?>
-<configuration>
- <packageSources>
- <!--To inherit the global NuGet package sources remove the <clear/> line below -->
- <clear />
- <add key="nuget" value="https://api.nuget.org/v3/index.json" />
- <add key="dotnet6" value="https://dnceng.pkgs.visualstudio.com/public/_packaging/dotnet6/nuget/v3/index.json" />
- </packageSources>
-</configuration>
-```
-
-After you publish you will find you all the binaries needed to run your application under `bin\Debug\netcoreapp3.0\win-x64\publish\`.
-
-```
-.\bin\Debug\netcoreapp3.0\win-x64\publish\HelloWorld.exe
-```
-
-**But we are not done yet, you need to replace the published runtime files with the files from your local build!**
-
-## Update CoreCLR from raw binary output
-
-Updating CoreCLR from raw binary output is easier for quick one-off testing which is what this set of instructions
-outline but for consuming in a real .NET application you should use the nuget package instructions below.
-
-The 'dotnet publish' step above creates a directory that has all the files necessary to run your app
-including the CoreCLR and the parts of CoreFX that were needed. You can use this fact to skip some steps if
-you wish to update the DLLs. For example typically when you update CoreCLR you end up updating one of two DLLs
-
-* coreclr.dll - Most modifications (with the exception of the JIT compiler and tools) that are C++ code update
- this DLL.
-* System.Private.CoreLib.dll - If you modified C# it will end up here.
-
-Thus after making a change and building, you can simply copy the updated binary from the `artifacts\bin\coreclr\<OS>.<arch>.<flavor>`
-directory to your publication directory (e.g. `helloWorld\bin\Debug\netcoreapp3.0\win-x64\publish`) to quickly
-deploy your new bits. In a lot of cases it is easiest to just copy everything from here to your publication directory.
-
-You can build just the .NET Library part of the build by doing (debug, for release add 'release' qualifier)
-(on Linux / OSX us ./build.sh)
-```cmd
- .\build skiptests skipnative
-```
-Which builds System.Private.CoreLib.dll if you modify C# code. If you wish to only compile the coreclr.dll you can do
- ```cmd
- .\build skiptests skipmscorlib
-```
-Note that this technique does not work on .NET Apps that have not been published (that is you have not created
-a directory with all DLLs needed to run the all) That is because the runtime is either fetched from the system-wide
-location that dotnet.exe installed, OR it is fetched from the local nuget package cache (which is where your
-build was put when you did a 'dotnet restore' and had a dependency on your particular runtime). In theory you
-could update these locations in place, but that is not recommended since they are shared more widely.
-
-## (Optional) Confirm that the app used your new runtime
-
-Congratulations, you have successfully used your newly built runtime.
-
-As a hint you could add some code like:
-```
-var coreAssemblyInfo = System.Diagnostics.FileVersionInfo.GetVersionInfo(typeof(object).Assembly.Location);
-Console.WriteLine($"Hello World from Core {coreAssemblyInfo.ProductVersion}");
-Console.WriteLine($"The location is {typeof(object).Assembly.Location}");
-```
-
-That should tell you the version and which user and machine build the assembly as well as the commit hash of the code
-at the time of building:
-
-```
-Hello World from Core 4.6.26210.0 @BuiltBy: adsitnik-MININT-O513E3V @SrcCode: https://github.com/dotnet/runtime/tree/3d6da797d1f7dc47d5934189787a4e8006ab3a04
-The location is C:\coreclr\helloWorld\bin\Debug\netcoreapp3.0\win-x64\publish\System.Private.CoreLib.dll
-```
-
-### If it's not using your copied binary
-
-Make sure you are running the exe directly. If you use `dotnet run` it will overwrite your custom binaries before executing the app:
-
-```
-.\bin\Debug\netcoreapp3.0\win-x64\publish\HelloWorld.exe
-```
-
-### If you get a consistency check assertion failure
-
-Something like this happens if you copied coreclr.dll but not System.Private.Corelib.dll as well.
-
-```
-Assert failure(PID 13452 [0x0000348c], Thread: 10784 [0x2a20]): Consistency check failed: AV in clr at this callstack:
-```
-
-## Using .NET SDK to run your .NET Application
-
-If you don't like the idea of copying files manually you can follow [these instructions](../using-dotnet-cli.md) to use dotnet cli to do this for you.
-However the steps described here are the simplest and most commonly used by runtime developers for ad-hoc testing.
-
-## Using CoreRun to run your .NET Application
-
-Generally using dotnet.exe tool to run your .NET application is the preferred mechanism to run .NET Apps.
-However there is a simpler 'host' for .NET applications called 'CoreRun' that can also be used. The value
-of this host is that it is simpler (in particular it knows nothing about NuGet), but precisely because of this
-it can be harder to use (since you are responsible for insuring all the dependencies you need are gather together)
-See [Using CoreRun To Run .NET Application](using-corerun.md) for more.
+++ /dev/null
-
-# Using your .NET Runtime build with .NET SDK
-
-This walkthrough explains how to run your own app against your local build using only the .NET SDK.
-
-Testing your local build this way is quite realistic - more like a real user. However it takes longer because you have to build the package. Each build can take 10 minutes altogether.
-
-If you want to use a faster method, you may want to use one of these walkthroughs instead:
-
-- [Using Your Build - Update from raw build output](./testing/using-your-build.md)
-- [Using CoreRun To Run .NET Application](./testing/using-corerun.md)
-- [Dogfooding .NET SDK](https://github.com/dotnet/runtime/blob/main/docs/project/dogfooding.md).
-
-## Prerequisites
-
-All paths in examples below are Windows-style but the procedure is otherwise exactly the same on Unix.
-
-1. Successfully built this repository including the shared framework package and thus have files of the form shown below. From now on we call this folder your NuGet package folder.
-
-```
- <your-repo-root>\artifacts\packages\<configuration>\Shipping\
-```
-
-If you don't have this folder, you may have built binaries but not packages. Try building from the root with a command like `build.cmd clr+libs+host+packs -c release`.
-
-2. Acquired the latest development .NET SDK from [here](https://github.com/dotnet/installer) and added its root folder to your [path](requirements/windows-requirements.md#adding-to-the-default-path-variable)
-
-## First Run
-
-### 1. Create new folder for the app
-
-`mkdir helloWorld`
-
-From now on all instructions relate to this folder as "app folder".
-
-### 2. Create NuGet.Config file
-
-The build script creates NuGet packages and puts them to `artifacts\packages\<configuration>\Shipping\`. .NET SDK has no idea about its existence and we need to tell it where to search for the packages.
-
-Please run `dotnet new nugetconfig` in the app folder and replace the created `NuGet.Config` file content with:
-
-```xml
-<configuration>
- <config>
- <!-- CHANGE THIS PATH BELOW to any empty folder. NuGet will cache things here, and that's convenient because you can delete it to reset things -->
- <add key="globalPackagesFolder" value="c:\localcache" />
- </config>
- <packageSources>
- <!--To inherit the global NuGet package sources remove the <clear/> line below -->
- <clear />
- <!-- This feed is for any packages you didn't build. See https://github.com/dotnet/installer#installers-and-binaries -->
- <add key="dotnet6" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet6/nuget/v3/index.json" />
- <!-- CHANGE THIS PATH BELOW to your local output path -->
- <add key="local runtime" value="C:\runtime\artifacts\packages\Release\Shipping\" />
- </packageSources>
-</configuration>
-```
-
-### 3. Create and update the Project file
-
-Please run `dotnet new console` in the app folder and update the created `.csproj` file:
-
-```xml
-<Project Sdk="Microsoft.NET.Sdk">
-
- <PropertyGroup>
- <OutputType>Exe</OutputType>
- <TargetFramework>net6.0</TargetFramework>
- <RuntimeIdentifier>win-x64</RuntimeIdentifier>
- </PropertyGroup>
-
- <ItemGroup>
- <FrameworkReference Update="Microsoft.NETCore.App" RuntimeFrameworkVersion="6.0.0-dev" />
- </ItemGroup>
-
-</Project>
-```
-
-**You have to set the correct values for `RuntimeIdentifier` (RI) and `RuntimeFrameworkVersion`.**
-
-You can generally figure that out by looking at the packages you found in your output.
-In our example you will see there is a package with the name `Microsoft.NETCore.App.Runtime.win-x64.6.0.0-dev.nupkg`
-
-```
-Microsoft.NETCore.App.Runtime.win-x64.6.0.0-dev.nupkg
- ^-RI--^ ^version^
-```
-
-### 4. Change Program.cs
-
-To make sure that you run against your local build of this repo please change your `Main` method in `Program.cs` file to:
-
-```cs
-static void Main(string[] args)
-{
- var coreAssemblyInfo = System.Diagnostics.FileVersionInfo.GetVersionInfo(typeof(object).Assembly.Location);
- Console.WriteLine($"Hello World from .NET {coreAssemblyInfo.ProductVersion}");
- Console.WriteLine($"The location is {typeof(object).Assembly.Location}");
-}
-```
-
-### 5. Publish
-
-Now is the time to publish. The publish step will trigger restore and build. You can iterate on build by calling `dotnet build` as
-needed.
-
-```cmd
-dotnet publish
-```
-
-Make sure that restoring done by `dotnet publish` installed the explicit version of the Runtime that you have specified:
-
-```
-c:\runtime\helloworld>dotnet publish
-Microsoft (R) Build Engine version 16.7.0-preview-20360-03+188921e2f for .NET
-Copyright (C) Microsoft Corporation. All rights reserved.
-
- Determining projects to restore...
- Restored c:\runtime\helloworld\helloworld.csproj (in 114 ms).
- You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
- helloworld -> c:\runtime\helloworld\bin\Debug\net6.0\win-x64\helloworld.dll
- helloworld -> c:\runtime\helloworld\bin\Debug\net6.0\win-x64\publish\
-```
-
-#### Troubleshooting Publish
-
-If you see something like the message below it means that it has failed to restore your local runtime packages. Check your `NuGet.config` file and paths used in it.
-
-```
-c:\runtime\helloworld>dotnet publish
-Microsoft (R) Build Engine version 16.7.0-preview-20360-03+188921e2f for .NET
-Copyright (C) Microsoft Corporation. All rights reserved.
-
- Determining projects to restore...
-c:\runtime\helloworld\helloworld.csproj : error NU1102: Unable to find package Microsoft.NETCore.App.Runtime.win-x64 with version (= 6.0.0-does-not-exist)
-c:\runtime\helloworld\helloworld.csproj : error NU1102: - Found 25 version(s) in nuget [ Nearest version: 6.0.0-preview.1.20120.5 ]
-c:\runtime\helloworld\helloworld.csproj : error NU1102: - Found 1 version(s) in local runtime [ Nearest version: 6.0.0-dev ]
-c:\runtime\helloworld\helloworld.csproj : error NU1102: Unable to find package Microsoft.NETCore.App.Host.win-x64 with version (= 6.0.0-does-not-exist)
-c:\runtime\helloworld\helloworld.csproj : error NU1102: - Found 27 version(s) in nuget [ Nearest version: 6.0.0-preview.1.20120.5 ]
-c:\runtime\helloworld\helloworld.csproj : error NU1102: - Found 1 version(s) in local runtime [ Nearest version: 6.0.0-dev ]
- Failed to restore c:\runtime\helloworld\helloworld.csproj (in 519 ms).
-```
-
-If you see error messages like these below, it means it has failed to restore other packages you need that you didn't build. Check your `NuGet.config` file includes the other feed.
-```
-c:\runtime\helloworld\helloworld.csproj : error NU1101: Unable to find package Microsoft.WindowsDesktop.App.Runtime.win-x64. No packages exist with this id in source(s): local runtime
-c:\runtime\helloworld\helloworld.csproj : error NU1101: Unable to find package Microsoft.AspNetCore.App.Runtime.win-x64. No packages exist with this id in source(s): local runtime
-```
-
-If you see error messages like this, it means you do not have a new enough SDK. Please visit https://github.com/dotnet/installer#installers-and-binaries and install a newer SDK.
-
-```
-C:\Program Files\dotnet\sdk\5.0.100\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.TargetFrameworkInference.targets(141,5): error NETSDK1045: The current .NET SDK does not support targeting .NET Core 6.0. Either target .NET Core 5.0 or lower, or use a version of the .NET SDK that supports .NET Core 6.0. [c:\runtime\helloworld\helloworld.csproj]
-```
-
-### 6. Run the app
-
-After you publish you will find all the binaries needed to run your application under `bin\Debug\net6.0\win-x64\publish\`.
-To run the application simply run the EXE that is in this publish directory (it is the name of the app, or specified in the project file).
-
-```
-.\bin\Debug\net6.0\win-x64\publish\HelloWorld.exe
-```
-
-Running the app should tell you the version and where the location of System.Private.CoreLib in the publish directory:
-
-```
-Hello World from .NET 6.0.0-dev
-The location is c:\runtime\helloworld\bin\Debug\net6.0\win-x64\publish\System.Private.CoreLib.dll
-```
-
-**Congratulations! You have just run your first app against your local build of this repo**
-
-## How to then consume updated packages
-
-Once you have successfully consumed a package, you probably want to make changes, update the package, and have your app consume it again. Normally NuGet would ignore your updated package, because its version number hasn't changed. The easiest way to avoid that is to simply delete your NuGet cache. To make this easy, in the `NuGet.config` file above, we used `globalPackagesFolder` to set a local package cache. Simply delete that folder and publish again and your app will pick up the new package.
-
-So the steps are:
-
-### 1. Build the package again
-
-```cmd
-build.cmd clr+libs+host+packs -c release
-```
-
-If you only changed libraries, `build.cmd libs+host+packs -c release` is a little faster; if you only changed clr, then `build.cmd clr+host+packs -c release`
-
-### 2. Delete your local package cache
-
-```cmd
-rd /s /q c:\localcache
-```
-
-### 3. Publish again
-
-```cmd
-dotnet publish
-```
-
-Now your app will use your updated package.
projects / platform / upstream / dotnet / runtime.git / commitdiff