Address Sanitizer support in CoreCLR (#74623)
authorJeremy Koritzinsky <jekoritz@microsoft.com>
Tue, 18 Jul 2023 03:32:03 +0000 (20:32 -0700)
committerGitHub <noreply@github.com>
Tue, 18 Jul 2023 03:32:03 +0000 (20:32 -0700)
commit2811e7c7551c5ec56f1afbfd1749e1f42d264dbf
tree701b30e8e794008d0791ef22e437c7d289b0182a
parenta654a772409a8d20b2557a558a019f7e08528239
Address Sanitizer support in CoreCLR (#74623)

* Enable AddressSanitizer in CoreCLR, Libs, and Host and update runtime-sanitized to run tests with ASAN on Linux and Mac

* Unify on HAS_ADDRESS_SANITIZER define name

Centralize setting up sanitizer flags.

Remove last usage of CLR_CMAKE_ENABLE_ASAN outside of eng/native

Remove unnecessary diffs

Use the cpuid intrinsic and enable ASAN on InitJitHelpers1.

Add comments for the places where we have ASAN disabled

Undo changes to src/coreclr/jit/CMakeLists.txt

Add docs and fix the docs in the build script.

Add docs for the SkipVCEnvInit hook

* Remove CRT runtime changes. We won't need this when we finally onboard to Windows with some changes they have coming down the pipeline, so remove them for now to reduce the diff of this PR.

* Remove suppression now that we've fixed the underlying issue.

* Remove some Windows-only sanitizer CMake that we won't need when ASAN is ready for us to consume on Windows.

* Set schedule for runtime-sanitized pipeline

* Remove workaround in JIT memory set/copy helpers now that the JIT bug has been fixed.

* Add missing helix queues setup template.

* Fix missing command to build nativeaot runtime tests as nativeaot.

* Add a scenario name to trigger the extended timeout in the libraries test helix configuration.

* Remove extraneous whitespace.

* Fix using the cross-targetting ILC when sanitizers are enabled. Also pass through our TargetOS and TargetArchitecture variables to the publish command for our native sanitizers targets to correctly do their extra logic.

* Disable LSAN on the CustomMain test.

* Disable some tests on sanitized runtimes.

* Copy the sanitizer runtime for OSX NativeAOT runtime tests.

* A little cleanup to try to get the build right now that we are always doing cross-builds on Linux.

* Fix crossgen-corelib.proj syntax.

* Fix cross-os dac builds to not include the host architecture in the output path.

* Split the debugger components into a separate component and make the "unsanitized cross components" build into an "unsanitized debugger components" build as it can't be part of the regular cross-components build

* Always write out the host-arch path on Windows and update the cross-dac build script to expect that.

* Change to use the dynamic runtime on Windows, as ASAN in VS is moving to a dynamic-only model.

* Update docker images to include the sanitizer runtimes in the crossrootfs images.

* Fix explicit image tags

* The unsanitized build should be of the target architecture, not the host architecture. As a result, we still need a cross-arch build for the cross-arch use cases when sanitized, as well as an unsanitized target arch build for the debugger tools.

* Turn off using the sigaltstack for NativeAOT tests that don't use the asansupport.cpp default options

* Make sure the shared ASAN runtime is present for the nativeaot/SmokeTests/SharedLibrary test.

* Disable crossgen2 tests with sanitizers as they don't get us interesting coverage.

* Fix custom default options and disable some more crossgen2-based tests.

* Simplify lookup of asan runtime on mac and fix copying the shared runtime for the SharedLibrary NativeAOT test.

* Disable test that's failing for weird reasons.

* Fix one more alloc-dealloc mismatch that only started to show up after test merging increased allocations in the runtime

* Disable the System.Text.Json test suite on sanitized builds as it causes SO failures on Mac

* Fix test exclusion

* PR feedback.

* Fix mac build

* Do review changes

* Add libbootstrapper object files to the platform manifest now that NativeAOT has

* Disable tests that check size on sanitized builds

* Use the built-in `include_guard` option

* Disable use-after-return checking in ASAN. CoreCLR doesn't do well with parallel stacks.

* Only pass the no UAR flag on C and CXX with Clang (not AppleClang).
108 files changed:
docs/workflow/building/coreclr/linux-instructions.md
docs/workflow/building/coreclr/macos-instructions.md
docs/workflow/building/coreclr/nativeaot.md
docs/workflow/building/coreclr/windows-instructions.md
docs/workflow/building/libraries/README.md
eng/Subsets.props
eng/build.ps1
eng/build.sh
eng/liveBuilds.targets
eng/native/build-commons.sh
eng/native/configurecompiler.cmake
eng/native/functions.cmake
eng/native/ijw/IJW.cmake
eng/native/init-vs-env.cmd
eng/nativeSanitizers.targets [new file with mode: 0644]
eng/pipelines/common/global-build-job.yml
eng/pipelines/common/templates/runtimes/build-runtime-tests-and-send-to-helix.yml
eng/pipelines/common/templates/runtimes/build-runtime-tests.yml
eng/pipelines/common/templates/runtimes/test-variables.yml
eng/pipelines/coreclr/nativeaot-post-build-steps.yml
eng/pipelines/coreclr/templates/crossdac-build.yml
eng/pipelines/runtime-sanitized.yml
eng/pipelines/runtime.yml
eng/testing/linker/project.csproj.template
eng/testing/linker/trimmingTests.targets
eng/testing/tests.singlefile.targets
src/coreclr/build-runtime.cmd
src/coreclr/build-runtime.sh
src/coreclr/components.cmake
src/coreclr/crossgen-corelib.proj
src/coreclr/dlls/mscordac/CMakeLists.txt
src/coreclr/dlls/mscordbi/CMakeLists.txt
src/coreclr/enablesanitizers.sh
src/coreclr/hosts/corerun/CMakeLists.txt
src/coreclr/nativeaot/Bootstrap/CMakeLists.txt
src/coreclr/nativeaot/Bootstrap/base/CMakeLists.txt
src/coreclr/nativeaot/Bootstrap/dll/CMakeLists.txt
src/coreclr/nativeaot/Bootstrap/main.cpp
src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets
src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Windows.targets
src/coreclr/nativeaot/Runtime/windows/PalRedhawkMinWin.cpp
src/coreclr/pal/src/arch/amd64/context2.S
src/coreclr/pal/src/arch/amd64/exceptionhelper.S
src/coreclr/pal/src/arch/arm/context2.S
src/coreclr/pal/src/arch/arm/exceptionhelper.S
src/coreclr/pal/src/arch/arm64/context2.S
src/coreclr/pal/src/arch/arm64/exceptionhelper.S
src/coreclr/pal/src/arch/i386/context2.S
src/coreclr/pal/src/arch/i386/exceptionhelper.S
src/coreclr/pal/src/arch/loongarch64/context2.S
src/coreclr/pal/src/arch/loongarch64/exceptionhelper.S
src/coreclr/pal/src/exception/machexception.cpp
src/coreclr/pal/src/thread/thread.cpp
src/coreclr/runtime.proj
src/coreclr/tools/aot/crossgen2/crossgen2.csproj
src/coreclr/utilcode/clrhost_nodependencies.cpp
src/coreclr/vm/frames.h
src/coreclr/vm/i386/asmhelpers.asm
src/coreclr/vm/i386/jitinterfacex86.cpp
src/coreclr/vm/i386/stublinkerx86.cpp
src/coreclr/vm/jithelpers.cpp
src/coreclr/vm/threads.cpp
src/coreclr/vm/threadsuspend.cpp
src/coreclr/vm/wks/CMakeLists.txt
src/installer/pkg/sfx/Microsoft.NETCore.App/Directory.Build.props
src/libraries/Directory.Build.targets
src/libraries/externals.csproj
src/libraries/tests.proj
src/mono/CMakeLists.txt
src/native/corehost/CMakeLists.txt
src/native/corehost/apphost/static/CMakeLists.txt
src/native/corehost/build.cmd
src/native/corehost/corehost.proj
src/native/corehost/exe.cmake
src/native/corehost/test/testexe.cmake
src/native/libs/CMakeLists.txt
src/native/libs/build-native.cmd
src/native/libs/build-native.proj
src/native/libs/verify-so.sh
src/native/minipal/asansupport.cpp [new file with mode: 0644]
src/native/minipal/utils.h
src/tests/Common/Directory.Build.targets
src/tests/Common/helixpublishwitharcade.proj
src/tests/Common/publishdependency.targets
src/tests/Directory.Build.props
src/tests/Directory.Build.targets
src/tests/Interop/COM/NativeClients/DefaultInterfaces/CMakeLists.txt
src/tests/Interop/COM/NativeClients/Dispatch/CMakeLists.txt
src/tests/Interop/COM/NativeClients/Events/CMakeLists.txt
src/tests/Interop/COM/NativeClients/Licensing/CMakeLists.txt
src/tests/Interop/COM/NativeClients/Primitives/CMakeLists.txt
src/tests/build.cmd
src/tests/build.sh
src/tests/nativeaot/CustomMain/CustomMainNative.cpp
src/tests/nativeaot/GenerateUnmanagedEntryPoints/GenerateUnmanagedEntryPoints.csproj
src/tests/nativeaot/SmokeTests/DwarfDump/DwarfDump.csproj
src/tests/nativeaot/SmokeTests/HardwareIntrinsics/X64Avx.csproj
src/tests/nativeaot/SmokeTests/HardwareIntrinsics/X64Avx2.csproj
src/tests/nativeaot/SmokeTests/HardwareIntrinsics/X64Avx512.csproj
src/tests/nativeaot/SmokeTests/HardwareIntrinsics/X64Avx_NoAvx2.csproj
src/tests/nativeaot/SmokeTests/HardwareIntrinsics/X64Baseline.csproj
src/tests/nativeaot/SmokeTests/HardwareIntrinsics/X64Sse42.csproj
src/tests/nativeaot/SmokeTests/SharedLibrary/CMakeLists.txt
src/tests/nativeaot/SmokeTests/SharedLibrary/SharedLibrary.cpp
src/tests/nativeaot/SmokeTests/SharedLibrary/SharedLibrary.csproj
src/tests/readytorun/coreroot_determinism/coreroot_determinism.csproj
src/tests/readytorun/determinism/crossgen2determinism.csproj
src/tests/readytorun/multifolder/multifolder.csproj