Allow the loader to enable ASAN outside of testing
authorCharles Giessen <charles@lunarg.com>
Tue, 7 Mar 2023 22:14:43 +0000 (15:14 -0700)
committerCharles Giessen <46324611+charles-lunarg@users.noreply.github.com>
Tue, 7 Mar 2023 23:39:27 +0000 (16:39 -0700)
This commit promotes the build option TEST_USE_ADDRESS_SANITIZER and
TEST_USE_THREAD_SANITIZER to not be behind BUILD_TESTS. This allows users
of the loader to enable ASAN & TSAN support on the loader without enabling
the loader's tests.

The new options are called LOADER_ENABLE_ADDRESS_SANITIZER and
LOADER_ENABLE_THREAD_SANITIZER.

There is also a new build option: `LOADER_DISABLE_DYNAMIC_LIBRARY_UNLOADING`.
This is to allow layers & drivers that use Address Sanitizer to get useful
stack traces when running their code. Because vkDestroyInstance unloads all
layer and driver dynamic libraries, leak sanitizer cannot create complete
stack traces, making debugging the leaks much more difficult than necessary.
This option is only available when LOADER_ENABLE_ADDRESS_SANITIZER is also
enabled, due to that being the only reason to prevent the unloading of
libraries.

.github/workflows/build.yml
BUILD.md
CMakeLists.txt
loader/CMakeLists.txt
loader/vk_loader_platform.h
tests/CMakeLists.txt
tests/README.md
tests/framework/CMakeLists.txt
tests/framework/README.md
tests/loader_regression_tests.cpp

index 47a14b85db7a0b8fdb9742577c0b488edcbc5647..e66f13384270d3244cffe85d734ef1d3972f91e8 100644 (file)
@@ -52,7 +52,7 @@ jobs:
               run: sudo apt install --yes --no-install-recommends libwayland-dev libxrandr-dev
 
             - name: Generate build files
-              run: cmake -S. -B build -D CMAKE_BUILD_TYPE=${{matrix.config}} -D BUILD_TESTS=ON -D UPDATE_DEPS=ON -D TEST_USE_ADDRESS_SANITIZER=ON
+              run: cmake -S. -B build -D CMAKE_BUILD_TYPE=${{matrix.config}} -D BUILD_TESTS=ON -D UPDATE_DEPS=ON -D LOADER_ENABLE_ADDRESS_SANITIZER=ON
               env:
                 CC: ${{matrix.cc}}
                 CXX: ${{matrix.cxx}}
@@ -166,7 +166,7 @@ jobs:
                 python-version: '3.7'
             - uses: lukka/get-cmake@latest
             - name: Generate build files
-              run: cmake -S. -B build -DCMAKE_BUILD_TYPE=${{matrix.config}} -D${{matrix.static_build}} -DBUILD_TESTS=ON -DUPDATE_DEPS=ON -DTEST_USE_ADDRESS_SANITIZER=ON -G "Ninja"
+              run: cmake -S. -B build -D CMAKE_BUILD_TYPE=${{matrix.config}} -D ${{matrix.static_build}} -D BUILD_TESTS=ON -D UPDATE_DEPS=ON -D LOADER_ENABLE_ADDRESS_SANITIZER=ON -G "Ninja"
               env:
                 # Specify the minimum version of macOS on which the target binaries are to be deployed.
                 # https://cmake.org/cmake/help/latest/envvar/MACOSX_DEPLOYMENT_TARGET.html
index a9260ddf3878b25b0eedd489983da5b2b0e08eb6..b1fa677e7176223ea412e4f28d98054a0c650dfb 100644 (file)
--- a/BUILD.md
+++ b/BUILD.md
@@ -23,6 +23,7 @@ Instructions for building this repository on Linux, Windows, and MacOS.
         - [Notes About the Automatic Option](#notes-about-the-automatic-option)
     - [Generated source code](#generated-source-code)
     - [Build Options](#build-options)
+    - [CCACHE](#ccache)
   - [Building On Windows](#building-on-windows)
     - [Windows Development Environment Requirements](#windows-development-environment-requirements)
     - [Windows Build - Microsoft Visual Studio](#windows-build---microsoft-visual-studio)
@@ -49,7 +50,6 @@ Instructions for building this repository on Linux, Windows, and MacOS.
     - [MacOS Development Environment Requirements](#macos-development-environment-requirements)
     - [Clone the Repository](#clone-the-repository)
     - [MacOS build](#macos-build)
-      - [CMake Generators](#cmake-generators)
       - [Building with the Unix Makefiles Generator](#building-with-the-unix-makefiles-generator)
       - [Building with the Xcode Generator](#building-with-the-xcode-generator)
     - [Using the new macOS loader](#using-the-new-macos-loader)
@@ -238,18 +238,22 @@ be specified to customize the build. Some of the options are binary on/off
 options, while others take a string as input. The following is a table of all
 on/off options currently supported by this repository:
 
-| Option                       | Platform | Default | Description                                                                                                                                                                       |
-| ---------------------------- | -------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| BUILD_TESTS                  | All      | `OFF`   | Controls whether or not the loader tests are built.                                                                                                                               |
-| BUILD_WSI_XCB_SUPPORT        | Linux    | `ON`    | Build the loader with the XCB entry points enabled. Without this, the XCB headers should not be needed, but the extension `VK_KHR_xcb_surface` won't be available.                |
-| BUILD_WSI_XLIB_SUPPORT       | Linux    | `ON`    | Build the loader with the Xlib entry points enabled. Without this, the X11 headers should not be needed, but the extension `VK_KHR_xlib_surface` won't be available.              |
-| BUILD_WSI_WAYLAND_SUPPORT    | Linux    | `ON`    | Build the loader with the Wayland entry points enabled. Without this, the Wayland headers should not be needed, but the extension `VK_KHR_wayland_surface` won't be available.    |
-| BUILD_WSI_DIRECTFB_SUPPORT   | Linux    | `OFF`   | Build the loader with the DirectFB entry points enabled. Without this, the DirectFB headers should not be needed, but the extension `VK_EXT_directfb_surface` won't be available. |
-| BUILD_WSI_SCREEN_QNX_SUPPORT | QNX      | `OFF`   | Build the loader with the QNX Screen entry points enabled. Without this the extension `VK_QNX_screen_surface` won't be available.                                                 |
-| ENABLE_WIN10_ONECORE         | Windows  | `OFF`   | Link the loader to the [OneCore](https://msdn.microsoft.com/en-us/library/windows/desktop/mt654039.aspx) umbrella library, instead of the standard Win32 ones.                    |
-| USE_GAS                      | Linux    | `ON`    | Controls whether to build assembly files with the GNU assembler, else fallback to C code.                                                                                         |
-| USE_MASM                     | Windows  | `ON`    | Controls whether to build assembly files with MS assembler, else fallback to C code                                                                                               |
-| BUILD_STATIC_LOADER          | macOS    | `OFF`   | This allows the loader to be built as a static library on macOS. Not tested, use at your own risk.                                                                                |
+| Option                                   | Platform      | Default | Description                                                                                                                                                                       |
+| ---------------------------------------- | ------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| BUILD_TESTS                              | All           | `OFF`   | Controls whether or not the loader tests are built.                                                                                                                               |
+| BUILD_WSI_XCB_SUPPORT                    | Linux         | `ON`    | Build the loader with the XCB entry points enabled. Without this, the XCB headers should not be needed, but the extension `VK_KHR_xcb_surface` won't be available.                |
+| BUILD_WSI_XLIB_SUPPORT                   | Linux         | `ON`    | Build the loader with the Xlib entry points enabled. Without this, the X11 headers should not be needed, but the extension `VK_KHR_xlib_surface` won't be available.              |
+| BUILD_WSI_WAYLAND_SUPPORT                | Linux         | `ON`    | Build the loader with the Wayland entry points enabled. Without this, the Wayland headers should not be needed, but the extension `VK_KHR_wayland_surface` won't be available.    |
+| BUILD_WSI_DIRECTFB_SUPPORT               | Linux         | `OFF`   | Build the loader with the DirectFB entry points enabled. Without this, the DirectFB headers should not be needed, but the extension `VK_EXT_directfb_surface` won't be available. |
+| BUILD_WSI_SCREEN_QNX_SUPPORT             | QNX           | `OFF`   | Build the loader with the QNX Screen entry points enabled. Without this the extension `VK_QNX_screen_surface` won't be available.                                                 |
+| ENABLE_WIN10_ONECORE                     | Windows       | `OFF`   | Link the loader to the [OneCore](https://msdn.microsoft.com/en-us/library/windows/desktop/mt654039.aspx) umbrella library, instead of the standard Win32 ones.                    |
+| USE_GAS                                  | Linux         | `ON`    | Controls whether to build assembly files with the GNU assembler, else fallback to C code.                                                                                         |
+| USE_MASM                                 | Windows       | `ON`    | Controls whether to build assembly files with MS assembler, else fallback to C code                                                                                               |
+| BUILD_STATIC_LOADER                      | macOS         | `OFF`   | This allows the loader to be built as a static library on macOS. Not tested, use at your own risk.                                                                                |
+| LOADER_ENABLE_ADDRESS_SANITIZER          | Linux & macOS | `OFF`   | Enables Address Sanitizer in the loader and tests.                                                                                                                                |
+| LOADER_ENABLE_THREAD_SANITIZER           | Linux & macOS | `OFF`   | Enables Thread Sanitizer in the loader and tests.                                                                                                                                 |
+| LOADER_DISABLE_DYNAMIC_LIBRARY_UNLOADING | Linux & macOS | `OFF`   | Causes the loader to not unload dynamic libraries. Only works when LOADER_ENABLE_ADDRESS_SANITIZER is enabled. This option allows leak sanitizer to have full stack traces.       |
+
 The following is a table of all string options currently supported by this repository:
 
 | Option                | Platform    | Default                       | Description                                                                                                                                          |
index 058f1f84c9a5da4d934fbca7f51b44b1f24793a2..2db64a95c228f8d92fee01b123196fd9a7970d0a 100644 (file)
@@ -25,6 +25,13 @@ find_package(PythonInterp 3 QUIET)
 set(THREADS_PREFER_PTHREAD_FLAG ON)
 find_package(Threads REQUIRED)
 
+# By default, loader & tests are built without sanitizers
+# Use these options to force a specific sanitizer on the loader and test executables
+if (UNIX)
+    option(LOADER_ENABLE_ADDRESS_SANITIZER "Linux & macOS only: Advanced memory checking" OFF)
+    option(LOADER_ENABLE_THREAD_SANITIZER "Linux & macOS only: Advanced thread checking" OFF)
+endif()
+
 if(APPLE)
     option(BUILD_STATIC_LOADER "Build a loader that can be statically linked" OFF)
 endif()
index babe02138e152cdcb71d2e3243bf306539293658..b386c839271afc529f04298075febbc836f58538 100644 (file)
@@ -321,6 +321,18 @@ else()
         target_link_libraries(vulkan PRIVATE Threads::Threads)
     endif()
 
+    if (LOADER_ENABLE_ADDRESS_SANITIZER)
+        target_compile_options(vulkan PUBLIC -fsanitize=address)
+        target_link_options(vulkan PUBLIC -fsanitize=address)
+        if (LOADER_DISABLE_DYNAMIC_LIBRARY_UNLOADING)
+            target_compile_definitions(vulkan PRIVATE LOADER_DISABLE_DYNAMIC_LIBRARY_UNLOADING=1)
+        endif()
+    endif()
+    if (LOADER_ENABLE_THREAD_SANITIZER)
+        target_compile_options(vulkan PUBLIC -fsanitize=thread)
+        target_link_options(vulkan PUBLIC -fsanitize=thread)
+    endif()
+
     if(APPLE)
         find_library(COREFOUNDATION_LIBRARY NAMES CoreFoundation)
         target_link_libraries(vulkan PRIVATE "-framework CoreFoundation")
index 44120f2c2b22476a06f8ca8d61e0d1b307d777b0..96fa3aea0b8a686ca3b5333a914f4a6e512e724c 100644 (file)
@@ -328,7 +328,15 @@ static inline char *loader_platform_executable_path(char *buffer, size_t size) {
 #endif
 
 #if __has_feature(address_sanitizer) || defined(__SANITIZE_ADDRESS__)
-#define LOADER_ADDRESS_SANITIZER  // TODO: Add proper build flag for ASAN support
+#define LOADER_ADDRESS_SANITIZER_ACTIVE  // TODO: Add proper build flag for ASAN support
+#endif
+
+// We should always unload dynamic libraries in loader_platform_close_library() unless both Address Sanitizer is active and
+// LOADER_DISABLE_DYNAMIC_LIBRARY_UNLOADING is set
+#if defined(LOADER_ADDRESS_SANITIZER_ACTIVE) && defined(LOADER_DISABLE_DYNAMIC_LIBRARY_UNLOADING)
+#define SHOULD_UNLOAD_DYNAMIC_LIBRARIES 0
+#else
+#define SHOULD_UNLOAD_DYNAMIC_LIBRARIES 1
 #endif
 
 // When loading the library, we use RTLD_LAZY so that not all symbols have to be
@@ -357,7 +365,12 @@ static inline const char *loader_platform_open_library_error(const char *libPath
     return dlerror();
 #endif
 }
-static inline void loader_platform_close_library(loader_platform_dl_handle library) { dlclose(library); }
+static inline void loader_platform_close_library(loader_platform_dl_handle library) {
+// We only want to remove dlclose if both Asan is active and LOADER_DISABLE_DYNAMIC_LIBRARY_UNLOADING is set
+#if SHOULD_UNLOAD_DYNAMIC_LIBRARIES == 1
+    dlclose(library);
+#endif
+}
 static inline void *loader_platform_get_proc_address(loader_platform_dl_handle library, const char *name) {
     assert(library);
     assert(name);
index f9677bcd36d2fc7012dacc2bc35af91ec4898d32..9f899f07a1c464c6dbfbaf2e2a400efb9ddbfff5 100644 (file)
 # ~~~
 
 
-# By default, tests and loader are built without sanitizers
-# Use these options to force a specific sanitizer on all test executables
-option(TEST_USE_ADDRESS_SANITIZER "Linux only: Advanced memory checking" OFF)
-option(TEST_USE_THREAD_SANITIZER "Linux only: Advanced thread checking" OFF)
 option(ENABLE_LIVE_VERIFICATION_TESTS "Enable tests which expect to run on live drivers. Meant for manual verification only" OFF)
 
 include(GoogleTest)
index a12fde3ec47dc9bc8c0eee892f00d8cfd374ba7a..7a09938f934ad796e225ab36e09834655154b799 100644 (file)
@@ -11,8 +11,6 @@ These tests are not exhaustive &mdash; they are expected to be supplemented with
 | ------------------------------ | -------- | ------- | -------------------------------------------------------- |
 | BUILD_TESTS                    | All      | `OFF`   | Controls whether or not the loader tests are built.      |
 | ENABLE_LIVE_VERIFICATION_TESTS | All      | `OFF`   | Enables building of tests meant to run with live drivers |
-| TEST_USE_ADDRESS_SANITIZER     | Linux    | `OFF`   | Enables Address Sanitizer in the loader and tests        |
-| TEST_USE_THREAD_SANITIZER      | Linux    | `OFF`   | Enables Thread Sanitizer in the loader and tests         |
 
 ## Running Tests
 
@@ -51,4 +49,4 @@ Then run the test again but with this env-var set to the older loader without th
 Basic usage example:
 ```c
 VK_LOADER_TEST_LOADER_PATH="/path/to/older/loader/build" ctest --output-on-failure
-```
\ No newline at end of file
+```
index c04ebae89b9ff11a86387afa5baa24b98bf93d76..af604054ef2d4018d43ee9ac02e9969c98a36e7c 100644 (file)
@@ -30,17 +30,13 @@ endif()
 target_include_directories(testing_framework_util PUBLIC ${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_SOURCE_DIR})
 
 if (UNIX)
-    if (TEST_USE_ADDRESS_SANITIZER)
+    if (LOADER_ENABLE_ADDRESS_SANITIZER)
         target_compile_options(testing_framework_util PUBLIC -fsanitize=address)
         target_link_options(testing_framework_util PUBLIC -fsanitize=address)
-        target_compile_options(vulkan PUBLIC -fsanitize=address)
-        target_link_options(vulkan PUBLIC -fsanitize=address)
     endif()
-    if (TEST_USE_THREAD_SANITIZER)
+    if (LOADER_ENABLE_THREAD_SANITIZER)
         target_compile_options(testing_framework_util PUBLIC -fsanitize=thread)
         target_link_options(testing_framework_util PUBLIC -fsanitize=thread)
-        target_compile_options(vulkan PUBLIC -fsanitize=thread)
-        target_link_options(vulkan PUBLIC -fsanitize=thread)
         target_compile_options(gtest PUBLIC -fsanitize=thread)
         target_link_options(gtest PUBLIC -fsanitize=thread)
     endif()
@@ -101,4 +97,4 @@ set_target_properties(testing_dependencies ${LOADER_STANDARD_CXX_PROPERTIES})
 if (APPLE AND BUILD_STATIC_LOADER)
     target_compile_definitions(testing_dependencies PUBLIC "BUILD_STATIC_LOADER=1")
     target_link_libraries(testing_dependencies PUBLIC vulkan)
-endif()
\ No newline at end of file
+endif()
index 3148e81fcfe90f9f80381b3c84a39697645bdb4b..7ae04c22e714295f4bc99f3ba2a3dc19b1d35e02 100644 (file)
@@ -22,8 +22,8 @@ Use the CMake configuration `UPDATE_DEPS=ON` to automatically get all required t
 Or Ensure that `googletest` is in the `external` directory.
 And on Windows only, ensure that the `Detours` library is in the `external` directory.
 
-Linux only: The CMake Configuration `TEST_USE_ADDRESS_SANITIZER` can be used to
-enable Address Sanitizer inside the testing framework.
+Linux & macOS only: The CMake Configuration `LOADER_ENABLE_ADDRESS_SANITIZER` can be used to
+enable Address Sanitizer.
 
 Run the test executables as normal
 
index 0f1d436f273eaa62e023c0451b9a0ce2e74ae273..e2b30176bd90ebc531bd98dcf9b8de848bae3b53 100644 (file)
@@ -1025,11 +1025,14 @@ TEST(CreateDevice, UnmatchInstanceAndDeviceLayers) {
 
     dev.CheckCreate(phys_dev);
 
-    ASSERT_TRUE(log.find("loader_create_device_chain: Using deprecated and ignored 'ppEnabledLayerNames' member of 'VkDeviceCreateInfo' when creating a Vulkan device."));
+    ASSERT_TRUE(
+        log.find("loader_create_device_chain: Using deprecated and ignored 'ppEnabledLayerNames' member of 'VkDeviceCreateInfo' "
+                 "when creating a Vulkan device."));
 }
 
 // Device layers are deprecated.
-// Ensure that when VkInstanceCreateInfo is deleted, the check of the instance layer lists is running correctly during VkDevice creation
+// Ensure that when VkInstanceCreateInfo is deleted, the check of the instance layer lists is running correctly during VkDevice
+// creation
 // https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/vkspec.html#extendingvulkan-layers-devicelayerdeprecation
 TEST(CreateDevice, CheckCopyOfInstanceLayerNames) {
     FrameworkEnvironment env{};
@@ -1044,7 +1047,8 @@ TEST(CreateDevice, CheckCopyOfInstanceLayerNames) {
 
     InstWrapper inst{env.vulkan_functions};
     {
-        // We intentionally create a local InstanceCreateInfo that goes out of scope at the } so that when dev.CheckCreate is called the layer name pointers are no longer valid
+        // We intentionally create a local InstanceCreateInfo that goes out of scope at the } so that when dev.CheckCreate is called
+        // the layer name pointers are no longer valid
         InstanceCreateInfo create_info{};
         create_info.add_layer(layer_name);
         inst.CheckCreateWithInfo(create_info);
@@ -1951,7 +1955,7 @@ TEST(EnumeratePhysicalDeviceGroups, CallThriceAddGroupInBetween) {
         for (uint32_t group2 = 0; group2 < group_props_after.size(); ++group2) {
             if (group_props_before[group1].physicalDeviceCount == group_props_after[group2].physicalDeviceCount) {
                 uint32_t found_count = 0;
-                bool found;
+                bool found = false;
                 for (uint32_t dev1 = 0; dev1 < group_props_before[group1].physicalDeviceCount; ++dev1) {
                     found = false;
                     for (uint32_t dev2 = 0; dev2 < group_props_after[group2].physicalDeviceCount; ++dev2) {
@@ -2036,7 +2040,7 @@ TEST(EnumeratePhysicalDeviceGroups, CallTwiceRemoveGroupInBetween) {
         for (uint32_t group2 = 0; group2 < group_props_before.size(); ++group2) {
             if (group_props_after[group1].physicalDeviceCount == group_props_before[group2].physicalDeviceCount) {
                 uint32_t found_count = 0;
-                bool found;
+                bool found = false;
                 for (uint32_t dev1 = 0; dev1 < group_props_after[group1].physicalDeviceCount; ++dev1) {
                     found = false;
                     for (uint32_t dev2 = 0; dev2 < group_props_before[group2].physicalDeviceCount; ++dev2) {
@@ -2118,7 +2122,7 @@ TEST(EnumeratePhysicalDeviceGroups, CallTwiceAddDeviceInBetween) {
     for (uint32_t group1 = 0; group1 < group_props_before.size(); ++group1) {
         for (uint32_t group2 = 0; group2 < group_props_after.size(); ++group2) {
             uint32_t found_count = 0;
-            bool found;
+            bool found = false;
             for (uint32_t dev1 = 0; dev1 < group_props_before[group1].physicalDeviceCount; ++dev1) {
                 found = false;
                 for (uint32_t dev2 = 0; dev2 < group_props_after[group2].physicalDeviceCount; ++dev2) {
@@ -2213,7 +2217,7 @@ TEST(EnumeratePhysicalDeviceGroups, CallTwiceRemoveDeviceInBetween) {
     for (uint32_t group1 = 0; group1 < group_props_after.size(); ++group1) {
         for (uint32_t group2 = 0; group2 < group_props_before.size(); ++group2) {
             uint32_t found_count = 0;
-            bool found;
+            bool found = false;
             for (uint32_t dev1 = 0; dev1 < group_props_after[group1].physicalDeviceCount; ++dev1) {
                 found = false;
                 for (uint32_t dev2 = 0; dev2 < group_props_before[group2].physicalDeviceCount; ++dev2) {