Update build.md and README.md for the tests

diff --git a/BUILD.md b/BUILD.md
index b3dc2ebcbe8e32b5b73c366509aff95adbf43475..45972f4dbc448c39bb2f634057a8a813bca557f1 100644 (file)
--- a/BUILD.md
+++ b/BUILD.md
@@ -211,21 +211,20 @@ be specified to customize the build. Some of the options are binary on/off
 options, while others take a string as input. The following is a table of all
 on/off options currently supported by this repository:
 
-| Option                       | Platform | Default | Description                                                                                                                                                                        |
-| ---------------------------- | -------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| BUILD_LOADER                 | All      | `ON`    | Controls whether or not the loader is built. Setting this to `OFF` will allow building the tests against a loader that is installed to the system.                                 |
-| BUILD_TESTS                  | All      | `???`   | Controls whether or not the loader tests are built. The default is `ON` when the Google Test repository is cloned into the `external` directory.  Otherwise, the default is `OFF`. |
-| BUILD_WSI_XCB_SUPPORT        | Linux    | `ON`    | Build the loader with the XCB entry points enabled. Without this, the XCB headers should not be needed, but the extension `VK_KHR_xcb_surface` won't be available.                 |
-| BUILD_WSI_XLIB_SUPPORT       | Linux    | `ON`    | Build the loader with the Xlib entry points enabled. Without this, the X11 headers should not be needed, but the extension `VK_KHR_xlib_surface` won't be available.               |
-| BUILD_WSI_WAYLAND_SUPPORT    | Linux    | `ON`    | Build the loader with the Wayland entry points enabled. Without this, the Wayland headers should not be needed, but the extension `VK_KHR_wayland_surface` won't be available.     |
-| BUILD_WSI_DIRECTFB_SUPPORT   | Linux    | `OFF`   | Build the loader with the DirectFB entry points enabled. Without this, the DirectFB headers should not be needed, but the extension `VK_EXT_directfb_surface` won't be available.  |
-| BUILD_WSI_SCREEN_QNX_SUPPORT | QNX      | `OFF`   | Build the loader with the QNX Screen entry points enabled. Without this the extension `VK_QNX_screen_surface` won't be available.                                                  |
-| ENABLE_WIN10_ONECORE         | Windows  | `OFF`   | Link the loader to the [OneCore](https://msdn.microsoft.com/en-us/library/windows/desktop/mt654039.aspx) umbrella library, instead of the standard Win32 ones.                     |
-| USE_CCACHE                   | Linux    | `OFF`   | Enable caching with the CCache program.                                                                                                                                            |
-| USE_GAS                      | Linux    | `ON`    | Controls whether to build assembly files with the GNU assembler, else fallback to C code.                                                                                          |
-| USE_MASM                     | Windows  | `ON`    | Controls whether to build assembly files with MS assembler, else fallback to C code                                                                                                |
-| BUILD_STATIC_LOADER          | macOS    | `OFF`   | This allows the loader to be built as a static library on macOS. Not tested, use at your own risk.                                                                                 |
-
+| Option                       | Platform | Default | Description                                                                                                                                                                       |
+| ---------------------------- | -------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| BUILD_LOADER                 | All      | `ON`    | Controls whether or not the loader is built.                                                                                                                                      |
+| BUILD_TESTS                  | All      | `OFF`   | Controls whether or not the loader tests are built.                                                                                                                               |
+| BUILD_WSI_XCB_SUPPORT        | Linux    | `ON`    | Build the loader with the XCB entry points enabled. Without this, the XCB headers should not be needed, but the extension `VK_KHR_xcb_surface` won't be available.                |
+| BUILD_WSI_XLIB_SUPPORT       | Linux    | `ON`    | Build the loader with the Xlib entry points enabled. Without this, the X11 headers should not be needed, but the extension `VK_KHR_xlib_surface` won't be available.              |
+| BUILD_WSI_WAYLAND_SUPPORT    | Linux    | `ON`    | Build the loader with the Wayland entry points enabled. Without this, the Wayland headers should not be needed, but the extension `VK_KHR_wayland_surface` won't be available.    |
+| BUILD_WSI_DIRECTFB_SUPPORT   | Linux    | `OFF`   | Build the loader with the DirectFB entry points enabled. Without this, the DirectFB headers should not be needed, but the extension `VK_EXT_directfb_surface` won't be available. |
+| BUILD_WSI_SCREEN_QNX_SUPPORT | QNX      | `OFF`   | Build the loader with the QNX Screen entry points enabled. Without this the extension `VK_QNX_screen_surface` won't be available.                                                 |
+| ENABLE_WIN10_ONECORE         | Windows  | `OFF`   | Link the loader to the [OneCore](https://msdn.microsoft.com/en-us/library/windows/desktop/mt654039.aspx) umbrella library, instead of the standard Win32 ones.                    |
+| USE_CCACHE                   | Linux    | `OFF`   | Enable caching with the CCache program.                                                                                                                                           |
+| USE_GAS                      | Linux    | `ON`    | Controls whether to build assembly files with the GNU assembler, else fallback to C code.                                                                                         |
+| USE_MASM                     | Windows  | `ON`    | Controls whether to build assembly files with MS assembler, else fallback to C code                                                                                               |
+| BUILD_STATIC_LOADER          | macOS    | `OFF`   | This allows the loader to be built as a static library on macOS. Not tested, use at your own risk.                                                                                |
 The following is a table of all string options currently supported by this repository:
 
 | Option                      | Platform    | Default                       | Description                                                                                                                                          |

diff --git a/tests/README.md b/tests/README.md
index a91ef481d462dfd988d4424529d26b99f7f1a615..a12fde3ec47dc9bc8c0eee892f00d8cfd374ba7a 100644 (file)
--- a/tests/README.md
+++ b/tests/README.md
@@ -5,6 +5,15 @@ This directory contains a test suite for the Vulkan loader.
 These tests are not exhaustive — they are expected to be supplemented with other tests, such as CTS.
 
 
+## Test specific CMake Configuration
+
+| Option                         | Platform | Default | Description                                              |
+| ------------------------------ | -------- | ------- | -------------------------------------------------------- |
+| BUILD_TESTS                    | All      | `OFF`   | Controls whether or not the loader tests are built.      |
+| ENABLE_LIVE_VERIFICATION_TESTS | All      | `OFF`   | Enables building of tests meant to run with live drivers |
+| TEST_USE_ADDRESS_SANITIZER     | Linux    | `OFF`   | Enables Address Sanitizer in the loader and tests        |
+| TEST_USE_THREAD_SANITIZER      | Linux    | `OFF`   | Enables Thread Sanitizer in the loader and tests         |
+
 ## Running Tests
 
 For most purposes `ctest` is the desired method of running tests.

diff --git a/tests/framework/README.md b/tests/framework/README.md
index 5994c36b04d33158efce5dcae9c2d32447fd41e6..5267a5a2e7778661e8db87cc16d40d6b37db0623 100644 (file)
--- a/tests/framework/README.md
+++ b/tests/framework/README.md
@@ -18,21 +18,22 @@ By default the Vulkan-Loader repo doesn't enable testing.
 
 To turn on building of the tests, set `BUILD_TESTS=ON` in the CMake configuration.
 
-Ensure that `googletest` is in the `external` directory.
+Use the CMake configuration `UPDATE_DEPS=ON` to automatically get all required test dependencies.
+Or Ensure that `googletest` is in the `external` directory.
+And on Windows only, ensure that the `Detours` library is in the `external` directory.
 
-Windows only: Ensure that the `Detours` library is in the `external` directory.
-
-Linux only: Optionally enable the CMake Configuration `TEST_USE_ADDRESS_SANITIZER` to
+Linux only: The CMake Configuration `TEST_USE_ADDRESS_SANITIZER` can be used to
 enable Address Sanitizer inside the testing framework.
 
 Run the test executables as normal
 
 The executables available are:
 * `test_regression`
-* `test_wsi`
 
 Alternatively, in the build directory run `ctest` to start the test framework.
 
+Use the `ctest` command line parameter `--output-on-failure` to printout logs in failing tests
+
 Note: The test framework was not designed to allow multiple tests to be run in parallel due to the extensive use of files and folders on the system.
 
 
@@ -150,8 +151,6 @@ There are many utilities that the test framework and tests have access to. These
   * InstanceCreateInfo
   * DeviceCreateInfo
   * DeviceQueueCreateInfo
-* InstWrapper - helper to construct and then destroy a vulkan instance during tests
-* DeviceWrapper - helper to construct and then destroy a vulkan device during tests
 * Comparison operators for various vulkan structs
 
 ### Test Environment
@@ -159,6 +158,8 @@ There are many utilities that the test framework and tests have access to. These
 The `test_environment.h/.cpp` contains classes which organize all the disparate parts of the framework into an easy to use entity that allows setting up and configuring the environment tests run in.
 
 The core components are:
+* InstWrapper - helper to construct and then destroy a vulkan instance during tests
+* DeviceWrapper - helper to construct and then destroy a vulkan device during tests
 * PlatformShimWrapper - opens and configures the platform specific shim library
   * Sets up the overrides
   * Resets state (clearing out previous test state)
@@ -180,9 +181,14 @@ The core components are:
   * Allows adding ICD's and Layers
     * Writes the json manifest file to the correct folder
 
-The `FrameworkEnvironment` class is then derived from to create easy to use 'environments'.
-Extends it to provide the custom environment setup behavior necessary
-Currently there are the following extension classes:
-* EnvVarICDOverrideShim - uses `VK_ICD_FILENAMES` to tell the loader where to look for a single ICD
-* SingleICDShim - Pass in a single ICDDetails to set the icd to use
-* MultiICDShim - Pass in a vector of ICDDetails to set the icds to use
+The `FrameworkEnvironment` class is used to easily create 'environments'.
+
+The `add_XXX()` member functions of `FrameworkEnvironment` make it easy to add drivers and layers to the environment a test runs in.
+
+The `get_test_icd()` and `get_test_layer()` functions allow querying references to the underlying
+drivers and layers that are in the environment, allowing quick modification of their behavior.
+
+The `reset_test_icd()` and `reset_test_layer()` are similar to the above functions but additionally
+reset the layer or driver to its initial state.
+Use this if you need to reset a driver during a test.
+These functions are called on the drivers and layers when the framework is being create in each test.
\ No newline at end of file