This document describes how to build and run Vulkan Conformance Test suite.
-Vulkan CTS is built on dEQP framework. General dEQP documentation is available
+Vulkan CTS is built on dEQP framework. dEQP documentation is available
at http://source.android.com/devices/graphics/testing.html
* CMake 2.8 or newer
Win32:
- * Visual Studio 2013 (glslang uses several C++11 features)
+ * Visual Studio 2013 or newer (glslang uses several C++11 features)
Linux:
* Standard toolchain (make, gcc/clang)
+Android:
+ * Android NDK r10e
+ * Android SDK with following packages:
+ + SDK Tools
+ + SDK Platform-tools
+ + SDK Build-tools
+ + API 22
+ * Apache Ant
+ * Windows: either NMake or JOM in PATH
+
-Building
---------
+Building CTS
+------------
To build dEQP, you need first to download sources for zlib, libpng, glslang,
and spirv-tools.
NOTE: glslang integration is not yet available on Android due to a toolchain
bug, so pre-compiled SPIR-V binaries must be used. See instructions below.
+With CMake out-of-source builds are always recommended. Create a build directory
+of your choosing, and in that directory generate Makefiles or IDE project
+using cmake.
-Running
--------
-Win32:
+Windows x86-32:
-> cd builddir/external/vulkancts/modules/vulkan
-> Debug/deqp-vk.exe
+ > cmake <path to vulkancts> -G"Visual Studio 12"
+ > start dEQP-Core-default.sln
-Linux:
-$ cd builddir/external/vulkancts/modules/vulkan
-$ ./deqp-vk
+Windows x86-64:
+
+ > cmake <path to vulkancts> -G"Visual Studio 12 Win64"
+ > start dEQP-Core-default.sln
+
+
+Linux 32-bit Debug:
+
+ $ cmake <path to vulkancts> -DCMAKE_BUILD_TYPE=Debug -DCMAKE_C_FLAGS=-m32 -DCMAKE_CXX_FLAGS=-m32
+ $ make -j
+
+Release build can be done by using -DCMAKE_BUILD_TYPE=Release
+
+
+Linux 64-bit Debug:
+
+ $ cmake <path to vulkancts> -DCMAKE_BUILD_TYPE=Debug -DCMAKE_C_FLAGS=-m64 -DCMAKE_CXX_FLAGS=-m64
+ $ make -j
+
Android:
-Using Cherry is recommended. Alternatively you can follow instructions at
-http://source.android.com/devices/graphics/run-tests.html
+Following command will build CTS into android/package/bin/dEQP-debug.apk.
+
+ $ python android/scripts/build.py
+
+The package can be installed by either running:
+
+ $ python android/scripts/install.py
+
+By default the CTS package will contain libdeqp.so built for armeabi-v7a, arm64-v8a,
+and x86 ABIs, but that can be changed in android/scripts/common.py script.
+
+To pick which ABI to use at install time, following commands must be used
+instead:
+
+ $ adb install --abi <ABI name> android/package/bin/dEQP-debug.apk /data/local/tmp/dEQP-debug.apk
+
+
+Building Mustpass
+-----------------
+
+Current mustpass is checked into repository and can be found at:
+
+ external/vulkancts/mustpass/1.0.0/vk-default.txt
+
+Vulkan CTS mustpass can be re-generated by running:
+
+ $ python <vulkancts>/external/vulkancts/build_mustpass.py
Pre-compiling SPIR-V binaries
For distribution, and platforms that don't support GLSL to SPIR-V compilation,
SPIR-V binaries must be pre-built with following command:
-$ python external/vulkancts/build_spirv_binaries.py
+ $ python external/vulkancts/build_spirv_binaries.py
Binaries will be written to external/vulkancts/data/vulkan/prebuilt/.
SPIR-V programs in order for the binaries to be available.
+Running CTS
+-----------
+
+Following command line options MUST be used when running CTS:
+
+ --deqp-caselist-file=<vulkancts>/external/vulkancts/mustpass/1.0.0/vk-default.txt
+ --deqp-log-images=disable
+ --deqp-log-shader-sources=disable
+
+In addition on multi-device systems the device for which conformance is claimed
+can be selected with:
+
+ --deqp-vk-device-id=<value>
+
+No other command line options are allowed.
+
+
+Win32:
+
+ > cd <builddir>/external/vulkancts/modules/vulkan
+ > Debug/deqp-vk.exe --deqp-caselist-file=...
+
+Test log will be written into TestResults.qpa
+
+
+Linux:
+
+ $ cd <builddir>/external/vulkancts/modules/vulkan
+ $ ./deqp-vk --deqp-vk-caselist-file=...
+
+
+Android:
+
+ $ adb push <vulkancts>/external/vulkancts/mustpass/1.0.0/vk-default.txt /sdcard/vk-default.txt
+ $ adb shell
+
+In device shell:
+
+ $ am start -n com.drawelements.deqp/android.app.NativeActivity -e cmdLine "deqp --deqp-caselist-file=/sdcard/vk-default.txt --deqp-log-images=disable --deqp-log-shader-sources=disable --deqp-log-filename=/sdcard/TestResults.qpa"
+
+Process can be followed by running:
+
+ $ adb logcat -s dEQP
+
+Test log will be written into /sdcard/TestResults.qpa
+
+
+Conformance Submission Package Requirements
+-------------------------------------------
+
+The conformance submission package must contain the following:
+
+1) Full test logs (TestResults.qpa) from CTS runs against all driver builds
+2) Result of "git status" and "git log" from CTS source directory
+3) Any patches used on top of release tag
+4) Conformance statement
+
+Test logs (1) should be named <submission pkg dir>/TestResults-<driver build type>.qpa,
+for example TestResults-armeabi-v7a.qpa. On platforms where multiple different driver
+builds (for example 64-bit and 32-bit) are present, CTS logs must be provided
+for each driver build.
+
+The CTS build must always be done from clean git repository that doesn't have any
+uncommitted changes. Thus it is necessary to run and capture output of "git
+status" and "git log" (2) in the source directory:
+
+ git status > <submission pkg dir>/git-status.txt
+ git log <release tag>..HEAD > <submission pkg dir>/git-log.txt
+
+Any changes made to CTS must be committed to the local repository, and provided
+as part of the submission package (3). This can be done by running:
+
+ git format-patch -o <submission pkg dir> <release tag>..HEAD
+
+In general, bugfixes and changes to platform-specific code (mostly under
+framework/platform) are allowed. The commit message for each change must
+include a clear description of the change and why it is necessary. Non-porting
+related changes must be accompanied by a waiver (see below).
+
+Note: When cherry-picking patches on top of release tag, please use "git cherry-pick -x"
+to include original commit hash in the commit message.
+
+Conformance statement (4) must be included in a file called STATEMENT-<adopter>
+and must contain following:
+
+ CONFORM_VERSION: <git tag of CTS release>
+ PRODUCT: <string-value>
+ CPU: <string-value>
+ OS: <string-value>
+
+Note that product/cpu/os information is also captured in dEQP-VK.info.* tests
+if vk::Platform::describePlatform() is implemented.
+
+The actual submission package consists of the above set of files which must
+be bundled into a gzipped tar file named VK10_<adopter><_info>.tgz. <adopter>
+is the name of the Adopting member company, or some recognizable abbreviation.
+The <_info> field is optional. It may be used to uniquely identify a submission
+by OS, platform, date, or other criteria when making multiple submissions.
+
+One way to create a suiteable gzipped tar file is to execute the command:
+
+$ tar -cvzf <filename.tgz> -C <submission pkg dir> .
+
+where <submission pkg dir> is the directory containing the files from (1)-(4)
+from above. A submission package must contain all of the files listed above,
+and only those files.
+
+
+Waivers
+-------
+
+The process for requesting a waiver is to report the issue by filing a bug
+report in the Gitlab VulkanCTS project (TODO Github?). When creating the
+submission package, include references to the waiver in the commit message of
+the relevant change. Including as much information as possible in your bug
+report (including a unified diff or a merge request of suggested file changes)
+will ensure the issue can be progressed as rapidly as possible. Issues must
+be labeled "Waiver" (TODO!) and identify the version of the CTS and affected
+tests.
+
+Conformance Criteria
+--------------------
+
+Conformance run is considered passing if all tests finish with allowed result
+codes. The test results are contained in the TestResults.qpa log. Each
+test case section contains XML tag Result, for example:
+
+ <Result StatusCode="Pass">Not validated</Result>
+
+The result code is the value of the StatusCode attribute. Following status
+codes are allowed:
+
+ Pass, NotSupported, QualityWarning, CompatibilityWarning
+
+TODO: Create script for verifying logs.
+
+
Vulkan platform port
--------------------
In cherry/testrunner.go, add following line to testPackageDescriptors list
(line 608 in NewTestRunner function):
-{"dEQP-VK", "deqp-vk", "../external/vulkancts/modules/vulkan", dataDir + "dEQP-VK-cases.xml"},
+ {"dEQP-VK", "deqp-vk", "../external/vulkancts/modules/vulkan", dataDir + "dEQP-VK-cases.xml"},
Before first launch, and every time test hierarchy has been modified, test
case list must be refreshed by running:
-$ python scripts/build_caselists.py path/to/cherry/data
+ $ python scripts/build_caselists.py path/to/cherry/data
Cherry must be restarted for the case list update to take effect.
projects / platform / upstream / VK-GL-CTS.git / blobdiff