From 41833f93fa1a551affefa71a16fc73e7afdf2ef8 Mon Sep 17 00:00:00 2001 From: Alexander Galazin Date: Fri, 13 Jan 2017 22:41:32 +0100 Subject: [PATCH] Update GL CTS README to point to GitHub VK GL CTS wiki Change-Id: Ia06ec8801cb840e25dbe6012de34ce38ffd79467 --- external/openglcts/README.md | 301 +------------------------------------------ 1 file changed, 7 insertions(+), 294 deletions(-) diff --git a/external/openglcts/README.md b/external/openglcts/README.md index 4661dac..7b78d65 100644 --- a/external/openglcts/README.md +++ b/external/openglcts/README.md @@ -33,15 +33,6 @@ Contents - [Debugging Test Failures](#debugging-test-failures) - [Waivers](#waivers) - [Creating a Submission Package](#creating-a-submission-package) - - [Obtaining the sources](#obtaining-the-sources) - - [Building and installing the CTS](#building-and-installing-the-cts) - - [Running the CTS](#running-the-cts) - - [Creating a package](#creating-a-package) - - [1) Statement of Conformance](#1-statement-of-conformance) - - [2) Conformant Products List](#2-conformant-products-list) - - [3) Result Logs](#3-result-logs) - - [4) Changes](#4-changes) - - [5) Explanation of Changes](#5-explanation-of-changes) - [Submission Update Package](#submission-update-package) - [Passing Criteria](#passing-criteria) - [Troubleshooting](#troubleshooting) @@ -702,306 +693,28 @@ debugger should help. Waivers ------------------------ The procedure for requesting a waiver is to report the issue by filing a bug -report in the Gitlab OpenGL & OpenGL ES Conformance Test Suite project -(https://gitlab.khronos.org/opengl/oss-cts). When you create your submission +report in the Gitlab VK GL CTS project +(https://gitlab.khronos.org/Tracker/vk-gl-cts). When you create your submission package, include references to the waivers as described in the adopters' agreement. [Fully-qualified links](https://en.wikipedia.org/wiki/Fully_qualified_domain_name) to bug reports are highly recommended. Including as much information as possible in your bug report will ensure the issue can be progressed as speedily as possible. Such bug report must -include a link to suggested file changes. Issues must be labeled `Waiver_OGLES` -(for OpenGL ES submissions) or `Waiver_OGL` (for OpenGL submissions) and +include a link to suggested file changes. Issues must be labeled `Waiver` and `OpenGL-ES` +(for OpenGL ES submissions) or `Waiver` and `OpenGL` (for OpenGL submissions) and identify the CTS release tag and affected tests. Creating a Submission Package ------------------------ - -### Obtaining the sources - -The CTS releases are git tags of form - `opengl-cts--...` -where -- `` is either `es` for OpenGL ES releases or `gl` for OpenGL releases; -- `` and `` match OpenGL (ES) API version; -- `.` is the CTS version. - -To clone the repository and check out a specific release, run: - - git clone https://github.com/KhronosGroup/.git -b opengl-cts- - -If you have existing clone of the repository you can check out specific release with: - - git checkout opengl-cts- - -### Building and installing the CTS - -Refer to [Building the tests](#building-the-tests) for detailed instructions. -The essential steps are: -1. Download zlib, libpng and other necessary sources by running -``` - python external/fetch_sources.py -``` -2. Download Khronos Confidential Conformance Test Suite by running -``` - python external/fetch_kc_cts.py -``` -3. Build the conformance tests by running - * Linux/Windows -``` - cmake -DDEQP_TARGET= -DGLCTS_GTF_TARGET= -G"" - cmake --build . -``` - The binaries and data files will be placed to -``` - /external/openglcts/modules -``` - * Android -``` - python external/openglcts/scripts/build_android.py -``` - Run one of the following commands to install the binary on your device -``` - python android/scripts/install.py -``` - or -``` - adb install --abi android/openglcts/bin/dEQP-debug.apk /data/local/tmp/dEQP-debug.apk -``` - -### Running the CTS -This section provides a short summary on how to launch a conformance run. For - detailed instructions refer to [Runnnig the tests](#running-the-tests). - -1. Linux -``` - ./cts-runner --type= -``` - -2. Windows -``` - cts-runner.exe --type= -``` - -3. Android -``` - am start -n org.khronos.gl_cts/org.khronos.cts. -e logdir "/sdcard/logs" -``` - -**NOTE**: A valid GLES 3.2 submission is sufficient to prove GLES 3.1 , -GLES 3.0 and GLES 2.0 conformance; it is not necessary to make separate -submissions for the older APIs. Similarly a valid GLES 3.1 submission is proof -of GLES3.0 and GLES 2.0 conformance. - -### Creating a package -Once the tests are all passing or all observed failures have appropriate -waivers, run the tests with the `cts-runner` executable or the Android -conformance activity. The command must not include the parameters for verbose -output. Refer to Section [Running Tests](#running-tests) its Subsection -[Android](#android-1) to learn more. - -The actual submission package consists of a set of files, which should be -bundled into a gzipped tar file named `ESMn__.tgz` -(For OpenGL ES) or `GLMn__.tgz` (for OpenGL). -Here `` is the name of the Adopting member company, or some -recognizable abbreviation (e.g. the company's extension prefix). -The `` field is optional. It may be used to uniquely identify -a submission by OS, platform, date or other criteria when you are making -multiple submissions within a short period of time. For example, company XYZ -might make a group of submissions titled `ES32_XYZ_Android4.1.1`, -`ES32_XYZ_SupercoreV8`, et cetera. The `` field is not required, and -it is permissible to make multiple distinct submissions which have the same -package file name. `` refers to the OpenGL or OpenGL ES major and minor -version for which results are being submitted. For example, a submission for -an OpenGL 3.3 implementation might be named `GL33_XYZ_Windows7.tgz`, -a submission for an OpenGL ES 3.1 implementation might be named -`ES31_XYZ_Windows7.tgz` . - -One way to create a suitable gzipped tar file is to execute the command: - - tar -cvzf -C srcDirectory . - -where `` is the name of the directory containing the package -files. A submission package should contain the files listed in the following -five sections and only them. - -#### 1) Statement of Conformance - -Statement of Conformance: Include a file called `STATEMENT-` that -describes the Implementation for which you are claiming conformance based on -this submission. This should be a simple text file containing the following -keywords and their values, with at most one keyword/value pair per line. - - CONFORM_VERSION: - PRODUCT: - CPU: - OS: - -`CONFORM_VERSION` is the CTS git release tag that you are using for your submission. - -`PRODUCT` is the name of the product that was used to generate the test results. -The product name should be the name that the product is commonly known by, and -should be specific enough to identify it unambiguously. -For example, hardware implementations might describe their product as -"Nokia N95 mobile phone handset", or "TI OMAP2420 H4 development board". -Software implementations should use the name under which the software is -distributed, such as "AMD RenderMonkey 1.8 SDK". Submissions covering simulated -hardware (e.g. silicon IP) should use the trade name of the IP package, e.g. -"GC500 version . RTL package". - -`CPU` and `OS` describe the platform on which the OpenGL ES driver client -interface is exposed. Note that only one CPU and OS can be listed for a given -submission. It is permissible to identify a CPU by its instruction set -(e.g. ARMv8), or an OS by its major and minor release numbers (e.g. iOS 6.1). -If an implementation makes use of optional CPU features, they should be -identified in the CPU name (e.g. ARMv7-NEON). - -#### 2) Conformant Products List - -Conformant Products List: Optionally, include a file named `PRODUCTS-` -describing any Products other than the one identified in the -Statement of Conformance for which conformance is claimed under the current -submission. If there are no such other Products, this file can be omitted. -Conformant Products are products that are so similar to -the Conformant Implementation that a separate submission is not required. - -Section [A8 of the Conformance Procedures Document](https://www.khronos.org/files/conformance_procedures.pdf) -describes the similarity criteria a Product must satisfy to be included on the -Conformant Products List of a submission. Additional information is given below. - -The Conformant Products List should be a text file containing a series of text -blocks separated from each other by blank lines. - -Each text block should have the form: - - PRODUCT: - CPU: - OS: - -The meanings of the fields are the same as in the Statement of Conformance -([see above](#1-statement-of-conformance)). - -Note that to be claimed as a Conformant Product under a submission, a -Product's CPU must be sufficiently similar to the CPU of -the Conformant Implementation that they can execute the same driver binary, -or different builds of the same driver source compiled with the same compiler -options. For example, products based on ARM 1136 can be claimed as part of -a submission for an ARM 1176 implementation, provided the driver does not use -any features that are not found in both CPUs. See section -[A8 of the Conformance Procedures Document](https://www.khronos.org/files/conformance_procedures.pdf). - -Note also that a Conformant Product's OS must be the same as -the Conformant Implementation described in the Statement of Conformance, -or must differ by at most a minor revision that does not materially affect -the driver binary. For example, a Product using Linux 2.6.5 may be claimed if -it is not materially different from a Conformant Implementation using Linux 2.6.3. - -Display parameters for a Conformant Product are not required to be the same as -those of the Conformant Implementation described in the Statement of Conformance. - -New Conformant Products may be added to the Conformant Products List of -a previous submission using the Submission Update process -([see below](#submission-update-package)). - -#### 3) Result Logs - -The submission package must contain all `.qpa` files produced by the -conformance test run, without the `--verbose` flag. The verbose logs are -large and the verbosity complicates the review process. The submission package -must also contain the summary log file, `cts-run-summary.xml`. - -#### 4) Changes - -The CTS build must always be done from clean git repository that doesn't have -any uncommitted changes. Thus it is necessary to run and capture output of -`git status` and `git log` in the root source directory and in -Khronos Confidential CTS source directory: - - git status > /git-status.txt - git log --first-parent ^..HEAD > /git-log.txt - - cd external/kc-cts/src - git status > /kc-cts-git-status.txt - git log --first-parent ^..HEAD > /kc-cts-git-log.txt - -Any changes made to CTS must be committed to the local repository, and provided -as part of the submission package. This can be done by running: - - git format-patch -o ..HEAD - - cd external/kc-cts/src - git format-patch -o ..HEAD - - -**NOTE**: When cherry-picking patches on top of release tag, please use -`git cherry-pick -x` to include original commit hash in the commit message. - -#### 5) Explanation of Changes - -Explanation of Changes: A text file named `README-` summarizing any -changes you made beyond the make system and the files listed above under -[Porting](#porting). Your summary should explain which files you changed, -the nature of the changes, and why it was necessary to change them. If you are -applying for a waiver for any change you made, include a reference to the bug -report containing the waiver request. [Fully-qualified links](https://en.wikipedia.org/wiki/Fully_qualified_domain_name) -to bug reports are highly recommended. - -Absence of any of these files (except the optional `PRODUCTS-` file) -from the package will invalidate the submission. Presence of extraneous files -will invalidate a submission. +Please see the [Creating a Submission Package page](https://github.com/KhronosGroup/VK-GL-CTS/wiki/Creating-a-OpenGL-and-OpenGL-ES-Submission-Package). Submission Update Package ------------------------ -A Submission Update adds products to the Conformant Products List of a previous -successful Submission. A submission update package is identical to a submission -package (see above), except that the `.tgz` file contains only a -Conformant Products List. The new Conformant Products List should include -all of the entries from the List in the previous submission, as well as -the new products. - -Adopters are not required to use the Update process. That is, it is permissible -to make new submissions for products that could have been claimed as Conformant -Products relative to a previous successful submission. +Please see the [Submission Update Package page](https://github.com/KhronosGroup/VK-GL-CTS/wiki/Submission-Update-Package). Passing Criteria ------------------------ -A submission is considered passing if the following statements are true: -1. The submission package contains exactly the files described above under -[Creating a Submission Package](#creating-a-submission-package), optionally -including a Conformant Products List, all correctly named and formatted. -2. The `git log` and `git status` files included with the submission show no -changes other than those explicitly allowed under [Porting](#porting), above; -OR, any such changes are accompanied by waiver requests as described above under [Waivers](#waivers). -3. The test logs show no failures -(see [Understanding the Results](#understanding-the-results) above). -4. The test logs are produced as described in -the [Running the Tests](#running-the-tests) section on a implementation where -a default (window) framebuffer supports the following configuration: -
    -
  1. at least a 16-bit color buffer, at least a 15-bit depth buffer, and at - least a 1-bit stencil buffer;
    2. -
  2. exactly a RGBA color buffer with 8 bits per channel, a 24-bit depth buffer, - a 8-bit stencil buffer, and no multisample buffer;
    3. -
  3. exactly a RGBA color buffer with 8 bits per channel, a 24-bit depth buffer, - a 8-bit stencil buffer, and a multisample buffer with 4 samples;
    4. -
  4. exactly a RGB color buffer with 5 bits for the R and B channels and 6 bits - for the G channel, no depth buffer, no stencil buffer, and no multisample buffer.
    5. -
- Waivers for specific configurations may be granted to accommodate limitations - in the underlying platform or implementation. -5. Test cases included in Khronos Confidential CTS must be run and present -in the test logs. -6. For implementations that support multiple window configurations (e.g. those -exposed by EGL, WGL, or similar binding layer), the number of non-multisampled -configurations identified as `CONFORMANT` by the binding layer is greater than -or equal to the number identified as `NON-CONFORMANT`. -7. An implementation must expose the same set of conformant configurations for -all API versions covered by the submission. For example, an OpenGL ES 3.2 -submission also covers OpenGL ES 3.1, ES 3.0 and 2.0 conformance, and therefore -for all these APIs the implementation must expose the same conformant configurations. -8. The appropriate OpenGL or OpenGL ES Review Committee agrees to grant any -requested waivers and confirms the validity of the test results, OR -the Review Period (30 days from date of submission) expires without comment -from the Review Committee. +Please see the [Conformance Submission Passing Criteria page](https://github.com/KhronosGroup/VK-GL-CTS/wiki/OpenGL-and-OpenGL-ES-Conformance-Submission-Passing-Criteria). Troubleshooting ------------------------ -- 2.7.4