From 2549b8bdae67ab8c1a6ecd974b17b3e3413ae622 Mon Sep 17 00:00:00 2001 From: Paul Robinson Date: Wed, 21 Dec 2022 07:09:35 -0800 Subject: [PATCH] [docs] Add tips on writing test constraints --- llvm/docs/TestingGuide.rst | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/llvm/docs/TestingGuide.rst b/llvm/docs/TestingGuide.rst index f610747..0601f9b 100644 --- a/llvm/docs/TestingGuide.rst +++ b/llvm/docs/TestingGuide.rst @@ -533,6 +533,46 @@ As a special case, ``XFAIL: *`` is expected to fail everywhere. ; XFAIL: target=powerpc{{.*}}, system-darwin +Tips for writing constraints +---------------------------- + +**``REQUIRES`` and ``UNSUPPORTED``** + +These are logical inverses. In principle, ``UNSUPPORTED`` isn't absolutely +necessary (the logical negation could be used with ``REQUIRES`` to get +exactly the same effect), but it can make these clauses easier to read and +understand. Generally, people use ``REQUIRES`` to state things that the test +depends on to operate correctly, and ``UNSUPPORTED`` to exclude cases where +the test is expected never to work. + +**``UNSUPPORTED`` and ``XFAIL``** + +Both of these indicate that the test isn't expected to work; however, they +have different effects. ``UNSUPPORTED`` causes the test to be skipped; +this saves execution time, but then you'll never know whether the test +actually would start working. Conversely, ``XFAIL`` actually runs the test +but expects a failure output, taking extra execution time but alerting you +if/when the test begins to behave correctly (an XPASS test result). You +need to decide which is more appropriate in each case. + +**Using ``target=...``** + +Checking the target triple can be tricky; it's easy to mis-specify. For +example, ``target=mips{{.*}}`` will match not only mips, but also mipsel, +mips64, and mips64el. ``target={{.*}}-linux-gnu`` will match +x86_64-unknown-linux-gnu, but not armv8l-unknown-linux-gnueabihf. +Prefer to use hyphens to delimit triple components (``target=mips-{{.*}}``) +and it's generally a good idea to use a trailing wildcard to allow for +unexpected suffixes. + +Also, it's generally better to write regular expressions that use entire +triple components, than to do something clever to shorten them. For +example, to match both freebsd and netbsd in an expression, you could write +``target={{.*(free|net)bsd.*}}`` and that would work. However, it would +prevent a ``grep freebsd`` from finding this test. Better to use +``target={{.*(freebsd|netbsd).*}}`` in this case. + + Substitutions ------------- -- 2.7.4