<p>Sometimes it is necessary to mark a test case as "expected fail" or XFAIL.
You can easily mark a test as XFAIL just by including <tt>XFAIL: </tt> on a
line near the top of the file. This signals that the test case should succeed
- if the test fails. Such test cases are counted separately by the testing tool. To
- specify an expected fail, use the XFAIL keyword in the comments of the test
- program followed by a colon and one or more regular expressions (separated by
- a comma). The regular expressions allow you to XFAIL the test conditionally by
- host platform. The regular expressions following the : are matched against the
- target triplet for the host machine. If there is a match, the test is expected
- to fail. If not, the test is expected to succeed. To XFAIL everywhere just
- specify <tt>XFAIL: *</tt>. Here is an example of an <tt>XFAIL</tt> line:</p>
+ if the test fails. Such test cases are counted separately by the testing
+ tool. To specify an expected fail, use the XFAIL keyword in the comments of
+ the test program followed by a colon and one or more failure patterns. Each
+ failure pattern can be either '*' (to specify fail everywhere), or a part of a
+ target triple (indicating the test should fail on that platfomr), or the name
+ of a configurable feature (for example, "loadable_module").. If there is a
+ match, the test is expected to fail. If not, the test is expected to
+ succeed. To XFAIL everywhere just specify <tt>XFAIL: *</tt>. Here is an
+ example of an <tt>XFAIL</tt> line:</p>
<div class="doc_code">
<pre>
--- /dev/null
+// This test should XPASS.
+
+// RUN: true
+// XFAIL: some-feature-name
return executeCommand(command, cwd=cwd, env=test.config.environment)
-def isExpectedFail(xfails, xtargets, target_triple):
- # Check if any xfail matches this target.
+def isExpectedFail(test, xfails, xtargets):
+ # If the xfail matches an available feature, it always fails.
for item in xfails:
- if item == '*' or item in target_triple:
+ if item in test.config.available_features:
+ return True
+
+ # Otherwise, check if any xfail matches this target.
+ for item in xfails:
+ if item == '*' or item in test.suite.config.target_triple:
break
else:
return False
#
# FIXME: Rename XTARGET to something that makes sense, like XPASS.
for item in xtargets:
- if item == '*' or item in target_triple:
+ if item == '*' or item in test.suite.config.target_triple:
return False
return True
return (Test.UNSUPPORTED,
"Test requires the following features: %s" % msg)
- isXFail = isExpectedFail(xfails, xtargets, test.suite.config.target_triple)
+ isXFail = isExpectedFail(test, xfails, xtargets)
return script,isXFail,tmpBase,execdir
def formatTestOutput(status, out, err, exitCode, failDueToStderr, script):