libs/test/doc/html/boost_test/tests_organization/tests_dependencies.html

   1 <html>
   2 <head>
   3 <meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
   4 <title>Managing test dependencies</title>
   5 <link rel="stylesheet" href="../../boostbook.css" type="text/css">
   6 <meta name="generator" content="DocBook XSL Stylesheets V1.79.1">
   7 <link rel="home" href="../../index.html" title="Boost.Test">
   8 <link rel="up" href="../tests_organization.html" title="Declaring and organizing tests">
   9 <link rel="prev" href="fixtures/global.html" title="Global fixture">
  10 <link rel="next" href="tests_grouping.html" title="Grouping tests into logical units by labels">
  11 </head>
  12 <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
  13 <table cellpadding="2" width="100%"><tr>
  14 <td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../boost.png"></td>
  15 <td align="center"><a href="../../../../../../index.html">Home</a></td>
  16 <td align="center"><a href="../../../../../../libs/libraries.htm">Libraries</a></td>
  17 <td align="center"><a href="http://www.boost.org/users/people.html">People</a></td>
  18 <td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td>
  19 <td align="center"><a href="../../../../../../more/index.htm">More</a></td>
  20 </tr></table>
  21 <hr>
  22 <div class="spirit-nav">
  23 <a accesskey="p" href="fixtures/global.html"><img src="../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../tests_organization.html"><img src="../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../index.html"><img src="../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="tests_grouping.html"><img src="../../../../../../doc/src/images/next.png" alt="Next"></a>
  24 </div>
  25 <div class="section">
  26 <div class="titlepage"><div><div><h3 class="title">
  27 <a name="boost_test.tests_organization.tests_dependencies"></a><a class="link" href="tests_dependencies.html" title="Managing test dependencies">Managing
  28       test dependencies</a>
  29 </h3></div></div></div>
  30 <p>
  31         In general, it is a good practice to write any test as independent as possible
  32         from any other, there are however cases where a dependency cannot be avoided
  33         and an order for executing the tests is needed.
  34       </p>
  35 <p>
  36         In the general setup and for any two test cases <code class="computeroutput"><span class="identifier">TA</span></code>
  37         and <code class="computeroutput"><span class="identifier">TB</span></code>, <code class="computeroutput"><span class="identifier">TB</span></code>
  38         should not take for granted that <code class="computeroutput"><span class="identifier">TA</span></code>
  39         has already executed, even if <code class="computeroutput"><span class="identifier">TA</span></code>
  40         is declared before <code class="computeroutput"><span class="identifier">TB</span></code> in
  41         the same translation unit. The only ordering-related guarantee that <span class="emphasis"><em>Unit
  42         Test Framework</em></span> makes by default is that if test cases <code class="computeroutput"><span class="identifier">TA</span></code> and <code class="computeroutput"><span class="identifier">TB</span></code>
  43         are declared in the same test suite, no test case (call it <code class="computeroutput"><span class="identifier">TX</span></code>) from any other test suite is executed
  44         between <code class="computeroutput"><span class="identifier">TA</span></code> and <code class="computeroutput"><span class="identifier">TB</span></code>, even if the declaration of <code class="computeroutput"><span class="identifier">TX</span></code> appears between the declarations of
  45         <code class="computeroutput"><span class="identifier">TA</span></code> and <code class="computeroutput"><span class="identifier">TB</span></code>.
  46         In other words, all tests from a suite are executed in one go, even if the
  47         test suite namespace is opened multiple times.
  48       </p>
  49 <p>
  50         Even though the order is not guaranteed, it may accidentally be preserved
  51         across the different runs. In order to make sure the test cases do not depend
  52         on one another, the test module may be called with an additional command-line
  53         argument, <a class="link" href="../utf_reference/rt_param_reference/random.html" title="random"><code class="computeroutput"><span class="identifier">random</span></code></a>, to shuffle the tests unit
  54         ordering and to be more robust against an erroneous implicit ordering.
  55       </p>
  56 <h4>
  57 <a name="boost_test.tests_organization.tests_dependencies.h0"></a>
  58         <span class="phrase"><a name="boost_test.tests_organization.tests_dependencies.declaring_a_test_case_dependency"></a></span><a class="link" href="tests_dependencies.html#boost_test.tests_organization.tests_dependencies.declaring_a_test_case_dependency">Declaring
  59         a test case dependency</a>
  60       </h4>
  61 <p>
  62         If there exist a dependency between test units, and an ordering is required
  63         between the execution of those tests, it has to be declared explicitly. Dependencies
  64         in the <span class="emphasis"><em>Unit Test Framework</em></span> affect two dimensions of
  65         test units, which are:
  66       </p>
  67 <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
  68 <li class="listitem">
  69             the order of execution of these units
  70           </li>
  71 <li class="listitem">
  72             the execution of a test unit, which is conditioned by the state of its
  73             parents
  74           </li>
  75 </ul></div>
  76 <p>
  77         <a class="link" href="decorators.html" title="Decorators">Decorator</a>
  78         <a class="link" href="../utf_reference/test_org_reference/decorator_depends_on.html" title="depends_on (decorator)"><code class="computeroutput"><span class="identifier">depends_on</span></code></a> associates the decorated
  79         test case (call it <code class="computeroutput"><span class="identifier">TB</span></code>) with
  80         another test case (call it <code class="computeroutput"><span class="identifier">TA</span></code>)
  81         specified by name. This affects the processing the test tree in two ways:
  82       </p>
  83 <div class="orderedlist"><ol class="orderedlist" type="1">
  84 <li class="listitem">
  85             first, test case <code class="computeroutput"><span class="identifier">TA</span></code> is
  86             ordered to be run before <code class="computeroutput"><span class="identifier">TB</span></code>,
  87             irrespective of the order in which they were declared or added to the
  88             test tree,
  89           </li>
  90 <li class="listitem">
  91             second, the execution of <code class="computeroutput"><span class="identifier">TB</span></code>
  92             is skipped if <code class="computeroutput"><span class="identifier">TA</span></code> is either
  93             disabled or skipped or is executed and marked as failed.
  94           </li>
  95 </ol></div>
  96 <h6>
  97 <a name="boost_test.tests_organization.tests_dependencies.h1"></a>
  98         <span class="phrase"><a name="boost_test.tests_organization.tests_dependencies.example_descr"></a></span><a class="link" href="tests_dependencies.html#boost_test.tests_organization.tests_dependencies.example_descr">Example:
  99         decorator depends_on</a>
 100       </h6>
 101 <div class="informaltable"><table class="table">
 102 <colgroup><col></colgroup>
 103 <thead><tr><th>
 104                 <p>
 105                   Code
 106                 </p>
 107               </th></tr></thead>
 108 <tbody><tr><td>
 109 <pre xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" class="table-programlisting"><span class="preprocessor">#define</span> <span class="identifier">BOOST_TEST_MODULE</span> <span class="identifier">decorator_07</span>
 110 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">test</span><span class="special">/</span><span class="identifier">included</span><span class="special">/</span><span class="identifier">unit_test</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 111
 112 <span class="keyword">namespace</span> <span class="identifier">utf</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">unit_test</span><span class="special">;</span>
 113
 114 <span class="comment">// test1 and test2 defined at the bottom</span>
 115
 116 <span class="identifier">BOOST_AUTO_TEST_CASE</span><span class="special">(</span><span class="identifier">test3</span><span class="special">,</span> <span class="special">*</span> <span class="identifier">utf</span><span class="special">::</span><span class="identifier">depends_on</span><span class="special">(</span><span class="string">"s1/test1"</span><span class="special">))</span>
 117 <span class="special">{</span>
 118   <span class="identifier">BOOST_TEST</span><span class="special">(</span><span class="keyword">false</span><span class="special">);</span>
 119 <span class="special">}</span>
 120
 121 <span class="identifier">BOOST_AUTO_TEST_CASE</span><span class="special">(</span><span class="identifier">test4</span><span class="special">,</span> <span class="special">*</span> <span class="identifier">utf</span><span class="special">::</span><span class="identifier">depends_on</span><span class="special">(</span><span class="string">"test3"</span><span class="special">))</span>
 122 <span class="special">{</span>
 123   <span class="identifier">BOOST_TEST</span><span class="special">(</span><span class="keyword">false</span><span class="special">);</span>
 124 <span class="special">}</span>
 125
 126 <span class="identifier">BOOST_AUTO_TEST_CASE</span><span class="special">(</span><span class="identifier">test5</span><span class="special">,</span> <span class="special">*</span> <span class="identifier">utf</span><span class="special">::</span><span class="identifier">depends_on</span><span class="special">(</span><span class="string">"s1/test2"</span><span class="special">))</span>
 127 <span class="special">{</span>
 128   <span class="identifier">BOOST_TEST</span><span class="special">(</span><span class="keyword">false</span><span class="special">);</span>
 129 <span class="special">}</span>
 130
 131 <span class="identifier">BOOST_AUTO_TEST_SUITE</span><span class="special">(</span><span class="identifier">s1</span><span class="special">)</span>
 132
 133   <span class="identifier">BOOST_AUTO_TEST_CASE</span><span class="special">(</span><span class="identifier">test1</span><span class="special">)</span>
 134   <span class="special">{</span>
 135     <span class="identifier">BOOST_TEST</span><span class="special">(</span><span class="keyword">true</span><span class="special">);</span>
 136   <span class="special">}</span>
 137
 138   <span class="identifier">BOOST_AUTO_TEST_CASE</span><span class="special">(</span><span class="identifier">test2</span><span class="special">,</span> <span class="special">*</span> <span class="identifier">utf</span><span class="special">::</span><span class="identifier">disabled</span><span class="special">())</span>
 139   <span class="special">{</span>
 140     <span class="identifier">BOOST_TEST</span><span class="special">(</span><span class="keyword">false</span><span class="special">);</span>
 141   <span class="special">}</span>
 142
 143 <span class="identifier">BOOST_AUTO_TEST_SUITE_END</span><span class="special">()</span>
 144 </pre>
 145               </td></tr></tbody>
 146 </table></div>
 147 <div class="informaltable"><table class="table">
 148 <colgroup><col></colgroup>
 149 <thead><tr><th>
 150                 <p>
 151                   Output
 152                 </p>
 153               </th></tr></thead>
 154 <tbody><tr><td>
 155 <pre xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" class="table-programlisting"><span class="special">&gt;</span> <span class="identifier">decorator_07</span> <span class="special">--</span><span class="identifier">report_level</span><span class="special">=</span><span class="identifier">detailed</span> <span class="special">--</span><span class="identifier">log_level</span><span class="special">=</span><span class="identifier">all</span>
 156 <span class="identifier">Running</span> <span class="number">4</span> <span class="identifier">test</span> <span class="identifier">cases</span><span class="special">...</span>
 157 <span class="identifier">Entering</span> <span class="identifier">test</span> <span class="identifier">module</span> <span class="string">"decorator_07"</span>
 158 <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">:</span><span class="number">31</span><span class="special">:</span> <span class="identifier">Entering</span> <span class="identifier">test</span> <span class="identifier">suite</span> <span class="string">"s1"</span>
 159 <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">:</span><span class="number">33</span><span class="special">:</span> <span class="identifier">Entering</span> <span class="identifier">test</span> <span class="keyword">case</span> <span class="string">"test1"</span>
 160 <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">:</span><span class="number">35</span><span class="special">:</span> <span class="identifier">info</span><span class="special">:</span> <span class="identifier">check</span> <span class="keyword">true</span> <span class="identifier">has</span> <span class="identifier">passed</span>
 161 <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">:</span><span class="number">33</span><span class="special">:</span> <span class="identifier">Leaving</span> <span class="identifier">test</span> <span class="keyword">case</span> <span class="string">"test1"</span><span class="special">;</span> <span class="identifier">testing</span> <span class="identifier">time</span><span class="special">:</span> <span class="number">100u</span><span class="identifier">s</span>
 162 <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">:</span><span class="number">31</span><span class="special">:</span> <span class="identifier">Leaving</span> <span class="identifier">test</span> <span class="identifier">suite</span> <span class="string">"s1"</span><span class="special">;</span> <span class="identifier">testing</span> <span class="identifier">time</span><span class="special">:</span> <span class="number">129u</span><span class="identifier">s</span>
 163 <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">:</span><span class="number">16</span><span class="special">:</span> <span class="identifier">Entering</span> <span class="identifier">test</span> <span class="keyword">case</span> <span class="string">"test3"</span>
 164 <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">:</span><span class="number">18</span><span class="special">:</span> <span class="identifier">error</span><span class="special">:</span> <span class="identifier">in</span> <span class="string">"test3"</span><span class="special">:</span> <span class="identifier">check</span> <span class="keyword">false</span> <span class="identifier">has</span> <span class="identifier">failed</span>
 165 <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">:</span><span class="number">16</span><span class="special">:</span> <span class="identifier">Leaving</span> <span class="identifier">test</span> <span class="keyword">case</span> <span class="string">"test3"</span><span class="special">;</span> <span class="identifier">testing</span> <span class="identifier">time</span><span class="special">:</span> <span class="number">48u</span><span class="identifier">s</span>
 166 <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">:</span><span class="number">26</span><span class="special">:</span> <span class="identifier">Test</span> <span class="keyword">case</span> <span class="string">"test5"</span> <span class="identifier">is</span> <span class="identifier">skipped</span> <span class="identifier">because</span> <span class="identifier">dependency</span> <span class="identifier">test</span> <span class="keyword">case</span> <span class="string">"s1/test2"</span> <span class="identifier">is</span> <span class="identifier">disabled</span>
 167 <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">:</span><span class="number">21</span><span class="special">:</span> <span class="identifier">Test</span> <span class="keyword">case</span> <span class="string">"test4"</span> <span class="identifier">is</span> <span class="identifier">skipped</span> <span class="identifier">because</span> <span class="identifier">dependency</span> <span class="identifier">test</span> <span class="keyword">case</span> <span class="string">"test3"</span> <span class="identifier">has</span> <span class="identifier">failed</span>
 168 <span class="identifier">Leaving</span> <span class="identifier">test</span> <span class="identifier">module</span> <span class="string">"decorator_07"</span><span class="special">;</span> <span class="identifier">testing</span> <span class="identifier">time</span><span class="special">:</span> <span class="number">263u</span><span class="identifier">s</span>
 169
 170 <span class="identifier">Test</span> <span class="identifier">module</span> <span class="string">"decorator_07"</span> <span class="identifier">has</span> <span class="identifier">failed</span> <span class="identifier">with</span><span class="special">:</span>
 171   <span class="number">1</span> <span class="identifier">test</span> <span class="keyword">case</span> <span class="identifier">out</span> <span class="identifier">of</span> <span class="number">4</span> <span class="identifier">passed</span>
 172   <span class="number">1</span> <span class="identifier">test</span> <span class="keyword">case</span> <span class="identifier">out</span> <span class="identifier">of</span> <span class="number">4</span> <span class="identifier">failed</span>
 173   <span class="number">2</span> <span class="identifier">test</span> <span class="identifier">cases</span> <span class="identifier">out</span> <span class="identifier">of</span> <span class="number">4</span> <span class="identifier">skipped</span>
 174   <span class="number">1</span> <span class="identifier">assertion</span> <span class="identifier">out</span> <span class="identifier">of</span> <span class="number">2</span> <span class="identifier">passed</span>
 175   <span class="number">1</span> <span class="identifier">assertion</span> <span class="identifier">out</span> <span class="identifier">of</span> <span class="number">2</span> <span class="identifier">failed</span>
 176
 177   <span class="identifier">Test</span> <span class="keyword">case</span> <span class="string">"test3"</span> <span class="identifier">has</span> <span class="identifier">failed</span> <span class="identifier">with</span><span class="special">:</span>
 178     <span class="number">1</span> <span class="identifier">assertion</span> <span class="identifier">out</span> <span class="identifier">of</span> <span class="number">1</span> <span class="identifier">failed</span>
 179
 180   <span class="identifier">Test</span> <span class="keyword">case</span> <span class="string">"test4"</span> <span class="identifier">was</span> <span class="identifier">skipped</span>
 181   <span class="identifier">Test</span> <span class="keyword">case</span> <span class="string">"test5"</span> <span class="identifier">was</span> <span class="identifier">skipped</span>
 182   <span class="identifier">Test</span> <span class="identifier">suite</span> <span class="string">"s1"</span> <span class="identifier">has</span> <span class="identifier">passed</span> <span class="identifier">with</span><span class="special">:</span>
 183     <span class="number">1</span> <span class="identifier">test</span> <span class="keyword">case</span> <span class="identifier">out</span> <span class="identifier">of</span> <span class="number">1</span> <span class="identifier">passed</span>
 184     <span class="number">1</span> <span class="identifier">assertion</span> <span class="identifier">out</span> <span class="identifier">of</span> <span class="number">1</span> <span class="identifier">passed</span>
 185
 186     <span class="identifier">Test</span> <span class="keyword">case</span> <span class="string">"s1/test1"</span> <span class="identifier">has</span> <span class="identifier">passed</span> <span class="identifier">with</span><span class="special">:</span>
 187       <span class="number">1</span> <span class="identifier">assertion</span> <span class="identifier">out</span> <span class="identifier">of</span> <span class="number">1</span> <span class="identifier">passed</span>
 188 </pre>
 189               </td></tr></tbody>
 190 </table></div>
 191 <p>
 192         In the above scenario:
 193       </p>
 194 <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
 195 <li class="listitem">
 196             test case <code class="computeroutput"><span class="identifier">test3</span></code> is run
 197             (and fails) because <code class="computeroutput"><span class="identifier">s1</span><span class="special">/</span><span class="identifier">test1</span></code>
 198             has been run and succeeded,
 199           </li>
 200 <li class="listitem">
 201             <code class="computeroutput"><span class="identifier">test4</span></code> is skipped because
 202             <code class="computeroutput"><span class="identifier">test3</span></code> has failed,
 203           </li>
 204 <li class="listitem">
 205             <code class="computeroutput"><span class="identifier">test5</span></code> is skipped because
 206             <code class="computeroutput"><span class="identifier">s1</span><span class="special">/</span><span class="identifier">test2</span></code> is disabled.
 207           </li>
 208 </ul></div>
 209 </div>
 210 <table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
 211 <td align="left"></td>
 212 <td align="right"><div class="copyright-footer">Copyright &#169; 2001-2019 Boost.Test
 213       contributors<p>
 214         Distributed under the Boost Software License, Version 1.0. (See accompanying
 215         file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>)
 216       </p>
 217 </div></td>
 218 </tr></table>
 219 <hr>
 220 <div class="spirit-nav">
 221 <a accesskey="p" href="fixtures/global.html"><img src="../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../tests_organization.html"><img src="../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../index.html"><img src="../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="tests_grouping.html"><img src="../../../../../../doc/src/images/next.png" alt="Next"></a>
 222 </div>
 223 </body>
 224 </html>