libs/test/doc/html/boost_test/testing_tools/tools_assertion_severity_level.html

   1 <html>
   2 <head>
   3 <meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
   4 <title>Assertion severity level</title>
   5 <link rel="stylesheet" href="../../boostbook.css" type="text/css">
   6 <meta name="generator" content="DocBook XSL Stylesheets V1.79.1">
   7 <link rel="home" href="../../index.html" title="Boost.Test">
   8 <link rel="up" href="../testing_tools.html" title="Writing unit tests">
   9 <link rel="prev" href="../testing_tools.html" title="Writing unit tests">
  10 <link rel="next" href="boost_test_universal_macro.html" title="BOOST_TEST: universal and general purpose assertions">
  11 </head>
  12 <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
  13 <table cellpadding="2" width="100%"><tr>
  14 <td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../boost.png"></td>
  15 <td align="center"><a href="../../../../../../index.html">Home</a></td>
  16 <td align="center"><a href="../../../../../../libs/libraries.htm">Libraries</a></td>
  17 <td align="center"><a href="http://www.boost.org/users/people.html">People</a></td>
  18 <td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td>
  19 <td align="center"><a href="../../../../../../more/index.htm">More</a></td>
  20 </tr></table>
  21 <hr>
  22 <div class="spirit-nav">
  23 <a accesskey="p" href="../testing_tools.html"><img src="../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../testing_tools.html"><img src="../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../index.html"><img src="../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="boost_test_universal_macro.html"><img src="../../../../../../doc/src/images/next.png" alt="Next"></a>
  24 </div>
  25 <div class="section">
  26 <div class="titlepage"><div><div><h3 class="title">
  27 <a name="boost_test.testing_tools.tools_assertion_severity_level"></a><a class="link" href="tools_assertion_severity_level.html" title="Assertion severity level">Assertion
  28       severity level</a>
  29 </h3></div></div></div>
  30 <p>
  31         There are three <span class="bold"><strong>levels</strong></span> of assertions and
  32         all the testing tools are supplied in these three flavours/levels. These
  33         levels have different meaning on the consistency of the test case:
  34       </p>
  35 <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
  36 <li class="listitem">
  37             <code class="computeroutput"><span class="identifier">REQUIRE</span></code> which implements
  38             a <span class="bold"><strong>requirements</strong></span> : this is a strong condition
  39             for the operations following the assertion to be valid. This type of
  40             assertions should be used when a pre-condition for running the test is
  41             not met or when the test-case cannot continue. If such as assertion fails,
  42             the test case execution stops immediately, and the test-case is flagged
  43             as <span class="emphasis"><em>failed</em></span>.
  44           </li>
  45 <li class="listitem">
  46             <code class="computeroutput"><span class="identifier">CHECK</span></code> for standard <span class="bold"><strong>checks</strong></span>: this is the most commonly used assertion
  47             level. If the statement evaluates to <code class="computeroutput"><span class="keyword">false</span></code>,
  48             the test case is flagged as failed but its execution continues.
  49           </li>
  50 <li class="listitem">
  51             <code class="computeroutput"><span class="identifier">WARN</span></code> which stands for
  52             <span class="bold"><strong>warnings</strong></span>: this is an assertion providing
  53             information. The test case execution continues and a warning message
  54             is logged. The warning does not change the success status of a test case.
  55             This level of assertion can be used to validate aspects less important
  56             then correctness: performance, portability, usability, etc.
  57           </li>
  58 </ul></div>
  59 <p>
  60         For example:
  61       </p>
  62 <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
  63 <li class="listitem">
  64             <a class="link" href="../utf_reference/testing_tool_ref/assertion_boost_level_throw.html" title="BOOST_&lt;level&gt;_THROW"><code class="computeroutput"><span class="identifier">BOOST_REQUIRE_THROW</span></code></a>, <a class="link" href="../utf_reference/testing_tool_ref/assertion_boost_test_universal_macro.html" title="BOOST_TEST"><code class="computeroutput"><span class="identifier">BOOST_TEST_REQUIRE</span></code></a>
  65           </li>
  66 <li class="listitem">
  67             <code class="computeroutput"><span class="identifier">BOOST_CHECK_THROW</span></code>, <code class="computeroutput"><span class="identifier">BOOST_TEST</span></code> <a href="#ftn.boost_test.testing_tools.tools_assertion_severity_level.f0" class="footnote" name="boost_test.testing_tools.tools_assertion_severity_level.f0"><sup class="footnote">[6]</sup></a>
  68           </li>
  69 <li class="listitem">
  70             <code class="computeroutput"><span class="identifier">BOOST_WARN_THROW</span></code>, <code class="computeroutput"><span class="identifier">BOOST_TEST_WARN</span></code>
  71           </li>
  72 </ul></div>
  73 <p>
  74         These three levels of assertions are filtered by the framework and reported
  75         into the test log and output:
  76       </p>
  77 <div class="orderedlist"><ol class="orderedlist" type="1">
  78 <li class="listitem">
  79             If an assertion designated by the tool passes, confirmation message can
  80             be printed in log output <a href="#ftn.boost_test.testing_tools.tools_assertion_severity_level.f1" class="footnote" name="boost_test.testing_tools.tools_assertion_severity_level.f1"><sup class="footnote">[7]</sup></a>.
  81           </li>
  82 <li class="listitem">
  83             If an assertion designated by the tool fails, the following will happen,
  84             depending on the assertion level <a href="#ftn.boost_test.testing_tools.tools_assertion_severity_level.f2" class="footnote" name="boost_test.testing_tools.tools_assertion_severity_level.f2"><sup class="footnote">[8]</sup></a>:
  85           </li>
  86 </ol></div>
  87 <div class="table">
  88 <a name="boost_test.testing_tools.tools_assertion_severity_level.assertions_severity_levels"></a><p class="title"><b>Table&#160;3.&#160;Assertions severity levels</b></p>
  89 <div class="table-contents"><table class="table" summary="Assertions severity levels">
  90 <colgroup>
  91 <col>
  92 <col>
  93 <col>
  94 <col>
  95 </colgroup>
  96 <thead><tr>
  97 <th>
  98                 <p>
  99                   Level
 100                 </p>
 101               </th>
 102 <th>
 103                 <p>
 104                   Test log content
 105                 </p>
 106               </th>
 107 <th>
 108                 <p>
 109                   Errors counter
 110                 </p>
 111               </th>
 112 <th>
 113                 <p>
 114                   Test execution
 115                 </p>
 116               </th>
 117 </tr></thead>
 118 <tbody>
 119 <tr>
 120 <td>
 121                 <p>
 122                   WARN
 123                 </p>
 124               </td>
 125 <td>
 126                 <p>
 127                   warning in <code class="computeroutput"><span class="special">&lt;</span><span class="identifier">test</span><span class="special">-</span><span class="keyword">case</span><span class="special">-</span><span class="identifier">name</span><span class="special">&gt;</span></code>: condition <code class="computeroutput"><span class="special">&lt;</span><span class="identifier">assertion</span> <span class="identifier">description</span><span class="special">&gt;</span></code> is not satisfied
 128                 </p>
 129               </td>
 130 <td>
 131                 <p>
 132                   not affected
 133                 </p>
 134               </td>
 135 <td>
 136                 <p>
 137                   continues
 138                 </p>
 139               </td>
 140 </tr>
 141 <tr>
 142 <td>
 143                 <p>
 144                   CHECK
 145                 </p>
 146               </td>
 147 <td>
 148                 <p>
 149                   error in <code class="computeroutput"><span class="special">&lt;</span><span class="identifier">test</span><span class="special">-</span><span class="keyword">case</span><span class="special">-</span><span class="identifier">name</span><span class="special">&gt;</span></code>: test <code class="computeroutput"><span class="special">&lt;</span><span class="identifier">assertion</span> <span class="identifier">description</span><span class="special">&gt;</span></code> failed
 150                 </p>
 151               </td>
 152 <td>
 153                 <p>
 154                   increased
 155                 </p>
 156               </td>
 157 <td>
 158                 <p>
 159                   continues
 160                 </p>
 161               </td>
 162 </tr>
 163 <tr>
 164 <td>
 165                 <p>
 166                   REQUIRE
 167                 </p>
 168               </td>
 169 <td>
 170                 <p>
 171                   fatal error in <code class="computeroutput"><span class="special">&lt;</span><span class="identifier">test</span><span class="special">-</span><span class="keyword">case</span><span class="special">-</span><span class="identifier">name</span><span class="special">&gt;</span></code>:
 172                   critical test <code class="computeroutput"><span class="special">&lt;</span><span class="identifier">assertion</span> <span class="identifier">description</span><span class="special">&gt;</span></code> failed
 173                 </p>
 174               </td>
 175 <td>
 176                 <p>
 177                   increased
 178                 </p>
 179               </td>
 180 <td>
 181                 <p>
 182                   aborts
 183                 </p>
 184               </td>
 185 </tr>
 186 </tbody>
 187 </table></div>
 188 </div>
 189 <br class="table-break"><p>
 190         The granularity of the report depends on the current <a class="link" href="../utf_reference/rt_param_reference/log_level.html" title="log_level">log
 191         level</a> and <a class="link" href="../utf_reference/rt_param_reference/report_level.html" title="report_level">report
 192         level</a>.
 193       </p>
 194 <div class="note"><table border="0" summary="Note">
 195 <tr>
 196 <td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../../../../doc/src/images/note.png"></td>
 197 <th align="left">Note</th>
 198 </tr>
 199 <tr><td align="left" valign="top"><p>
 200           in the above table, the <span class="emphasis"><em>test execution</em></span> is related
 201           to the current test case <span class="emphasis"><em>only</em></span>. Hence <span class="emphasis"><em>"aborts"</em></span>
 202           means that the current test case is aborted, but other test cases in the
 203           test tree are still executed.
 204         </p></td></tr>
 205 </table></div>
 206 <div class="footnotes">
 207 <br><hr style="width:100; text-align:left;margin-left: 0">
 208 <div id="ftn.boost_test.testing_tools.tools_assertion_severity_level.f0" class="footnote"><p><a href="#boost_test.testing_tools.tools_assertion_severity_level.f0" class="para"><sup class="para">[6] </sup></a>
 209               <a class="link" href="../utf_reference/testing_tool_ref/assertion_boost_test_universal_macro.html" title="BOOST_TEST"><code class="computeroutput"><span class="identifier">BOOST_TEST</span></code></a> is equivalent to
 210               <code class="computeroutput"><span class="identifier">BOOST_TEST_CHECK</span></code>
 211             </p></div>
 212 <div id="ftn.boost_test.testing_tools.tools_assertion_severity_level.f1" class="footnote"><p><a href="#boost_test.testing_tools.tools_assertion_severity_level.f1" class="para"><sup class="para">[7] </sup></a>
 213               to manage what messages appear in the test log stream, set the proper
 214               <a class="link" href="../utf_reference/rt_param_reference/log_level.html" title="log_level"><code class="computeroutput"><span class="identifier">log_level</span></code></a>
 215             </p></div>
 216 <div id="ftn.boost_test.testing_tools.tools_assertion_severity_level.f2" class="footnote"><p><a href="#boost_test.testing_tools.tools_assertion_severity_level.f2" class="para"><sup class="para">[8] </sup></a>
 217               in some cases log message can be slightly different to reflect failed
 218               tool specifics, see <a class="link" href="reports.html" title="Reported information">here</a>
 219             </p></div>
 220 </div>
 221 </div>
 222 <table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
 223 <td align="left"></td>
 224 <td align="right"><div class="copyright-footer">Copyright &#169; 2001-2019 Boost.Test
 225       contributors<p>
 226         Distributed under the Boost Software License, Version 1.0. (See accompanying
 227         file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>)
 228       </p>
 229 </div></td>
 230 </tr></table>
 231 <hr>
 232 <div class="spirit-nav">
 233 <a accesskey="p" href="../testing_tools.html"><img src="../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../testing_tools.html"><img src="../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../index.html"><img src="../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="boost_test_universal_macro.html"><img src="../../../../../../doc/src/images/next.png" alt="Next"></a>
 234 </div>
 235 </body>
 236 </html>