libs/test/doc/html/boost_test/testing_tools/output_stream_testing.html

   1 <html>
   2 <head>
   3 <meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
   4 <title>Output streams testing tool</title>
   5 <link rel="stylesheet" href="../../boostbook.css" type="text/css">
   6 <meta name="generator" content="DocBook XSL Stylesheets V1.79.1">
   7 <link rel="home" href="../../index.html" title="Boost.Test">
   8 <link rel="up" href="../testing_tools.html" title="Writing unit tests">
   9 <link rel="prev" href="custom_predicates.html" title="Custom predicate support">
  10 <link rel="next" href="internal_details.html" title="BOOST_TEST: details on expressions">
  11 </head>
  12 <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
  13 <table cellpadding="2" width="100%"><tr>
  14 <td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../boost.png"></td>
  15 <td align="center"><a href="../../../../../../index.html">Home</a></td>
  16 <td align="center"><a href="../../../../../../libs/libraries.htm">Libraries</a></td>
  17 <td align="center"><a href="http://www.boost.org/users/people.html">People</a></td>
  18 <td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td>
  19 <td align="center"><a href="../../../../../../more/index.htm">More</a></td>
  20 </tr></table>
  21 <hr>
  22 <div class="spirit-nav">
  23 <a accesskey="p" href="custom_predicates.html"><img src="../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../testing_tools.html"><img src="../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../index.html"><img src="../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="internal_details.html"><img src="../../../../../../doc/src/images/next.png" alt="Next"></a>
  24 </div>
  25 <div class="section">
  26 <div class="titlepage"><div><div><h3 class="title">
  27 <a name="boost_test.testing_tools.output_stream_testing"></a><a class="link" href="output_stream_testing.html" title="Output streams testing tool">Output
  28       streams testing tool</a>
  29 </h3></div></div></div>
  30 <p>
  31         How would you perform correctness test for
  32       </p>
  33 <pre class="programlisting"><span class="keyword">operator</span><span class="special">&lt;&lt;</span> <span class="special">(</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">ostream</span> <span class="special">&amp;,</span> <span class="special">...</span> <span class="special">)</span></pre>
  34 <p>
  35         operations?
  36       </p>
  37 <p>
  38         You can print into the standard output stream and manually check that it
  39         is matching your expectations. Unfortunately, this is not really acceptable
  40         for the regression testing and doesn't serve a long term purpose of a unit
  41         test.
  42       </p>
  43 <p>
  44         You can use <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">stringstream</span></code> and compare resulting output
  45         buffer with the expected pattern string, but you are required to perform
  46         several additional operations with every check you do. So it becomes tedious
  47         very fast.
  48       </p>
  49 <p>
  50         The class <code class="computeroutput"><a class="link" href="../../boost/test_tools/output_test_stream.html" title="Class output_test_stream">boost::test_tools::output_test_stream</a></code>
  51         is designed to automate these tasks for you. This is a simple, but powerful
  52         tool for testing standard <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">ostream</span></code>
  53         based output operation. The class <code class="computeroutput"><span class="identifier">output_test_stream</span></code>
  54         complies to <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">ostream</span></code> interface so it can be used in
  55         place of any <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">ostream</span></code> parameter. It provides several
  56         test methods to validate output content, including test for match to expected
  57         output content or test for expected output length. Flushing, synchronizing,
  58         string comparison and error message generation is automated by the tool implementation.
  59       </p>
  60 <p>
  61         All <code class="computeroutput"><span class="identifier">output_test_stream</span></code> validation
  62         member functions by default flush the stream once the check is performed.
  63         If you want to perform several checks with the same output, specify parameter
  64         <code class="computeroutput"><span class="identifier">flush_stream</span></code> with value
  65         <code class="computeroutput"><span class="keyword">false</span></code> <a href="#ftn.boost_test.testing_tools.output_stream_testing.f0" class="footnote" name="boost_test.testing_tools.output_stream_testing.f0"><sup class="footnote">[16]</sup></a>.
  66       </p>
  67 <p>
  68         In some cases manual generation of expected output is either too time consuming
  69         or is impossible at all because of sheer volume. A possible way to address
  70         that issue is to split the test in two steps:
  71       </p>
  72 <div class="orderedlist"><ol class="orderedlist" type="1">
  73 <li class="listitem">
  74             first by checking the expected output manually
  75           </li>
  76 <li class="listitem">
  77             second to save this output to ensure that future checks produce the same
  78             output
  79           </li>
  80 </ol></div>
  81 <p>
  82         The class <code class="computeroutput"><span class="identifier">output_test_stream</span></code>
  83         allows both the matching of the output content versus a <span class="emphasis"><em>pattern
  84         file</em></span> and generation of this pattern file. The command line parameter
  85         <a class="link" href="../utf_reference/rt_param_reference/save_pattern.html" title="save_pattern"><code class="computeroutput"><span class="identifier">save_pattern</span></code></a> may be used to either
  86         generate a new pattern file, or to check against an existing pattern.
  87       </p>
  88 <h4>
  89 <a name="boost_test.testing_tools.output_stream_testing.h0"></a>
  90         <span class="phrase"><a name="boost_test.testing_tools.output_stream_testing.usages"></a></span><a class="link" href="output_stream_testing.html#boost_test.testing_tools.output_stream_testing.usages">Usage</a>
  91       </h4>
  92 <p>
  93         There are two ways to employ the class <code class="computeroutput"><span class="identifier">output_test_stream</span></code>:
  94       </p>
  95 <div class="orderedlist"><ol class="orderedlist" type="1">
  96 <li class="listitem">
  97             explicit output checks and
  98           </li>
  99 <li class="listitem">
 100             pattern file matching
 101           </li>
 102 </ol></div>
 103 <h5>
 104 <a name="boost_test.testing_tools.output_stream_testing.h1"></a>
 105         <span class="phrase"><a name="boost_test.testing_tools.output_stream_testing.explicit_output_checks"></a></span><a class="link" href="output_stream_testing.html#boost_test.testing_tools.output_stream_testing.explicit_output_checks">Explicit
 106         output checks</a>
 107       </h5>
 108 <p>
 109         Use the instance of class <code class="computeroutput"><span class="identifier">output_test_stream</span></code>
 110         as an output stream and check output content using tool's methods.
 111       </p>
 112 <h6>
 113 <a name="boost_test.testing_tools.output_stream_testing.h2"></a>
 114         <span class="phrase"><a name="boost_test.testing_tools.output_stream_testing.example_descr"></a></span><a class="link" href="output_stream_testing.html#boost_test.testing_tools.output_stream_testing.example_descr">Example:
 115         Explicit output checks with <code class="computeroutput"><span class="identifier">output_test_stream</span></code></a>
 116       </h6>
 117 <div class="informaltable"><table class="table">
 118 <colgroup><col></colgroup>
 119 <thead><tr><th>
 120                 <p>
 121                   Code
 122                 </p>
 123               </th></tr></thead>
 124 <tbody><tr><td>
 125 <pre xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" class="table-programlisting"><span class="preprocessor">#define</span> <span class="identifier">BOOST_TEST_MODULE</span> <span class="identifier">example</span>
 126 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">test</span><span class="special">/</span><span class="identifier">included</span><span class="special">/</span><span class="identifier">unit_test</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 127 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">test</span><span class="special">/</span><span class="identifier">tools</span><span class="special">/</span><span class="identifier">output_test_stream</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 128 <span class="keyword">using</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">test_tools</span><span class="special">::</span><span class="identifier">output_test_stream</span><span class="special">;</span>
 129
 130 <span class="identifier">BOOST_AUTO_TEST_CASE</span><span class="special">(</span> <span class="identifier">test</span> <span class="special">)</span>
 131 <span class="special">{</span>
 132   <span class="identifier">output_test_stream</span> <span class="identifier">output</span><span class="special">;</span>
 133   <span class="keyword">int</span> <span class="identifier">i</span><span class="special">=</span><span class="number">2</span><span class="special">;</span>
 134   <span class="identifier">output</span> <span class="special">&lt;&lt;</span> <span class="string">"i="</span> <span class="special">&lt;&lt;</span> <span class="identifier">i</span><span class="special">;</span>
 135   <span class="identifier">BOOST_TEST</span><span class="special">(</span> <span class="special">!</span><span class="identifier">output</span><span class="special">.</span><span class="identifier">is_empty</span><span class="special">(</span> <span class="keyword">false</span> <span class="special">)</span> <span class="special">);</span>
 136   <span class="identifier">BOOST_TEST</span><span class="special">(</span> <span class="identifier">output</span><span class="special">.</span><span class="identifier">check_length</span><span class="special">(</span> <span class="number">3</span><span class="special">,</span> <span class="keyword">false</span> <span class="special">)</span> <span class="special">);</span>
 137   <span class="identifier">BOOST_TEST</span><span class="special">(</span> <span class="identifier">output</span><span class="special">.</span><span class="identifier">is_equal</span><span class="special">(</span> <span class="string">"i=3"</span> <span class="special">)</span> <span class="special">);</span>
 138 <span class="special">}</span>
 139 </pre>
 140               </td></tr></tbody>
 141 </table></div>
 142 <div class="informaltable"><table class="table">
 143 <colgroup><col></colgroup>
 144 <thead><tr><th>
 145                 <p>
 146                   Output
 147                 </p>
 148               </th></tr></thead>
 149 <tbody><tr><td>
 150 <pre xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" class="table-programlisting"><span class="special">&gt;</span> <span class="identifier">example</span>
 151 <span class="identifier">Running</span> <span class="number">1</span> <span class="identifier">test</span> <span class="keyword">case</span><span class="special">...</span>
 152 <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">15</span><span class="special">):</span> <span class="identifier">error</span> <span class="identifier">in</span> <span class="string">"test"</span><span class="special">:</span> <span class="identifier">check</span> <span class="identifier">output</span><span class="special">.</span><span class="identifier">is_equal</span><span class="special">(</span> <span class="string">"i=3"</span> <span class="special">)</span> <span class="identifier">has</span> <span class="identifier">failed</span><span class="special">.</span> <span class="identifier">Output</span> <span class="identifier">content</span><span class="special">:</span> <span class="string">"i=2"</span>
 153
 154 <span class="special">***</span> <span class="number">1</span> <span class="identifier">failures</span> <span class="identifier">is</span> <span class="identifier">detected</span> <span class="identifier">in</span> <span class="identifier">test</span> <span class="identifier">suite</span> <span class="string">"example"</span>
 155 </pre>
 156               </td></tr></tbody>
 157 </table></div>
 158 <div class="note"><table border="0" summary="Note">
 159 <tr>
 160 <td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../../../../doc/src/images/note.png"></td>
 161 <th align="left">Note</th>
 162 </tr>
 163 <tr><td align="left" valign="top"><p>
 164           Use of <code class="computeroutput"><span class="keyword">false</span></code> to prevent output
 165           flushing in first two invocation of check functions. Unless you want to
 166           perform several different checks for the same output you wouldn't need
 167           to use it though. Your test will look like a sequence of output operators
 168           followed by one check.
 169         </p></td></tr>
 170 </table></div>
 171 <div class="tip"><table border="0" summary="Tip">
 172 <tr>
 173 <td rowspan="2" align="center" valign="top" width="25"><img alt="[Tip]" src="../../../../../../doc/src/images/tip.png"></td>
 174 <th align="left">Tip</th>
 175 </tr>
 176 <tr><td align="left" valign="top"><p>
 177           Try to perform checks as frequently as possible. It not only simplifies
 178           patterns you compare with, but also allows you to more closely identify
 179           possible source of failure.
 180         </p></td></tr>
 181 </table></div>
 182 <h5>
 183 <a name="boost_test.testing_tools.output_stream_testing.h3"></a>
 184         <span class="phrase"><a name="boost_test.testing_tools.output_stream_testing.pattern_file_matching"></a></span><a class="link" href="output_stream_testing.html#boost_test.testing_tools.output_stream_testing.pattern_file_matching">Pattern
 185         file matching</a>
 186       </h5>
 187 <p>
 188         The <span class="emphasis"><em>pattern file</em></span> is a companion file containing the
 189         patterns that the stream should match. Your testing will look like a series
 190         of output operators followed by match pattern checks repeated several times.
 191       </p>
 192 <p>
 193         In the example below, the file <code class="computeroutput"><span class="identifier">pattern_file</span></code>
 194         contains the patterns that should match.
 195       </p>
 196 <pre class="programlisting">i=2
 197 File: test.cpp Line:XXX
 198 </pre>
 199 <h6>
 200 <a name="boost_test.testing_tools.output_stream_testing.h4"></a>
 201         <span class="phrase"><a name="boost_test.testing_tools.output_stream_testing.example_descr0"></a></span><a class="link" href="output_stream_testing.html#boost_test.testing_tools.output_stream_testing.example_descr0">Example:
 202         Pattern file matching with <code class="computeroutput"><span class="identifier">output_test_stream</span></code></a>
 203       </h6>
 204 <div class="informaltable"><table class="table">
 205 <colgroup><col></colgroup>
 206 <thead><tr><th>
 207                 <p>
 208                   Code
 209                 </p>
 210               </th></tr></thead>
 211 <tbody><tr><td>
 212 <pre xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" class="table-programlisting"><span class="preprocessor">#define</span> <span class="identifier">BOOST_TEST_MODULE</span> <span class="identifier">example</span>
 213 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">test</span><span class="special">/</span><span class="identifier">included</span><span class="special">/</span><span class="identifier">unit_test</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 214 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">test</span><span class="special">/</span><span class="identifier">tools</span><span class="special">/</span><span class="identifier">output_test_stream</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 215 <span class="keyword">using</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">test_tools</span><span class="special">::</span><span class="identifier">output_test_stream</span><span class="special">;</span>
 216
 217 <span class="identifier">BOOST_AUTO_TEST_CASE</span><span class="special">(</span> <span class="identifier">test</span> <span class="special">)</span>
 218 <span class="special">{</span>
 219   <span class="identifier">output_test_stream</span> <span class="identifier">output</span><span class="special">(</span> <span class="string">"pattern_file"</span><span class="special">,</span> <span class="special">!</span><span class="identifier">boost</span><span class="special">::</span><span class="identifier">unit_test</span><span class="special">::</span><span class="identifier">runtime_config</span><span class="special">::</span><span class="identifier">save_pattern</span><span class="special">()</span> <span class="special">);</span>
 220   <span class="keyword">int</span> <span class="identifier">i</span><span class="special">=</span><span class="number">2</span><span class="special">;</span>
 221   <span class="identifier">output</span> <span class="special">&lt;&lt;</span> <span class="string">"i="</span> <span class="special">&lt;&lt;</span> <span class="identifier">i</span><span class="special">;</span>
 222   <span class="identifier">BOOST_TEST</span><span class="special">(</span> <span class="identifier">output</span><span class="special">.</span><span class="identifier">match_pattern</span><span class="special">()</span> <span class="special">);</span>
 223
 224   <span class="identifier">output</span> <span class="special">&lt;&lt;</span> <span class="string">"\nFile: "</span> <span class="special">&lt;&lt;</span> <span class="identifier">__FILE__</span> <span class="special">&lt;&lt;</span> <span class="string">" Line:YYY"</span><span class="special">;</span>
 225   <span class="identifier">BOOST_TEST</span><span class="special">(</span> <span class="identifier">output</span><span class="special">.</span><span class="identifier">match_pattern</span><span class="special">()</span> <span class="special">);</span> <a class="co" name="boost_test.testing_tools.output_stream_testing.c0" href="output_stream_testing.html#boost_test.testing_tools.output_stream_testing.c1"><img src="../../../../../../doc/src/images/callouts/1.png" alt="1" border="0"></a>
 226 <span class="special">}</span>
 227 </pre>
 228                 <div class="calloutlist"><table border="0" summary="Callout list"><tr>
 229 <td width="5%" valign="top" align="left"><p><a name="boost_test.testing_tools.output_stream_testing.c1"></a><a href="#boost_test.testing_tools.output_stream_testing.c0"><img src="../../../../../../doc/src/images/callouts/1.png" alt="1" border="0"></a> </p></td>
 230 <td valign="top" align="left"><p>
 231                       This line generates the error Line:YYY != Line:XXX
 232                     </p></td>
 233 </tr></table></div>
 234               </td></tr></tbody>
 235 </table></div>
 236 <div class="informaltable"><table class="table">
 237 <colgroup><col></colgroup>
 238 <thead><tr><th>
 239                 <p>
 240                   Output
 241                 </p>
 242               </th></tr></thead>
 243 <tbody><tr><td>
 244 <pre xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" class="table-programlisting"><span class="special">&gt;</span> <span class="identifier">example</span>
 245 <span class="identifier">Running</span> <span class="number">1</span> <span class="identifier">test</span> <span class="keyword">case</span><span class="special">...</span>
 246 <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">16</span><span class="special">):</span> <span class="identifier">error</span> <span class="identifier">in</span> <span class="string">"test"</span><span class="special">:</span> <span class="identifier">check</span> <span class="identifier">output</span><span class="special">.</span><span class="identifier">match_pattern</span><span class="special">()</span> <span class="identifier">has</span> <span class="identifier">failed</span><span class="special">.</span> <span class="identifier">Mismatch</span> <span class="identifier">at</span> <span class="identifier">position</span> <span class="number">23</span>
 247 <span class="special">...</span><span class="number">5.</span><span class="special">..</span>
 248 <span class="special">...</span><span class="number">4.</span><span class="special">..</span>
 249
 250 <span class="special">***</span> <span class="number">1</span> <span class="identifier">failures</span> <span class="identifier">is</span> <span class="identifier">detected</span> <span class="identifier">in</span> <span class="identifier">test</span> <span class="identifier">suite</span> <span class="string">"example"</span>
 251 </pre>
 252               </td></tr></tbody>
 253 </table></div>
 254 <div class="tip"><table border="0" summary="Tip">
 255 <tr>
 256 <td rowspan="2" align="center" valign="top" width="25"><img alt="[Tip]" src="../../../../../../doc/src/images/tip.png"></td>
 257 <th align="left">Tip</th>
 258 </tr>
 259 <tr><td align="left" valign="top"><p>
 260           Try to perform checks as frequently as possible, because it allows you
 261           to more closely identify possible source of failure
 262         </p></td></tr>
 263 </table></div>
 264 <div class="footnotes">
 265 <br><hr style="width:100; text-align:left;margin-left: 0">
 266 <div id="ftn.boost_test.testing_tools.output_stream_testing.f0" class="footnote"><p><a href="#boost_test.testing_tools.output_stream_testing.f0" class="para"><sup class="para">[16] </sup></a>
 267           This parameter is supported on all comparison methods, see the class <code class="computeroutput"><a class="link" href="../../boost/test_tools/output_test_stream.html" title="Class output_test_stream">documentation.</a></code>
 268         </p></div>
 269 </div>
 270 </div>
 271 <table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
 272 <td align="left"></td>
 273 <td align="right"><div class="copyright-footer">Copyright &#169; 2001-2019 Boost.Test
 274       contributors<p>
 275         Distributed under the Boost Software License, Version 1.0. (See accompanying
 276         file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>)
 277       </p>
 278 </div></td>
 279 </tr></table>
 280 <hr>
 281 <div class="spirit-nav">
 282 <a accesskey="p" href="custom_predicates.html"><img src="../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../testing_tools.html"><img src="../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../index.html"><img src="../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="internal_details.html"><img src="../../../../../../doc/src/images/next.png" alt="Next"></a>
 283 </div>
 284 </body>
 285 </html>