libs/math/doc/html/math_toolkit/stat_tut/weg/st_eg/tut_mean_test.html

   1 <html>
   2 <head>
   3 <meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
   4 <title>Testing a sample mean for difference from a "true" mean</title>
   5 <link rel="stylesheet" href="../../../../math.css" type="text/css">
   6 <meta name="generator" content="DocBook XSL Stylesheets V1.79.1">
   7 <link rel="home" href="../../../../index.html" title="Math Toolkit 2.11.0">
   8 <link rel="up" href="../st_eg.html" title="Student's t Distribution Examples">
   9 <link rel="prev" href="tut_mean_intervals.html" title="Calculating confidence intervals on the mean with the Students-t distribution">
  10 <link rel="next" href="tut_mean_size.html" title="Estimating how large a sample size would have to become in order to give a significant Students-t test result with a single sample test">
  11 </head>
  12 <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
  13 <table cellpadding="2" width="100%"><tr>
  14 <td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../../../boost.png"></td>
  15 <td align="center"><a href="../../../../../../../../index.html">Home</a></td>
  16 <td align="center"><a href="../../../../../../../../libs/libraries.htm">Libraries</a></td>
  17 <td align="center"><a href="http://www.boost.org/users/people.html">People</a></td>
  18 <td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td>
  19 <td align="center"><a href="../../../../../../../../more/index.htm">More</a></td>
  20 </tr></table>
  21 <hr>
  22 <div class="spirit-nav">
  23 <a accesskey="p" href="tut_mean_intervals.html"><img src="../../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../st_eg.html"><img src="../../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../../index.html"><img src="../../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="tut_mean_size.html"><img src="../../../../../../../../doc/src/images/next.png" alt="Next"></a>
  24 </div>
  25 <div class="section">
  26 <div class="titlepage"><div><div><h5 class="title">
  27 <a name="math_toolkit.stat_tut.weg.st_eg.tut_mean_test"></a><a class="link" href="tut_mean_test.html" title='Testing a sample mean for difference from a "true" mean'>Testing
  28           a sample mean for difference from a "true" mean</a>
  29 </h5></div></div></div>
  30 <p>
  31             When calibrating or comparing a scientific instrument or measurement
  32             method of some kind, we want to be answer the question "Does an
  33             observed sample mean differ from the "true" mean in any significant
  34             way?". If it does, then we have evidence of a systematic difference.
  35             This question can be answered with a Students-t test: more information
  36             can be found <a href="http://www.itl.nist.gov/div898/handbook/eda/section3/eda352.htm" target="_top">on
  37             the NIST site</a>.
  38           </p>
  39 <p>
  40             Of course, the assignment of "true" to one mean may be quite
  41             arbitrary, often this is simply a "traditional" method of measurement.
  42           </p>
  43 <p>
  44             The following example code is taken from the example program <a href="../../../../../../example/students_t_single_sample.cpp" target="_top">students_t_single_sample.cpp</a>.
  45           </p>
  46 <p>
  47             We'll begin by defining a procedure to determine which of the possible
  48             hypothesis are rejected or not-rejected at a given significance level:
  49           </p>
  50 <div class="note"><table border="0" summary="Note">
  51 <tr>
  52 <td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../../../../../../doc/src/images/note.png"></td>
  53 <th align="left">Note</th>
  54 </tr>
  55 <tr><td align="left" valign="top"><p>
  56               Non-statisticians might say 'not-rejected' means 'accepted', (often
  57               of the null-hypothesis) implying, wrongly, that there really <span class="bold"><strong>IS</strong></span> no difference, but statisticans eschew this
  58               to avoid implying that there is positive evidence of 'no difference'.
  59               'Not-rejected' here means there is <span class="bold"><strong>no evidence</strong></span>
  60               of difference, but there still might well be a difference. For example,
  61               see <a href="http://en.wikipedia.org/wiki/Argument_from_ignorance" target="_top">argument
  62               from ignorance</a> and <a href="http://www.bmj.com/cgi/content/full/311/7003/485" target="_top">Absence
  63               of evidence does not constitute evidence of absence.</a>
  64             </p></td></tr>
  65 </table></div>
  66 <pre class="programlisting"><span class="comment">// Needed includes:</span>
  67 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">math</span><span class="special">/</span><span class="identifier">distributions</span><span class="special">/</span><span class="identifier">students_t</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
  68 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">iostream</span><span class="special">&gt;</span>
  69 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">iomanip</span><span class="special">&gt;</span>
  70 <span class="comment">// Bring everything into global namespace for ease of use:</span>
  71 <span class="keyword">using</span> <span class="keyword">namespace</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">math</span><span class="special">;</span>
  72 <span class="keyword">using</span> <span class="keyword">namespace</span> <span class="identifier">std</span><span class="special">;</span>
  73
  74 <span class="keyword">void</span> <span class="identifier">single_sample_t_test</span><span class="special">(</span><span class="keyword">double</span> <span class="identifier">M</span><span class="special">,</span> <span class="keyword">double</span> <span class="identifier">Sm</span><span class="special">,</span> <span class="keyword">double</span> <span class="identifier">Sd</span><span class="special">,</span> <span class="keyword">unsigned</span> <span class="identifier">Sn</span><span class="special">,</span> <span class="keyword">double</span> <span class="identifier">alpha</span><span class="special">)</span>
  75 <span class="special">{</span>
  76    <span class="comment">//</span>
  77    <span class="comment">// M = true mean.</span>
  78    <span class="comment">// Sm = Sample Mean.</span>
  79    <span class="comment">// Sd = Sample Standard Deviation.</span>
  80    <span class="comment">// Sn = Sample Size.</span>
  81    <span class="comment">// alpha = Significance Level.</span>
  82 </pre>
  83 <p>
  84             Most of the procedure is pretty-printing, so let's just focus on the
  85             calculation, we begin by calculating the t-statistic:
  86           </p>
  87 <pre class="programlisting"><span class="comment">// Difference in means:</span>
  88 <span class="keyword">double</span> <span class="identifier">diff</span> <span class="special">=</span> <span class="identifier">Sm</span> <span class="special">-</span> <span class="identifier">M</span><span class="special">;</span>
  89 <span class="comment">// Degrees of freedom:</span>
  90 <span class="keyword">unsigned</span> <span class="identifier">v</span> <span class="special">=</span> <span class="identifier">Sn</span> <span class="special">-</span> <span class="number">1</span><span class="special">;</span>
  91 <span class="comment">// t-statistic:</span>
  92 <span class="keyword">double</span> <span class="identifier">t_stat</span> <span class="special">=</span> <span class="identifier">diff</span> <span class="special">*</span> <span class="identifier">sqrt</span><span class="special">(</span><span class="keyword">double</span><span class="special">(</span><span class="identifier">Sn</span><span class="special">))</span> <span class="special">/</span> <span class="identifier">Sd</span><span class="special">;</span>
  93 </pre>
  94 <p>
  95             Finally calculate the probability from the t-statistic. If we're interested
  96             in simply whether there is a difference (either less or greater) or not,
  97             we don't care about the sign of the t-statistic, and we take the complement
  98             of the probability for comparison to the significance level:
  99           </p>
 100 <pre class="programlisting"><span class="identifier">students_t</span> <span class="identifier">dist</span><span class="special">(</span><span class="identifier">v</span><span class="special">);</span>
 101 <span class="keyword">double</span> <span class="identifier">q</span> <span class="special">=</span> <span class="identifier">cdf</span><span class="special">(</span><span class="identifier">complement</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span> <span class="identifier">fabs</span><span class="special">(</span><span class="identifier">t_stat</span><span class="special">)));</span>
 102 </pre>
 103 <p>
 104             The procedure then prints out the results of the various tests that can
 105             be done, these can be summarised in the following table:
 106           </p>
 107 <div class="informaltable"><table class="table">
 108 <colgroup>
 109 <col>
 110 <col>
 111 </colgroup>
 112 <thead><tr>
 113 <th>
 114                     <p>
 115                       Hypothesis
 116                     </p>
 117                   </th>
 118 <th>
 119                     <p>
 120                       Test
 121                     </p>
 122                   </th>
 123 </tr></thead>
 124 <tbody>
 125 <tr>
 126 <td>
 127                     <p>
 128                       The Null-hypothesis: there is <span class="bold"><strong>no difference</strong></span>
 129                       in means
 130                     </p>
 131                   </td>
 132 <td>
 133                     <p>
 134                       Reject if complement of CDF for |t| &lt; significance level
 135                       / 2:
 136                     </p>
 137                     <p>
 138                       <code class="computeroutput"><span class="identifier">cdf</span><span class="special">(</span><span class="identifier">complement</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span>
 139                       <span class="identifier">fabs</span><span class="special">(</span><span class="identifier">t</span><span class="special">)))</span>
 140                       <span class="special">&lt;</span> <span class="identifier">alpha</span>
 141                       <span class="special">/</span> <span class="number">2</span></code>
 142                     </p>
 143                   </td>
 144 </tr>
 145 <tr>
 146 <td>
 147                     <p>
 148                       The Alternative-hypothesis: there <span class="bold"><strong>is
 149                       difference</strong></span> in means
 150                     </p>
 151                   </td>
 152 <td>
 153                     <p>
 154                       Reject if complement of CDF for |t| &gt; significance level
 155                       / 2:
 156                     </p>
 157                     <p>
 158                       <code class="computeroutput"><span class="identifier">cdf</span><span class="special">(</span><span class="identifier">complement</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span>
 159                       <span class="identifier">fabs</span><span class="special">(</span><span class="identifier">t</span><span class="special">)))</span>
 160                       <span class="special">&gt;</span> <span class="identifier">alpha</span>
 161                       <span class="special">/</span> <span class="number">2</span></code>
 162                     </p>
 163                   </td>
 164 </tr>
 165 <tr>
 166 <td>
 167                     <p>
 168                       The Alternative-hypothesis: the sample mean <span class="bold"><strong>is
 169                       less</strong></span> than the true mean.
 170                     </p>
 171                   </td>
 172 <td>
 173                     <p>
 174                       Reject if CDF of t &gt; 1 - significance level:
 175                     </p>
 176                     <p>
 177                       <code class="computeroutput"><span class="identifier">cdf</span><span class="special">(</span><span class="identifier">complement</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span>
 178                       <span class="identifier">t</span><span class="special">))</span>
 179                       <span class="special">&lt;</span> <span class="identifier">alpha</span></code>
 180                     </p>
 181                   </td>
 182 </tr>
 183 <tr>
 184 <td>
 185                     <p>
 186                       The Alternative-hypothesis: the sample mean <span class="bold"><strong>is
 187                       greater</strong></span> than the true mean.
 188                     </p>
 189                   </td>
 190 <td>
 191                     <p>
 192                       Reject if complement of CDF of t &lt; significance level:
 193                     </p>
 194                     <p>
 195                       <code class="computeroutput"><span class="identifier">cdf</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span>
 196                       <span class="identifier">t</span><span class="special">)</span>
 197                       <span class="special">&lt;</span> <span class="identifier">alpha</span></code>
 198                     </p>
 199                   </td>
 200 </tr>
 201 </tbody>
 202 </table></div>
 203 <div class="note"><table border="0" summary="Note">
 204 <tr>
 205 <td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../../../../../../doc/src/images/note.png"></td>
 206 <th align="left">Note</th>
 207 </tr>
 208 <tr><td align="left" valign="top"><p>
 209               Notice that the comparisons are against <code class="computeroutput"><span class="identifier">alpha</span>
 210               <span class="special">/</span> <span class="number">2</span></code>
 211               for a two-sided test and against <code class="computeroutput"><span class="identifier">alpha</span></code>
 212               for a one-sided test
 213             </p></td></tr>
 214 </table></div>
 215 <p>
 216             Now that we have all the parts in place, let's take a look at some sample
 217             output, first using the <a href="http://www.itl.nist.gov/div898/handbook/eda/section4/eda428.htm" target="_top">Heat
 218             flow data</a> from the NIST site. The data set was collected by Bob
 219             Zarr of NIST in January, 1990 from a heat flow meter calibration and
 220             stability analysis. The corresponding dataplot output for this test can
 221             be found in <a href="http://www.itl.nist.gov/div898/handbook/eda/section3/eda352.htm" target="_top">section
 222             3.5.2</a> of the <a href="http://www.itl.nist.gov/div898/handbook/" target="_top">NIST/SEMATECH
 223             e-Handbook of Statistical Methods.</a>.
 224           </p>
 225 <pre class="programlisting">__________________________________
 226 Student t test for a single sample
 227 __________________________________
 228
 229 Number of Observations                                 =  195
 230 Sample Mean                                            =  9.26146
 231 Sample Standard Deviation                              =  0.02279
 232 Expected True Mean                                     =  5.00000
 233
 234 Sample Mean - Expected Test Mean                       =  4.26146
 235 Degrees of Freedom                                     =  194
 236 T Statistic                                            =  2611.28380
 237 Probability that difference is due to chance           =  0.000e+000
 238
 239 Results for Alternative Hypothesis and alpha           =  0.0500
 240
 241 Alternative Hypothesis     Conclusion
 242 Mean != 5.000            NOT REJECTED
 243 Mean  &lt; 5.000            REJECTED
 244 Mean  &gt; 5.000            NOT REJECTED
 245 </pre>
 246 <p>
 247             You will note the line that says the probability that the difference
 248             is due to chance is zero. From a philosophical point of view, of course,
 249             the probability can never reach zero. However, in this case the calculated
 250             probability is smaller than the smallest representable double precision
 251             number, hence the appearance of a zero here. Whatever its "true"
 252             value is, we know it must be extraordinarily small, so the alternative
 253             hypothesis - that there is a difference in means - is not rejected.
 254           </p>
 255 <p>
 256             For comparison the next example data output is taken from <span class="emphasis"><em>P.K.Hou,
 257             O. W. Lau &amp; M.C. Wong, Analyst (1983) vol. 108, p 64. and from Statistics
 258             for Analytical Chemistry, 3rd ed. (1994), pp 54-55 J. C. Miller and J.
 259             N. Miller, Ellis Horwood ISBN 0 13 0309907.</em></span> The values result
 260             from the determination of mercury by cold-vapour atomic absorption.
 261           </p>
 262 <pre class="programlisting">__________________________________
 263 Student t test for a single sample
 264 __________________________________
 265
 266 Number of Observations                                 =  3
 267 Sample Mean                                            =  37.80000
 268 Sample Standard Deviation                              =  0.96437
 269 Expected True Mean                                     =  38.90000
 270
 271 Sample Mean - Expected Test Mean                       =  -1.10000
 272 Degrees of Freedom                                     =  2
 273 T Statistic                                            =  -1.97566
 274 Probability that difference is due to chance           =  1.869e-001
 275
 276 Results for Alternative Hypothesis and alpha           =  0.0500
 277
 278 Alternative Hypothesis     Conclusion
 279 Mean != 38.900            REJECTED
 280 Mean  &lt; 38.900            NOT REJECTED
 281 Mean  &gt; 38.900            NOT REJECTED
 282 </pre>
 283 <p>
 284             As you can see the small number of measurements (3) has led to a large
 285             uncertainty in the location of the true mean. So even though there appears
 286             to be a difference between the sample mean and the expected true mean,
 287             we conclude that there is no significant difference, and are unable to
 288             reject the null hypothesis. However, if we were to lower the bar for
 289             acceptance down to alpha = 0.1 (a 90% confidence level) we see a different
 290             output:
 291           </p>
 292 <pre class="programlisting">__________________________________
 293 Student t test for a single sample
 294 __________________________________
 295
 296 Number of Observations                                 =  3
 297 Sample Mean                                            =  37.80000
 298 Sample Standard Deviation                              =  0.96437
 299 Expected True Mean                                     =  38.90000
 300
 301 Sample Mean - Expected Test Mean                       =  -1.10000
 302 Degrees of Freedom                                     =  2
 303 T Statistic                                            =  -1.97566
 304 Probability that difference is due to chance           =  1.869e-001
 305
 306 Results for Alternative Hypothesis and alpha           =  0.1000
 307
 308 Alternative Hypothesis     Conclusion
 309 Mean != 38.900            REJECTED
 310 Mean  &lt; 38.900            NOT REJECTED
 311 Mean  &gt; 38.900            REJECTED
 312 </pre>
 313 <p>
 314             In this case, we really have a borderline result, and more data (and/or
 315             more accurate data), is needed for a more convincing conclusion.
 316           </p>
 317 </div>
 318 <table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
 319 <td align="left"></td>
 320 <td align="right"><div class="copyright-footer">Copyright &#169; 2006-2019 Nikhar
 321       Agrawal, Anton Bikineev, Paul A. Bristow, Marco Guazzone, Christopher Kormanyos,
 322       Hubert Holin, Bruno Lalande, John Maddock, Jeremy Murphy, Matthew Pulver, Johan
 323       R&#229;de, Gautam Sewani, Benjamin Sobotta, Nicholas Thompson, Thijs van den Berg,
 324       Daryle Walker and Xiaogang Zhang<p>
 325         Distributed under the Boost Software License, Version 1.0. (See accompanying
 326         file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>)
 327       </p>
 328 </div></td>
 329 </tr></table>
 330 <hr>
 331 <div class="spirit-nav">
 332 <a accesskey="p" href="tut_mean_intervals.html"><img src="../../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../st_eg.html"><img src="../../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../../index.html"><img src="../../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="tut_mean_size.html"><img src="../../../../../../../../doc/src/images/next.png" alt="Next"></a>
 333 </div>
 334 </body>
 335 </html>