libs/math/doc/html/math_toolkit/stat_tut/weg/cs_eg/chi_sq_intervals.html

   1 <html>
   2 <head>
   3 <meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
   4 <title>Confidence Intervals on the Standard Deviation</title>
   5 <link rel="stylesheet" href="../../../../math.css" type="text/css">
   6 <meta name="generator" content="DocBook XSL Stylesheets V1.79.1">
   7 <link rel="home" href="../../../../index.html" title="Math Toolkit 2.11.0">
   8 <link rel="up" href="../cs_eg.html" title="Chi Squared Distribution Examples">
   9 <link rel="prev" href="../cs_eg.html" title="Chi Squared Distribution Examples">
  10 <link rel="next" href="chi_sq_test.html" title="Chi-Square Test for the Standard Deviation">
  11 </head>
  12 <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
  13 <table cellpadding="2" width="100%"><tr>
  14 <td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../../../boost.png"></td>
  15 <td align="center"><a href="../../../../../../../../index.html">Home</a></td>
  16 <td align="center"><a href="../../../../../../../../libs/libraries.htm">Libraries</a></td>
  17 <td align="center"><a href="http://www.boost.org/users/people.html">People</a></td>
  18 <td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td>
  19 <td align="center"><a href="../../../../../../../../more/index.htm">More</a></td>
  20 </tr></table>
  21 <hr>
  22 <div class="spirit-nav">
  23 <a accesskey="p" href="../cs_eg.html"><img src="../../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../cs_eg.html"><img src="../../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../../index.html"><img src="../../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="chi_sq_test.html"><img src="../../../../../../../../doc/src/images/next.png" alt="Next"></a>
  24 </div>
  25 <div class="section">
  26 <div class="titlepage"><div><div><h5 class="title">
  27 <a name="math_toolkit.stat_tut.weg.cs_eg.chi_sq_intervals"></a><a class="link" href="chi_sq_intervals.html" title="Confidence Intervals on the Standard Deviation">Confidence
  28           Intervals on the Standard Deviation</a>
  29 </h5></div></div></div>
  30 <p>
  31             Once you have calculated the standard deviation for your data, a legitimate
  32             question to ask is "How reliable is the calculated standard deviation?".
  33             For this situation the Chi Squared distribution can be used to calculate
  34             confidence intervals for the standard deviation.
  35           </p>
  36 <p>
  37             The full example code &amp; sample output is in <a href="../../../../../../example/chi_square_std_dev_test.cpp" target="_top">chi_square_std_dev_test.cpp</a>.
  38           </p>
  39 <p>
  40             We'll begin by defining the procedure that will calculate and print out
  41             the confidence intervals:
  42           </p>
  43 <pre class="programlisting"><span class="keyword">void</span> <span class="identifier">confidence_limits_on_std_deviation</span><span class="special">(</span>
  44      <span class="keyword">double</span> <span class="identifier">Sd</span><span class="special">,</span>    <span class="comment">// Sample Standard Deviation</span>
  45      <span class="keyword">unsigned</span> <span class="identifier">N</span><span class="special">)</span>   <span class="comment">// Sample size</span>
  46 <span class="special">{</span>
  47 </pre>
  48 <p>
  49             We'll begin by printing out some general information:
  50           </p>
  51 <pre class="programlisting"><span class="identifier">cout</span> <span class="special">&lt;&lt;</span>
  52    <span class="string">"________________________________________________\n"</span>
  53    <span class="string">"2-Sided Confidence Limits For Standard Deviation\n"</span>
  54    <span class="string">"________________________________________________\n\n"</span><span class="special">;</span>
  55 <span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="identifier">setprecision</span><span class="special">(</span><span class="number">7</span><span class="special">);</span>
  56 <span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="identifier">setw</span><span class="special">(</span><span class="number">40</span><span class="special">)</span> <span class="special">&lt;&lt;</span> <span class="identifier">left</span> <span class="special">&lt;&lt;</span> <span class="string">"Number of Observations"</span> <span class="special">&lt;&lt;</span> <span class="string">"=  "</span> <span class="special">&lt;&lt;</span> <span class="identifier">N</span> <span class="special">&lt;&lt;</span> <span class="string">"\n"</span><span class="special">;</span>
  57 <span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="identifier">setw</span><span class="special">(</span><span class="number">40</span><span class="special">)</span> <span class="special">&lt;&lt;</span> <span class="identifier">left</span> <span class="special">&lt;&lt;</span> <span class="string">"Standard Deviation"</span> <span class="special">&lt;&lt;</span> <span class="string">"=  "</span> <span class="special">&lt;&lt;</span> <span class="identifier">Sd</span> <span class="special">&lt;&lt;</span> <span class="string">"\n"</span><span class="special">;</span>
  58 </pre>
  59 <p>
  60             and then define a table of significance levels for which we'll calculate
  61             intervals:
  62           </p>
  63 <pre class="programlisting"><span class="keyword">double</span> <span class="identifier">alpha</span><span class="special">[]</span> <span class="special">=</span> <span class="special">{</span> <span class="number">0.5</span><span class="special">,</span> <span class="number">0.25</span><span class="special">,</span> <span class="number">0.1</span><span class="special">,</span> <span class="number">0.05</span><span class="special">,</span> <span class="number">0.01</span><span class="special">,</span> <span class="number">0.001</span><span class="special">,</span> <span class="number">0.0001</span><span class="special">,</span> <span class="number">0.00001</span> <span class="special">};</span>
  64 </pre>
  65 <p>
  66             The distribution we'll need to calculate the confidence intervals is
  67             a Chi Squared distribution, with N-1 degrees of freedom:
  68           </p>
  69 <pre class="programlisting"><span class="identifier">chi_squared</span> <span class="identifier">dist</span><span class="special">(</span><span class="identifier">N</span> <span class="special">-</span> <span class="number">1</span><span class="special">);</span>
  70 </pre>
  71 <p>
  72             For each value of alpha, the formula for the confidence interval is given
  73             by:
  74           </p>
  75 <div class="blockquote"><blockquote class="blockquote"><p>
  76               <span class="inlinemediaobject"><img src="../../../../../equations/chi_squ_tut1.svg"></span>
  77
  78             </p></blockquote></div>
  79 <p>
  80             Where
  81           </p>
  82 <div class="blockquote"><blockquote class="blockquote"><p>
  83               <span class="inlinemediaobject"><img src="../../../../../equations/chi_squ_tut2.svg"></span>
  84
  85             </p></blockquote></div>
  86 <p>
  87             is the upper critical value, and
  88           </p>
  89 <div class="blockquote"><blockquote class="blockquote"><p>
  90               <span class="inlinemediaobject"><img src="../../../../../equations/chi_squ_tut3.svg"></span>
  91
  92             </p></blockquote></div>
  93 <p>
  94             is the lower critical value of the Chi Squared distribution.
  95           </p>
  96 <p>
  97             In code we begin by printing out a table header:
  98           </p>
  99 <pre class="programlisting"><span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="string">"\n\n"</span>
 100         <span class="string">"_____________________________________________\n"</span>
 101         <span class="string">"Confidence          Lower          Upper\n"</span>
 102         <span class="string">" Value (%)          Limit          Limit\n"</span>
 103         <span class="string">"_____________________________________________\n"</span><span class="special">;</span>
 104 </pre>
 105 <p>
 106             and then loop over the values of alpha and calculate the intervals for
 107             each: remember that the lower critical value is the same as the quantile,
 108             and the upper critical value is the same as the quantile from the complement
 109             of the probability:
 110           </p>
 111 <pre class="programlisting"><span class="keyword">for</span><span class="special">(</span><span class="keyword">unsigned</span> <span class="identifier">i</span> <span class="special">=</span> <span class="number">0</span><span class="special">;</span> <span class="identifier">i</span> <span class="special">&lt;</span> <span class="keyword">sizeof</span><span class="special">(</span><span class="identifier">alpha</span><span class="special">)/</span><span class="keyword">sizeof</span><span class="special">(</span><span class="identifier">alpha</span><span class="special">[</span><span class="number">0</span><span class="special">]);</span> <span class="special">++</span><span class="identifier">i</span><span class="special">)</span>
 112 <span class="special">{</span>
 113    <span class="comment">// Confidence value:</span>
 114    <span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="identifier">fixed</span> <span class="special">&lt;&lt;</span> <span class="identifier">setprecision</span><span class="special">(</span><span class="number">3</span><span class="special">)</span> <span class="special">&lt;&lt;</span> <span class="identifier">setw</span><span class="special">(</span><span class="number">10</span><span class="special">)</span> <span class="special">&lt;&lt;</span> <span class="identifier">right</span> <span class="special">&lt;&lt;</span> <span class="number">100</span> <span class="special">*</span> <span class="special">(</span><span class="number">1</span><span class="special">-</span><span class="identifier">alpha</span><span class="special">[</span><span class="identifier">i</span><span class="special">]);</span>
 115    <span class="comment">// Calculate limits:</span>
 116    <span class="keyword">double</span> <span class="identifier">lower_limit</span> <span class="special">=</span> <span class="identifier">sqrt</span><span class="special">((</span><span class="identifier">N</span> <span class="special">-</span> <span class="number">1</span><span class="special">)</span> <span class="special">*</span> <span class="identifier">Sd</span> <span class="special">*</span> <span class="identifier">Sd</span> <span class="special">/</span> <span class="identifier">quantile</span><span class="special">(</span><span class="identifier">complement</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span> <span class="identifier">alpha</span><span class="special">[</span><span class="identifier">i</span><span class="special">]</span> <span class="special">/</span> <span class="number">2</span><span class="special">)));</span>
 117    <span class="keyword">double</span> <span class="identifier">upper_limit</span> <span class="special">=</span> <span class="identifier">sqrt</span><span class="special">((</span><span class="identifier">N</span> <span class="special">-</span> <span class="number">1</span><span class="special">)</span> <span class="special">*</span> <span class="identifier">Sd</span> <span class="special">*</span> <span class="identifier">Sd</span> <span class="special">/</span> <span class="identifier">quantile</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span> <span class="identifier">alpha</span><span class="special">[</span><span class="identifier">i</span><span class="special">]</span> <span class="special">/</span> <span class="number">2</span><span class="special">));</span>
 118    <span class="comment">// Print Limits:</span>
 119    <span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="identifier">fixed</span> <span class="special">&lt;&lt;</span> <span class="identifier">setprecision</span><span class="special">(</span><span class="number">5</span><span class="special">)</span> <span class="special">&lt;&lt;</span> <span class="identifier">setw</span><span class="special">(</span><span class="number">15</span><span class="special">)</span> <span class="special">&lt;&lt;</span> <span class="identifier">right</span> <span class="special">&lt;&lt;</span> <span class="identifier">lower_limit</span><span class="special">;</span>
 120    <span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="identifier">fixed</span> <span class="special">&lt;&lt;</span> <span class="identifier">setprecision</span><span class="special">(</span><span class="number">5</span><span class="special">)</span> <span class="special">&lt;&lt;</span> <span class="identifier">setw</span><span class="special">(</span><span class="number">15</span><span class="special">)</span> <span class="special">&lt;&lt;</span> <span class="identifier">right</span> <span class="special">&lt;&lt;</span> <span class="identifier">upper_limit</span> <span class="special">&lt;&lt;</span> <span class="identifier">endl</span><span class="special">;</span>
 121 <span class="special">}</span>
 122 <span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="identifier">endl</span><span class="special">;</span>
 123 </pre>
 124 <p>
 125             To see some example output we'll use the <a href="http://www.itl.nist.gov/div898/handbook/eda/section3/eda3581.htm" target="_top">gear
 126             data</a> from the <a href="http://www.itl.nist.gov/div898/handbook/" target="_top">NIST/SEMATECH
 127             e-Handbook of Statistical Methods.</a>. The data represents measurements
 128             of gear diameter from a manufacturing process.
 129           </p>
 130 <pre class="programlisting">________________________________________________
 131 2-Sided Confidence Limits For Standard Deviation
 132 ________________________________________________
 133
 134 Number of Observations                  =  100
 135 Standard Deviation                      =  0.006278908
 136
 137
 138 _____________________________________________
 139 Confidence          Lower          Upper
 140  Value (%)          Limit          Limit
 141 _____________________________________________
 142     50.000        0.00601        0.00662
 143     75.000        0.00582        0.00685
 144     90.000        0.00563        0.00712
 145     95.000        0.00551        0.00729
 146     99.000        0.00530        0.00766
 147     99.900        0.00507        0.00812
 148     99.990        0.00489        0.00855
 149     99.999        0.00474        0.00895
 150 </pre>
 151 <p>
 152             So at the 95% confidence level we conclude that the standard deviation
 153             is between 0.00551 and 0.00729.
 154           </p>
 155 <h5>
 156 <a name="math_toolkit.stat_tut.weg.cs_eg.chi_sq_intervals.h0"></a>
 157             <span class="phrase"><a name="math_toolkit.stat_tut.weg.cs_eg.chi_sq_intervals.confidence_intervals_as_a_functi"></a></span><a class="link" href="chi_sq_intervals.html#math_toolkit.stat_tut.weg.cs_eg.chi_sq_intervals.confidence_intervals_as_a_functi">Confidence
 158             intervals as a function of the number of observations</a>
 159           </h5>
 160 <p>
 161             Similarly, we can also list the confidence intervals for the standard
 162             deviation for the common confidence levels 95%, for increasing numbers
 163             of observations.
 164           </p>
 165 <p>
 166             The standard deviation used to compute these values is unity, so the
 167             limits listed are <span class="bold"><strong>multipliers</strong></span> for any
 168             particular standard deviation. For example, given a standard deviation
 169             of 0.0062789 as in the example above; for 100 observations the multiplier
 170             is 0.8780 giving the lower confidence limit of 0.8780 * 0.006728 = 0.00551.
 171           </p>
 172 <pre class="programlisting">____________________________________________________
 173 Confidence level (two-sided)            =  0.0500000
 174 Standard Deviation                      =  1.0000000
 175 ________________________________________
 176 Observations        Lower          Upper
 177                     Limit          Limit
 178 ________________________________________
 179          2         0.4461        31.9102
 180          3         0.5207         6.2847
 181          4         0.5665         3.7285
 182          5         0.5991         2.8736
 183          6         0.6242         2.4526
 184          7         0.6444         2.2021
 185          8         0.6612         2.0353
 186          9         0.6755         1.9158
 187         10         0.6878         1.8256
 188         15         0.7321         1.5771
 189         20         0.7605         1.4606
 190         30         0.7964         1.3443
 191         40         0.8192         1.2840
 192         50         0.8353         1.2461
 193         60         0.8476         1.2197
 194        100         0.8780         1.1617
 195        120         0.8875         1.1454
 196       1000         0.9580         1.0459
 197      10000         0.9863         1.0141
 198      50000         0.9938         1.0062
 199     100000         0.9956         1.0044
 200    1000000         0.9986         1.0014
 201 </pre>
 202 <p>
 203             With just 2 observations the limits are from <span class="bold"><strong>0.445</strong></span>
 204             up to to <span class="bold"><strong>31.9</strong></span>, so the standard deviation
 205             might be about <span class="bold"><strong>half</strong></span> the observed value
 206             up to <span class="bold"><strong>30 times</strong></span> the observed value!
 207           </p>
 208 <p>
 209             Estimating a standard deviation with just a handful of values leaves
 210             a very great uncertainty, especially the upper limit. Note especially
 211             how far the upper limit is skewed from the most likely standard deviation.
 212           </p>
 213 <p>
 214             Even for 10 observations, normally considered a reasonable number, the
 215             range is still from 0.69 to 1.8, about a range of 0.7 to 2, and is still
 216             highly skewed with an upper limit <span class="bold"><strong>twice</strong></span>
 217             the median.
 218           </p>
 219 <p>
 220             When we have 1000 observations, the estimate of the standard deviation
 221             is starting to look convincing, with a range from 0.95 to 1.05 - now
 222             near symmetrical, but still about + or - 5%.
 223           </p>
 224 <p>
 225             Only when we have 10000 or more repeated observations can we start to
 226             be reasonably confident (provided we are sure that other factors like
 227             drift are not creeping in).
 228           </p>
 229 <p>
 230             For 10000 observations, the interval is 0.99 to 1.1 - finally a really
 231             convincing + or -1% confidence.
 232           </p>
 233 </div>
 234 <table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
 235 <td align="left"></td>
 236 <td align="right"><div class="copyright-footer">Copyright &#169; 2006-2019 Nikhar
 237       Agrawal, Anton Bikineev, Paul A. Bristow, Marco Guazzone, Christopher Kormanyos,
 238       Hubert Holin, Bruno Lalande, John Maddock, Jeremy Murphy, Matthew Pulver, Johan
 239       R&#229;de, Gautam Sewani, Benjamin Sobotta, Nicholas Thompson, Thijs van den Berg,
 240       Daryle Walker and Xiaogang Zhang<p>
 241         Distributed under the Boost Software License, Version 1.0. (See accompanying
 242         file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>)
 243       </p>
 244 </div></td>
 245 </tr></table>
 246 <hr>
 247 <div class="spirit-nav">
 248 <a accesskey="p" href="../cs_eg.html"><img src="../../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../cs_eg.html"><img src="../../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../../index.html"><img src="../../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="chi_sq_test.html"><img src="../../../../../../../../doc/src/images/next.png" alt="Next"></a>
 249 </div>
 250 </body>
 251 </html>