libs/math/doc/html/math_toolkit/diff.html

   1 <html>
   2 <head>
   3 <meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
   4 <title>Numerical Differentiation</title>
   5 <link rel="stylesheet" href="../math.css" type="text/css">
   6 <meta name="generator" content="DocBook XSL Stylesheets V1.79.1">
   7 <link rel="home" href="../index.html" title="Math Toolkit 2.11.0">
   8 <link rel="up" href="../quadrature.html" title="Chapter&#160;13.&#160;Quadrature and Differentiation">
   9 <link rel="prev" href="naive_monte_carlo.html" title="Naive Monte Carlo Integration">
  10 <link rel="next" href="autodiff.html" title="Automatic Differentiation">
  11 </head>
  12 <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
  13 <table cellpadding="2" width="100%"><tr>
  14 <td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../boost.png"></td>
  15 <td align="center"><a href="../../../../../index.html">Home</a></td>
  16 <td align="center"><a href="../../../../../libs/libraries.htm">Libraries</a></td>
  17 <td align="center"><a href="http://www.boost.org/users/people.html">People</a></td>
  18 <td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td>
  19 <td align="center"><a href="../../../../../more/index.htm">More</a></td>
  20 </tr></table>
  21 <hr>
  22 <div class="spirit-nav">
  23 <a accesskey="p" href="naive_monte_carlo.html"><img src="../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../quadrature.html"><img src="../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="autodiff.html"><img src="../../../../../doc/src/images/next.png" alt="Next"></a>
  24 </div>
  25 <div class="section">
  26 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
  27 <a name="math_toolkit.diff"></a><a class="link" href="diff.html" title="Numerical Differentiation">Numerical Differentiation</a>
  28 </h2></div></div></div>
  29 <h4>
  30 <a name="math_toolkit.diff.h0"></a>
  31       <span class="phrase"><a name="math_toolkit.diff.synopsis"></a></span><a class="link" href="diff.html#math_toolkit.diff.synopsis">Synopsis</a>
  32     </h4>
  33 <pre class="programlisting"><span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">math</span><span class="special">/</span><span class="identifier">differentiaton</span><span class="special">/</span><span class="identifier">finite_difference</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
  34
  35
  36 <span class="keyword">namespace</span> <span class="identifier">boost</span> <span class="special">{</span> <span class="keyword">namespace</span> <span class="identifier">math</span> <span class="special">{</span> <span class="keyword">namespace</span> <span class="identifier">differentiation</span> <span class="special">{</span>
  37
  38     <span class="keyword">template</span> <span class="special">&lt;</span><span class="keyword">class</span> <span class="identifier">F</span><span class="special">,</span> <span class="keyword">class</span> <span class="identifier">Real</span><span class="special">&gt;</span>
  39     <span class="identifier">Real</span> <span class="identifier">complex_step_derivative</span><span class="special">(</span><span class="keyword">const</span> <span class="identifier">F</span> <span class="identifier">f</span><span class="special">,</span> <span class="identifier">Real</span> <span class="identifier">x</span><span class="special">);</span>
  40
  41     <span class="keyword">template</span> <span class="special">&lt;</span><span class="keyword">class</span> <span class="identifier">F</span><span class="special">,</span> <span class="keyword">class</span> <span class="identifier">Real</span><span class="special">,</span> <span class="identifier">size_t</span> <span class="identifier">order</span> <span class="special">=</span> <span class="number">6</span><span class="special">&gt;</span>
  42     <span class="identifier">Real</span> <span class="identifier">finite_difference_derivative</span><span class="special">(</span><span class="keyword">const</span> <span class="identifier">F</span> <span class="identifier">f</span><span class="special">,</span> <span class="identifier">Real</span> <span class="identifier">x</span><span class="special">,</span> <span class="identifier">Real</span><span class="special">*</span> <span class="identifier">error</span> <span class="special">=</span> <span class="keyword">nullptr</span><span class="special">);</span>
  43
  44 <span class="special">}}}</span> <span class="comment">// namespaces</span>
  45 </pre>
  46 <h4>
  47 <a name="math_toolkit.diff.h1"></a>
  48       <span class="phrase"><a name="math_toolkit.diff.description"></a></span><a class="link" href="diff.html#math_toolkit.diff.description">Description</a>
  49     </h4>
  50 <p>
  51       The function <code class="computeroutput"><span class="identifier">finite_difference_derivative</span></code>
  52       calculates a finite-difference approximation to the derivative of a function
  53       <span class="emphasis"><em>f</em></span> at point <span class="emphasis"><em>x</em></span>. A basic usage is
  54     </p>
  55 <pre class="programlisting"><span class="keyword">auto</span> <span class="identifier">f</span> <span class="special">=</span> <span class="special">[](</span><span class="keyword">double</span> <span class="identifier">x</span><span class="special">)</span> <span class="special">{</span> <span class="keyword">return</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">exp</span><span class="special">(</span><span class="identifier">x</span><span class="special">);</span> <span class="special">};</span>
  56 <span class="keyword">double</span> <span class="identifier">x</span> <span class="special">=</span> <span class="number">1.7</span><span class="special">;</span>
  57 <span class="keyword">double</span> <span class="identifier">dfdx</span> <span class="special">=</span> <span class="identifier">finite_difference_derivative</span><span class="special">(</span><span class="identifier">f</span><span class="special">,</span> <span class="identifier">x</span><span class="special">);</span>
  58 </pre>
  59 <p>
  60       Finite differencing is complicated, as differentiation is <span class="emphasis"><em>infinitely</em></span>
  61       ill-conditioned. In addition, for any function implemented in finite-precision
  62       arithmetic, the "true" derivative is <span class="emphasis"><em>zero</em></span> almost
  63       everywhere, and undefined at representables. However, some tricks allow for
  64       reasonable results to be obtained in many cases.
  65     </p>
  66 <p>
  67       There are two sources of error from finite differences: One, the truncation
  68       error arising from using a finite number of samples to cancel out higher order
  69       terms in the Taylor series. The second is the roundoff error involved in evaluating
  70       the function. The truncation error goes to zero as <span class="emphasis"><em>h</em></span> &#8594;
  71       0, but the roundoff error becomes unbounded. By balancing these two sources
  72       of error, we can choose a value of <span class="emphasis"><em>h</em></span> that minimizes the
  73       maximum total error. For this reason boost's <code class="computeroutput"><span class="identifier">finite_difference_derivative</span></code>
  74       does not require the user to input a stepsize. For more details about the theoretical
  75       error analysis involved in finite-difference approximations to the derivative,
  76       see <a href="http://web.archive.org/web/20150420195907/http://www.uio.no/studier/emner/matnat/math/MAT-INF1100/h08/kompendiet/diffint.pdf" target="_top">here</a>.
  77     </p>
  78 <p>
  79       Despite the effort that has went into choosing a reasonable value of <span class="emphasis"><em>h</em></span>,
  80       the problem is still fundamentally ill-conditioned, and hence an error estimate
  81       is essential. It can be queried as follows
  82     </p>
  83 <pre class="programlisting"><span class="keyword">double</span> <span class="identifier">error_estimate</span><span class="special">;</span>
  84 <span class="keyword">double</span> <span class="identifier">d</span> <span class="special">=</span> <span class="identifier">finite_difference_derivative</span><span class="special">(</span><span class="identifier">f</span><span class="special">,</span> <span class="identifier">x</span><span class="special">,</span> <span class="special">&amp;</span><span class="identifier">error_estimate</span><span class="special">);</span>
  85 </pre>
  86 <p>
  87       N.B.: Producing an error estimate requires additional function evaluations
  88       and as such is slower than simple evaluation of the derivative. It also expands
  89       the domain over which the function must be differentiable and requires the
  90       function to have two more continuous derivatives. The error estimate is computed
  91       under the assumption that <span class="emphasis"><em>f</em></span> is evaluated to 1ULP. This
  92       might seem an extreme assumption, but it is the only sensible one, as the routine
  93       cannot know the functions rounding error. If the function cannot be evaluated
  94       with very great accuracy, Lanczos's smoothing differentiation is recommended
  95       as an alternative.
  96     </p>
  97 <p>
  98       The default order of accuracy is 6, which reflects that fact that people tend
  99       to be interested in functions with many continuous derivatives. If your function
 100       does not have 7 continuous derivatives, is may be of interest to use a lower
 101       order method, which can be achieved via (say)
 102     </p>
 103 <pre class="programlisting"><span class="keyword">double</span> <span class="identifier">d</span> <span class="special">=</span> <span class="identifier">finite_difference_derivative</span><span class="special">&lt;</span><span class="keyword">decltype</span><span class="special">(</span><span class="identifier">f</span><span class="special">),</span> <span class="identifier">Real</span><span class="special">,</span> <span class="number">2</span><span class="special">&gt;(</span><span class="identifier">f</span><span class="special">,</span> <span class="identifier">x</span><span class="special">);</span>
 104 </pre>
 105 <p>
 106       This requests a second-order accurate derivative be computed.
 107     </p>
 108 <p>
 109       It is emphatically <span class="emphasis"><em>not</em></span> the case that higher order methods
 110       always give higher accuracy for smooth functions. Higher order methods require
 111       more addition of positive and negative terms, which can lead to catastrophic
 112       cancellation. A function which is very good at making a mockery of finite-difference
 113       differentiation is exp(x)/(cos(x)<sup>3</sup> + sin(x)<sup>3</sup>). Differentiating this function
 114       by <code class="computeroutput"><span class="identifier">finite_difference_derivative</span></code>
 115       in double precision at <span class="emphasis"><em>x=5.5</em></span> gives zero correct digits
 116       at order 4, 6, and 8, but recovers 5 correct digits at order 2. These are dangerous
 117       waters; use the error estimates to tread carefully.
 118     </p>
 119 <p>
 120       For a finite-difference method of order <span class="emphasis"><em>k</em></span>, the error is
 121       <span class="emphasis"><em>C</em></span> &#949;<sup>k/k+1</sup>. In the limit <span class="emphasis"><em>k</em></span> &#8594;
 122       &#8734;, we see that the error tends to &#949;, recovering the full precision
 123       for the type. However, this ignores the fact that higher-order methods require
 124       subtracting more nearly-equal (perhaps noisy) terms, so the constant <span class="emphasis"><em>C</em></span>
 125       grows with <span class="emphasis"><em>k</em></span>. Since <span class="emphasis"><em>C</em></span> grows quickly
 126       and &#949;<sup>k/k+1</sup> approaches &#949; slowly, we can see there is a compromise
 127       between high-order accuracy and conditioning of the difference quotient. In
 128       practice we have found that <span class="emphasis"><em>k=6</em></span> seems to be a good compromise
 129       between the two (and have made this the default), but users are encouraged
 130       to examine the error estimates to choose an optimal order of accuracy for the
 131       given problem.
 132     </p>
 133 <div class="table">
 134 <a name="math_toolkit.diff.id"></a><p class="title"><b>Table&#160;13.1.&#160;Cost of Finite-Difference Numerical Differentiation</b></p>
 135 <div class="table-contents"><table class="table" summary="Cost of Finite-Difference Numerical Differentiation">
 136 <colgroup>
 137 <col>
 138 <col>
 139 <col>
 140 <col>
 141 <col>
 142 </colgroup>
 143 <thead><tr>
 144 <th>
 145               <p>
 146                 Order of Accuracy
 147               </p>
 148             </th>
 149 <th>
 150               <p>
 151                 Function Evaluations
 152               </p>
 153             </th>
 154 <th>
 155               <p>
 156                 Error
 157               </p>
 158             </th>
 159 <th>
 160               <p>
 161                 Continuous Derivatives Required for Error Estimate to Hold
 162               </p>
 163             </th>
 164 <th>
 165               <p>
 166                 Additional Function Evaluations to Produce Error Estimates
 167               </p>
 168             </th>
 169 </tr></thead>
 170 <tbody>
 171 <tr>
 172 <td>
 173               <p>
 174                 1
 175               </p>
 176             </td>
 177 <td>
 178               <p>
 179                 2
 180               </p>
 181             </td>
 182 <td>
 183               <p>
 184                 &#949;<sup>1/2</sup>
 185               </p>
 186             </td>
 187 <td>
 188               <p>
 189                 2
 190               </p>
 191             </td>
 192 <td>
 193               <p>
 194                 1
 195               </p>
 196             </td>
 197 </tr>
 198 <tr>
 199 <td>
 200               <p>
 201                 2
 202               </p>
 203             </td>
 204 <td>
 205               <p>
 206                 2
 207               </p>
 208             </td>
 209 <td>
 210               <p>
 211                 &#949;<sup>2/3</sup>
 212               </p>
 213             </td>
 214 <td>
 215               <p>
 216                 3
 217               </p>
 218             </td>
 219 <td>
 220               <p>
 221                 2
 222               </p>
 223             </td>
 224 </tr>
 225 <tr>
 226 <td>
 227               <p>
 228                 4
 229               </p>
 230             </td>
 231 <td>
 232               <p>
 233                 4
 234               </p>
 235             </td>
 236 <td>
 237               <p>
 238                 &#949;<sup>4/5</sup>
 239               </p>
 240             </td>
 241 <td>
 242               <p>
 243                 5
 244               </p>
 245             </td>
 246 <td>
 247               <p>
 248                 2
 249               </p>
 250             </td>
 251 </tr>
 252 <tr>
 253 <td>
 254               <p>
 255                 6
 256               </p>
 257             </td>
 258 <td>
 259               <p>
 260                 6
 261               </p>
 262             </td>
 263 <td>
 264               <p>
 265                 &#949;<sup>6/7</sup>
 266               </p>
 267             </td>
 268 <td>
 269               <p>
 270                 7
 271               </p>
 272             </td>
 273 <td>
 274               <p>
 275                 2
 276               </p>
 277             </td>
 278 </tr>
 279 <tr>
 280 <td>
 281               <p>
 282                 8
 283               </p>
 284             </td>
 285 <td>
 286               <p>
 287                 8
 288               </p>
 289             </td>
 290 <td>
 291               <p>
 292                 &#949;<sup>8/9</sup>
 293               </p>
 294             </td>
 295 <td>
 296               <p>
 297                 9
 298               </p>
 299             </td>
 300 <td>
 301               <p>
 302                 2
 303               </p>
 304             </td>
 305 </tr>
 306 </tbody>
 307 </table></div>
 308 </div>
 309 <br class="table-break"><p>
 310       Given all the caveats which must be kept in mind for successful use of finite-difference
 311       differentiation, it is reasonable to try to avoid it if possible. Boost provides
 312       two possibilities: The Chebyshev transform (see <a class="link" href="sf_poly/chebyshev.html" title="Chebyshev Polynomials">here</a>)
 313       and the complex step derivative. If your function is the restriction to the
 314       real line of a holomorphic function which takes real values at real argument,
 315       then the <span class="bold"><strong>complex step derivative</strong></span> can be used.
 316       The idea is very simple: Since <span class="emphasis"><em>f</em></span> is complex-differentiable,
 317       <span class="emphasis"><em>f(x+&#8520; h) = f(x) + &#8520; hf'(x) - h<sup>2</sup>f''(x) + &#119926;(h<sup>3</sup>)</em></span>.
 318       As long as <span class="emphasis"><em>f(x)</em></span> &#8712; &#8477;, then <span class="emphasis"><em>f'(x)
 319       = &#8465; f(x+&#8520; h)/h + &#119926;(h<sup>2</sup>)</em></span>. This method requires a single
 320       complex function evaluation and is not subject to the catastrophic subtractive
 321       cancellation that plagues finite-difference calculations.
 322     </p>
 323 <p>
 324       An example usage:
 325     </p>
 326 <pre class="programlisting"><span class="keyword">double</span> <span class="identifier">x</span> <span class="special">=</span> <span class="number">7.2</span><span class="special">;</span>
 327 <span class="keyword">double</span> <span class="identifier">e_prime</span> <span class="special">=</span> <span class="identifier">complex_step_derivative</span><span class="special">(</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">exp</span><span class="special">&lt;</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">complex</span><span class="special">&lt;</span><span class="keyword">double</span><span class="special">&gt;&gt;,</span> <span class="identifier">x</span><span class="special">);</span>
 328 </pre>
 329 <p>
 330       References:
 331     </p>
 332 <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
 333 <li class="listitem">
 334           Squire, William, and George Trapp. <span class="emphasis"><em>Using complex variables to
 335           estimate derivatives of real functions.</em></span> Siam Review 40.1 (1998):
 336           110-112.
 337         </li>
 338 <li class="listitem">
 339           Fornberg, Bengt. <span class="emphasis"><em>Generation of finite difference formulas on
 340           arbitrarily spaced grids.</em></span> Mathematics of computation 51.184
 341           (1988): 699-706.
 342         </li>
 343 <li class="listitem">
 344           Corless, Robert M., and Nicolas Fillion. <span class="emphasis"><em>A graduate introduction
 345           to numerical methods.</em></span> AMC 10 (2013): 12.
 346         </li>
 347 </ul></div>
 348 </div>
 349 <table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
 350 <td align="left"></td>
 351 <td align="right"><div class="copyright-footer">Copyright &#169; 2006-2019 Nikhar
 352       Agrawal, Anton Bikineev, Paul A. Bristow, Marco Guazzone, Christopher Kormanyos,
 353       Hubert Holin, Bruno Lalande, John Maddock, Jeremy Murphy, Matthew Pulver, Johan
 354       R&#229;de, Gautam Sewani, Benjamin Sobotta, Nicholas Thompson, Thijs van den Berg,
 355       Daryle Walker and Xiaogang Zhang<p>
 356         Distributed under the Boost Software License, Version 1.0. (See accompanying
 357         file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>)
 358       </p>
 359 </div></td>
 360 </tr></table>
 361 <hr>
 362 <div class="spirit-nav">
 363 <a accesskey="p" href="naive_monte_carlo.html"><img src="../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../quadrature.html"><img src="../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="autodiff.html"><img src="../../../../../doc/src/images/next.png" alt="Next"></a>
 364 </div>
 365 </body>
 366 </html>