doc/html/mpi/python.html

   1 <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
   2 <html>
   3 <head>
   4 <meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
   5 <title>Python Bindings</title>
   6 <link rel="stylesheet" href="../../../doc/src/boostbook.css" type="text/css">
   7 <meta name="generator" content="DocBook XSL Stylesheets V1.79.1">
   8 <link rel="home" href="../index.html" title="The Boost C++ Libraries BoostBook Documentation Subset">
   9 <link rel="up" href="../mpi.html" title="Chapter&#160;26.&#160;Boost.MPI">
  10 <link rel="prev" href="../boost/mpi/timer.html" title="Class timer">
  11 <link rel="next" href="../multi_array.html" title="Chapter&#160;27.&#160;Boost.MultiArray Reference Manual">
  12 </head>
  13 <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
  14 <table cellpadding="2" width="100%"><tr>
  15 <td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../boost.png"></td>
  16 <td align="center"><a href="../../../index.html">Home</a></td>
  17 <td align="center"><a href="../../../libs/libraries.htm">Libraries</a></td>
  18 <td align="center"><a href="http://www.boost.org/users/people.html">People</a></td>
  19 <td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td>
  20 <td align="center"><a href="../../../more/index.htm">More</a></td>
  21 </tr></table>
  22 <hr>
  23 <div class="spirit-nav">
  24 <a accesskey="p" href="../boost/mpi/timer.html"><img src="../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../mpi.html"><img src="../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="../multi_array.html"><img src="../../../doc/src/images/next.png" alt="Next"></a>
  25 </div>
  26 <div class="section">
  27 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
  28 <a name="mpi.python"></a>Python Bindings</h2></div></div></div>
  29 <div class="toc"><dl class="toc">
  30 <dt><span class="section"><a href="python.html#mpi.python_quickstart">Quickstart</a></span></dt>
  31 <dt><span class="section"><a href="python.html#mpi.python_user_data">Transmitting User-Defined Data</a></span></dt>
  32 <dt><span class="section"><a href="python.html#mpi.python_collectives">Collectives</a></span></dt>
  33 <dt><span class="section"><a href="python.html#mpi.python_skeleton_content">Skeleton/Content Mechanism</a></span></dt>
  34 <dt><span class="section"><a href="python.html#mpi.design">Design Philosophy</a></span></dt>
  35 <dt><span class="section"><a href="python.html#mpi.threading">Threads</a></span></dt>
  36 <dt><span class="section"><a href="python.html#mpi.performance">Performance Evaluation</a></span></dt>
  37 <dt><span class="section"><a href="python.html#mpi.history">Revision History</a></span></dt>
  38 <dt><span class="section"><a href="python.html#mpi.acknowledge">Acknowledgments</a></span></dt>
  39 </dl></div>
  40 <p>
  41       Boost.MPI provides an alternative MPI interface from the <a href="http://www.python.org" target="_top">Python</a>
  42       programming language via the <code class="computeroutput"><span class="identifier">boost</span><span class="special">.</span><span class="identifier">mpi</span></code> module.
  43       The Boost.MPI Python bindings, built on top of the C++ Boost.MPI using the
  44       <a href="http://www.boost.org/libs/python/doc" target="_top">Boost.Python</a> library,
  45       provide nearly all of the functionality of Boost.MPI within a dynamic, object-oriented
  46       language.
  47     </p>
  48 <p>
  49       The Boost.MPI Python module can be built and installed from the <code class="computeroutput"><span class="identifier">libs</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">build</span></code> directory.
  50       Just follow the <a class="link" href="getting_started.html#mpi.config" title="Configure and Build">configuration</a> and <a class="link" href="getting_started.html#mpi.installation" title="Build and Install">installation</a>
  51       instructions for the C++ Boost.MPI. Once you have installed the Python module,
  52       be sure that the installation location is in your <code class="computeroutput"><span class="identifier">PYTHONPATH</span></code>.
  53     </p>
  54 <div class="section">
  55 <div class="titlepage"><div><div><h3 class="title">
  56 <a name="mpi.python_quickstart"></a>Quickstart</h3></div></div></div>
  57 <p>
  58         Getting started with the Boost.MPI Python module is as easy as importing
  59         <code class="computeroutput"><span class="identifier">boost</span><span class="special">.</span><span class="identifier">mpi</span></code>. Our first "Hello, World!"
  60         program is just two lines long:
  61       </p>
  62 <pre class="programlisting"><span class="keyword">import</span> <span class="identifier">boost</span><span class="special">.</span><span class="identifier">mpi</span> <span class="keyword">as</span> <span class="identifier">mpi</span>
  63 <span class="keyword">print</span> <span class="string">"I am process %d of %d."</span> <span class="special">%</span> <span class="special">(</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">rank</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">.</span><span class="identifier">size</span><span class="special">)</span>
  64 </pre>
  65 <p>
  66         Go ahead and run this program with several processes. Be sure to invoke the
  67         <code class="computeroutput"><span class="identifier">python</span></code> interpreter from
  68         <code class="computeroutput"><span class="identifier">mpirun</span></code>, e.g.,
  69       </p>
  70 <pre class="programlisting">mpirun -np 5 python hello_world.py
  71 </pre>
  72 <p>
  73         This will return output such as:
  74       </p>
  75 <pre class="programlisting">I am process 1 of 5.
  76 I am process 3 of 5.
  77 I am process 2 of 5.
  78 I am process 4 of 5.
  79 I am process 0 of 5.
  80 </pre>
  81 <p>
  82         Point-to-point operations in Boost.MPI have nearly the same syntax in Python
  83         as in C++. We can write a simple two-process Python program that prints "Hello,
  84         world!" by transmitting Python strings:
  85       </p>
  86 <pre class="programlisting"><span class="keyword">import</span> <span class="identifier">boost</span><span class="special">.</span><span class="identifier">mpi</span> <span class="keyword">as</span> <span class="identifier">mpi</span>
  87
  88 <span class="keyword">if</span> <span class="identifier">mpi</span><span class="special">.</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span> <span class="special">==</span> <span class="number">0</span><span class="special">:</span>
  89   <span class="identifier">mpi</span><span class="special">.</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="string">'Hello'</span><span class="special">)</span>
  90   <span class="identifier">msg</span> <span class="special">=</span> <span class="identifier">mpi</span><span class="special">.</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">1</span><span class="special">)</span>
  91   <span class="keyword">print</span> <span class="identifier">msg</span><span class="special">,</span><span class="string">'!'</span>
  92 <span class="keyword">else</span><span class="special">:</span>
  93   <span class="identifier">msg</span> <span class="special">=</span> <span class="identifier">mpi</span><span class="special">.</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">0</span><span class="special">)</span>
  94   <span class="keyword">print</span> <span class="special">(</span><span class="identifier">msg</span> <span class="special">+</span> <span class="string">', '</span><span class="special">),</span>
  95   <span class="identifier">mpi</span><span class="special">.</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">1</span><span class="special">,</span> <span class="string">'world'</span><span class="special">)</span>
  96 </pre>
  97 <p>
  98         There are only a few notable differences between this Python code and the
  99         example <a class="link" href="tutorial.html#mpi.point_to_point" title="Point-to-Point communication">in the C++ tutorial</a>. First
 100         of all, we don't need to write any initialization code in Python: just loading
 101         the <code class="computeroutput"><span class="identifier">boost</span><span class="special">.</span><span class="identifier">mpi</span></code> module makes the appropriate <code class="computeroutput"><span class="identifier">MPI_Init</span></code> and <code class="computeroutput"><span class="identifier">MPI_Finalize</span></code>
 102         calls. Second, we're passing Python objects from one process to another through
 103         MPI. Any Python object that can be pickled can be transmitted; the next section
 104         will describe in more detail how the Boost.MPI Python layer transmits objects.
 105         Finally, when we receive objects with <code class="computeroutput"><span class="identifier">recv</span></code>,
 106         we don't need to specify the type because transmission of Python objects
 107         is polymorphic.
 108       </p>
 109 <p>
 110         When experimenting with Boost.MPI in Python, don't forget that help is always
 111         available via <code class="computeroutput"><span class="identifier">pydoc</span></code>: just
 112         pass the name of the module or module entity on the command line (e.g.,
 113         <code class="computeroutput"><span class="identifier">pydoc</span> <span class="identifier">boost</span><span class="special">.</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">communicator</span></code>) to receive complete reference
 114         documentation. When in doubt, try it!
 115       </p>
 116 </div>
 117 <div class="section">
 118 <div class="titlepage"><div><div><h3 class="title">
 119 <a name="mpi.python_user_data"></a>Transmitting User-Defined Data</h3></div></div></div>
 120 <p>
 121         Boost.MPI can transmit user-defined data in several different ways. Most
 122         importantly, it can transmit arbitrary <a href="http://www.python.org" target="_top">Python</a>
 123         objects by pickling them at the sender and unpickling them at the receiver,
 124         allowing arbitrarily complex Python data structures to interoperate with
 125         MPI.
 126       </p>
 127 <p>
 128         Boost.MPI also supports efficient serialization and transmission of C++ objects
 129         (that have been exposed to Python) through its C++ interface. Any C++ type
 130         that provides (de-)serialization routines that meet the requirements of the
 131         Boost.Serialization library is eligible for this optimization, but the type
 132         must be registered in advance. To register a C++ type, invoke the C++ function
 133         <code class="computeroutput"><a class="link" href="../boost/mpi/python/register_serialized.html" title="Function template register_serialized">register_serialized</a></code>.
 134         If your C++ types come from other Python modules (they probably will!), those
 135         modules will need to link against the <code class="computeroutput"><span class="identifier">boost_mpi</span></code>
 136         and <code class="computeroutput"><span class="identifier">boost_mpi_python</span></code> libraries
 137         as described in the <a class="link" href="getting_started.html#mpi.installation" title="Build and Install">installation section</a>.
 138         Note that you do <span class="bold"><strong>not</strong></span> need to link against
 139         the Boost.MPI Python extension module.
 140       </p>
 141 <p>
 142         Finally, Boost.MPI supports separation of the structure of an object from
 143         the data it stores, allowing the two pieces to be transmitted separately.
 144         This "skeleton/content" mechanism, described in more detail in
 145         a later section, is a communication optimization suitable for problems with
 146         fixed data structures whose internal data changes frequently.
 147       </p>
 148 </div>
 149 <div class="section">
 150 <div class="titlepage"><div><div><h3 class="title">
 151 <a name="mpi.python_collectives"></a>Collectives</h3></div></div></div>
 152 <p>
 153         Boost.MPI supports all of the MPI collectives (<code class="computeroutput"><span class="identifier">scatter</span></code>,
 154         <code class="computeroutput"><span class="identifier">reduce</span></code>, <code class="computeroutput"><span class="identifier">scan</span></code>,
 155         <code class="computeroutput"><span class="identifier">broadcast</span></code>, etc.) for any
 156         type of data that can be transmitted with the point-to-point communication
 157         operations. For the MPI collectives that require a user-specified operation
 158         (e.g., <code class="computeroutput"><span class="identifier">reduce</span></code> and <code class="computeroutput"><span class="identifier">scan</span></code>), the operation can be an arbitrary
 159         Python function. For instance, one could concatenate strings with <code class="computeroutput"><span class="identifier">all_reduce</span></code>:
 160       </p>
 161 <pre class="programlisting"><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">all_reduce</span><span class="special">(</span><span class="identifier">my_string</span><span class="special">,</span> <span class="keyword">lambda</span> <span class="identifier">x</span><span class="special">,</span><span class="identifier">y</span><span class="special">:</span> <span class="identifier">x</span> <span class="special">+</span> <span class="identifier">y</span><span class="special">)</span>
 162 </pre>
 163 <p>
 164         The following module-level functions implement MPI collectives: all_gather
 165         Gather the values from all processes. all_reduce Combine the results from
 166         all processes. all_to_all Every process sends data to every other process.
 167         broadcast Broadcast data from one process to all other processes. gather
 168         Gather the values from all processes to the root. reduce Combine the results
 169         from all processes to the root. scan Prefix reduction of the values from
 170         all processes. scatter Scatter the values stored at the root to all processes.
 171       </p>
 172 </div>
 173 <div class="section">
 174 <div class="titlepage"><div><div><h3 class="title">
 175 <a name="mpi.python_skeleton_content"></a>Skeleton/Content Mechanism</h3></div></div></div>
 176 <div class="toc"><dl class="toc">
 177 <dt><span class="section"><a href="python.html#mpi.python_compatibility">C++/Python MPI Compatibility</a></span></dt>
 178 <dt><span class="section"><a href="python.html#mpi.pythonref">Reference</a></span></dt>
 179 </dl></div>
 180 <p>
 181         Boost.MPI provides a skeleton/content mechanism that allows the transfer
 182         of large data structures to be split into two separate stages, with the skeleton
 183         (or, "shape") of the data structure sent first and the content
 184         (or, "data") of the data structure sent later, potentially several
 185         times, so long as the structure has not changed since the skeleton was transferred.
 186         The skeleton/content mechanism can improve performance when the data structure
 187         is large and its shape is fixed, because while the skeleton requires serialization
 188         (it has an unknown size), the content transfer is fixed-size and can be done
 189         without extra copies.
 190       </p>
 191 <p>
 192         To use the skeleton/content mechanism from Python, you must first register
 193         the type of your data structure with the skeleton/content mechanism <span class="bold"><strong>from C++</strong></span>. The registration function is <code class="computeroutput"><a class="link" href="../boost/mpi/python/register_skel_idp669096704.html" title="Function template register_skeleton_and_content">register_skeleton_and_content</a></code>
 194         and resides in the <code class="computeroutput"><a class="link" href="reference.html#header.boost.mpi.python_hpp" title="Header &lt;boost/mpi/python.hpp&gt;">&lt;boost/mpi/python.hpp&gt;</a></code>
 195         header.
 196       </p>
 197 <p>
 198         Once you have registered your C++ data structures, you can extract the skeleton
 199         for an instance of that data structure with <code class="computeroutput"><span class="identifier">skeleton</span><span class="special">()</span></code>. The resulting <code class="computeroutput"><span class="identifier">skeleton_proxy</span></code>
 200         can be transmitted via the normal send routine, e.g.,
 201       </p>
 202 <pre class="programlisting"><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">skeleton</span><span class="special">(</span><span class="identifier">my_data_structure</span><span class="special">))</span>
 203 </pre>
 204 <p>
 205         <code class="computeroutput"><span class="identifier">skeleton_proxy</span></code> objects can
 206         be received on the other end via <code class="computeroutput"><span class="identifier">recv</span><span class="special">()</span></code>, which stores a newly-created instance
 207         of your data structure with the same "shape" as the sender in its
 208         <code class="computeroutput"><span class="string">"object"</span></code> attribute:
 209       </p>
 210 <pre class="programlisting"><span class="identifier">shape</span> <span class="special">=</span> <span class="identifier">mpi</span><span class="special">.</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">0</span><span class="special">)</span>
 211 <span class="identifier">my_data_structure</span> <span class="special">=</span> <span class="identifier">shape</span><span class="special">.</span><span class="identifier">object</span>
 212 </pre>
 213 <p>
 214         Once the skeleton has been transmitted, the content (accessed via <code class="computeroutput"><span class="identifier">get_content</span></code>) can be transmitted in much
 215         the same way. Note, however, that the receiver also specifies <code class="computeroutput"><span class="identifier">get_content</span><span class="special">(</span><span class="identifier">my_data_structure</span><span class="special">)</span></code>
 216         in its call to receive:
 217       </p>
 218 <pre class="programlisting"><span class="keyword">if</span> <span class="identifier">mpi</span><span class="special">.</span><span class="identifier">rank</span> <span class="special">==</span> <span class="number">0</span><span class="special">:</span>
 219   <span class="identifier">mpi</span><span class="special">.</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">get_content</span><span class="special">(</span><span class="identifier">my_data_structure</span><span class="special">))</span>
 220 <span class="keyword">else</span><span class="special">:</span>
 221   <span class="identifier">mpi</span><span class="special">.</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">get_content</span><span class="special">(</span><span class="identifier">my_data_structure</span><span class="special">))</span>
 222 </pre>
 223 <p>
 224         Of course, this transmission of content can occur repeatedly, if the values
 225         in the data structure--but not its shape--changes.
 226       </p>
 227 <p>
 228         The skeleton/content mechanism is a structured way to exploit the interaction
 229         between custom-built MPI datatypes and <code class="computeroutput"><span class="identifier">MPI_BOTTOM</span></code>,
 230         to eliminate extra buffer copies.
 231       </p>
 232 <div class="section">
 233 <div class="titlepage"><div><div><h4 class="title">
 234 <a name="mpi.python_compatibility"></a>C++/Python MPI Compatibility</h4></div></div></div>
 235 <p>
 236           Boost.MPI is a C++ library whose facilities have been exposed to Python
 237           via the Boost.Python library. Since the Boost.MPI Python bindings are build
 238           directly on top of the C++ library, and nearly every feature of C++ library
 239           is available in Python, hybrid C++/Python programs using Boost.MPI can
 240           interact, e.g., sending a value from Python but receiving that value in
 241           C++ (or vice versa). However, doing so requires some care. Because Python
 242           objects are dynamically typed, Boost.MPI transfers type information along
 243           with the serialized form of the object, so that the object can be received
 244           even when its type is not known. This mechanism differs from its C++ counterpart,
 245           where the static types of transmitted values are always known.
 246         </p>
 247 <p>
 248           The only way to communicate between the C++ and Python views on Boost.MPI
 249           is to traffic entirely in Python objects. For Python, this is the normal
 250           state of affairs, so nothing will change. For C++, this means sending and
 251           receiving values of type <code class="computeroutput"><span class="identifier">boost</span><span class="special">::</span><span class="identifier">python</span><span class="special">::</span><span class="identifier">object</span></code>,
 252           from the <a href="http://www.boost.org/libs/python/doc" target="_top">Boost.Python</a>
 253           library. For instance, say we want to transmit an integer value from Python:
 254         </p>
 255 <pre class="programlisting"><span class="identifier">comm</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="number">17</span><span class="special">)</span>
 256 </pre>
 257 <p>
 258           In C++, we would receive that value into a Python object and then <code class="computeroutput"><span class="identifier">extract</span></code> an integer value:
 259         </p>
 260 <pre class="programlisting"><span class="identifier">boost</span><span class="special">::</span><span class="identifier">python</span><span class="special">::</span><span class="identifier">object</span> <span class="identifier">value</span><span class="special">;</span>
 261 <span class="identifier">comm</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">value</span><span class="special">);</span>
 262 <span class="keyword">int</span> <span class="identifier">int_value</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">python</span><span class="special">::</span><span class="identifier">extract</span><span class="special">&lt;</span><span class="keyword">int</span><span class="special">&gt;(</span><span class="identifier">value</span><span class="special">);</span>
 263 </pre>
 264 <p>
 265           In the future, Boost.MPI will be extended to allow improved interoperability
 266           with the C++ Boost.MPI and the C MPI bindings.
 267         </p>
 268 </div>
 269 <div class="section">
 270 <div class="titlepage"><div><div><h4 class="title">
 271 <a name="mpi.pythonref"></a>Reference</h4></div></div></div>
 272 <p>
 273           The Boost.MPI Python module, <code class="computeroutput"><span class="identifier">boost</span><span class="special">.</span><span class="identifier">mpi</span></code>,
 274           has its own <a href="../boost.mpi.html" target="_top">reference documentation</a>,
 275           which is also available using <code class="computeroutput"><span class="identifier">pydoc</span></code>
 276           (from the command line) or <code class="computeroutput"><span class="identifier">help</span><span class="special">(</span><span class="identifier">boost</span><span class="special">.</span><span class="identifier">mpi</span><span class="special">)</span></code> (from the Python interpreter).
 277         </p>
 278 </div>
 279 </div>
 280 <div class="section">
 281 <div class="titlepage"><div><div><h3 class="title">
 282 <a name="mpi.design"></a>Design Philosophy</h3></div></div></div>
 283 <p>
 284         The design philosophy of the Parallel MPI library is very simple: be both
 285         convenient and efficient. MPI is a library built for high-performance applications,
 286         but it's FORTRAN-centric, performance-minded design makes it rather inflexible
 287         from the C++ point of view: passing a string from one process to another
 288         is inconvenient, requiring several messages and explicit buffering; passing
 289         a container of strings from one process to another requires an extra level
 290         of manual bookkeeping; and passing a map from strings to containers of strings
 291         is positively infuriating. The Parallel MPI library allows all of these data
 292         types to be passed using the same simple <code class="computeroutput"><span class="identifier">send</span><span class="special">()</span></code> and <code class="computeroutput"><span class="identifier">recv</span><span class="special">()</span></code> primitives. Likewise, collective operations
 293         such as <code class="computeroutput"><a class="link" href="../boost/mpi/reduce.html" title="Function reduce">reduce()</a></code> allow arbitrary data types
 294         and function objects, much like the C++ Standard Library would.
 295       </p>
 296 <p>
 297         The higher-level abstractions provided for convenience must not have an impact
 298         on the performance of the application. For instance, sending an integer via
 299         <code class="computeroutput"><span class="identifier">send</span></code> must be as efficient
 300         as a call to <code class="computeroutput"><span class="identifier">MPI_Send</span></code>, which
 301         means that it must be implemented by a simple call to <code class="computeroutput"><span class="identifier">MPI_Send</span></code>;
 302         likewise, an integer <code class="computeroutput"><a class="link" href="../boost/mpi/reduce.html" title="Function reduce">reduce()</a></code>
 303         using <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">plus</span><span class="special">&lt;</span><span class="keyword">int</span><span class="special">&gt;</span></code> must
 304         be implemented with a call to <code class="computeroutput"><span class="identifier">MPI_Reduce</span></code>
 305         on integers using the <code class="computeroutput"><span class="identifier">MPI_SUM</span></code>
 306         operation: anything less will impact performance. In essence, this is the
 307         "don't pay for what you don't use" principle: if the user is not
 308         transmitting strings, s/he should not pay the overhead associated with strings.
 309       </p>
 310 <p>
 311         Sometimes, achieving maximal performance means foregoing convenient abstractions
 312         and implementing certain functionality using lower-level primitives. For
 313         this reason, it is always possible to extract enough information from the
 314         abstractions in Boost.MPI to minimize the amount of effort required to interface
 315         between Boost.MPI and the C MPI library.
 316       </p>
 317 </div>
 318 <div class="section">
 319 <div class="titlepage"><div><div><h3 class="title">
 320 <a name="mpi.threading"></a>Threads</h3></div></div></div>
 321 <p>
 322         There are an increasing number of hybrid parallel applications that mix distributed
 323         and shared memory parallelism. To know how to support that model, one need
 324         to know what level of threading support is guaranteed by the MPI implementation.
 325         There are 4 ordered level of possible threading support described by <code class="computeroutput"><a class="link" href="../boost/mpi/threading/level.html" title="Type level">mpi::threading::level</a></code>. At the
 326         lowest level, you should not use threads at all, at the highest level, any
 327         thread can perform MPI call.
 328       </p>
 329 <p>
 330         If you want to use multi-threading in your MPI application, you should indicate
 331         in the environment constructor your preferred threading support. Then probe
 332         the one the library did provide, and decide what you can do with it (it could
 333         be nothing, then aborting is a valid option):
 334       </p>
 335 <pre class="programlisting"><span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">environment</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 336 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">communicator</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 337 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">iostream</span><span class="special">&gt;</span>
 338 <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
 339 <span class="keyword">namespace</span> <span class="identifier">mt</span>  <span class="special">=</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">threading</span><span class="special">;</span>
 340
 341 <span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span>
 342 <span class="special">{</span>
 343   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">(</span><span class="identifier">mt</span><span class="special">::</span><span class="identifier">funneled</span><span class="special">);</span>
 344   <span class="keyword">if</span> <span class="special">(</span><span class="identifier">env</span><span class="special">.</span><span class="identifier">thread_level</span><span class="special">()</span> <span class="special">&lt;</span> <span class="identifier">mt</span><span class="special">::</span><span class="identifier">funneled</span><span class="special">)</span> <span class="special">{</span>
 345      <span class="identifier">env</span><span class="special">.</span><span class="identifier">abort</span><span class="special">(-</span><span class="number">1</span><span class="special">);</span>
 346   <span class="special">}</span>
 347   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
 348   <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="string">"I am process "</span> <span class="special">&lt;&lt;</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">&lt;&lt;</span> <span class="string">" of "</span> <span class="special">&lt;&lt;</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span>
 349             <span class="special">&lt;&lt;</span> <span class="string">"."</span> <span class="special">&lt;&lt;</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
 350   <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
 351 <span class="special">}</span>
 352 </pre>
 353 </div>
 354 <div class="section">
 355 <div class="titlepage"><div><div><h3 class="title">
 356 <a name="mpi.performance"></a>Performance Evaluation</h3></div></div></div>
 357 <p>
 358         Message-passing performance is crucial in high-performance distributed computing.
 359         To evaluate the performance of Boost.MPI, we modified the standard <a href="http://www.scl.ameslab.gov/netpipe/" target="_top">NetPIPE</a> benchmark (version
 360         3.6.2) to use Boost.MPI and compared its performance against raw MPI. We
 361         ran five different variants of the NetPIPE benchmark:
 362       </p>
 363 <div class="orderedlist"><ol class="orderedlist" type="1">
 364 <li class="listitem">
 365             MPI: The unmodified NetPIPE benchmark.
 366           </li>
 367 <li class="listitem">
 368             Boost.MPI: NetPIPE modified to use Boost.MPI calls for communication.
 369           </li>
 370 <li class="listitem">
 371             MPI (Datatypes): NetPIPE modified to use a derived datatype (which itself
 372             contains a single <code class="computeroutput"><span class="identifier">MPI_BYTE</span></code>)
 373             rather than a fundamental datatype.
 374           </li>
 375 <li class="listitem">
 376             Boost.MPI (Datatypes): NetPIPE modified to use a user-defined type <code class="computeroutput"><span class="identifier">Char</span></code> in place of the fundamental <code class="computeroutput"><span class="keyword">char</span></code> type. The <code class="computeroutput"><span class="identifier">Char</span></code>
 377             type contains a single <code class="computeroutput"><span class="keyword">char</span></code>,
 378             a <code class="computeroutput"><span class="identifier">serialize</span><span class="special">()</span></code>
 379             method to make it serializable, and specializes <code class="computeroutput"><a class="link" href="../boost/mpi/is_mpi_datatype.html" title="Struct template is_mpi_datatype">is_mpi_datatype</a></code>
 380             to force Boost.MPI to build a derived MPI data type for it.
 381           </li>
 382 <li class="listitem">
 383             Boost.MPI (Serialized): NetPIPE modified to use a user-defined type
 384             <code class="computeroutput"><span class="identifier">Char</span></code> in place of the
 385             fundamental <code class="computeroutput"><span class="keyword">char</span></code> type. This
 386             <code class="computeroutput"><span class="identifier">Char</span></code> type contains a
 387             single <code class="computeroutput"><span class="keyword">char</span></code> and is serializable.
 388             Unlike the Datatypes case, <code class="computeroutput"><a class="link" href="../boost/mpi/is_mpi_datatype.html" title="Struct template is_mpi_datatype">is_mpi_datatype</a></code>
 389             is <span class="bold"><strong>not</strong></span> specialized, forcing Boost.MPI
 390             to perform many, many serialization calls.
 391           </li>
 392 </ol></div>
 393 <p>
 394         The actual tests were performed on the Odin cluster in the <a href="http://www.cs.indiana.edu/" target="_top">Department
 395         of Computer Science</a> at <a href="http://www.iub.edu" target="_top">Indiana University</a>,
 396         which contains 128 nodes connected via Infiniband. Each node contains 4GB
 397         memory and two AMD Opteron processors. The NetPIPE benchmarks were compiled
 398         with Intel's C++ Compiler, version 9.0, Boost 1.35.0 (prerelease), and <a href="http://www.open-mpi.org/" target="_top">Open MPI</a> version 1.1. The NetPIPE
 399         results follow:
 400       </p>
 401 <p>
 402         <span class="inlinemediaobject"><img src="../../../libs/mpi/doc/netpipe.png" alt="netpipe"></span>
 403       </p>
 404 <p>
 405         There are a some observations we can make about these NetPIPE results. First
 406         of all, the top two plots show that Boost.MPI performs on par with MPI for
 407         fundamental types. The next two plots show that Boost.MPI performs on par
 408         with MPI for derived data types, even though Boost.MPI provides a much more
 409         abstract, completely transparent approach to building derived data types
 410         than raw MPI. Overall performance for derived data types is significantly
 411         worse than for fundamental data types, but the bottleneck is in the underlying
 412         MPI implementation itself. Finally, when forcing Boost.MPI to serialize characters
 413         individually, performance suffers greatly. This particular instance is the
 414         worst possible case for Boost.MPI, because we are serializing millions of
 415         individual characters. Overall, the additional abstraction provided by Boost.MPI
 416         does not impair its performance.
 417       </p>
 418 </div>
 419 <div class="section">
 420 <div class="titlepage"><div><div><h3 class="title">
 421 <a name="mpi.history"></a>Revision History</h3></div></div></div>
 422 <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
 423 <li class="listitem">
 424             <span class="bold"><strong>Boost 1.36.0</strong></span>:
 425             <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem">
 426                   Support for non-blocking operations in Python, from Andreas Kl&#246;ckner
 427                 </li></ul></div>
 428           </li>
 429 <li class="listitem">
 430             <span class="bold"><strong>Boost 1.35.0</strong></span>: Initial release, containing
 431             the following post-review changes
 432             <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: circle; ">
 433 <li class="listitem">
 434                   Support for arrays in all collective operations
 435                 </li>
 436 <li class="listitem">
 437                   Support default-construction of <code class="computeroutput"><a class="link" href="../boost/mpi/environment.html" title="Class environment">environment</a></code>
 438                 </li>
 439 </ul></div>
 440           </li>
 441 <li class="listitem">
 442             <span class="bold"><strong>2006-09-21</strong></span>: Boost.MPI accepted into
 443             Boost.
 444           </li>
 445 </ul></div>
 446 </div>
 447 <div class="section">
 448 <div class="titlepage"><div><div><h3 class="title">
 449 <a name="mpi.acknowledge"></a>Acknowledgments</h3></div></div></div>
 450 <p>
 451         Boost.MPI was developed with support from Zurcher Kantonalbank. Daniel Egloff
 452         and Michael Gauckler contributed many ideas to Boost.MPI's design, particularly
 453         in the design of its abstractions for MPI data types and the novel skeleton/context
 454         mechanism for large data structures. Prabhanjan (Anju) Kambadur developed
 455         the predecessor to Boost.MPI that proved the usefulness of the Serialization
 456         library in an MPI setting and the performance benefits of specialization
 457         in a C++ abstraction layer for MPI. Jeremy Siek managed the formal review
 458         of Boost.MPI.
 459       </p>
 460 </div>
 461 </div>
 462 <table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
 463 <td align="left"></td>
 464 <td align="right"><div class="copyright-footer">Copyright &#169; 2005-2007 Douglas Gregor,
 465       Matthias Troyer, Trustees of Indiana University<p>
 466         Distributed under the Boost Software License, Version 1.0. (See accompanying
 467         file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">
 468         http://www.boost.org/LICENSE_1_0.txt </a>)
 469       </p>
 470 </div></td>
 471 </tr></table>
 472 <hr>
 473 <div class="spirit-nav">
 474 <a accesskey="p" href="../boost/mpi/timer.html"><img src="../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../mpi.html"><img src="../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="../multi_array.html"><img src="../../../doc/src/images/next.png" alt="Next"></a>
 475 </div>
 476 </body>
 477 </html>