doc/html/mpi/tutorial.html

   1 <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
   2 <html>
   3 <head>
   4 <meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
   5 <title>Tutorial</title>
   6 <link rel="stylesheet" href="../../../doc/src/boostbook.css" type="text/css">
   7 <meta name="generator" content="DocBook XSL Stylesheets V1.79.1">
   8 <link rel="home" href="../index.html" title="The Boost C++ Libraries BoostBook Documentation Subset">
   9 <link rel="up" href="../mpi.html" title="Chapter&#160;26.&#160;Boost.MPI">
  10 <link rel="prev" href="getting_started.html" title="Getting started">
  11 <link rel="next" href="c_mapping.html" title="Mapping from C MPI to Boost.MPI">
  12 </head>
  13 <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
  14 <table cellpadding="2" width="100%"><tr>
  15 <td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../boost.png"></td>
  16 <td align="center"><a href="../../../index.html">Home</a></td>
  17 <td align="center"><a href="../../../libs/libraries.htm">Libraries</a></td>
  18 <td align="center"><a href="http://www.boost.org/users/people.html">People</a></td>
  19 <td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td>
  20 <td align="center"><a href="../../../more/index.htm">More</a></td>
  21 </tr></table>
  22 <hr>
  23 <div class="spirit-nav">
  24 <a accesskey="p" href="getting_started.html"><img src="../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../mpi.html"><img src="../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="c_mapping.html"><img src="../../../doc/src/images/next.png" alt="Next"></a>
  25 </div>
  26 <div class="section">
  27 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
  28 <a name="mpi.tutorial"></a><a class="link" href="tutorial.html" title="Tutorial">Tutorial</a>
  29 </h2></div></div></div>
  30 <div class="toc"><dl class="toc">
  31 <dt><span class="section"><a href="tutorial.html#mpi.tutorial.point_to_point">Point-to-Point communication</a></span></dt>
  32 <dt><span class="section"><a href="tutorial.html#mpi.tutorial.collectives">Collective operations</a></span></dt>
  33 <dt><span class="section"><a href="tutorial.html#mpi.tutorial.user_data_types">User-defined data types</a></span></dt>
  34 <dt><span class="section"><a href="tutorial.html#mpi.tutorial.communicators">Communicators</a></span></dt>
  35 <dt><span class="section"><a href="tutorial.html#mpi.tutorial.threading">Threads</a></span></dt>
  36 <dt><span class="section"><a href="tutorial.html#mpi.tutorial.skeleton_and_content">Separating structure
  37       from content</a></span></dt>
  38 <dt><span class="section"><a href="tutorial.html#mpi.tutorial.performance_optimizations">Performance optimizations</a></span></dt>
  39 </dl></div>
  40 <p>
  41       A Boost.MPI program consists of many cooperating processes (possibly running
  42       on different computers) that communicate among themselves by passing messages.
  43       Boost.MPI is a library (as is the lower-level MPI), not a language, so the
  44       first step in a Boost.MPI is to create an <code class="computeroutput"><a class="link" href="../boost/mpi/environment.html" title="Class environment">mpi::environment</a></code>
  45       object that initializes the MPI environment and enables communication among
  46       the processes. The <code class="computeroutput"><a class="link" href="../boost/mpi/environment.html" title="Class environment">mpi::environment</a></code>
  47       object is initialized with the program arguments (which it may modify) in your
  48       main program. The creation of this object initializes MPI, and its destruction
  49       will finalize MPI. In the vast majority of Boost.MPI programs, an instance
  50       of <code class="computeroutput"><a class="link" href="../boost/mpi/environment.html" title="Class environment">mpi::environment</a></code> will
  51       be declared in <code class="computeroutput"><span class="identifier">main</span></code> at the
  52       very beginning of the program.
  53     </p>
  54 <div class="warning"><table border="0" summary="Warning">
  55 <tr>
  56 <td rowspan="2" align="center" valign="top" width="25"><img alt="[Warning]" src="../../../doc/src/images/warning.png"></td>
  57 <th align="left">Warning</th>
  58 </tr>
  59 <tr><td align="left" valign="top"><p>
  60         Declaring an <code class="computeroutput"><a class="link" href="../boost/mpi/environment.html" title="Class environment">mpi::environment</a></code>
  61         at global scope is undefined behavior. <a href="#ftn.mpi.tutorial.f0" class="footnote" name="mpi.tutorial.f0"><sup class="footnote">[11]</sup></a>
  62       </p></td></tr>
  63 </table></div>
  64 <p>
  65       Communication with MPI always occurs over a <span class="bold"><strong>communicator</strong></span>,
  66       which can be created be simply default-constructing an object of type <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html" title="Class communicator">mpi::communicator</a></code>. This communicator
  67       can then be queried to determine how many processes are running (the "size"
  68       of the communicator) and to give a unique number to each process, from zero
  69       to the size of the communicator (i.e., the "rank" of the process):
  70     </p>
  71 <pre class="programlisting"><span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">environment</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
  72 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">communicator</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
  73 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">iostream</span><span class="special">&gt;</span>
  74 <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
  75
  76 <span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span>
  77 <span class="special">{</span>
  78   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span>
  79   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
  80   <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="string">"I am process "</span> <span class="special">&lt;&lt;</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">&lt;&lt;</span> <span class="string">" of "</span> <span class="special">&lt;&lt;</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span>
  81             <span class="special">&lt;&lt;</span> <span class="string">"."</span> <span class="special">&lt;&lt;</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
  82   <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
  83 <span class="special">}</span>
  84 </pre>
  85 <p>
  86       If you run this program with 7 processes, for instance, you will receive output
  87       such as:
  88     </p>
  89 <pre class="programlisting">I am process 5 of 7.
  90 I am process 0 of 7.
  91 I am process 1 of 7.
  92 I am process 6 of 7.
  93 I am process 2 of 7.
  94 I am process 4 of 7.
  95 I am process 3 of 7.
  96 </pre>
  97 <p>
  98       Of course, the processes can execute in a different order each time, so the
  99       ranks might not be strictly increasing. More interestingly, the text could
 100       come out completely garbled, because one process can start writing "I
 101       am a process" before another process has finished writing "of 7.".
 102     </p>
 103 <p>
 104       If you should still have an MPI library supporting only MPI 1.1 you will need
 105       to pass the command line arguments to the environment constructor as shown
 106       in this example:
 107     </p>
 108 <pre class="programlisting"><span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">environment</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 109 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">communicator</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 110 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">iostream</span><span class="special">&gt;</span>
 111 <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
 112
 113 <span class="keyword">int</span> <span class="identifier">main</span><span class="special">(</span><span class="keyword">int</span> <span class="identifier">argc</span><span class="special">,</span> <span class="keyword">char</span><span class="special">*</span> <span class="identifier">argv</span><span class="special">[])</span>
 114 <span class="special">{</span>
 115   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">(</span><span class="identifier">argc</span><span class="special">,</span> <span class="identifier">argv</span><span class="special">);</span>
 116   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
 117   <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="string">"I am process "</span> <span class="special">&lt;&lt;</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">&lt;&lt;</span> <span class="string">" of "</span> <span class="special">&lt;&lt;</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span>
 118             <span class="special">&lt;&lt;</span> <span class="string">"."</span> <span class="special">&lt;&lt;</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
 119   <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
 120 <span class="special">}</span>
 121 </pre>
 122 <div class="section">
 123 <div class="titlepage"><div><div><h3 class="title">
 124 <a name="mpi.tutorial.point_to_point"></a><a class="link" href="tutorial.html#mpi.tutorial.point_to_point" title="Point-to-Point communication">Point-to-Point communication</a>
 125 </h3></div></div></div>
 126 <div class="toc"><dl class="toc">
 127 <dt><span class="section"><a href="tutorial.html#mpi.tutorial.point_to_point.blocking">Blocking communication</a></span></dt>
 128 <dt><span class="section"><a href="tutorial.html#mpi.tutorial.point_to_point.nonblocking">Non-blocking
 129         communication</a></span></dt>
 130 </dl></div>
 131 <div class="section">
 132 <div class="titlepage"><div><div><h4 class="title">
 133 <a name="mpi.tutorial.point_to_point.blocking"></a><a class="link" href="tutorial.html#mpi.tutorial.point_to_point.blocking" title="Blocking communication">Blocking communication</a>
 134 </h4></div></div></div>
 135 <p>
 136           As a message passing library, MPI's primary purpose is to routine messages
 137           from one process to another, i.e., point-to-point. MPI contains routines
 138           that can send messages, receive messages, and query whether messages are
 139           available. Each message has a source process, a target process, a tag,
 140           and a payload containing arbitrary data. The source and target processes
 141           are the ranks of the sender and receiver of the message, respectively.
 142           Tags are integers that allow the receiver to distinguish between different
 143           messages coming from the same sender.
 144         </p>
 145 <p>
 146           The following program uses two MPI processes to write "Hello, world!"
 147           to the screen (<code class="computeroutput"><span class="identifier">hello_world</span><span class="special">.</span><span class="identifier">cpp</span></code>):
 148         </p>
 149 <pre class="programlisting"><span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 150 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">iostream</span><span class="special">&gt;</span>
 151 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">string</span><span class="special">&gt;</span>
 152 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span><span class="special">/</span><span class="identifier">string</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 153 <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
 154
 155 <span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span>
 156 <span class="special">{</span>
 157   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span>
 158   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
 159
 160   <span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span>
 161     <span class="identifier">world</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">(</span><span class="string">"Hello"</span><span class="special">));</span>
 162     <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">msg</span><span class="special">;</span>
 163     <span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">1</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">);</span>
 164     <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="identifier">msg</span> <span class="special">&lt;&lt;</span> <span class="string">"!"</span> <span class="special">&lt;&lt;</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
 165   <span class="special">}</span> <span class="keyword">else</span> <span class="special">{</span>
 166     <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">msg</span><span class="special">;</span>
 167     <span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">);</span>
 168     <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="identifier">msg</span> <span class="special">&lt;&lt;</span> <span class="string">", "</span><span class="special">;</span>
 169     <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span><span class="special">.</span><span class="identifier">flush</span><span class="special">();</span>
 170     <span class="identifier">world</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">1</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">(</span><span class="string">"world"</span><span class="special">));</span>
 171   <span class="special">}</span>
 172
 173   <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
 174 <span class="special">}</span>
 175 </pre>
 176 <p>
 177           The first processor (rank 0) passes the message "Hello" to the
 178           second processor (rank 1) using tag 0. The second processor prints the
 179           string it receives, along with a comma, then passes the message "world"
 180           back to processor 0 with a different tag. The first processor then writes
 181           this message with the "!" and exits. All sends are accomplished
 182           with the <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_4-bb">communicator::send</a></code>
 183           method and all receives use a corresponding <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_9-bb">communicator::recv</a></code>
 184           call.
 185         </p>
 186 </div>
 187 <div class="section">
 188 <div class="titlepage"><div><div><h4 class="title">
 189 <a name="mpi.tutorial.point_to_point.nonblocking"></a><a class="link" href="tutorial.html#mpi.tutorial.point_to_point.nonblocking" title="Non-blocking communication">Non-blocking
 190         communication</a>
 191 </h4></div></div></div>
 192 <p>
 193           The default MPI communication operations--<code class="computeroutput"><span class="identifier">send</span></code>
 194           and <code class="computeroutput"><span class="identifier">recv</span></code>--may have to wait
 195           until the entire transmission is completed before they can return. Sometimes
 196           this <span class="bold"><strong>blocking</strong></span> behavior has a negative
 197           impact on performance, because the sender could be performing useful computation
 198           while it is waiting for the transmission to occur. More important, however,
 199           are the cases where several communication operations must occur simultaneously,
 200           e.g., a process will both send and receive at the same time.
 201         </p>
 202 <p>
 203           Let's revisit our "Hello, world!" program from the previous
 204           <a class="link" href="tutorial.html#mpi.tutorial.point_to_point.blocking" title="Blocking communication">section</a>. The
 205           core of this program transmits two messages:
 206         </p>
 207 <pre class="programlisting"><span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span>
 208   <span class="identifier">world</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">(</span><span class="string">"Hello"</span><span class="special">));</span>
 209   <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">msg</span><span class="special">;</span>
 210   <span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">1</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">);</span>
 211   <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="identifier">msg</span> <span class="special">&lt;&lt;</span> <span class="string">"!"</span> <span class="special">&lt;&lt;</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
 212 <span class="special">}</span> <span class="keyword">else</span> <span class="special">{</span>
 213   <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">msg</span><span class="special">;</span>
 214   <span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">);</span>
 215   <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="identifier">msg</span> <span class="special">&lt;&lt;</span> <span class="string">", "</span><span class="special">;</span>
 216   <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span><span class="special">.</span><span class="identifier">flush</span><span class="special">();</span>
 217   <span class="identifier">world</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">1</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">(</span><span class="string">"world"</span><span class="special">));</span>
 218 <span class="special">}</span>
 219 </pre>
 220 <p>
 221           The first process passes a message to the second process, then prepares
 222           to receive a message. The second process does the send and receive in the
 223           opposite order. However, this sequence of events is just that--a <span class="bold"><strong>sequence</strong></span>--meaning that there is essentially no parallelism.
 224           We can use non-blocking communication to ensure that the two messages are
 225           transmitted simultaneously (<code class="computeroutput"><span class="identifier">hello_world_nonblocking</span><span class="special">.</span><span class="identifier">cpp</span></code>):
 226         </p>
 227 <pre class="programlisting"><span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 228 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">iostream</span><span class="special">&gt;</span>
 229 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">string</span><span class="special">&gt;</span>
 230 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span><span class="special">/</span><span class="identifier">string</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 231 <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
 232
 233 <span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span>
 234 <span class="special">{</span>
 235   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span>
 236   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
 237
 238   <span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span>
 239     <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">request</span> <span class="identifier">reqs</span><span class="special">[</span><span class="number">2</span><span class="special">];</span>
 240     <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">msg</span><span class="special">,</span> <span class="identifier">out_msg</span> <span class="special">=</span> <span class="string">"Hello"</span><span class="special">;</span>
 241     <span class="identifier">reqs</span><span class="special">[</span><span class="number">0</span><span class="special">]</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">isend</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">out_msg</span><span class="special">);</span>
 242     <span class="identifier">reqs</span><span class="special">[</span><span class="number">1</span><span class="special">]</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">irecv</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">1</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">);</span>
 243     <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">wait_all</span><span class="special">(</span><span class="identifier">reqs</span><span class="special">,</span> <span class="identifier">reqs</span> <span class="special">+</span> <span class="number">2</span><span class="special">);</span>
 244     <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="identifier">msg</span> <span class="special">&lt;&lt;</span> <span class="string">"!"</span> <span class="special">&lt;&lt;</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
 245   <span class="special">}</span> <span class="keyword">else</span> <span class="special">{</span>
 246     <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">request</span> <span class="identifier">reqs</span><span class="special">[</span><span class="number">2</span><span class="special">];</span>
 247     <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">msg</span><span class="special">,</span> <span class="identifier">out_msg</span> <span class="special">=</span> <span class="string">"world"</span><span class="special">;</span>
 248     <span class="identifier">reqs</span><span class="special">[</span><span class="number">0</span><span class="special">]</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">isend</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">1</span><span class="special">,</span> <span class="identifier">out_msg</span><span class="special">);</span>
 249     <span class="identifier">reqs</span><span class="special">[</span><span class="number">1</span><span class="special">]</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">irecv</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">);</span>
 250     <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">wait_all</span><span class="special">(</span><span class="identifier">reqs</span><span class="special">,</span> <span class="identifier">reqs</span> <span class="special">+</span> <span class="number">2</span><span class="special">);</span>
 251     <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="identifier">msg</span> <span class="special">&lt;&lt;</span> <span class="string">", "</span><span class="special">;</span>
 252   <span class="special">}</span>
 253
 254   <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
 255 <span class="special">}</span>
 256 </pre>
 257 <p>
 258           We have replaced calls to the <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_4-bb">communicator::send</a></code>
 259           and <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_9-bb">communicator::recv</a></code>
 260           members with similar calls to their non-blocking counterparts, <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_16-bb">communicator::isend</a></code>
 261           and <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_21-bb">communicator::irecv</a></code>.
 262           The prefix <span class="bold"><strong>i</strong></span> indicates that the operations
 263           return immediately with a <code class="computeroutput"><a class="link" href="../boost/mpi/request.html" title="Class request">mpi::request</a></code>
 264           object, which allows one to query the status of a communication request
 265           (see the <code class="computeroutput"><a class="link" href="../boost/mpi/request.html#id-1_3_27_7_21_2_1_1_7_2-bb">test</a></code>
 266           method) or wait until it has completed (see the <code class="computeroutput"><a class="link" href="../boost/mpi/request.html#id-1_3_27_7_21_2_1_1_7_1-bb">wait</a></code>
 267           method). Multiple requests can be completed at the same time with the
 268           <code class="computeroutput"><a class="link" href="../boost/mpi/wait_all.html" title="Function wait_all">wait_all</a></code> operation.
 269         </p>
 270 <div class="important"><table border="0" summary="Important">
 271 <tr>
 272 <td rowspan="2" align="center" valign="top" width="25"><img alt="[Important]" src="../../../doc/src/images/important.png"></td>
 273 <th align="left">Important</th>
 274 </tr>
 275 <tr><td align="left" valign="top"><p>
 276             Regarding communication completion/progress: The MPI standard requires
 277             users to keep the request handle for a non-blocking communication, and
 278             to call the "wait" operation (or successfully test for completion)
 279             to complete the send or receive. nlike most C MPI implementations, which
 280             allow the user to discard the request for a non-blocking send, Boost.MPI
 281             requires the user to call "wait" or "test", since
 282             the request object might contain temporary buffers that have to be kept
 283             until the send is completed. Moreover, the MPI standard does not guarantee
 284             that the receive makes any progress before a call to "wait"
 285             or "test", although most implementations of the C MPI do allow
 286             receives to progress before the call to "wait" or "test".
 287             Boost.MPI, on the other hand, generally requires "test" or
 288             "wait" calls to make progress. More specifically, Boost.MPI
 289             guarantee that calling "test" multiple time will eventually
 290             complete the communication (this is due to the fact that serialized communication
 291             are potentially a multi step operation.).
 292           </p></td></tr>
 293 </table></div>
 294 <p>
 295           If you run this program multiple times, you may see some strange results:
 296           namely, some runs will produce:
 297         </p>
 298 <pre class="programlisting"><span class="identifier">Hello</span><span class="special">,</span> <span class="identifier">world</span><span class="special">!</span>
 299 </pre>
 300 <p>
 301           while others will produce:
 302         </p>
 303 <pre class="programlisting"><span class="identifier">world</span><span class="special">!</span>
 304 <span class="identifier">Hello</span><span class="special">,</span>
 305 </pre>
 306 <p>
 307           or even some garbled version of the letters in "Hello" and "world".
 308           This indicates that there is some parallelism in the program, because after
 309           both messages are (simultaneously) transmitted, both processes will concurrent
 310           execute their print statements. For both performance and correctness, non-blocking
 311           communication operations are critical to many parallel applications using
 312           MPI.
 313         </p>
 314 </div>
 315 </div>
 316 <div class="section">
 317 <div class="titlepage"><div><div><h3 class="title">
 318 <a name="mpi.tutorial.collectives"></a><a class="link" href="tutorial.html#mpi.tutorial.collectives" title="Collective operations">Collective operations</a>
 319 </h3></div></div></div>
 320 <div class="toc"><dl class="toc">
 321 <dt><span class="section"><a href="tutorial.html#mpi.tutorial.collectives.broadcast">Broadcast</a></span></dt>
 322 <dt><span class="section"><a href="tutorial.html#mpi.tutorial.collectives.gather">Gather</a></span></dt>
 323 <dt><span class="section"><a href="tutorial.html#mpi.tutorial.collectives.scatter">Scatter</a></span></dt>
 324 <dt><span class="section"><a href="tutorial.html#mpi.tutorial.collectives.reduce">Reduce</a></span></dt>
 325 </dl></div>
 326 <p>
 327         <a class="link" href="tutorial.html#mpi.tutorial.point_to_point" title="Point-to-Point communication">Point-to-point operations</a>
 328         are the core message passing primitives in Boost.MPI. However, many message-passing
 329         applications also require higher-level communication algorithms that combine
 330         or summarize the data stored on many different processes. These algorithms
 331         support many common tasks such as "broadcast this value to all processes",
 332         "compute the sum of the values on all processors" or "find
 333         the global minimum."
 334       </p>
 335 <div class="section">
 336 <div class="titlepage"><div><div><h4 class="title">
 337 <a name="mpi.tutorial.collectives.broadcast"></a><a class="link" href="tutorial.html#mpi.tutorial.collectives.broadcast" title="Broadcast">Broadcast</a>
 338 </h4></div></div></div>
 339 <p>
 340           The <code class="computeroutput"><a class="link" href="../boost/mpi/broadcast.html" title="Function broadcast">broadcast</a></code>
 341           algorithm is by far the simplest collective operation. It broadcasts a
 342           value from a single process to all other processes within a <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html" title="Class communicator">communicator</a></code>. For instance,
 343           the following program broadcasts "Hello, World!" from process
 344           0 to every other process. (<code class="computeroutput"><span class="identifier">hello_world_broadcast</span><span class="special">.</span><span class="identifier">cpp</span></code>)
 345         </p>
 346 <pre class="programlisting"><span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 347 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">iostream</span><span class="special">&gt;</span>
 348 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">string</span><span class="special">&gt;</span>
 349 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span><span class="special">/</span><span class="identifier">string</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 350 <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
 351
 352 <span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span>
 353 <span class="special">{</span>
 354   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span>
 355   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
 356
 357   <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">value</span><span class="special">;</span>
 358   <span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span>
 359     <span class="identifier">value</span> <span class="special">=</span> <span class="string">"Hello, World!"</span><span class="special">;</span>
 360   <span class="special">}</span>
 361
 362   <span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">value</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span>
 363
 364   <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="string">"Process #"</span> <span class="special">&lt;&lt;</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">&lt;&lt;</span> <span class="string">" says "</span> <span class="special">&lt;&lt;</span> <span class="identifier">value</span>
 365             <span class="special">&lt;&lt;</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
 366   <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
 367 <span class="special">}</span>
 368 </pre>
 369 <p>
 370           Running this program with seven processes will produce a result such as:
 371         </p>
 372 <pre class="programlisting">Process #0 says Hello, World!
 373 Process #2 says Hello, World!
 374 Process #1 says Hello, World!
 375 Process #4 says Hello, World!
 376 Process #3 says Hello, World!
 377 Process #5 says Hello, World!
 378 Process #6 says Hello, World!
 379 </pre>
 380 </div>
 381 <div class="section">
 382 <div class="titlepage"><div><div><h4 class="title">
 383 <a name="mpi.tutorial.collectives.gather"></a><a class="link" href="tutorial.html#mpi.tutorial.collectives.gather" title="Gather">Gather</a>
 384 </h4></div></div></div>
 385 <p>
 386           The <code class="computeroutput"><a class="link" href="../boost/mpi/gather.html" title="Function gather">gather</a></code>
 387           collective gathers the values produced by every process in a communicator
 388           into a vector of values on the "root" process (specified by an
 389           argument to <code class="computeroutput"><span class="identifier">gather</span></code>). The
 390           /i/th element in the vector will correspond to the value gathered from
 391           the /i/th process. For instance, in the following program each process
 392           computes its own random number. All of these random numbers are gathered
 393           at process 0 (the "root" in this case), which prints out the
 394           values that correspond to each processor. (<code class="computeroutput"><span class="identifier">random_gather</span><span class="special">.</span><span class="identifier">cpp</span></code>)
 395         </p>
 396 <pre class="programlisting"><span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 397 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">iostream</span><span class="special">&gt;</span>
 398 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">vector</span><span class="special">&gt;</span>
 399 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">cstdlib</span><span class="special">&gt;</span>
 400 <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
 401
 402 <span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span>
 403 <span class="special">{</span>
 404   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span>
 405   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
 406
 407   <span class="identifier">std</span><span class="special">::</span><span class="identifier">srand</span><span class="special">(</span><span class="identifier">time</span><span class="special">(</span><span class="number">0</span><span class="special">)</span> <span class="special">+</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">());</span>
 408   <span class="keyword">int</span> <span class="identifier">my_number</span> <span class="special">=</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">rand</span><span class="special">();</span>
 409   <span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span>
 410     <span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special">&lt;</span><span class="keyword">int</span><span class="special">&gt;</span> <span class="identifier">all_numbers</span><span class="special">;</span>
 411     <span class="identifier">gather</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">my_number</span><span class="special">,</span> <span class="identifier">all_numbers</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span>
 412     <span class="keyword">for</span> <span class="special">(</span><span class="keyword">int</span> <span class="identifier">proc</span> <span class="special">=</span> <span class="number">0</span><span class="special">;</span> <span class="identifier">proc</span> <span class="special">&lt;</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">();</span> <span class="special">++</span><span class="identifier">proc</span><span class="special">)</span>
 413       <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="string">"Process #"</span> <span class="special">&lt;&lt;</span> <span class="identifier">proc</span> <span class="special">&lt;&lt;</span> <span class="string">" thought of "</span>
 414                 <span class="special">&lt;&lt;</span> <span class="identifier">all_numbers</span><span class="special">[</span><span class="identifier">proc</span><span class="special">]</span> <span class="special">&lt;&lt;</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
 415   <span class="special">}</span> <span class="keyword">else</span> <span class="special">{</span>
 416     <span class="identifier">gather</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">my_number</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span>
 417   <span class="special">}</span>
 418
 419   <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
 420 <span class="special">}</span>
 421 </pre>
 422 <p>
 423           Executing this program with seven processes will result in output such
 424           as the following. Although the random values will change from one run to
 425           the next, the order of the processes in the output will remain the same
 426           because only process 0 writes to <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span></code>.
 427         </p>
 428 <pre class="programlisting">Process #0 thought of 332199874
 429 Process #1 thought of 20145617
 430 Process #2 thought of 1862420122
 431 Process #3 thought of 480422940
 432 Process #4 thought of 1253380219
 433 Process #5 thought of 949458815
 434 Process #6 thought of 650073868
 435 </pre>
 436 <p>
 437           The <code class="computeroutput"><span class="identifier">gather</span></code> operation collects
 438           values from every process into a vector at one process. If instead the
 439           values from every process need to be collected into identical vectors on
 440           every process, use the <code class="computeroutput"><a class="link" href="../boost/mpi/all_gather.html" title="Function all_gather">all_gather</a></code> algorithm,
 441           which is semantically equivalent to calling <code class="computeroutput"><span class="identifier">gather</span></code>
 442           followed by a <code class="computeroutput"><span class="identifier">broadcast</span></code>
 443           of the resulting vector.
 444         </p>
 445 </div>
 446 <div class="section">
 447 <div class="titlepage"><div><div><h4 class="title">
 448 <a name="mpi.tutorial.collectives.scatter"></a><a class="link" href="tutorial.html#mpi.tutorial.collectives.scatter" title="Scatter">Scatter</a>
 449 </h4></div></div></div>
 450 <p>
 451           The <code class="computeroutput"><a class="link" href="../boost/mpi/scatter.html" title="Function scatter">scatter</a></code>
 452           collective scatters the values from a vector in the "root" process
 453           in a communicator into values in all the processes of the communicator.
 454           The /i/th element in the vector will correspond to the value received by
 455           the /i/th process. For instance, in the following program, the root process
 456           produces a vector of random nomber and send one value to each process that
 457           will print it. (<code class="computeroutput"><span class="identifier">random_scatter</span><span class="special">.</span><span class="identifier">cpp</span></code>)
 458         </p>
 459 <pre class="programlisting"><span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 460 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">collectives</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 461 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">iostream</span><span class="special">&gt;</span>
 462 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">cstdlib</span><span class="special">&gt;</span>
 463 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">vector</span><span class="special">&gt;</span>
 464
 465 <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
 466
 467 <span class="keyword">int</span> <span class="identifier">main</span><span class="special">(</span><span class="keyword">int</span> <span class="identifier">argc</span><span class="special">,</span> <span class="keyword">char</span><span class="special">*</span> <span class="identifier">argv</span><span class="special">[])</span>
 468 <span class="special">{</span>
 469   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">(</span><span class="identifier">argc</span><span class="special">,</span> <span class="identifier">argv</span><span class="special">);</span>
 470   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
 471
 472   <span class="identifier">std</span><span class="special">::</span><span class="identifier">srand</span><span class="special">(</span><span class="identifier">time</span><span class="special">(</span><span class="number">0</span><span class="special">)</span> <span class="special">+</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">());</span>
 473   <span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special">&lt;</span><span class="keyword">int</span><span class="special">&gt;</span> <span class="identifier">all</span><span class="special">;</span>
 474   <span class="keyword">int</span> <span class="identifier">mine</span> <span class="special">=</span> <span class="special">-</span><span class="number">1</span><span class="special">;</span>
 475   <span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span>
 476     <span class="identifier">all</span><span class="special">.</span><span class="identifier">resize</span><span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">());</span>
 477     <span class="identifier">std</span><span class="special">::</span><span class="identifier">generate</span><span class="special">(</span><span class="identifier">all</span><span class="special">.</span><span class="identifier">begin</span><span class="special">(),</span> <span class="identifier">all</span><span class="special">.</span><span class="identifier">end</span><span class="special">(),</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">rand</span><span class="special">);</span>
 478   <span class="special">}</span>
 479   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">scatter</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">all</span><span class="special">,</span> <span class="identifier">mine</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span>
 480   <span class="keyword">for</span> <span class="special">(</span><span class="keyword">int</span> <span class="identifier">r</span> <span class="special">=</span> <span class="number">0</span><span class="special">;</span> <span class="identifier">r</span> <span class="special">&lt;</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">();</span> <span class="special">++</span><span class="identifier">r</span><span class="special">)</span> <span class="special">{</span>
 481     <span class="identifier">world</span><span class="special">.</span><span class="identifier">barrier</span><span class="special">();</span>
 482     <span class="keyword">if</span> <span class="special">(</span><span class="identifier">r</span> <span class="special">==</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">())</span> <span class="special">{</span>
 483       <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="string">"Rank "</span> <span class="special">&lt;&lt;</span> <span class="identifier">r</span> <span class="special">&lt;&lt;</span> <span class="string">" got "</span> <span class="special">&lt;&lt;</span> <span class="identifier">mine</span> <span class="special">&lt;&lt;</span> <span class="char">'\n'</span><span class="special">;</span>
 484     <span class="special">}</span>
 485   <span class="special">}</span>
 486   <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
 487 <span class="special">}</span>
 488 </pre>
 489 <p>
 490           Executing this program with seven processes will result in output such
 491           as the following. Although the random values will change from one run to
 492           the next, the order of the processes in the output will remain the same
 493           because of the barrier.
 494         </p>
 495 <pre class="programlisting">Rank 0 got 1409381269
 496 Rank 1 got 17045268
 497 Rank 2 got 440120016
 498 Rank 3 got 936998224
 499 Rank 4 got 1827129182
 500 Rank 5 got 1951746047
 501 Rank 6 got 2117359639
 502 </pre>
 503 </div>
 504 <div class="section">
 505 <div class="titlepage"><div><div><h4 class="title">
 506 <a name="mpi.tutorial.collectives.reduce"></a><a class="link" href="tutorial.html#mpi.tutorial.collectives.reduce" title="Reduce">Reduce</a>
 507 </h4></div></div></div>
 508 <p>
 509           The <code class="computeroutput"><a class="link" href="../boost/mpi/reduce.html" title="Function reduce">reduce</a></code>
 510           collective summarizes the values from each process into a single value
 511           at the user-specified "root" process. The Boost.MPI <code class="computeroutput"><span class="identifier">reduce</span></code> operation is similar in spirit
 512           to the STL <a href="http://www.sgi.com/tech/stl/accumulate.html" target="_top"><code class="computeroutput"><span class="identifier">accumulate</span></code></a> operation, because
 513           it takes a sequence of values (one per process) and combines them via a
 514           function object. For instance, we can randomly generate values in each
 515           process and the compute the minimum value over all processes via a call
 516           to <code class="computeroutput"><a class="link" href="../boost/mpi/reduce.html" title="Function reduce">reduce</a></code>
 517           (<code class="computeroutput"><span class="identifier">random_min</span><span class="special">.</span><span class="identifier">cpp</span></code>):
 518         </p>
 519 <pre class="programlisting"><span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 520 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">iostream</span><span class="special">&gt;</span>
 521 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">cstdlib</span><span class="special">&gt;</span>
 522 <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
 523
 524 <span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span>
 525 <span class="special">{</span>
 526   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span>
 527   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
 528
 529   <span class="identifier">std</span><span class="special">::</span><span class="identifier">srand</span><span class="special">(</span><span class="identifier">time</span><span class="special">(</span><span class="number">0</span><span class="special">)</span> <span class="special">+</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">());</span>
 530   <span class="keyword">int</span> <span class="identifier">my_number</span> <span class="special">=</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">rand</span><span class="special">();</span>
 531
 532   <span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span>
 533     <span class="keyword">int</span> <span class="identifier">minimum</span><span class="special">;</span>
 534     <span class="identifier">reduce</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">my_number</span><span class="special">,</span> <span class="identifier">minimum</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">minimum</span><span class="special">&lt;</span><span class="keyword">int</span><span class="special">&gt;(),</span> <span class="number">0</span><span class="special">);</span>
 535     <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="string">"The minimum value is "</span> <span class="special">&lt;&lt;</span> <span class="identifier">minimum</span> <span class="special">&lt;&lt;</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
 536   <span class="special">}</span> <span class="keyword">else</span> <span class="special">{</span>
 537     <span class="identifier">reduce</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">my_number</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">minimum</span><span class="special">&lt;</span><span class="keyword">int</span><span class="special">&gt;(),</span> <span class="number">0</span><span class="special">);</span>
 538   <span class="special">}</span>
 539
 540   <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
 541 <span class="special">}</span>
 542 </pre>
 543 <p>
 544           The use of <code class="computeroutput"><span class="identifier">mpi</span><span class="special">::</span><span class="identifier">minimum</span><span class="special">&lt;</span><span class="keyword">int</span><span class="special">&gt;</span></code>
 545           indicates that the minimum value should be computed. <code class="computeroutput"><span class="identifier">mpi</span><span class="special">::</span><span class="identifier">minimum</span><span class="special">&lt;</span><span class="keyword">int</span><span class="special">&gt;</span></code> is a binary function object that compares
 546           its two parameters via <code class="computeroutput"><span class="special">&lt;</span></code>
 547           and returns the smaller value. Any associative binary function or function
 548           object will work provided it's stateless. For instance, to concatenate
 549           strings with <code class="computeroutput"><span class="identifier">reduce</span></code> one
 550           could use the function object <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">plus</span><span class="special">&lt;</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">&gt;</span></code> (<code class="computeroutput"><span class="identifier">string_cat</span><span class="special">.</span><span class="identifier">cpp</span></code>):
 551         </p>
 552 <pre class="programlisting"><span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 553 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">iostream</span><span class="special">&gt;</span>
 554 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">string</span><span class="special">&gt;</span>
 555 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">functional</span><span class="special">&gt;</span>
 556 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span><span class="special">/</span><span class="identifier">string</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 557 <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
 558
 559 <span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span>
 560 <span class="special">{</span>
 561   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span>
 562   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
 563
 564   <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">names</span><span class="special">[</span><span class="number">10</span><span class="special">]</span> <span class="special">=</span> <span class="special">{</span> <span class="string">"zero "</span><span class="special">,</span> <span class="string">"one "</span><span class="special">,</span> <span class="string">"two "</span><span class="special">,</span> <span class="string">"three "</span><span class="special">,</span>
 565                             <span class="string">"four "</span><span class="special">,</span> <span class="string">"five "</span><span class="special">,</span> <span class="string">"six "</span><span class="special">,</span> <span class="string">"seven "</span><span class="special">,</span>
 566                             <span class="string">"eight "</span><span class="special">,</span> <span class="string">"nine "</span> <span class="special">};</span>
 567
 568   <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span> <span class="identifier">result</span><span class="special">;</span>
 569   <span class="identifier">reduce</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span>
 570          <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">&lt;</span> <span class="number">10</span><span class="special">?</span> <span class="identifier">names</span><span class="special">[</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()]</span>
 571                           <span class="special">:</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">(</span><span class="string">"many "</span><span class="special">),</span>
 572          <span class="identifier">result</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">plus</span><span class="special">&lt;</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">&gt;(),</span> <span class="number">0</span><span class="special">);</span>
 573
 574   <span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span>
 575     <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="string">"The result is "</span> <span class="special">&lt;&lt;</span> <span class="identifier">result</span> <span class="special">&lt;&lt;</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
 576
 577   <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
 578 <span class="special">}</span>
 579 </pre>
 580 <p>
 581           In this example, we compute a string for each process and then perform
 582           a reduction that concatenates all of the strings together into one, long
 583           string. Executing this program with seven processors yields the following
 584           output:
 585         </p>
 586 <pre class="programlisting">The result is zero one two three four five six
 587 </pre>
 588 <h5>
 589 <a name="mpi.tutorial.collectives.reduce.h0"></a>
 590           <span class="phrase"><a name="mpi.tutorial.collectives.reduce.binary_operations_for_reduce"></a></span><a class="link" href="tutorial.html#mpi.tutorial.collectives.reduce.binary_operations_for_reduce">Binary
 591           operations for reduce</a>
 592         </h5>
 593 <p>
 594           Any kind of binary function objects can be used with <code class="computeroutput"><span class="identifier">reduce</span></code>.
 595           For instance, and there are many such function objects in the C++ standard
 596           <code class="computeroutput"><span class="special">&lt;</span><span class="identifier">functional</span><span class="special">&gt;</span></code> header and the Boost.MPI header <code class="computeroutput"><span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">operations</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span></code>. Or, you can create your own function
 597           object. Function objects used with <code class="computeroutput"><span class="identifier">reduce</span></code>
 598           must be associative, i.e. <code class="computeroutput"><span class="identifier">f</span><span class="special">(</span><span class="identifier">x</span><span class="special">,</span>
 599           <span class="identifier">f</span><span class="special">(</span><span class="identifier">y</span><span class="special">,</span> <span class="identifier">z</span><span class="special">))</span></code> must be equivalent to <code class="computeroutput"><span class="identifier">f</span><span class="special">(</span><span class="identifier">f</span><span class="special">(</span><span class="identifier">x</span><span class="special">,</span> <span class="identifier">y</span><span class="special">),</span> <span class="identifier">z</span><span class="special">)</span></code>. If they are also commutative (i..e,
 600           <code class="computeroutput"><span class="identifier">f</span><span class="special">(</span><span class="identifier">x</span><span class="special">,</span> <span class="identifier">y</span><span class="special">)</span> <span class="special">==</span> <span class="identifier">f</span><span class="special">(</span><span class="identifier">y</span><span class="special">,</span>
 601           <span class="identifier">x</span><span class="special">)</span></code>),
 602           Boost.MPI can use a more efficient implementation of <code class="computeroutput"><span class="identifier">reduce</span></code>.
 603           To state that a function object is commutative, you will need to specialize
 604           the class <code class="computeroutput"><a class="link" href="../boost/mpi/is_commutative.html" title="Struct template is_commutative">is_commutative</a></code>.
 605           For instance, we could modify the previous example by telling Boost.MPI
 606           that string concatenation is commutative:
 607         </p>
 608 <pre class="programlisting"><span class="keyword">namespace</span> <span class="identifier">boost</span> <span class="special">{</span> <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">{</span>
 609
 610   <span class="keyword">template</span><span class="special">&lt;&gt;</span>
 611   <span class="keyword">struct</span> <span class="identifier">is_commutative</span><span class="special">&lt;</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">plus</span><span class="special">&lt;</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">&gt;,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span><span class="special">&gt;</span>
 612     <span class="special">:</span> <span class="identifier">mpl</span><span class="special">::</span><span class="identifier">true_</span> <span class="special">{</span> <span class="special">};</span>
 613
 614 <span class="special">}</span> <span class="special">}</span> <span class="comment">// end namespace boost::mpi</span>
 615 </pre>
 616 <p>
 617           By adding this code prior to <code class="computeroutput"><span class="identifier">main</span><span class="special">()</span></code>, Boost.MPI will assume that string concatenation
 618           is commutative and employ a different parallel algorithm for the <code class="computeroutput"><span class="identifier">reduce</span></code> operation. Using this algorithm,
 619           the program outputs the following when run with seven processes:
 620         </p>
 621 <pre class="programlisting">The result is zero one four five six two three
 622 </pre>
 623 <p>
 624           Note how the numbers in the resulting string are in a different order:
 625           this is a direct result of Boost.MPI reordering operations. The result
 626           in this case differed from the non-commutative result because string concatenation
 627           is not commutative: <code class="computeroutput"><span class="identifier">f</span><span class="special">(</span><span class="string">"x"</span><span class="special">,</span>
 628           <span class="string">"y"</span><span class="special">)</span></code>
 629           is not the same as <code class="computeroutput"><span class="identifier">f</span><span class="special">(</span><span class="string">"y"</span><span class="special">,</span>
 630           <span class="string">"x"</span><span class="special">)</span></code>,
 631           because argument order matters. For truly commutative operations (e.g.,
 632           integer addition), the more efficient commutative algorithm will produce
 633           the same result as the non-commutative algorithm. Boost.MPI also performs
 634           direct mappings from function objects in <code class="computeroutput"><span class="special">&lt;</span><span class="identifier">functional</span><span class="special">&gt;</span></code>
 635           to <code class="computeroutput"><span class="identifier">MPI_Op</span></code> values predefined
 636           by MPI (e.g., <code class="computeroutput"><span class="identifier">MPI_SUM</span></code>,
 637           <code class="computeroutput"><span class="identifier">MPI_MAX</span></code>); if you have your
 638           own function objects that can take advantage of this mapping, see the class
 639           template <code class="computeroutput"><a class="link" href="../boost/mpi/is_mpi_op.html" title="Struct template is_mpi_op">is_mpi_op</a></code>.
 640         </p>
 641 <div class="warning"><table border="0" summary="Warning">
 642 <tr>
 643 <td rowspan="2" align="center" valign="top" width="25"><img alt="[Warning]" src="../../../doc/src/images/warning.png"></td>
 644 <th align="left">Warning</th>
 645 </tr>
 646 <tr><td align="left" valign="top"><p>
 647             Due to the underlying MPI limitations, it is important to note that the
 648             operation must be stateless.
 649           </p></td></tr>
 650 </table></div>
 651 <h5>
 652 <a name="mpi.tutorial.collectives.reduce.h1"></a>
 653           <span class="phrase"><a name="mpi.tutorial.collectives.reduce.all_process_variant"></a></span><a class="link" href="tutorial.html#mpi.tutorial.collectives.reduce.all_process_variant">All process
 654           variant</a>
 655         </h5>
 656 <p>
 657           Like <a class="link" href="tutorial.html#mpi.tutorial.collectives.gather" title="Gather"><code class="computeroutput"><span class="identifier">gather</span></code></a>,
 658           <code class="computeroutput"><span class="identifier">reduce</span></code> has an "all"
 659           variant called <code class="computeroutput"><a class="link" href="../boost/mpi/all_reduce.html" title="Function all_reduce">all_reduce</a></code> that performs
 660           the reduction operation and broadcasts the result to all processes. This
 661           variant is useful, for instance, in establishing global minimum or maximum
 662           values.
 663         </p>
 664 <p>
 665           The following code (<code class="computeroutput"><span class="identifier">global_min</span><span class="special">.</span><span class="identifier">cpp</span></code>)
 666           shows a broadcasting version of the <code class="computeroutput"><span class="identifier">random_min</span><span class="special">.</span><span class="identifier">cpp</span></code>
 667           example:
 668         </p>
 669 <pre class="programlisting"><span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 670 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">iostream</span><span class="special">&gt;</span>
 671 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">cstdlib</span><span class="special">&gt;</span>
 672 <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
 673
 674 <span class="keyword">int</span> <span class="identifier">main</span><span class="special">(</span><span class="keyword">int</span> <span class="identifier">argc</span><span class="special">,</span> <span class="keyword">char</span><span class="special">*</span> <span class="identifier">argv</span><span class="special">[])</span>
 675 <span class="special">{</span>
 676   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">(</span><span class="identifier">argc</span><span class="special">,</span> <span class="identifier">argv</span><span class="special">);</span>
 677   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
 678
 679   <span class="identifier">std</span><span class="special">::</span><span class="identifier">srand</span><span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">());</span>
 680   <span class="keyword">int</span> <span class="identifier">my_number</span> <span class="special">=</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">rand</span><span class="special">();</span>
 681   <span class="keyword">int</span> <span class="identifier">minimum</span><span class="special">;</span>
 682
 683   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">all_reduce</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">my_number</span><span class="special">,</span> <span class="identifier">minimum</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">minimum</span><span class="special">&lt;</span><span class="keyword">int</span><span class="special">&gt;());</span>
 684
 685   <span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span>
 686       <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="string">"The minimum value is "</span> <span class="special">&lt;&lt;</span> <span class="identifier">minimum</span> <span class="special">&lt;&lt;</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
 687   <span class="special">}</span>
 688
 689   <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
 690 <span class="special">}</span>
 691 </pre>
 692 <p>
 693           In that example we provide both input and output values, requiring twice
 694           as much space, which can be a problem depending on the size of the transmitted
 695           data. If there is no need to preserve the input value, the output value
 696           can be omitted. In that case the input value will be overridden with the
 697           output value and Boost.MPI is able, in some situation, to implement the
 698           operation with a more space efficient solution (using the <code class="computeroutput"><span class="identifier">MPI_IN_PLACE</span></code> flag of the MPI C mapping),
 699           as in the following example (<code class="computeroutput"><span class="identifier">in_place_global_min</span><span class="special">.</span><span class="identifier">cpp</span></code>):
 700         </p>
 701 <pre class="programlisting"><span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 702 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">iostream</span><span class="special">&gt;</span>
 703 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">cstdlib</span><span class="special">&gt;</span>
 704 <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
 705
 706 <span class="keyword">int</span> <span class="identifier">main</span><span class="special">(</span><span class="keyword">int</span> <span class="identifier">argc</span><span class="special">,</span> <span class="keyword">char</span><span class="special">*</span> <span class="identifier">argv</span><span class="special">[])</span>
 707 <span class="special">{</span>
 708   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">(</span><span class="identifier">argc</span><span class="special">,</span> <span class="identifier">argv</span><span class="special">);</span>
 709   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
 710
 711   <span class="identifier">std</span><span class="special">::</span><span class="identifier">srand</span><span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">());</span>
 712   <span class="keyword">int</span> <span class="identifier">my_number</span> <span class="special">=</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">rand</span><span class="special">();</span>
 713
 714   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">all_reduce</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">my_number</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">minimum</span><span class="special">&lt;</span><span class="keyword">int</span><span class="special">&gt;());</span>
 715
 716   <span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">==</span> <span class="number">0</span><span class="special">)</span> <span class="special">{</span>
 717       <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="string">"The minimum value is "</span> <span class="special">&lt;&lt;</span> <span class="identifier">my_number</span> <span class="special">&lt;&lt;</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
 718   <span class="special">}</span>
 719
 720   <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
 721 <span class="special">}</span>
 722 </pre>
 723 </div>
 724 </div>
 725 <div class="section">
 726 <div class="titlepage"><div><div><h3 class="title">
 727 <a name="mpi.tutorial.user_data_types"></a><a class="link" href="tutorial.html#mpi.tutorial.user_data_types" title="User-defined data types">User-defined data types</a>
 728 </h3></div></div></div>
 729 <p>
 730         The inclusion of <code class="computeroutput"><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span><span class="special">/</span><span class="identifier">string</span><span class="special">.</span><span class="identifier">hpp</span></code> in the previous examples is very important:
 731         it makes values of type <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span></code>
 732         serializable, so that they can be be transmitted using Boost.MPI. In general,
 733         built-in C++ types (<code class="computeroutput"><span class="keyword">int</span></code>s, <code class="computeroutput"><span class="keyword">float</span></code>s, characters, etc.) can be transmitted
 734         over MPI directly, while user-defined and library-defined types will need
 735         to first be serialized (packed) into a format that is amenable to transmission.
 736         Boost.MPI relies on the <a href="../../../libs/serialization/doc" target="_top">Boost.Serialization</a>
 737         library to serialize and deserialize data types.
 738       </p>
 739 <p>
 740         For types defined by the standard library (such as <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span></code>
 741         or <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span></code>) and some types in Boost (such as
 742         <code class="computeroutput"><span class="identifier">boost</span><span class="special">::</span><span class="identifier">variant</span></code>), the <a href="../../../libs/serialization/doc" target="_top">Boost.Serialization</a>
 743         library already contains all of the required serialization code. In these
 744         cases, you need only include the appropriate header from the <code class="computeroutput"><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span></code> directory.
 745       </p>
 746 <p>
 747         For types that do not already have a serialization header, you will first
 748         need to implement serialization code before the types can be transmitted
 749         using Boost.MPI. Consider a simple class <a class="link" href="tutorial.html#gps_position"><code class="computeroutput"><span class="identifier">gps_position</span></code></a> that contains members
 750         <code class="computeroutput"><span class="identifier">degrees</span></code>, <code class="computeroutput"><span class="identifier">minutes</span></code>,
 751         and <code class="computeroutput"><span class="identifier">seconds</span></code>. This class is
 752         made serializable by making it a friend of <code class="computeroutput"><span class="identifier">boost</span><span class="special">::</span><span class="identifier">serialization</span><span class="special">::</span><span class="identifier">access</span></code>
 753         and introducing the templated <code class="computeroutput"><span class="identifier">serialize</span><span class="special">()</span></code> function, as follows:<a name="gps_position"></a>
 754       </p>
 755 <pre class="programlisting"><span class="keyword">class</span> <span class="identifier">gps_position</span>
 756 <span class="special">{</span>
 757 <span class="keyword">private</span><span class="special">:</span>
 758     <span class="keyword">friend</span> <span class="keyword">class</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">serialization</span><span class="special">::</span><span class="identifier">access</span><span class="special">;</span>
 759
 760     <span class="keyword">template</span><span class="special">&lt;</span><span class="keyword">class</span> <span class="identifier">Archive</span><span class="special">&gt;</span>
 761     <span class="keyword">void</span> <span class="identifier">serialize</span><span class="special">(</span><span class="identifier">Archive</span> <span class="special">&amp;</span> <span class="identifier">ar</span><span class="special">,</span> <span class="keyword">const</span> <span class="keyword">unsigned</span> <span class="keyword">int</span> <span class="identifier">version</span><span class="special">)</span>
 762     <span class="special">{</span>
 763         <span class="identifier">ar</span> <span class="special">&amp;</span> <span class="identifier">degrees</span><span class="special">;</span>
 764         <span class="identifier">ar</span> <span class="special">&amp;</span> <span class="identifier">minutes</span><span class="special">;</span>
 765         <span class="identifier">ar</span> <span class="special">&amp;</span> <span class="identifier">seconds</span><span class="special">;</span>
 766     <span class="special">}</span>
 767
 768     <span class="keyword">int</span> <span class="identifier">degrees</span><span class="special">;</span>
 769     <span class="keyword">int</span> <span class="identifier">minutes</span><span class="special">;</span>
 770     <span class="keyword">float</span> <span class="identifier">seconds</span><span class="special">;</span>
 771 <span class="keyword">public</span><span class="special">:</span>
 772     <span class="identifier">gps_position</span><span class="special">(){};</span>
 773     <span class="identifier">gps_position</span><span class="special">(</span><span class="keyword">int</span> <span class="identifier">d</span><span class="special">,</span> <span class="keyword">int</span> <span class="identifier">m</span><span class="special">,</span> <span class="keyword">float</span> <span class="identifier">s</span><span class="special">)</span> <span class="special">:</span>
 774         <span class="identifier">degrees</span><span class="special">(</span><span class="identifier">d</span><span class="special">),</span> <span class="identifier">minutes</span><span class="special">(</span><span class="identifier">m</span><span class="special">),</span> <span class="identifier">seconds</span><span class="special">(</span><span class="identifier">s</span><span class="special">)</span>
 775     <span class="special">{}</span>
 776 <span class="special">};</span>
 777 </pre>
 778 <p>
 779         Complete information about making types serializable is beyond the scope
 780         of this tutorial. For more information, please see the <a href="../../../libs/serialization/doc" target="_top">Boost.Serialization</a>
 781         library tutorial from which the above example was extracted. One important
 782         side benefit of making types serializable for Boost.MPI is that they become
 783         serializable for any other usage, such as storing the objects to disk and
 784         manipulated them in XML.
 785       </p>
 786 <p>
 787         Some serializable types, like <a class="link" href="tutorial.html#gps_position"><code class="computeroutput"><span class="identifier">gps_position</span></code></a> above, have a fixed
 788         amount of data stored at fixed offsets and are fully defined by the values
 789         of their data member (most POD with no pointers are a good example). When
 790         this is the case, Boost.MPI can optimize their serialization and transmission
 791         by avoiding extraneous copy operations. To enable this optimization, users
 792         must specialize the type trait <code class="computeroutput"><a class="link" href="../boost/mpi/is_mpi_datatype.html" title="Struct template is_mpi_datatype">is_mpi_datatype</a></code>, e.g.:
 793       </p>
 794 <pre class="programlisting"><span class="keyword">namespace</span> <span class="identifier">boost</span> <span class="special">{</span> <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">{</span>
 795   <span class="keyword">template</span> <span class="special">&lt;&gt;</span>
 796   <span class="keyword">struct</span> <span class="identifier">is_mpi_datatype</span><span class="special">&lt;</span><span class="identifier">gps_position</span><span class="special">&gt;</span> <span class="special">:</span> <span class="identifier">mpl</span><span class="special">::</span><span class="identifier">true_</span> <span class="special">{</span> <span class="special">};</span>
 797 <span class="special">}</span> <span class="special">}</span>
 798 </pre>
 799 <p>
 800         For non-template types we have defined a macro to simplify declaring a type
 801         as an MPI datatype
 802       </p>
 803 <pre class="programlisting"><span class="identifier">BOOST_IS_MPI_DATATYPE</span><span class="special">(</span><span class="identifier">gps_position</span><span class="special">)</span>
 804 </pre>
 805 <p>
 806         For composite traits, the specialization of <code class="computeroutput"><a class="link" href="../boost/mpi/is_mpi_datatype.html" title="Struct template is_mpi_datatype">is_mpi_datatype</a></code> may depend
 807         on <code class="computeroutput"><span class="identifier">is_mpi_datatype</span></code> itself.
 808         For instance, a <code class="computeroutput"><span class="identifier">boost</span><span class="special">::</span><span class="identifier">array</span></code> object is fixed only when the type
 809         of the parameter it stores is fixed:
 810       </p>
 811 <pre class="programlisting"><span class="keyword">namespace</span> <span class="identifier">boost</span> <span class="special">{</span> <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">{</span>
 812   <span class="keyword">template</span> <span class="special">&lt;</span><span class="keyword">typename</span> <span class="identifier">T</span><span class="special">,</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">size_t</span> <span class="identifier">N</span><span class="special">&gt;</span>
 813   <span class="keyword">struct</span> <span class="identifier">is_mpi_datatype</span><span class="special">&lt;</span><span class="identifier">array</span><span class="special">&lt;</span><span class="identifier">T</span><span class="special">,</span> <span class="identifier">N</span><span class="special">&gt;</span> <span class="special">&gt;</span>
 814     <span class="special">:</span> <span class="keyword">public</span> <span class="identifier">is_mpi_datatype</span><span class="special">&lt;</span><span class="identifier">T</span><span class="special">&gt;</span> <span class="special">{</span> <span class="special">};</span>
 815 <span class="special">}</span> <span class="special">}</span>
 816 </pre>
 817 <p>
 818         The redundant copy elimination optimization can only be applied when the
 819         shape of the data type is completely fixed. Variable-length types (e.g.,
 820         strings, linked lists) and types that store pointers cannot use the optimization,
 821         but Boost.MPI will be unable to detect this error at compile time. Attempting
 822         to perform this optimization when it is not correct will likely result in
 823         segmentation faults and other strange program behavior.
 824       </p>
 825 <p>
 826         Boost.MPI can transmit any user-defined data type from one process to another.
 827         Built-in types can be transmitted without any extra effort; library-defined
 828         types require the inclusion of a serialization header; and user-defined types
 829         will require the addition of serialization code. Fixed data types can be
 830         optimized for transmission using the <code class="computeroutput"><a class="link" href="../boost/mpi/is_mpi_datatype.html" title="Struct template is_mpi_datatype">is_mpi_datatype</a></code> type trait.
 831       </p>
 832 </div>
 833 <div class="section">
 834 <div class="titlepage"><div><div><h3 class="title">
 835 <a name="mpi.tutorial.communicators"></a><a class="link" href="tutorial.html#mpi.tutorial.communicators" title="Communicators">Communicators</a>
 836 </h3></div></div></div>
 837 <div class="toc"><dl class="toc">
 838 <dt><span class="section"><a href="tutorial.html#mpi.tutorial.communicators.managing">Managing communicators</a></span></dt>
 839 <dt><span class="section"><a href="tutorial.html#mpi.tutorial.communicators.cartesian_communicator">Cartesian
 840         communicator</a></span></dt>
 841 </dl></div>
 842 <div class="section">
 843 <div class="titlepage"><div><div><h4 class="title">
 844 <a name="mpi.tutorial.communicators.managing"></a><a class="link" href="tutorial.html#mpi.tutorial.communicators.managing" title="Managing communicators">Managing communicators</a>
 845 </h4></div></div></div>
 846 <p>
 847           Communication with Boost.MPI always occurs over a communicator. A communicator
 848           contains a set of processes that can send messages among themselves and
 849           perform collective operations. There can be many communicators within a
 850           single program, each of which contains its own isolated communication space
 851           that acts independently of the other communicators.
 852         </p>
 853 <p>
 854           When the MPI environment is initialized, only the "world" communicator
 855           (called <code class="computeroutput"><span class="identifier">MPI_COMM_WORLD</span></code>
 856           in the MPI C and Fortran bindings) is available. The "world"
 857           communicator, accessed by default-constructing a <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html" title="Class communicator">mpi::communicator</a></code>
 858           object, contains all of the MPI processes present when the program begins
 859           execution. Other communicators can then be constructed by duplicating or
 860           building subsets of the "world" communicator. For instance, in
 861           the following program we split the processes into two groups: one for processes
 862           generating data and the other for processes that will collect the data.
 863           (<code class="computeroutput"><span class="identifier">generate_collect</span><span class="special">.</span><span class="identifier">cpp</span></code>)
 864         </p>
 865 <pre class="programlisting"><span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 866 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">iostream</span><span class="special">&gt;</span>
 867 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">cstdlib</span><span class="special">&gt;</span>
 868 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">serialization</span><span class="special">/</span><span class="identifier">vector</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 869 <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
 870
 871 <span class="keyword">enum</span> <span class="identifier">message_tags</span> <span class="special">{</span><span class="identifier">msg_data_packet</span><span class="special">,</span> <span class="identifier">msg_broadcast_data</span><span class="special">,</span> <span class="identifier">msg_finished</span><span class="special">};</span>
 872
 873 <span class="keyword">void</span> <span class="identifier">generate_data</span><span class="special">(</span><span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">local</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">);</span>
 874 <span class="keyword">void</span> <span class="identifier">collect_data</span><span class="special">(</span><span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">local</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">);</span>
 875
 876 <span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span>
 877 <span class="special">{</span>
 878   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">;</span>
 879   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
 880
 881   <span class="keyword">bool</span> <span class="identifier">is_generator</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">&lt;</span> <span class="number">2</span> <span class="special">*</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span> <span class="special">/</span> <span class="number">3</span><span class="special">;</span>
 882   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">local</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">split</span><span class="special">(</span><span class="identifier">is_generator</span><span class="special">?</span> <span class="number">0</span> <span class="special">:</span> <span class="number">1</span><span class="special">);</span>
 883   <span class="keyword">if</span> <span class="special">(</span><span class="identifier">is_generator</span><span class="special">)</span> <span class="identifier">generate_data</span><span class="special">(</span><span class="identifier">local</span><span class="special">,</span> <span class="identifier">world</span><span class="special">);</span>
 884   <span class="keyword">else</span> <span class="identifier">collect_data</span><span class="special">(</span><span class="identifier">local</span><span class="special">,</span> <span class="identifier">world</span><span class="special">);</span>
 885
 886   <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
 887 <span class="special">}</span>
 888 </pre>
 889 <p>
 890           When communicators are split in this way, their processes retain membership
 891           in both the original communicator (which is not altered by the split) and
 892           the new communicator. However, the ranks of the processes may be different
 893           from one communicator to the next, because the rank values within a communicator
 894           are always contiguous values starting at zero. In the example above, the
 895           first two thirds of the processes become "generators" and the
 896           remaining processes become "collectors". The ranks of the "collectors"
 897           in the <code class="computeroutput"><span class="identifier">world</span></code> communicator
 898           will be 2/3 <code class="computeroutput"><span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span></code>
 899           and greater, whereas the ranks of the same collector processes in the
 900           <code class="computeroutput"><span class="identifier">local</span></code> communicator will
 901           start at zero. The following excerpt from <code class="computeroutput"><span class="identifier">collect_data</span><span class="special">()</span></code> (in <code class="computeroutput"><span class="identifier">generate_collect</span><span class="special">.</span><span class="identifier">cpp</span></code>)
 902           illustrates how to manage multiple communicators:
 903         </p>
 904 <pre class="programlisting"><span class="identifier">mpi</span><span class="special">::</span><span class="identifier">status</span> <span class="identifier">msg</span> <span class="special">=</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">probe</span><span class="special">();</span>
 905 <span class="keyword">if</span> <span class="special">(</span><span class="identifier">msg</span><span class="special">.</span><span class="identifier">tag</span><span class="special">()</span> <span class="special">==</span> <span class="identifier">msg_data_packet</span><span class="special">)</span> <span class="special">{</span>
 906   <span class="comment">// Receive the packet of data</span>
 907   <span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special">&lt;</span><span class="keyword">int</span><span class="special">&gt;</span> <span class="identifier">data</span><span class="special">;</span>
 908   <span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="identifier">msg</span><span class="special">.</span><span class="identifier">source</span><span class="special">(),</span> <span class="identifier">msg</span><span class="special">.</span><span class="identifier">tag</span><span class="special">(),</span> <span class="identifier">data</span><span class="special">);</span>
 909
 910   <span class="comment">// Tell each of the collectors that we'll be broadcasting some data</span>
 911   <span class="keyword">for</span> <span class="special">(</span><span class="keyword">int</span> <span class="identifier">dest</span> <span class="special">=</span> <span class="number">1</span><span class="special">;</span> <span class="identifier">dest</span> <span class="special">&lt;</span> <span class="identifier">local</span><span class="special">.</span><span class="identifier">size</span><span class="special">();</span> <span class="special">++</span><span class="identifier">dest</span><span class="special">)</span>
 912     <span class="identifier">local</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="identifier">dest</span><span class="special">,</span> <span class="identifier">msg_broadcast_data</span><span class="special">,</span> <span class="identifier">msg</span><span class="special">.</span><span class="identifier">source</span><span class="special">());</span>
 913
 914   <span class="comment">// Broadcast the actual data.</span>
 915   <span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">local</span><span class="special">,</span> <span class="identifier">data</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span>
 916 <span class="special">}</span>
 917 </pre>
 918 <p>
 919           The code in this except is executed by the "master" collector,
 920           e.g., the node with rank 2/3 <code class="computeroutput"><span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span></code> in the <code class="computeroutput"><span class="identifier">world</span></code>
 921           communicator and rank 0 in the <code class="computeroutput"><span class="identifier">local</span></code>
 922           (collector) communicator. It receives a message from a generator via the
 923           <code class="computeroutput"><span class="identifier">world</span></code> communicator, then
 924           broadcasts the message to each of the collectors via the <code class="computeroutput"><span class="identifier">local</span></code> communicator.
 925         </p>
 926 <p>
 927           For more control in the creation of communicators for subgroups of processes,
 928           the Boost.MPI <code class="computeroutput"><a class="link" href="../boost/mpi/group.html" title="Class group">group</a></code>
 929           provides facilities to compute the union (<code class="computeroutput"><span class="special">|</span></code>),
 930           intersection (<code class="computeroutput"><span class="special">&amp;</span></code>), and
 931           difference (<code class="computeroutput"><span class="special">-</span></code>) of two groups,
 932           generate arbitrary subgroups, etc.
 933         </p>
 934 </div>
 935 <div class="section">
 936 <div class="titlepage"><div><div><h4 class="title">
 937 <a name="mpi.tutorial.communicators.cartesian_communicator"></a><a class="link" href="tutorial.html#mpi.tutorial.communicators.cartesian_communicator" title="Cartesian communicator">Cartesian
 938         communicator</a>
 939 </h4></div></div></div>
 940 <p>
 941           A communicator can be organised as a cartesian grid, here a basic example:
 942         </p>
 943 <pre class="programlisting"><span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">vector</span><span class="special">&gt;</span>
 944 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">iostream</span><span class="special">&gt;</span>
 945
 946 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">communicator</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 947 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">collectives</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 948 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">environment</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 949 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">cartesian_communicator</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 950
 951 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">test</span><span class="special">/</span><span class="identifier">minimal</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 952
 953 <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
 954 <span class="keyword">int</span> <span class="identifier">test_main</span><span class="special">(</span><span class="keyword">int</span> <span class="identifier">argc</span><span class="special">,</span> <span class="keyword">char</span><span class="special">*</span> <span class="identifier">argv</span><span class="special">[])</span>
 955 <span class="special">{</span>
 956   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span>  <span class="identifier">env</span><span class="special">;</span>
 957   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
 958
 959   <span class="keyword">if</span> <span class="special">(</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span> <span class="special">!=</span> <span class="number">24</span><span class="special">)</span>  <span class="keyword">return</span> <span class="special">-</span><span class="number">1</span><span class="special">;</span>
 960   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">cartesian_dimension</span> <span class="identifier">dims</span><span class="special">[]</span> <span class="special">=</span> <span class="special">{{</span><span class="number">2</span><span class="special">,</span> <span class="keyword">true</span><span class="special">},</span> <span class="special">{</span><span class="number">3</span><span class="special">,</span><span class="keyword">true</span><span class="special">},</span> <span class="special">{</span><span class="number">4</span><span class="special">,</span><span class="keyword">true</span><span class="special">}};</span>
 961   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">cartesian_communicator</span> <span class="identifier">cart</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">cartesian_topology</span><span class="special">(</span><span class="identifier">dims</span><span class="special">));</span>
 962   <span class="keyword">for</span> <span class="special">(</span><span class="keyword">int</span> <span class="identifier">r</span> <span class="special">=</span> <span class="number">0</span><span class="special">;</span> <span class="identifier">r</span> <span class="special">&lt;</span> <span class="identifier">cart</span><span class="special">.</span><span class="identifier">size</span><span class="special">();</span> <span class="special">++</span><span class="identifier">r</span><span class="special">)</span> <span class="special">{</span>
 963     <span class="identifier">cart</span><span class="special">.</span><span class="identifier">barrier</span><span class="special">();</span>
 964     <span class="keyword">if</span> <span class="special">(</span><span class="identifier">r</span> <span class="special">==</span> <span class="identifier">cart</span><span class="special">.</span><span class="identifier">rank</span><span class="special">())</span> <span class="special">{</span>
 965       <span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special">&lt;</span><span class="keyword">int</span><span class="special">&gt;</span> <span class="identifier">c</span> <span class="special">=</span> <span class="identifier">cart</span><span class="special">.</span><span class="identifier">coordinates</span><span class="special">(</span><span class="identifier">r</span><span class="special">);</span>
 966       <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="string">"rk :"</span> <span class="special">&lt;&lt;</span> <span class="identifier">r</span> <span class="special">&lt;&lt;</span> <span class="string">" coords: "</span>
 967                 <span class="special">&lt;&lt;</span> <span class="identifier">c</span><span class="special">[</span><span class="number">0</span><span class="special">]</span> <span class="special">&lt;&lt;</span> <span class="char">' '</span> <span class="special">&lt;&lt;</span> <span class="identifier">c</span><span class="special">[</span><span class="number">1</span><span class="special">]</span> <span class="special">&lt;&lt;</span> <span class="char">' '</span> <span class="special">&lt;&lt;</span> <span class="identifier">c</span><span class="special">[</span><span class="number">2</span><span class="special">]</span> <span class="special">&lt;&lt;</span> <span class="char">'\n'</span><span class="special">;</span>
 968     <span class="special">}</span>
 969   <span class="special">}</span>
 970   <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
 971 <span class="special">}</span>
 972 </pre>
 973 </div>
 974 </div>
 975 <div class="section">
 976 <div class="titlepage"><div><div><h3 class="title">
 977 <a name="mpi.tutorial.threading"></a><a class="link" href="tutorial.html#mpi.tutorial.threading" title="Threads">Threads</a>
 978 </h3></div></div></div>
 979 <p>
 980         There are an increasing number of hybrid parallel applications that mix distributed
 981         and shared memory parallelism. To know how to support that model, one need
 982         to know what level of threading support is guaranteed by the MPI implementation.
 983         There are 4 ordered level of possible threading support described by <code class="computeroutput"><a class="link" href="../boost/mpi/threading/level.html" title="Type level">mpi::threading::level</a></code>. At the
 984         lowest level, you should not use threads at all, at the highest level, any
 985         thread can perform MPI call.
 986       </p>
 987 <p>
 988         If you want to use multi-threading in your MPI application, you should indicate
 989         in the environment constructor your preferred threading support. Then probe
 990         the one the library did provide, and decide what you can do with it (it could
 991         be nothing, then aborting is a valid option):
 992       </p>
 993 <pre class="programlisting"><span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">environment</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 994 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">communicator</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
 995 <span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">iostream</span><span class="special">&gt;</span>
 996 <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">mpi</span><span class="special">;</span>
 997 <span class="keyword">namespace</span> <span class="identifier">mt</span>  <span class="special">=</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">threading</span><span class="special">;</span>
 998
 999 <span class="keyword">int</span> <span class="identifier">main</span><span class="special">()</span>
1000 <span class="special">{</span>
1001   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">environment</span> <span class="identifier">env</span><span class="special">(</span><span class="identifier">mt</span><span class="special">::</span><span class="identifier">funneled</span><span class="special">);</span>
1002   <span class="keyword">if</span> <span class="special">(</span><span class="identifier">env</span><span class="special">.</span><span class="identifier">thread_level</span><span class="special">()</span> <span class="special">&lt;</span> <span class="identifier">mt</span><span class="special">::</span><span class="identifier">funneled</span><span class="special">)</span> <span class="special">{</span>
1003      <span class="identifier">env</span><span class="special">.</span><span class="identifier">abort</span><span class="special">(-</span><span class="number">1</span><span class="special">);</span>
1004   <span class="special">}</span>
1005   <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">communicator</span> <span class="identifier">world</span><span class="special">;</span>
1006   <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special">&lt;&lt;</span> <span class="string">"I am process "</span> <span class="special">&lt;&lt;</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span><span class="special">()</span> <span class="special">&lt;&lt;</span> <span class="string">" of "</span> <span class="special">&lt;&lt;</span> <span class="identifier">world</span><span class="special">.</span><span class="identifier">size</span><span class="special">()</span>
1007             <span class="special">&lt;&lt;</span> <span class="string">"."</span> <span class="special">&lt;&lt;</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
1008   <span class="keyword">return</span> <span class="number">0</span><span class="special">;</span>
1009 <span class="special">}</span>
1010 </pre>
1011 </div>
1012 <div class="section">
1013 <div class="titlepage"><div><div><h3 class="title">
1014 <a name="mpi.tutorial.skeleton_and_content"></a><a class="link" href="tutorial.html#mpi.tutorial.skeleton_and_content" title="Separating structure from content">Separating structure
1015       from content</a>
1016 </h3></div></div></div>
1017 <p>
1018         When communicating data types over MPI that are not fundamental to MPI (such
1019         as strings, lists, and user-defined data types), Boost.MPI must first serialize
1020         these data types into a buffer and then communicate them; the receiver then
1021         copies the results into a buffer before deserializing into an object on the
1022         other end. For some data types, this overhead can be eliminated by using
1023         <code class="computeroutput"><a class="link" href="../boost/mpi/is_mpi_datatype.html" title="Struct template is_mpi_datatype">is_mpi_datatype</a></code>.
1024         However, variable-length data types such as strings and lists cannot be MPI
1025         data types.
1026       </p>
1027 <p>
1028         Boost.MPI supports a second technique for improving performance by separating
1029         the structure of these variable-length data structures from the content stored
1030         in the data structures. This feature is only beneficial when the shape of
1031         the data structure remains the same but the content of the data structure
1032         will need to be communicated several times. For instance, in a finite element
1033         analysis the structure of the mesh may be fixed at the beginning of computation
1034         but the various variables on the cells of the mesh (temperature, stress,
1035         etc.) will be communicated many times within the iterative analysis process.
1036         In this case, Boost.MPI allows one to first send the "skeleton"
1037         of the mesh once, then transmit the "content" multiple times. Since
1038         the content need not contain any information about the structure of the data
1039         type, it can be transmitted without creating separate communication buffers.
1040       </p>
1041 <p>
1042         To illustrate the use of skeletons and content, we will take a somewhat more
1043         limited example wherein a master process generates random number sequences
1044         into a list and transmits them to several slave processes. The length of
1045         the list will be fixed at program startup, so the content of the list (i.e.,
1046         the current sequence of numbers) can be transmitted efficiently. The complete
1047         example is available in <code class="computeroutput"><span class="identifier">example</span><span class="special">/</span><span class="identifier">random_content</span><span class="special">.</span><span class="identifier">cpp</span></code>. We
1048         being with the master process (rank 0), which builds a list, communicates
1049         its structure via a <code class="computeroutput"><a class="link" href="reference.html#boost.mpi.skeleton">skeleton</a></code>, then repeatedly
1050         generates random number sequences to be broadcast to the slave processes
1051         via <code class="computeroutput">content</code>:
1052       </p>
1053 <pre class="programlisting"><span class="comment">// Generate the list and broadcast its structure</span>
1054 <span class="identifier">std</span><span class="special">::</span><span class="identifier">list</span><span class="special">&lt;</span><span class="keyword">int</span><span class="special">&gt;</span> <span class="identifier">l</span><span class="special">(</span><span class="identifier">list_len</span><span class="special">);</span>
1055 <span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">skeleton</span><span class="special">(</span><span class="identifier">l</span><span class="special">),</span> <span class="number">0</span><span class="special">);</span>
1056
1057 <span class="comment">// Generate content several times and broadcast out that content</span>
1058 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">content</span> <span class="identifier">c</span> <span class="special">=</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">get_content</span><span class="special">(</span><span class="identifier">l</span><span class="special">);</span>
1059 <span class="keyword">for</span> <span class="special">(</span><span class="keyword">int</span> <span class="identifier">i</span> <span class="special">=</span> <span class="number">0</span><span class="special">;</span> <span class="identifier">i</span> <span class="special">&lt;</span> <span class="identifier">iterations</span><span class="special">;</span> <span class="special">++</span><span class="identifier">i</span><span class="special">)</span> <span class="special">{</span>
1060   <span class="comment">// Generate new random values</span>
1061   <span class="identifier">std</span><span class="special">::</span><span class="identifier">generate</span><span class="special">(</span><span class="identifier">l</span><span class="special">.</span><span class="identifier">begin</span><span class="special">(),</span> <span class="identifier">l</span><span class="special">.</span><span class="identifier">end</span><span class="special">(),</span> <span class="special">&amp;</span><span class="identifier">random</span><span class="special">);</span>
1062
1063   <span class="comment">// Broadcast the new content of l</span>
1064   <span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">c</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span>
1065 <span class="special">}</span>
1066
1067 <span class="comment">// Notify the slaves that we're done by sending all zeroes</span>
1068 <span class="identifier">std</span><span class="special">::</span><span class="identifier">fill</span><span class="special">(</span><span class="identifier">l</span><span class="special">.</span><span class="identifier">begin</span><span class="special">(),</span> <span class="identifier">l</span><span class="special">.</span><span class="identifier">end</span><span class="special">(),</span> <span class="number">0</span><span class="special">);</span>
1069 <span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">c</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span>
1070 </pre>
1071 <p>
1072         The slave processes have a very similar structure to the master. They receive
1073         (via the <code class="computeroutput"><a class="link" href="../boost/mpi/broadcast.html" title="Function broadcast">broadcast()</a></code> call) the skeleton of the
1074         data structure, then use it to build their own lists of integers. In each
1075         iteration, they receive via another <code class="computeroutput"><span class="identifier">broadcast</span><span class="special">()</span></code> the new content in the data structure and
1076         compute some property of the data:
1077       </p>
1078 <pre class="programlisting"><span class="comment">// Receive the content and build up our own list</span>
1079 <span class="identifier">std</span><span class="special">::</span><span class="identifier">list</span><span class="special">&lt;</span><span class="keyword">int</span><span class="special">&gt;</span> <span class="identifier">l</span><span class="special">;</span>
1080 <span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">skeleton</span><span class="special">(</span><span class="identifier">l</span><span class="special">),</span> <span class="number">0</span><span class="special">);</span>
1081
1082 <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">content</span> <span class="identifier">c</span> <span class="special">=</span> <span class="identifier">mpi</span><span class="special">::</span><span class="identifier">get_content</span><span class="special">(</span><span class="identifier">l</span><span class="special">);</span>
1083 <span class="keyword">int</span> <span class="identifier">i</span> <span class="special">=</span> <span class="number">0</span><span class="special">;</span>
1084 <span class="keyword">do</span> <span class="special">{</span>
1085   <span class="identifier">broadcast</span><span class="special">(</span><span class="identifier">world</span><span class="special">,</span> <span class="identifier">c</span><span class="special">,</span> <span class="number">0</span><span class="special">);</span>
1086
1087   <span class="keyword">if</span> <span class="special">(</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">find_if</span>
1088        <span class="special">(</span><span class="identifier">l</span><span class="special">.</span><span class="identifier">begin</span><span class="special">(),</span> <span class="identifier">l</span><span class="special">.</span><span class="identifier">end</span><span class="special">(),</span>
1089         <span class="identifier">std</span><span class="special">::</span><span class="identifier">bind1st</span><span class="special">(</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">not_equal_to</span><span class="special">&lt;</span><span class="keyword">int</span><span class="special">&gt;(),</span> <span class="number">0</span><span class="special">))</span> <span class="special">==</span> <span class="identifier">l</span><span class="special">.</span><span class="identifier">end</span><span class="special">())</span>
1090     <span class="keyword">break</span><span class="special">;</span>
1091
1092   <span class="comment">// Compute some property of the data.</span>
1093
1094   <span class="special">++</span><span class="identifier">i</span><span class="special">;</span>
1095 <span class="special">}</span> <span class="keyword">while</span> <span class="special">(</span><span class="keyword">true</span><span class="special">);</span>
1096 </pre>
1097 <p>
1098         The skeletons and content of any Serializable data type can be transmitted
1099         either via the <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_4-bb">send</a></code> and <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html#id-1_3_27_7_6_2_1_1_3_9-bb">recv</a></code> members of the <code class="computeroutput"><a class="link" href="../boost/mpi/communicator.html" title="Class communicator">communicator</a></code>
1100         class (for point-to-point communicators) or broadcast via the <code class="computeroutput"><a class="link" href="../boost/mpi/broadcast.html" title="Function broadcast">broadcast()</a></code> collective. When separating
1101         a data structure into a skeleton and content, be careful not to modify the
1102         data structure (either on the sender side or the receiver side) without transmitting
1103         the skeleton again. Boost.MPI can not detect these accidental modifications
1104         to the data structure, which will likely result in incorrect data being transmitted
1105         or unstable programs.
1106       </p>
1107 </div>
1108 <div class="section">
1109 <div class="titlepage"><div><div><h3 class="title">
1110 <a name="mpi.tutorial.performance_optimizations"></a><a class="link" href="tutorial.html#mpi.tutorial.performance_optimizations" title="Performance optimizations">Performance optimizations</a>
1111 </h3></div></div></div>
1112 <div class="toc"><dl class="toc">
1113 <dt><span class="section"><a href="tutorial.html#mpi.tutorial.performance_optimizations.serialization_optimizations">Serialization
1114         optimizations</a></span></dt>
1115 <dt><span class="section"><a href="tutorial.html#mpi.tutorial.performance_optimizations.homogeneous_machines">Homogeneous
1116         Machines</a></span></dt>
1117 </dl></div>
1118 <div class="section">
1119 <div class="titlepage"><div><div><h4 class="title">
1120 <a name="mpi.tutorial.performance_optimizations.serialization_optimizations"></a><a class="link" href="tutorial.html#mpi.tutorial.performance_optimizations.serialization_optimizations" title="Serialization optimizations">Serialization
1121         optimizations</a>
1122 </h4></div></div></div>
1123 <p>
1124           To obtain optimal performance for small fixed-length data types not containing
1125           any pointers it is very important to mark them using the type traits of
1126           Boost.MPI and Boost.Serialization.
1127         </p>
1128 <p>
1129           It was already discussed that fixed length types containing no pointers
1130           can be using as <code class="computeroutput"><a class="link" href="../boost/mpi/is_mpi_datatype.html" title="Struct template is_mpi_datatype">is_mpi_datatype</a></code>, e.g.:
1131         </p>
1132 <pre class="programlisting"><span class="keyword">namespace</span> <span class="identifier">boost</span> <span class="special">{</span> <span class="keyword">namespace</span> <span class="identifier">mpi</span> <span class="special">{</span>
1133   <span class="keyword">template</span> <span class="special">&lt;&gt;</span>
1134   <span class="keyword">struct</span> <span class="identifier">is_mpi_datatype</span><span class="special">&lt;</span><span class="identifier">gps_position</span><span class="special">&gt;</span> <span class="special">:</span> <span class="identifier">mpl</span><span class="special">::</span><span class="identifier">true_</span> <span class="special">{</span> <span class="special">};</span>
1135 <span class="special">}</span> <span class="special">}</span>
1136 </pre>
1137 <p>
1138           or the equivalent macro
1139         </p>
1140 <pre class="programlisting"><span class="identifier">BOOST_IS_MPI_DATATYPE</span><span class="special">(</span><span class="identifier">gps_position</span><span class="special">)</span>
1141 </pre>
1142 <p>
1143           In addition it can give a substantial performance gain to turn off tracking
1144           and versioning for these types, if no pointers to these types are used,
1145           by using the traits classes or helper macros of Boost.Serialization:
1146         </p>
1147 <pre class="programlisting"><span class="identifier">BOOST_CLASS_TRACKING</span><span class="special">(</span><span class="identifier">gps_position</span><span class="special">,</span><span class="identifier">track_never</span><span class="special">)</span>
1148 <span class="identifier">BOOST_CLASS_IMPLEMENTATION</span><span class="special">(</span><span class="identifier">gps_position</span><span class="special">,</span><span class="identifier">object_serializable</span><span class="special">)</span>
1149 </pre>
1150 </div>
1151 <div class="section">
1152 <div class="titlepage"><div><div><h4 class="title">
1153 <a name="mpi.tutorial.performance_optimizations.homogeneous_machines"></a><a class="link" href="tutorial.html#mpi.tutorial.performance_optimizations.homogeneous_machines" title="Homogeneous Machines">Homogeneous
1154         Machines</a>
1155 </h4></div></div></div>
1156 <p>
1157           More optimizations are possible on homogeneous machines, by avoiding MPI_Pack/MPI_Unpack
1158           calls but using direct bitwise copy. This feature is enabled by default
1159           by defining the macro <code class="computeroutput"><a class="link" href="../BOOST_MPI_HOMOGENEOUS.html" title="Macro BOOST_MPI_HOMOGENEOUS">BOOST_MPI_HOMOGENEOUS</a></code>
1160           in the include file <code class="computeroutput"><span class="identifier">boost</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">config</span><span class="special">.</span><span class="identifier">hpp</span></code>.
1161           That definition must be consistent when building Boost.MPI and when building
1162           the application.
1163         </p>
1164 <p>
1165           In addition all classes need to be marked both as is_mpi_datatype and as
1166           is_bitwise_serializable, by using the helper macro of Boost.Serialization:
1167         </p>
1168 <pre class="programlisting"><span class="identifier">BOOST_IS_BITWISE_SERIALIZABLE</span><span class="special">(</span><span class="identifier">gps_position</span><span class="special">)</span>
1169 </pre>
1170 <p>
1171           Usually it is safe to serialize a class for which is_mpi_datatype is true
1172           by using binary copy of the bits. The exception are classes for which some
1173           members should be skipped for serialization.
1174         </p>
1175 </div>
1176 </div>
1177 <div class="footnotes">
1178 <br><hr style="width:100; text-align:left;margin-left: 0">
1179 <div id="ftn.mpi.tutorial.f0" class="footnote"><p><a href="#mpi.tutorial.f0" class="para"><sup class="para">[11] </sup></a>
1180           According to the MPI standard, initialization must take place at user's
1181           initiative after once the main function has been called.
1182         </p></div>
1183 </div>
1184 </div>
1185 <table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
1186 <td align="left"></td>
1187 <td align="right"><div class="copyright-footer">Copyright &#169; 2005-2007 Douglas Gregor,
1188       Matthias Troyer, Trustees of Indiana University<p>
1189         Distributed under the Boost Software License, Version 1.0. (See accompanying
1190         file LICENSE_1_0.txt or copy at &lt;ulink url="http://www.boost.org/LICENSE_1_0.txt"&gt;
1191         http://www.boost.org/LICENSE_1_0.txt &lt;/ulink&gt;)
1192       </p>
1193 </div></td>
1194 </tr></table>
1195 <hr>
1196 <div class="spirit-nav">
1197 <a accesskey="p" href="getting_started.html"><img src="../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../mpi.html"><img src="../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="c_mapping.html"><img src="../../../doc/src/images/next.png" alt="Next"></a>
1198 </div>
1199 </body>
1200 </html>