More documentation on caffe2::Operator

Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/16371

Reviewed By: dzhulgakov

Differential Revision: D13820472

fbshipit-source-id: efccea0e92c86d30ec2bdda50eb9aab8a3a1504d

diff --git a/caffe2/core/operator.h b/caffe2/core/operator.h
index 37d5733..41d83aa 100644 (file)
--- a/caffe2/core/operator.h
+++ b/caffe2/core/operator.h
@@ -602,6 +602,12 @@ class Operator : public OperatorBase {
   }
   ~Operator() noexcept override {}
 
+  /// Retrieve a non-owning reference to the input at position 'idx' for this
+  /// operator.  The returned reference is valid for the duration of the
+  /// RunOnDevice call.  The optional 'type' parameter can be used to assert a
+  /// required device type for the input (by default, we assert that the tensor
+  /// is consistent with the device type implied by the Context parameter of an
+  /// Operator.)
   inline const Tensor& Input(
       int idx,
       DeviceType type = Context::GetDeviceType()) {
@@ -617,6 +623,54 @@ class Operator : public OperatorBase {
     return OperatorBase::XOutputTensor(idx, dims, options);
   }
 
+  /// Retrieve a non-owning pointer to the output at position 'idx',
+  /// initializing it to have size 'dims' and properties 'options' if
+  /// there is no pre-existing output or the pre-existing output does
+  /// not have the correct options.  The returned pointer is valid for
+  /// the duration of the RunOnDevice call.  If device is not explicitly
+  /// specified in options, we default to allocating output on the
+  /// current device of the device type implied by the Context parameter
+  /// of this Operator.
+  ///
+  /// Note [Operator::Output what?]
+  /// ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+  /// The contract of Operator::Output is somewhat complex; it is perhaps better
+  /// understood in terms of what was historically an idiomatic Caffe2 operator
+  /// implementation:
+  ///
+  ///     void RunOnDevice() override {
+  ///         auto* output = Output(0, output_size, dtype<float>());
+  ///         float* output_ptr = output->data<float>();
+  ///         // write into output_ptr
+  ///     }
+  ///
+  /// In the simple case, this code does the following things:
+  ///
+  ///   1. Allocates a new tensor with size 'output_size' and dtype 'float'
+  ///      (and device type whatever the Operator's device type is)
+  ///   2. "Registers" this tensor as the 0th output tensor of this operator
+  ///      (Caffe2 operators don't "return" outputs; instead, outputs
+  ///      are shoved into an output vector which the executor reads out.)
+  ///   3. Returns the tensor, so the operator implementation can write
+  ///      the actual output data into the tensor.
+  ///
+  /// So what's this business with "pre-existing" outputs?  Caffe2
+  /// commonly applies an optimization whereby it reuses tensors on
+  /// subsequent runs of operators in a graph.  It doesn't know ahead
+  /// of time what intermediate tensors it will need, so the first
+  /// time it runs a graph it has all of the operators create the outputs
+  /// necessary (as described above).  However, the second time around,
+  /// it will reuse all of the tensors created from the first time.
+  /// If they are lucky, this time the Output() call is a no-op and
+  /// just returns the old tensor.
+  ///
+  /// However, we cannot /guarantee/ that the output size will be the
+  /// same the next time the Operator is called; for example, output
+  /// size may be data dependent and vary between runs.  In this case,
+  /// we have to resize it to the correct size.  Resizing is still
+  /// helpful, as we may be able to fit the output in the same
+  /// space that was previously used.
+  ///
   Tensor* Output(int idx, at::IntList dims, at::TensorOptions options) {
     // We'll default device to the device of the current Operator Context
     if (options.device_opt() == c10::nullopt) {