~~~~~~~~
Soft backend takes a pointer to a computational graph(Model IR) and
generates artifact in form of C++ source code file, header with artifact interfaces and
-binary file that containing parameters of compiled network.
+binary file that contains parameters of compiled network.
Example of generated artifact interface:
// Setter of NN input named "layer1"
// contents of in Tensor are copied to internals of Model
- void set_layer1(const Tensor &in);
+ bool set_layer1(const Tensor &in);
+
+ // This setter is generated if NN has only one input tensor
+ // It's name is independent from operation names
+ bool setInput(const Tensor &in);
// Getter of NN output named "result"
// Model creates result object during inference and
// or another inference executed
std::stared_ptr<Tensor> get_result();
+ // This getter is generated if NN has only one output
+ // It's name is independent from operation names
+ std::stared_ptr<Tensor> getOutput();
+
// This method exists in every aftifact,
// has fixed name that is not dependent on Model
// It is responsible for inference of result tensors from input data
for (int i = 0; i < numInputs; ++i)
{
fillInput(inputTensor);
- model.set_layer1(inputTensor);
+ bool initialized = model.set_layer1(inputTensor);
+ if (!initialized)
+ throw "invalid input tensor";
model.doInference();
std::shared_ptr<Tensor> result = model.get_result();
showResult(*result);
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Main backend sequence can be found in ``BaseCodeGenerator::generate`` method:
-1. Apply ``ShapeInference`` visitor to determine output Shapes of operations,
- this simplifies artifact code: no need to compute shapes during inference.
-2. Apply ``ModelAnalyzer`` visitor to generate inference sequence and
+1. Apply ``ModelAnalyzer`` visitor to generate inference sequence and
find artifact inputs, output and temporary Tensors.
-3. Apply ``Serializer`` visitor on inference sequence generated by ``ModelAnalyzer``
+2. Apply ``Serializer`` visitor on inference sequence generated by ``ModelAnalyzer``
to create binary array of parameters.
-4. Call ``formatTensorNames`` method that adjusts input and output names
+3. Call ``formatTensorNames`` method that adjusts input and output names
to target language naming convention(remove symbols(like '/') invalid in C++ names, etc.).
-5. Create artifact output directory(set by ``--output-dir`` option,
+4. Create artifact output directory(set by ``--output-dir`` option,
possibility of this operation should be checked by driver systems).
-6. Create header file in output directory, write it contents
+5. Create header file in output directory, write it contents
by calling virtual ``materializeHeader`` method overrided
in particular soft backend class(CPPCodeGenerator, CCodeGenerator, etc).
This phase consumes data gathered by ``ModelAnalyzer``.
-7. Create code file in output directory, write it contents
+6. Create code file in output directory, write it contents
by calling virtual ``materializeCode`` method overrided
in particular soft backend class(CPPCodeGenerator, CCodeGenerator, etc);
This phase consumes data gathered by ``ModelAnalyzer``.
-8. Create and fill file with model parameters.
+7. Create and fill file with model parameters.
This file contains a header (magic number, version of protocol, etc.)
to identify this file and avoid errors of wrong model + parameters combination
- and raw data colected by serializer.
+ and raw data collected by serializer.
Inference sequence construction
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-``ModelAnalyzer`` object walks computational graph on phase 2 from general sequence
+``ModelAnalyzer`` object walks computational graph on phase 1 from general sequence
in topological order and creates layout of it's operations. This sequence is represented
by list of ``ModelAnalyzer::OpDescr`` objects.
``ModelAnalyzer::OpDescr`` object contains name of operation,
-pointer to corresponding CG Node, ids of input and output Tensors.
+pointer to corresponding CG Node, identifiers of input and output Tensors.
-Information about artifact variables(Tensors) are stored in array of ``TensorDescription`` objects,
+Information about artifact variables(Tensors) is stored in array of ``TensorDescription`` objects,
Mentioned object holds name of ``Tensor`` and it's properties(is it input/output/temporary).
-Every ``Node<VariableOp>`` node emits "input" type ``Tensor`` variable.
-Every node with name(what is not ``Node<VariableOp>``) emits "output" type ``Tensor`` variable that holds operation output.
+Every ``VariableOp`` node emits "input" type ``Tensor`` variable.
+Every node with name(what is not ``VariableOp``) emits "output" type ``Tensor`` variable that holds operation output.
Node without particular name creates temporary ``Tensor``, that is not accessible outside Model;
Serialization
Pads are stored as vectors too, but they have additional value to save:
padding type. It is not used in inference yet, because all ``Tensor`` ``Shapes`` are available at compile time.
To serialize ``Tensor`` objects (like convolution kernel, fully-connected weights) ``Serializer`` dumps ``Shape`` first,
-then actual data in form of C multidimensional array(data stored continuously like ``int a[100][100][3]``).
+then actual data in form of C multidimensional array(data stored continuously like ``int a[1][100][100][3]``).
Def files
~~~~~~~~~
-Generator has number of ``.def`` files. This files contain code snipets used to generate the artifact.
+Generator has number of ``.def`` files. This files contain code snippets used to generate the artifact.
This is classes and type declarations for artifact, library of operations, support functions.
Build system converts them into headers containing char arrays with contents of ``def`` files.
``materializeHeader`` method of derivative of ``BaseCodeGenerator`` implements generation of header file.
C++ backend generates artifact class in header file that contains:
+
+ constructor that takes path to parameter file
+ destructor that frees taken resources
+ setters of model inputs. Every function has unique name taken from CG node.
These setters correspond to "input" tensors found by ``ModelAnalyzer`` object.
+ getters of model products. Every function has unique name taken from CG node;
-These getters corrspond to "output" tensors found by `ModelAnalyzer` object.
+These getters correspond to "output" tensors found by `ModelAnalyzer` object.
Also header file contains a number of helper functions and types(``Shape``, ``Tensor``) defined in ``.def`` files.
These types and methods are needed by users of the artifact.
Constructor and destructor call support functions from included snippets to manage parameters.
-Setters and getters are trivial and contain assignments or ``return`` statements.
+Setter checks input correctness and stores given data as input of NN.
+
+Getter is trivial and contains single ``return`` statement.
``doInference`` function contains "initilizer" section and actual inference.
"initializer" section resets all "output" variables so reference to them are dropped.
projects / platform / core / ml / nnfw.git / commitdiff