# TensorFlow™ Support
# Introduction
-[TensorFlow™](https://www.tensorflow.org/) is a deep learning framework pioneered by Google. The NCSDK introduced TensorFlow support with the 1.09.xx NCSDK release. Validation has been done with TensorFlow r1.3. As described on the TensorFlow website, "TensorFlow™ is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them."
+[TensorFlow™](https://www.tensorflow.org/) is a deep learning framework pioneered by Google. The NCSDK introduced TensorFlow™ support with the 1.09.xx NCSDK release and TensorFlow™ 1.3. Validation for each release happens on the TensorFlow™ version noted in the release notes. As described on the TensorFlow™ website, "TensorFlow™ is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them."
* Default installation location: /opt/movidius/tensorflow
TensorFlow has a model GitHub repo at https://github.com/tensorflow/models similar to the Caffe Zoo for Caffe. The TensorFlow models GitHub repository contains several models that are maintained by the respective authors, unlike Caffe, which is not a single GitHub repo.
# Save Session with Graph and Checkpoint Information
-
+The code below shows one way to save a tensorflow session which can be compiled for the NCS.
```python
import numpy as np
import tensorflow as tf
run('inception-v1', 224, 1001)
```
# Compile for TensorFlow
-
+The command here shows how to compile the saved session from the above code sample
```
mvNCCompile output/inception-v1.meta -in=input -on=InceptionV1/Logits/Predictions/Reshape_1 -s12
```
+# Guidence for Compiling TensorFlow Networks
+If you are training a TensorFlow network you will want to [follow the guidance for creating an inference only version of the network](tf_compile_guidance.md) that is suitable for compiling via the ncsdk compiler.
+
# TensorFlow Networks Supported
* Inception V1
* Inception V2
* MobileNet_v1_0.25_192
* MobileNet_v1_0.25_160
* MobileNet_v1_0.25_128
+
+_*see release notes for supported networks for a particular release_
--- /dev/null
+# Guidence for Compiling TensorFlow™ Networks
+Below you will find general guidance for compiling a TensorFlow™ network that was built for training rather than inference. The general guidance is illustrated with changes to make to the [mnist_deep.py available from the tensorflow github repository](https://github.com/tensorflow/tensorflow/blob/r1.4/tensorflow/examples/tutorials/mnist/mnist_deep.py). The changes are shown as typical diff output where a '-' at the front of a line indicates the line is removed, and a '+' at the front of a line indicates the line should be added. Lines without a '-' or '+' are unchanged and provided for context.
+
+In order to compile a TensorFlow™ network for the NCS you will need to save a version of the network that is specific to deployment/inference and omits the training features. The following list of steps includes what users need to do to compile a typical TensorFlow™ network for the NCS. Every step may not apply to every network, but should be taken as general guidence.
+
+
+- Make sure there is a name set for the first layer of the network. This is not strictly required but makes compiling much easier because if you don't explicitly name the first and last layer you will need to determine what name those layers were given and provide those to the compiler. For mnist_deep.py you would make the following change for the first node to give it the name "input":
+
+```python
+ # Create the model
+- x = tf.placeholder(tf.float32, [None, 784])
++ x = tf.placeholder(tf.float32, [None, 784], name="input")
+```
+
+- Add tensorflow code to save the trained network. For mnist_deep.py the change to save the trained network is:
+
+```python
++ saver = tf.train.Saver()
++
+ with tf.Session() as sess:
+...
+
+ print('test accuracy %g' % accuracy.eval(feed_dict={
+ x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0}))
++
++ graph_location = "."
++ save_path = saver.save(sess, graph_location + "/mnist_model")
+```
+
+- Run the code to train the network and make sure saver.save() is called to save the trained network. After the program completes, if it was successful, saver.save() will have created the following files:
+ - mnist_model.index
+ - mnist_model.data-00000-of-00001
+ - mnist_model.meta
+
+- Remove training specific code from the network, and add code to read in the previously saved network to create an inference only version. For this step its advised that you copy the original tensorflow code to a new file and modify the new file. For example if you are working with mnist_deep.py you could copy that to mnist_deep_inference.py. Things to remove from the inference code are:
+ - Dropout layers
+ - Training specific code
+ - Reading or importing training and testing data
+ - Cross entropy/accuracy code
+ - Placeholders except the input tensor.
+
+The ncsdk compiler does not resolve unknown placeholders. Often extra placeholders are used for training specific variables so they are not necessary for inference. Placeholder variables that cannot be removed should be replaced by constants in the inference graph.
+
+
+For mnist_deep.py you would make the following changes
+
+```python
+import tempfile
+- from tensorflow.examples.tutorials.mnist import input_data
+
+...
+- # Dropout - controls the complexity of the model, prevents co-adaptation of
+- # features.
+- with tf.name_scope('dropout'):
+- keep_prob = tf.placeholder(tf.float32)
+- h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)
+
+...
+
+- y_conv = tf.matmul(h_fc1_drop, W_fc2) + b_fc2
+- return y_conv, keep_prob
++ y_conv = tf.matmul(h_fc1, W_fc2) + b_fc2
++ return y_conv
+
+...
+
+- # Import data
+- mnist = input_data.read_data_sets(FLAGS.data_dir, one_hot=True)
+
+...
+
+- # Define loss and optimizer
+- y_ = tf.placeholder(tf.float32, [None, 10])
+
+...
+
+ # Build the graph for the deep net
+- y_conv, keep_prob = deepnn(x)
++ # No longer need keep_prob since removing dropout layers.
++ y_conv = deepnn(x)
+
+...
+
+- with tf.name_scope('loss'):
+- cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=y_,
+- logits=y_conv)
+- cross_entropy = tf.reduce_mean(cross_entropy)
+-
+- with tf.name_scope('adam_optimizer'):
+- train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
+-
+- with tf.name_scope('accuracy'):
+- correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y_, 1))
+- correct_prediction = tf.cast(correct_prediction, tf.float32)
+- accuracy = tf.reduce_mean(correct_prediction)
+-
+- graph_location = tempfile.mkdtemp()
+- print('Saving graph to: %s' % graph_location)
+- train_writer = tf.summary.FileWriter(graph_location)
+- train_writer.add_graph(tf.get_default_graph())
++
++ saver = tf.train.Saver(tf.global_variables())
++
+ with tf.Session() as sess:
+ sess.run(tf.global_variables_initializer())
++ sess.run(tf.local_variables_initializer())
++ # read the previously saved network.
++ saver.restore(sess, '.' + '/mnist_model')
++ # save the version of the network ready that can be compiled for NCS
++ saver.save(sess, '.' + '/mnist_inference')
+
+- for i in range(5000):
+- batch = mnist.train.next_batch(50)
+- if i % 100 == 0:
+- train_accuracy = accuracy.eval(feed_dict={
+- x: batch[0], y_: batch[1], keep_prob: 1.0})
+- print('step %d, training accuracy %g' % (i, train_accuracy))
+- train_step.run(feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5})
+-
+- print('test accuracy %g' % accuracy.eval(feed_dict={
+- x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0}))
+- save_path = saver.save(sess, "./model.ckpt")
+```
+
+- Make sure the last node is named. As with the first node, this is not strictly required but you need to know the name to compile it. This is the change to make to mnist_deep.py in order to have a last softmax layer with a node name of "output":
+
+```python
+ # Build the graph for the deep net
+- y_conv, keep_prob = deepnn(x)
++ y_conv = deepnn(x)
++ output = tf.nn.softmax(y_conv, name='output')
+```
+
+- Run the inference version of the code to save a session that is suitable for compiling via the ncsdk compiler. This will only take a second since its not actually training the network, just resaving it in an NCS-friendly way. After you run, if successfull, the following files will be created.
+ - mnist_inference.index
+ - mnist_inference.data-00000-of-00001
+ - mnist_inference.meta
+
+- Compile the final saved network with the following command and if it all works you should see the mnist_inference.graph file created in the current directory. Note you pass in only the weights file prefix "mnist_inference" for the -w option for a TensorFlow network on the compile command line. The full command is below.
+
+```bash
+mvNCCompile mnist_inference.meta -w mnist_inference -s 12 -in input -on output -o mnist_inference.graph
+```
See also| [mvNCCompile](compile.md), [mvNCProfile](profile.md), [TensorFlow™ Info](../TensorFlowUsage.md)
## Overview
-This command line tool checks the results for an inference on the connected Intel® Movidius™ Neural Compute Stick (Intel® Movidius™ NCS) vs. the caffe results on the host computer for the provided network.
+This command line tool validates (checks) a Caffe or TensorFlow™ nerual network on the Intel® Movidius™ Neural Compute Stick (Intel® Movidius™ NCS.) The check is done by running an inference on the NCS and also on the host computer in software using the supplied network and appropriate framework libraries. The results for both inferences (NCS results vs. framework's expected results) are compared to determine a if the network passes or fails and the top 5 inference results are provided as output.
## Syntax
Argument|Description
------------ | -------------
network.prototxt(caffe)<br>network.meta(TensorFlow™)|Name of the network file.
-[-w weights_file]|Weights filename from training. (Only applies to Caffe, not to be used with TensorFlow™.) If omitted, zero weights will be used.
+[-h --help] | Display help for the command
+[-w weights_file]|Weights filename from training. For Caffe this is the .caffemodel file name. For TensorFlow™ it must be the network name prefix. If omitted zero weights will be used for Caffe models.
[-s Max # of Shaves]|Default: 1<br><br>Selects the maximum number of SHAVEs (1,2,4,8, or 12) to use for network layers.<br><br>Note: The NCS runtime code may use less than the MAX SHAVE value for some layers where measurements have typically shown no inference performance degradation (and consequently a power benefit) of using fewer SHAVEs.
[-in Input Node Name]|By default the network is processed from the input tensor. This option allows a user to select an alternative start point in the network.<br><br>This enables partial network processing. When used together with the -on option, a user can isolate one or more layers in a network for analysis.
[-on Output Node Name]|By default the network is processed through to the output tensor. This option allows a user to select an alternative end point in the network.<br><br>This enables partial network processing. When used together with the -in option, a user can isolate one or more layers in a network for analysis.
### Caffe
```bash
-mvNCCompile network.prototxt [-w weights_file] [-s Max Number of Shaves] [-in Input Node Name] [-on Output Node Name] [-is Input-Width Input-Height] [-o Output Graph Filename]
+mvNCCompile network.prototxt [-w network.caffemodel] [-s Max Number of Shaves] [-in Input Node Name] [-on Output Node Name] [-is Input-Width Input-Height] [-o Output Graph Filename]
```
### TensorFlow™
```bash
-mvNCCompile network.meta [-s Max Number of Shaves] [-in Input Node Name] [-on Output Node Name] [-is Input-Width Input-Height] [-o Output Graph Filename]
+mvNCCompile network.meta [-w network] [-s Max Number of Shaves] [-in Input Node Name] [-on Output Node Name] [-is Input-Width Input-Height] [-o Output Graph Filename]
```
Argument|Description
------------ | -------------
network.prototxt(Caffe)<br>network.meta(TensorFlow™)|Name of the network file.
-[-w weights_file]|Weights filename from training (only applies to Caffe, not to be used with TensorFlow™.) If omitted, zero weights will be used.
+[-w weights_file]|Weights filename from training. For Caffe this is typically the .caffemodel file. For TensorFlow this is typically the network.data-xxx-of-yyy file, but you should only put the prefix of that file. ie. "network". If omitted, zero weights will be used.
[-s Max # of Shaves]|Default: 1<br><br>Selects the maximum number of SHAVEs (1, 2, 4, 8, or 12) to use for network layers.<br><br>Note: The NCS runtime code may use less than the MAX SHAVE value for some layers where measurements have typically shown no inference performance degradation (and consequently a power benefit) of using fewer SHAVEs.
-[-in Input Node Name]|By default the network is processed from the input tensor. This option allows a user to select an alternative start point in the network.<br><br>This enables partial network processing. When used together with the -on option, a user can isolate one or more layers in a network for analysis.
-[-on Output Node Name]|By default the network is processed through to the output tensor. This option allows a user to select an alternative end point in the network.<br><br>This enables partial network processing. When used together with the -in option, a user can isolate one or more layers in a network for analysis. Note: Beware that the parser stops at the first instance of the output node name (e.g., a Relu following a Conv will not be processed if it shares the same name).
-[-is Input-Width Input-Height]|Input size is typically described as a part of the network. For networks that do not have dimension constraints on the input tensor, this option can be used to set the desired input dimensions.<br><br>Only two dimensions are defined because the batch size is always 1 and the number of color planes is assumed to be 3.
+[-in Input Node Name]|By default the network is processed from the input tensor. This option allows a user to select an alternative start point in the network.<br><br>This enables partial network processing. When used together with the -on option, a user can isolate one or more layers in a network for analysis. Use the name parameter available for most tensorflow layers for the first node when creating your network and pass this name into this option. For example if this is your first layer x = tf.placeholder(tf.float32, [1, 784], name='input') then you can use "-in input"
+[-on Output Node Name]|By default the network is processed through to the output tensor. This option allows a user to select an alternative end point in the network.<br><br>This enables partial network processing. When used together with the -in option, a user can isolate one or more layers in a network for analysis. Note: Beware that the parser stops at the first instance of the output node name (e.g., a Relu following a Conv will not be processed if it shares the same name). Use the name parameter for most tensorflow layers to specify the name of the node. To add an output node with a known name that doesn't change the network you can use the following.
+output = tensorflow.identity(prev_tensor, name='output')
+[-is Input-Width Input-Height]|Input size is typically described as a part of the network. For networks that do not have dimension constraints on the input tensor, this option can be used to set the desired input dimensions.<br><br>Only two dimensions are defined because the batch size is always 1 and the number of color planes is assumed to be 3. if the number of color planes for the network is is not 3, this will need to be described as part of the network and the -is option can not be used.
[-o Output Graph Filename]|Default: "graph"<br><br>Output graph container filename. If not provided, “graph” will be used.
## Known Issues
projects / platform / adaptation / npu / intel-libmvnc.git / commitdiff