4 Last update: 2020-10-29
6 This document briefly explains how to use one-* commands.
7 Detailed options are not explained here. Run the command to see options.
9 Compilation flow for running with onert;
10 1) one-import will import model files generated from famous frameworks
11 2) one-optimize will optimize models. This step is optional.
12 3) one-quantize will quantize models. This step is also optional.
13 4) one-pack will pack to nnpkg so that we can run the model with our onert
16 Compilation flow for NPU
17 1) one-import will import model files generated from famous frameworks
18 2) one-optimize will optimize models. This step is optional.
19 3) one-quantize will quantize models. Depending on the NPUs.
20 4) one-codegen will compile to binary codes.
28 You can run one-commands with configuration file as well as command line parameters. The
29 configuration file should be written with the options the one-commands need to run.
32 # configuration_file.cfg
34 [The_driver_you_want_to_run]
35 input_path=/input/path/to/convert
43 You can see a template file for how to write a configuration file in `one-build.template.cfg`.
47 Sometimes you want to change certain options without touching the configuration file. If you
48 pass the option directly to the command line, the option is processed prior to the configuration
49 file. A list of options can be found in each driver's help message with `-h` option.
53 $ ./one-import tf -C my-conf.cfg -i path/to/overwrite.pb
60 one-build is an integrated driver that can execute one-commands at once. It's nice to run each
61 driver individually, but sometimes you'll want to put together the most frequently used commands
62 and run them all at once. You can do this with one-build and its configuration file.
64 For one-build, the configuration file needs 'one-build' section that consists of list of driver.
67 # one-build.template.cfg
70 one-import-tflite=False
87 See 'one-build.template.cfg' for more details.
93 one-import will invokes one-import-* commands.
95 Syntax: one-import [framework] [options]
97 Currently supported frameworks are 'tf', 'tflite' for TensorFlow and TensorFlow
104 This will convert Tensorflow model file (.pb) to our circle model file with applying BCQ.
105 To execute this command, original Tensorflow model file must include BCQ information.
107 This command invokes following scripts internally.
108 - generate_bcq_metadata : Generate BCQ metadata in the model
109 - generate_bcq_info : Designate BCQ information nodes as model output automatically
110 - tf2tfliteV2 : Convert Tensorflow model to tflite model
111 - tflite2circle : Convert Tensorflow Lite model to circle model
112 When this command is finished, BCQ information nodes will be removed if BCQ information
113 was valid and applying BCQ is done correctly without any errors.
115 As tf2tfliteV2.py runs TensorFlow lite converter, you need to have TensorFlow
116 installed in your system. We recommand to use 2.3.0 for now.
118 We provide python virtual environment and one-import-bcq will enter and leave
119 this environment so that you don't need to explictly 'activate' virtual
126 This will convert TensorFlow model (.pb) file to our circle model. You can also
127 directly call this command. one-import-tf invokes tf2tfliteV2.py script that
128 will internally use TensorFlow lite converter and then invoke tflite2circle
129 converter to convert tflite model to circle model.
131 As tf2tfliteV2.py runs TensorFlow lite converter, you need to have TensorFlow
132 installed in your system. We recommand to use 2.3.0 for now.
134 We provide python virtual environment and one-import-tf will enter and leave
135 this environment so that you don't need to explictly 'activate' virtual
142 You can use one-import-tflite to convert TensorFlow lite model (.tflite) file to
143 our circle model. Internally this will invoke tflite2circle.
149 one-optimize provides network or operator transformation shown below.
151 Current transformation options are
152 - fold_dequantize : This removes Dequantize operation which can be folded
153 - fuse_add_with_tconv: This fuses Add operator with the preceding TConv operator if possible
154 - fuse_bcq: This enables Binary-Coded-bases Quantized DNNs
155 - read https://arxiv.org/abs/2005.09904 for detailed information
156 - fuse_instnorm: This will convert instance normalization related operators to
157 one InstanceNormalization operator that our onert provides for faster
159 - fuse_preactivation_batchnorm: This fuses batch normalization operators of pre-activations to Conv operators.
160 - fuse_activation_function: This fuses Activation function to a preceding operator.
161 - make_batchnorm_gamma_positive: This makes negative gamma of batch normalization into a small positive value (1e-10).
162 Note that this pass can change the execution result of the model.
163 So, use it only when the impact is known to be acceptable.
164 - resolve_customop_add: This will convert Custom(Add) to normal Add operator
165 - resolve_customop_batchmatmul: This will convert Custom(BatchMatMul) to
166 normal BatchMatMul operator
167 - resolve_customop_matmul: This will convert Custom(MatMul) to normal MatMul
174 one-quantize will quantize float32 model to uint8 so that the model can benefit
175 for speed that our onert runtime and NPU provides. For convolution type
176 operators we currently support layer-wise quantization. Later we will support
177 int16 and channel-wise quantization.
179 Internally this calls circle-quantizer and record-minmax tools.
185 one-pack will generate a package from circle model to nnpackage for our onert
188 Output is a folder with the model(s) and meta information.
190 ex) if you have a model named '20200719.circle' and want to pack to 'testnnpack'
192 $ one-pack -i 20200709.circle -o testnnpack
205 one-codegen, like one-import will invoke backend code generation commands.
206 As of now, our ONE repo does not provide any code generation commands yet.
208 Syntax: one-codegen [target-backend] [options]
210 This will invoke [target-backend]-compile command if available.