## Compilation tests
-The tests are in the `vpu_classification_regression.cpp` file and contains only one scenario ` VpuNoRegressionWithCompilation `. To add a new test just update parameters just as in generalized scenarion of Classification/Detection test groups.
+The tests are in the `vpu_classification_regression.cpp` file and contain only one scenario ` VpuNoRegressionWithCompilation `. To add a new test just update parameters just as in generalized scenario of Classification/Detection test groups.
}
}
```
-This kernel can be rewritten to introduce special data binding `__dma_preload` and `__dma_postwrite intrinsics`. This means that instead of one kernel, a group of three kernels should be implemented: `kernelName`, `__dma_preload_kernelName` and `__dma_postwrite_kernelName`. `__dma_preload_kernelName` for a particular work group `n` is guaranteed to be executed before `n`-th work group itself, while `__dma_postwrite_kernelName` is guarantied to be executed after a corresponding work group. You can define one of those functions that are intended to be used to copy data from-to `__global` and `__local` memory. The syntactics requires exact functional signature match. The example below illustrates how to prepare your kernel for manual-DMA.
+This kernel can be rewritten to introduce special data binding `__dma_preload` and `__dma_postwrite intrinsics`. This means that instead of one kernel, a group of three kernels should be implemented: `kernelName`, `__dma_preload_kernelName` and `__dma_postwrite_kernelName`. `__dma_preload_kernelName` for a particular work group `n` is guaranteed to be executed before `n`-th work group itself, while `__dma_postwrite_kernelName` is guaranteed to be executed after a corresponding work group. You can define one of those functions that are intended to be used to copy data from-to `__global` and `__local` memory. The syntactics requires exact functional signature match. The example below illustrates how to prepare your kernel for manual-DMA.
```cpp
__kernel void __dma_preload_grn_NCHW(
__global const half* restrict src,
For 8-bit integer computations, a model must be quantized. If the model is not quantized then you can use the [Post-Training Optimization Tool](@ref pot_README) to quantize the model. The quantization process adds `FakeQuantize` layers on activations and weights for most layers. Read more about mathematical computations under the hood in the [white paper](https://intel.github.io/mkl-dnn/ex_int8_simplenet.html).
8-bit inference pipeline includes two stages (also refer to the figure below):
-1. *Offline stage*, or *model quantization*. During this stage, `FakeQuantize` layers are added before most layers to have quantized tensors before layers in a way that low-precision accuracy drop for 8-bit integer inference satisfies the specified threshold. The output of this stage is a quantized model. Quantized model precision is not changed, quantized tensors are in original precision range (`fp32`). `FakeQuantize` layer has `Quantization Levels` attribute whic defines quants count. Quants count defines precision which is used during inference. For `int8` range `Quantization Levels` attribute value has to be 255 or 256.
+1. *Offline stage*, or *model quantization*. During this stage, `FakeQuantize` layers are added before most layers to have quantized tensors before layers in a way that low-precision accuracy drop for 8-bit integer inference satisfies the specified threshold. The output of this stage is a quantized model. Quantized model precision is not changed, quantized tensors are in original precision range (`fp32`). `FakeQuantize` layer has `Quantization Levels` attribute which defines quants count. Quants count defines precision which is used during inference. For `int8` range `Quantization Levels` attribute value has to be 255 or 256.
2. *Run-time stage*. This stage is an internal procedure of the [CPU Plugin](supported_plugins/CPU.md). During this stage, the quantized model is loaded to the plugin. The plugin updates each `FakeQuantize` layer on activations and weights to have `FakeQuantize` output tensor values in low precision range.
![int8_flow]
Heterogeneous plugin was tested with the Intel® GNA as a primary device and CPU as a secondary device. To run inference of networks with layers unsupported by the GNA plugin (for example, Softmax), use the Heterogeneous plugin with the `HETERO:GNA,CPU` configuration. For the list of supported networks, see the [Supported Frameworks](#supported-frameworks).
-> **NOTE:** Due to limitation of the Intel® GNA backend library, heterogenous support is limited to cases where in the resulted sliced graph, only one subgraph is scheduled to run on GNA\_HW or GNA\_SW devices.
+> **NOTE:** Due to limitation of the Intel® GNA backend library, heterogeneous support is limited to cases where in the resulted sliced graph, only one subgraph is scheduled to run on GNA\_HW or GNA\_SW devices.
## Recovery from interruption by high-priority Windows audio processes\*
| KEY_VPU_HDDL_STREAM_ID | string | empty string | Allows to execute inference on a specified device. |
| KEY_VPU_HDDL_DEVICE_TAG | string | empty string | Allows to allocate/deallocate networks on specified devices. |
| KEY_VPU_HDDL_BIND_DEVICE | YES/NO | NO | Whether the network should bind to a device. Refer to vpu_plugin_config.hpp. |
-| KEY_VPU_HDDL_RUNTIME_PRIORITY | singed int | 0 | Specify the runtime priority of a device among all devices that running a same network Refer to vpu_plugin_config.hpp. |
+| KEY_VPU_HDDL_RUNTIME_PRIORITY | signed int | 0 | Specify the runtime priority of a device among all devices that running a same network Refer to vpu_plugin_config.hpp. |
## See Also
@snippet src/template_plugin.cpp plugin:transform_network
-> **NOTE**: After all these transformations, a `ngraph::Function` object cointains operations which can be perfectly mapped to backend kernels. E.g. if backend has kernel computing `A + B` operations at once, the `TransformNetwork` function should contain a pass which fuses operations `A` and `B` into a single custom operation `A + B` which fits backend kernels set.
+> **NOTE**: After all these transformations, a `ngraph::Function` object contains operations which can be perfectly mapped to backend kernels. E.g. if backend has kernel computing `A + B` operations at once, the `TransformNetwork` function should contain a pass which fuses operations `A` and `B` into a single custom operation `A + B` which fits backend kernels set.
### `QueryNetwork()`
Inference Engine Plugin tests are included in the `IE::funcSharedTests` CMake target which is built within the OpenVINO repository
(see [Build Plugin Using CMake](@ref plugin_build) guide). This library contains tests definitions (the tests bodies) which can be parametrized and instantiated in plugins depending on whether a plugin supports a particular feature, specific sets of parameters for test on supported operation set and so on.
-Test definitions are splitted into tests class declaration (see `inference_engine/tests/functional/plugin/shared/include`) and tests class implementation (see `inference_engine/tests/functional/plugin/shared/src`) and include the following scopes of plugin conformance tests:
+Test definitions are split into tests class declaration (see `inference_engine/tests/functional/plugin/shared/include`) and tests class implementation (see `inference_engine/tests/functional/plugin/shared/src`) and include the following scopes of plugin conformance tests:
1. **Behavior tests** (`behavior` sub-folder), which are a separate test group to check that a plugin satisfies basic Inference
Engine concepts: plugin creation, multiple executable networks support, multiple synchronous and asynchronous inference requests support, and so on. See the next section with details how to instantiate the tests definition class with plugin-specific parameters.
@snippet single_layer_tests/convolution.cpp test_convolution:instantiate
-3. **Sub-graph tests** (`subgraph_tests` sub-folder). This group of tests is designed to tests small patterns or combination of layers. E.g. when a particular topology is being enabled in a plugin e.g. TF ResNet-50, there is no need to add the whole topology to test tests. In opposite way, a particular repetative subgraph or pattern can be extracted from `ResNet-50` and added to the tests. The instantiation of the sub-graph tests is done in the same way as for single layer tests.
+3. **Sub-graph tests** (`subgraph_tests` sub-folder). This group of tests is designed to tests small patterns or combination of layers. E.g. when a particular topology is being enabled in a plugin e.g. TF ResNet-50, there is no need to add the whole topology to test tests. In opposite way, a particular repetitive subgraph or pattern can be extracted from `ResNet-50` and added to the tests. The instantiation of the sub-graph tests is done in the same way as for single layer tests.
> **Note**, such sub-graphs or patterns for sub-graph tests should be added to `IE::ngraphFunctions` library first (this library is a pre-defined set of small `ngraph::Function`) and re-used in sub-graph tests after.
4. **HETERO tests** (`subgraph_tests` sub-folder) contains tests for `HETERO` scenario (manual or automatic affinities settings, tests for `QueryNetwork`).
b. Add **Milestone** and **Labels** to the MR if it is possible.
- c. If your work is finished, assign the MR to a reviewer. If it is in progress, assing the MR to yourself (`[WIP]` case).
+ c. If your work is finished, assign the MR to a reviewer. If it is in progress, assign the MR to yourself (`[WIP]` case).
Example of an [MR](https://gitlab-icv.inn.intel.com/inference-engine/inference-engine/merge_requests/2512):
<br/>
* **Parameter name**: *end_axis*
- * **Description**: *end_axis* speficies the last dimension to flatten. The value can be negative meaning counting axes from the end.
+ * **Description**: *end_axis* specifies the last dimension to flatten. The value can be negative meaning counting axes from the end.
* **Range of values**: an integer
* **Type**: `int`
* **Default value**: -1
* **Parameter name**: *pads_end*
- * **Description**: *pads_end* specfies the number of padding elements at the end of each axis.
+ * **Description**: *pads_end* specifies the number of padding elements at the end of each axis.
* **Range of values**: a list of non-negative integers. The length of the list must be equal to the number of dimensions in the input blob.
* **Type**: `int[]`
* **Default value**: None
**Short description**: *ExperimentalSparseWeightedSum* extracts embedding vectors from the parameters table for each object feature value and sum up these embedding vectors multiplied by weights for each object.
-**Detailed description**: [Reference](https://www.tensorflow.org/api_docs/python/tf/nn/embedding_lookup_sparse). This is similar to *embedding_lookup_sparse* but it accepts objects with empty feature values for which it uses a defaut value to extract an embedding from the parameters table. In comparison with *embedding_lookup_sparse* it has a limitation to work only with two-dimensional indices tensor.
+**Detailed description**: [Reference](https://www.tensorflow.org/api_docs/python/tf/nn/embedding_lookup_sparse). This is similar to *embedding_lookup_sparse* but it accepts objects with empty feature values for which it uses a default value to extract an embedding from the parameters table. In comparison with *embedding_lookup_sparse* it has a limitation to work only with two-dimensional indices tensor.
**Inputs**:
* **1**: ND tensor. Data tensor from which rows are selected for the mean operation. Required.
* **2**: 1D tensor. Tensor of rows indices selected from the first input tensor along 0 dimension. Required.
-* **3**: 1D tensor. Tensor of segment IDs that rows selected for the operation belong to. Rows beloging to the same segment are summed up and divided by N, where N is a number of selected rows in a segment. This input has the same size as the second input. Values must be sorted in ascending order and can be repeated. Required.
+* **3**: 1D tensor. Tensor of segment IDs that rows selected for the operation belong to. Rows belonging to the same segment are summed up and divided by N, where N is a number of selected rows in a segment. This input has the same size as the second input. Values must be sorted in ascending order and can be repeated. Required.
**Outputs**:
```shell
python mo.py --input_model VGG16_faster_rcnn_final.caffemodel --input_proto test.prototxt
```
- You get the model successfuly converted to Intermediate Representation, and you can infer it with the Inference Engine.
+ You get the model successfully converted to Intermediate Representation, and you can infer it with the Inference Engine.
However, the aim of this tutorial is to demonstrate the way of supporting custom layers not yet supported by the Model Optimizer.
If you want to understand better how Model Optimizer works, remove the extension for layer `Proposal` and follow all steps of this tutorial.
* The priors tensor is not constant like in SSDs so the bounding boxes tensor must be scaled with variances [0.1, 0.1, 0.2, 0.2].
-The descibed above difference are resolved with the following code:
+The described above difference are resolved with the following code:
```python
# TF produces locations tensor without boxes for background.
jtagconfig --setparam 1 JtagClock 6M
```
-4. Store the Intel® Vision Accelerator Design with Intel® Arria® 10 FPGA bistream on the board:
+4. Store the Intel® Vision Accelerator Design with Intel® Arria® 10 FPGA bitstream on the board:
```sh
aocl flash acl0 /opt/intel/openvino/bitstreams/a10_vision_design_bitstreams/5-0_PL1_FP11_SqueezeNet.aocx
```
Each `ngraph::Node` has a unique name (used for nGraph internals) and a friendly name. In transformations we care only about friendly name because it represents the name from intermediate representation (IR).
Also friendly name is used as output tensor name (until we do not have other way to represent output tensor name) and user code that requests intermediate outputs based on these names.
-To avoid loosing friendly name when replacing node with other node or subgraph, set the original friendly name to the latest node in replacing subgraph. See the example below.
+To avoid losing friendly name when replacing node with other node or subgraph, set the original friendly name to the latest node in replacing subgraph. See the example below.
```cpp
// Replace Div operation with Power and Multiply sub-graph and set original friendly name to Multiply operation
**Short description**: *Round* performs element-wise round operation with given tensor.
-**Detailed description**: Operation takes one input tensor and rounds the values, element-wise, meaning it finds the nearest integer for each value. In case of halfs, the rule is to round them to the nearest even integer if `mode` attribute is `half_to_even` or rounding in such a way that the result heads away from zero if `mode` attribute is `half_away_from_zero`.
+**Detailed description**: Operation takes one input tensor and rounds the values, element-wise, meaning it finds the nearest integer for each value. In case of halves, the rule is to round them to the nearest even integer if `mode` attribute is `half_to_even` or rounding in such a way that the result heads away from zero if `mode` attribute is `half_away_from_zero`.
Input = [-4.5, -1.9, -1.5, 0.5, 0.9, 1.5, 2.3, 2.5]
* *mode*
- * **Description**: If set to `half_to_even` then the rule is to round halfs to the nearest even integer, if set to `half_away_from_zero` then rounding in such a way that the result heads away from zero.
+ * **Description**: If set to `half_to_even` then the rule is to round halves to the nearest even integer, if set to `half_away_from_zero` then rounding in such a way that the result heads away from zero.
* **Range of values**: `half_to_even` or `half_away_from_zero`
* **Type**: string
* **Default value**: `half_to_even`
*Example 1*
```xml
-<layer ... type="Substract">
+<layer ... type="Subtract">
<input>
<port id="0">
<dim>256</dim>
* *shape*
* **Description**: the shape of the output tensor
- * **Range of values**: list of non-negative integers, emty list is allowed that means 0D or scalar tensor
+ * **Range of values**: list of non-negative integers, empty list is allowed that means 0D or scalar tensor
* **Type**: int[]
* **Default value**: None
* **Required**: *Yes*
* **2**: `axis` - An axis along `data` to split. A scalar of type T2 with value from range `-rank(data) .. rank(data)-1`. Negative values address dimensions from the end.
**Required.**
-* **3**: `split_lengths` - A list containing the sizes of each output tensor along the split `axis`. Size of `split_lengths` should be equal to the number of outputs. The sum of sizes must match `data.shape[axis]`. A 1-D Tensor of type T2. `split_lenghts` can contain a single `-1` element, that means all remining items along specified `axis` that are not consumed by other parts. **Required.**
+* **3**: `split_lengths` - A list containing the sizes of each output tensor along the split `axis`. Size of `split_lengths` should be equal to the number of outputs. The sum of sizes must match `data.shape[axis]`. A 1-D Tensor of type T2. `split_lenghts` can contain a single `-1` element, that means all remaining items along specified `axis` that are not consumed by other parts. **Required.**
**Outputs**
*CTCLoss* operation is presented in [Connectionist Temporal Classification - Labeling Unsegmented Sequence Data with Recurrent Neural Networks: Graves et al., 2016](http://www.cs.toronto.edu/~graves/icml_2006.pdf)
-*CTCLoss* estimates likelyhood that a target `labels[i,:]` can occur (or is real) for given input sequence of logits `logits[i,:,:]`.
+*CTCLoss* estimates likelihood that a target `labels[i,:]` can occur (or is real) for given input sequence of logits `logits[i,:,:]`.
Briefly, *CTCLoss* operation finds all sequences aligned with a target `labels[i,:]`, computes log-probabilities of the aligned sequences using `logits[i,:,:]`
and computes a negative sum of these log-probabilies.
2. For a given `i`-th target from `labels[i,:]` find all aligned paths.
A path `S = (c1,c2,...,cT)` is aligned with a target `G=(g1,g2,...,gT)` if both chains are equal after decoding.
The decoding extracts substring of length `label_length[i]` from a target `G`, merges repeated characters in `G` in case *preprocess_collapse_repeated* equal to True and
-finds unique elements in the order of character occurence in case *unique* equal to True.
+finds unique elements in the order of character occurrence in case *unique* equal to True.
The decoding merges repeated characters in `S` in case *ctc_merge_repeated* equal to True and removes blank characters represented by `blank_index`.
By default, `blank_index` is equal to `C-1`, where `C` is a number of classes including the blank.
For example, in case default *ctc_merge_repeated*, *preprocess_collapse_repeated*, *unique* and `blank_index` a target sequence `G=(0,3,2,2,2,2,2,4,3)` of a length `label_length[i]=4` is processed
* *unique*
- * **Description**: *unique* is a flag to find unique elements for a target `labels[i,:]` before matching with potential alignments. Unique elements in the processed `labels[i,:]` are sorted in the order of their occurence in original `labels[i,:]`. For example, the processed sequence for `labels[i,:]=(0,1,1,0,1,3,3,2,2,3)` of length `label_length[i]=10` will be `(0,1,3,2)` in case *unique* equal to True.
+ * **Description**: *unique* is a flag to find unique elements for a target `labels[i,:]` before matching with potential alignments. Unique elements in the processed `labels[i,:]` are sorted in the order of their occurrence in original `labels[i,:]`. For example, the processed sequence for `labels[i,:]=(0,1,1,0,1,3,3,2,2,3)` of length `label_length[i]=10` will be `(0,1,3,2)` in case *unique* equal to True.
* **Range of values**: True or False
* **Type**: `boolean`
* **Default value**: False
* *activations_alpha, activations_beta*
- * **Description**: *activations_alpha, activations_beta* attributes of functions; applicability and meaning of these attributes depends on choosen activation functions
+ * **Description**: *activations_alpha, activations_beta* attributes of functions; applicability and meaning of these attributes depends on chosen activation functions
* **Range of values**: a list of floating-point numbers
* **Type**: `float[]`
* **Default value**: None
* *activations_alpha, activations_beta*
- * **Description**: *activations_alpha, activations_beta* attributes of functions; applicability and meaning of these attributes depends on choosen activation functions
+ * **Description**: *activations_alpha, activations_beta* attributes of functions; applicability and meaning of these attributes depends on chosen activation functions
* **Range of values**: a list of floating-point numbers
* **Type**: `float[]`
* **Default value**: None
* *activations_alpha, activations_beta*
- * **Description**: *activations_alpha, activations_beta* attributes of functions; applicability and meaning of these attributes depends on choosen activation functions
+ * **Description**: *activations_alpha, activations_beta* attributes of functions; applicability and meaning of these attributes depends on chosen activation functions
* **Range of values**: a list of floating-point numbers
* **Type**: `float[]`
* **Default value**: None
// and memory consumption can be larger than it is expected.
// So Inference Engone provides executors cache.
_taskExecutor = ExecutorManager::getInstance()->getIdleCPUStreamsExecutor(streamsExecutorConfig);
- // NOTE: callback Executor is not configured. So callback will be called in the thread of tha last stage of inference request pipeline
+ // NOTE: callback Executor is not configured. So callback will be called in the thread of the last stage of inference request pipeline
// _callbackExecutor = ExecutorManager::getInstance()->getIdleCPUStreamsExecutor({"TemplateCallbackExecutor"});
}
// ! [executable_network:init_executor]
- Usage examples:
- Create an `ie_core_t` t instance with a custom configuration location sepcified:
+ Create an `ie_core_t` t instance with a custom configuration location specified:
```
char *xml_config_file="/localdisk/plugins/my_custom_cfg.xml";
/**
* @struct c_mat
- * @brief OpenCV Mat Wraper
+ * @brief OpenCV Mat Wrapper
*/
typedef struct c_mat {
unsigned char *mat_data;
/**
* @brief Convert the numbers to char *;
-* @param str A pointer to the convered string .
+* @param str A pointer to the converted string.
* @param num The number to convert.
* @return none.
*/
Below is fragment of application output for CPU device:
```
-[Step 10/11] Measuring performance (Start inference asyncronously, 4 inference requests using 4 streams for CPU, limits: 60000 ms duration)
+[Step 10/11] Measuring performance (Start inference asynchronously, 4 inference requests using 4 streams for CPU, limits: 60000 ms duration)
[Step 11/11] Dumping statistics report
Count: 8904 iterations
Duration: 60045.87 ms
fillBlobs(inferRequestsQueue.requests, inputsInfo);
// ---------- 10. Measuring performance ----------------------------------------
- String ss = "Start inference " + api + "ronously";
+ String ss = "Start inference " + api + "hronously";
if (isAsync) {
if (!ss.isEmpty()) {
ss += ", ";
-nt NUMBER_TOP, --number_top NUMBER_TOP
Optional. Number of top results
--mean_val_r MEAN_VAL_R, -mean_val_r MEAN_VAL_R
- Optional. Mean value of red chanel for mean value subtraction in
+ Optional. Mean value of red channel for mean value subtraction in
postprocessing
--mean_val_g MEAN_VAL_G, -mean_val_g MEAN_VAL_G
- Optional. Mean value of green chanel for mean value subtraction
+ Optional. Mean value of green channel for mean value subtraction
in postprocessing
--mean_val_b MEAN_VAL_B, -mean_val_b MEAN_VAL_B
- Optional. Mean value of blue chanel for mean value subtraction
+ Optional. Mean value of blue channel for mean value subtraction
in postprocessing
```
type=str)
args.add_argument("-nt", "--number_top", help="Optional. Number of top results", default=10, type=int)
args.add_argument("--mean_val_r", "-mean_val_r",
- help="Optional. Mean value of red chanel for mean value subtraction in postprocessing ", default=0,
+ help="Optional. Mean value of red channel for mean value subtraction in postprocessing ", default=0,
type=float)
args.add_argument("--mean_val_g", "-mean_val_g",
- help="Optional. Mean value of green chanel for mean value subtraction in postprocessing ", default=0,
+ help="Optional. Mean value of green channel for mean value subtraction in postprocessing ", default=0,
type=float)
args.add_argument("--mean_val_b", "-mean_val_b",
- help="Optional. Mean value of blue chanel for mean value subtraction in postprocessing ", default=0,
+ help="Optional. Mean value of blue channel for mean value subtraction in postprocessing ", default=0,
type=float)
return parser
* For CPU:
```
- [Step 8/9] Measuring performance (Start inference asyncronously, 60000 ms duration, 4 inference requests in parallel using 4 streams)
+ [Step 8/9] Measuring performance (Start inference asynchronously, 60000 ms duration, 4 inference requests in parallel using 4 streams)
Progress: [....................] 100.00% done
[Step 9/9] Dumping statistics report
size_t iteration = 0;
std::stringstream ss;
- ss << "Start inference " << FLAGS_api << "ronously";
+ ss << "Start inference " << FLAGS_api << "hronously";
if (FLAGS_api == "async") {
if (!ss.str().empty()) {
ss << ", ";
return;
}
if (perfCounts.empty()) {
- slog::info << "Peformance counters are empty. No reports are dumped." << slog::endl;
+ slog::info << "Performance counters are empty. No reports are dumped." << slog::endl;
return;
}
CsvDumper dumper(true, _config.report_folder + _separator + "benchmark_" + _config.report_type + "_report.csv");
namespace FormatReader {
/**
* \class OCVMAT
- * \brief OpenCV Wraper
+ * \brief OpenCV Wrapper
*/
class OCVReader : public Reader {
private:
//
/**
- * @brief a header file with ouput classification results
+ * @brief a header file with output classification results
* @file classification_results.hpp
*/
#include <string>
scale factor (floating point scalar multiplier) required to scale the
maximum input value of the first utterance to 16384 (15 bits) is used
for all subsequent inputs. The neural network is quantized to
-accomodate the scaled input dynamic range. In user-defined
+accommodate the scaled input dynamic range. In user-defined
quantization mode, the user may specify a scale factor via the `-sf`
flag that will be used for static quantization. In dynamic
quantization mode, the scale factor for each input batch is computed
specified, the plugin will use 8-bit weights wherever possible in the
network. Note that it is not always possible to use 8-bit weights due
to GNA hardware limitations. For example, convolutional layers always
-use 16-bit weights (GNA harware verison 1 and 2). This limitation
+use 16-bit weights (GNA hardware version 1 and 2). This limitation
will be removed in GNA hardware version 3 and higher.
#### Execution Modes
projects / platform / upstream / dldt.git / commitdiff