[IE] Add batched blob support (#2203)
authorRafik Saliev <rafik.f.saliev@intel.com>
Tue, 3 Nov 2020 18:19:26 +0000 (19:19 +0100)
committerGitHub <noreply@github.com>
Tue, 3 Nov 2020 18:19:26 +0000 (21:19 +0300)
commitd225ba6e5369b6697a99e8b855b3aeb3941b5859
tree11a2ac0fd1111bd311f4a9c74dbb7c4047029bc4
parentd682950e6463109722899085004fefa20e773072
[IE] Add batched blob support (#2203)

* [IE] Add batched blob support

New `class BatchedBlob : public CompoundBlob` defined to allow to pass multiple blobs as 1 InferRequest input.

Motivation: There is the special user case when a number of plain images (e.g. `NV12Blob`) should be passed as one input for network which batch size > 1.

`class CompoundBlob` is not applicable for such cases due to:
1. `NV12Blob` is `CompoundBlob` which prevents to combine multiple NV12 images to a CompoundBlob
2. The default behavior in most of plugins - do not accept generic CompoundBlob as `SetBlob()` argument

Adding `SetBlob(name, vector<Blob::Ptr>...)` to `class IInferRequest`, `class InferRequest`, `class IInferRequestInternal`, ...  - is not effective solution due to limited and specific use cases for `batched inputs`.

+ Apply rule-of-zero to CompoundBlob and inherited classes.

* Add "BATCHED_BLOB" optimization capability metric

* Add BatchedBlob usage to hello_nv12_input_classification

* Apply offline code review outcome:

1. Revert CompoundBlob public .ctors signatures
2. Remove 'workaround' .ctor for `BatchedBlob`
3. Revert tensor descriptors of `I420Blob` `NV12Blob` back to the 'fake' value.

* Code review fix

* Add functional tests for CPU, GPU, MULTI, HETERO

* update doc comment

* Apply code review change requests.
inference-engine/include/ie_compound_blob.h
inference-engine/include/ie_plugin_config.hpp
inference-engine/samples/hello_nv12_input_classification/main.cpp
inference-engine/src/inference_engine/ie_compound_blob.cpp
inference-engine/tests/functional/plugin/cpu/shared_tests_instances/behavior/set_blob_of_kind.cpp [new file with mode: 0644]
inference-engine/tests/functional/plugin/gpu/shared_tests_instances/behavior/set_blob_of_kind.cpp [new file with mode: 0644]
inference-engine/tests/functional/plugin/shared/include/behavior/set_blob_of_kind.hpp [new file with mode: 0644]
inference-engine/tests/functional/plugin/shared/src/behavior/set_blob_of_kind.cpp [new file with mode: 0644]
inference-engine/tests/ie_test_utils/functional_test_utils/blob_utils.hpp