From: Gladilov, Gleb Date: Mon, 13 Jul 2020 15:19:05 +0000 (+0300) Subject: [IE][VPU][nGraph]: Enables dynamic Reshape with non-const pattern support in myriad... X-Git-Url: http://review.tizen.org/git/?a=commitdiff_plain;h=543559f58c5b6ee7494d3d3448b02dc6f263c802;p=platform%2Fupstream%2Fdldt.git [IE][VPU][nGraph]: Enables dynamic Reshape with non-const pattern support in myriad plugin (#1159) * [IE][nGraph]: Introduces PartialShape ctor from values vector Signed-off-by: Gladilov, Gleb * [IE][VPU][nGraph]: Moves evaluateTargetShape to common utilities The same functionality - get upper-bound shape estimation for dynamic input - is needed in dynamic Reshape along with dynamic Broadcast. Return value type has been changed from PartialShape to vector. The reason is Reshape encodes special values (0, -1) into input values that define output shape. Representing those values (which upper-bound provides evaluateTargetShape) as PartialShape leads to incorrect representation vector with -1 as dynamic shape - which is not expected. Signed-off-by: Gladilov, Gleb * [IE][VPU][nGraph]: Introduces StaticShapeReshape In comparison with original Reshape StaticShapeReshape propagates upper-bound shape through a function in case of dynamic input. To do so, shape inference method gets upper-bound shape from evaluateTargetShape, decodes special values (0, -1) in it and then propagate the result. Output shape processing happens only once, because if shape inference were called after ShapeOf operations have been optimized out on dynamic path, then evaluateTargetShape will require evaluate method for all operations that appear in function before current Reshape. Since evaluate method is implemented not for all operations it lead to Faster-RCNN compilation error. Signed-off-by: Gladilov, Gleb * [IE][VPU][nGraph]: Updates Reshape DTS on StaticShapeReshape In case of non-const Reshape input that defines output shape DTS uses StaticShapeReshape which propagates upper-bound shape evaluated from this input through a function. Signed-off-by: Gladilov, Gleb * [IE][VPU][nGraph][Tests]: Refactoring DTS Reshape tests The only changes are: * header files include reordering * indentation/wrapping fixing Signed-off-by: Gladilov, Gleb * [IE][VPU][nGraph]: Moves ShapeOf transformation out of DTS scope In comparison with DTS ShapeOf transformation needs to work on whole function. Separating these 2 transformations makes testing easier since now it's possible to call specific DTS without ShapeOf transformation and vice versa. Also DynamicToStaticShapeOf has been renamed into EliminateShapeOfAfterDSR since transformation doesn't introduce new DSR operations. Signed-off-by: Gladilov, Gleb * [VPU][Tests]: Introduces DTS Reshape tests with non-const pattern New StaticShapeReshape constructor has been added as well, since test fixture should create it from reshape parameters, not reshape itself. Signed-off-by: Gladilov, Gleb --- diff --git a/inference-engine/src/vpu/common/include/vpu/ngraph/operations/static_shape_reshape.hpp b/inference-engine/src/vpu/common/include/vpu/ngraph/operations/static_shape_reshape.hpp new file mode 100644 index 0000000..00b390f --- /dev/null +++ b/inference-engine/src/vpu/common/include/vpu/ngraph/operations/static_shape_reshape.hpp @@ -0,0 +1,29 @@ +// Copyright (C) 2020 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#pragma once + +#include "ngraph/op/util/attr_types.hpp" +#include "ngraph/node.hpp" +#include + +#include +#include + +namespace ngraph { namespace vpu { namespace op { + +class StaticShapeReshape : public ngraph::opset3::Reshape { +public: + StaticShapeReshape(const Output& arg, const Output& pattern, bool special_zero); + explicit StaticShapeReshape(const std::shared_ptr& reshape); + + static constexpr NodeTypeInfo type_info{"StaticShapeReshape", 0}; + const NodeTypeInfo& get_type_info() const override { return type_info; } + + void validate_and_infer_types() override; +}; + +} // namespace op +} // namespace vpu +} // namespace ngraph diff --git a/inference-engine/src/vpu/common/include/vpu/ngraph/transformations/dynamic_to_static_shape_shapeof.hpp b/inference-engine/src/vpu/common/include/vpu/ngraph/transformations/eliminate_shapeof_after_dsr.hpp similarity index 63% rename from inference-engine/src/vpu/common/include/vpu/ngraph/transformations/dynamic_to_static_shape_shapeof.hpp rename to inference-engine/src/vpu/common/include/vpu/ngraph/transformations/eliminate_shapeof_after_dsr.hpp index a8be57a..78afeca 100644 --- a/inference-engine/src/vpu/common/include/vpu/ngraph/transformations/dynamic_to_static_shape_shapeof.hpp +++ b/inference-engine/src/vpu/common/include/vpu/ngraph/transformations/eliminate_shapeof_after_dsr.hpp @@ -8,9 +8,9 @@ namespace vpu { -class DynamicToStaticShapeShapeOf : public ngraph::pass::GraphRewrite { +class EliminateShapeOfAfterDSR : public ngraph::pass::GraphRewrite { public: - DynamicToStaticShapeShapeOf(); + EliminateShapeOfAfterDSR(); }; } //namespace vpu diff --git a/inference-engine/src/vpu/common/include/vpu/ngraph/utilities.hpp b/inference-engine/src/vpu/common/include/vpu/ngraph/utilities.hpp new file mode 100644 index 0000000..ffd6e80 --- /dev/null +++ b/inference-engine/src/vpu/common/include/vpu/ngraph/utilities.hpp @@ -0,0 +1,9 @@ +// Copyright (C) 2020 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#pragma once + +#include "ngraph/node.hpp" + +std::vector evaluateTargetShape(const ngraph::Output& value); diff --git a/inference-engine/src/vpu/common/src/ngraph/operations/static_shape_broadcast.cpp b/inference-engine/src/vpu/common/src/ngraph/operations/static_shape_broadcast.cpp index aed6a2d..afdcf7a 100644 --- a/inference-engine/src/vpu/common/src/ngraph/operations/static_shape_broadcast.cpp +++ b/inference-engine/src/vpu/common/src/ngraph/operations/static_shape_broadcast.cpp @@ -3,6 +3,7 @@ // #include "vpu/ngraph/operations/static_shape_broadcast.hpp" +#include "vpu/ngraph/utilities.hpp" #include "vpu/utils/error.hpp" @@ -11,61 +12,6 @@ namespace ngraph { namespace vpu { namespace op { -namespace { - -HostTensorVector evaluateShapeOf(Node* node, const HostTensorVector&) { - auto shapeOf = as_type(node); - const auto inputValue = shapeOf->input_value(0); - const auto outputValue = shapeOf->output(0); - const auto inputTensors = - HostTensorVector{std::make_shared(inputValue)}; - const auto outputTensors = - HostTensorVector{std::make_shared(outputValue)}; - - shapeOf->evaluate(outputTensors, inputTensors); - return outputTensors; -} - -HostTensorVector evaluateConstant(Node* node, const HostTensorVector&) { - const auto constantNode = as_type(node); - const auto constant = std::make_shared(*constantNode); - - const auto outputTensor = std::make_shared(constant); - - return {outputTensor}; -} - -HostTensorVector evaluateOp(Node* node, const HostTensorVector& inputTensors) { - HostTensorVector outputTensors; - for (const auto& output : node->outputs()) { - outputTensors.push_back(std::make_shared(output)); - } - - node->evaluate(outputTensors, inputTensors); - return outputTensors; -} - -PartialShape evaluateTargetShape(const Output& value) { - static Evaluator::op_handler_map handlers = { - {opset3::ShapeOf::type_info, evaluateShapeOf}, - {opset3::Constant::type_info, evaluateConstant}, - {opset3::Gather::type_info, evaluateOp}, - {opset3::Concat::type_info, evaluateOp}}; - Evaluator::value_map value_map; - Evaluator evaluator(handlers, value_map); - - const auto shapeTensor = evaluator.evaluate(value); - if (!shapeTensor || !shapeTensor->get_is_allocated()) { - return PartialShape::dynamic(); - } - const auto shapeConstNode = std::make_shared(shapeTensor); - const auto resultShape = Shape{shapeConstNode->cast_vector()}; - - return resultShape; -} - -} // namespace - constexpr NodeTypeInfo StaticShapeBroadcast::type_info; StaticShapeBroadcast::StaticShapeBroadcast(const Output& arg, @@ -106,14 +52,20 @@ void StaticShapeBroadcast::validate_and_infer_types() { ::ngraph::op::util::BroadcastBase::validate_and_infer_types(); // Try to evaluate output shape. After some transformations further, we may not be able // to evaluate the target shape again, then we will leave the evaluated shape unchanged. - // For example, DynamicToStaticShapeShapeOf remove ShapeOf and pass the second input of DSR. - const auto evaluatedTargetShape = evaluateTargetShape(input_value(1)); + // For example, EliminateShapeOfAfterDSR remove ShapeOf and pass the second input of DSR. + const auto evaluatedDimensionValues = evaluateTargetShape(input_value(1)); + NODE_VALIDATION_CHECK(this, !evaluatedDimensionValues.empty(), "StaticShapeBroadcast (", get_friendly_name(), ") can't evaluate output shape"); + + const auto evaluatedTargetShape = ngraph::PartialShape(evaluatedDimensionValues); if (evaluatedTargetShape.is_static()) { m_evaluatedOutputShape = evaluatedTargetShape; } NODE_VALIDATION_CHECK(this, m_evaluatedOutputShape.is_static(), "StaticShapeBroadcast (", get_friendly_name(), ") ", "can't evaluate output shape, got: ", m_evaluatedOutputShape); + NODE_VALIDATION_CHECK(this, m_evaluatedOutputShape.all_non_negative(), + "StaticShapeBroadcast (", get_friendly_name(), ") ", + "expects non-negative shape, got: ", m_evaluatedOutputShape); set_output_type(0, get_input_element_type(0), m_evaluatedOutputShape); } } diff --git a/inference-engine/src/vpu/common/src/ngraph/operations/static_shape_reshape.cpp b/inference-engine/src/vpu/common/src/ngraph/operations/static_shape_reshape.cpp new file mode 100644 index 0000000..682de1c --- /dev/null +++ b/inference-engine/src/vpu/common/src/ngraph/operations/static_shape_reshape.cpp @@ -0,0 +1,89 @@ +// Copyright (C) 2020 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include +#include "vpu/ngraph/operations/static_shape_reshape.hpp" +#include "vpu/ngraph/utilities.hpp" + +namespace ngraph { namespace vpu { namespace op { + +constexpr NodeTypeInfo StaticShapeReshape::type_info; + +StaticShapeReshape::StaticShapeReshape(const Output& arg, const Output& pattern, bool special_zero) + : ::ngraph::opset3::Reshape(arg, pattern, special_zero) { + constructor_validate_and_infer_types(); +} + +StaticShapeReshape::StaticShapeReshape(const std::shared_ptr& reshape) + : StaticShapeReshape(reshape->get_argument(0), reshape->get_argument(1), reshape->get_special_zero()) { +} + +void StaticShapeReshape::validate_and_infer_types() { + const auto& targetShape = input_value(1); + if (as_type_ptr(targetShape.get_node_shared_ptr())) { + opset3::Reshape::validate_and_infer_types(); + return; + } + + NODE_VALIDATION_CHECK(this, get_input_element_type(1).is_integral_number(), "Pattern must be an integral number."); + NODE_VALIDATION_CHECK(this, get_input_partial_shape(1).rank().compatible(1), "Pattern must have rank 1, got ", get_input_partial_shape(1).rank(), "."); + + set_input_is_relevant_to_shape(1); + + NODE_VALIDATION_CHECK(this, get_input_partial_shape(0).is_static(), "StaticShapeReshape (", get_friendly_name(), ") ", + "input#0 is expected to be of static shape, got: ", get_input_partial_shape(0)); + + if (get_output_partial_shape(0).is_static()) { + return; + } + + const auto& inputShape = get_input_shape(0); + + auto outputDimensionsValues = evaluateTargetShape(targetShape); + NODE_VALIDATION_CHECK(this, !outputDimensionsValues.empty(), "StaticShapeReshape (", get_friendly_name(), ") can't evaluate output shape"); + + for (std::size_t i = 0; i < outputDimensionsValues.size(); ++i) { + if (outputDimensionsValues[i] == 0 && m_special_zero) { + NODE_VALIDATION_CHECK(this, inputShape[i] <= static_cast(std::numeric_limits::max()), + "StaticShapeReshape (", get_friendly_name(), ") out of range input shape dimension value: ", inputShape[i]); + outputDimensionsValues[i] = static_cast(inputShape[i]); + } + } + + NODE_VALIDATION_CHECK(this, std::none_of(outputDimensionsValues.cbegin(), outputDimensionsValues.cend(), + [](std::int64_t dimension) { return dimension < -1; }), "Dim size cannot be less than -1, got ", ngraph::PartialShape(outputDimensionsValues)); + const auto negativeDimsCount = std::count_if(outputDimensionsValues.cbegin(), outputDimensionsValues.cend(), + [](std::int64_t dimension) { return dimension == -1; }); + NODE_VALIDATION_CHECK(this, negativeDimsCount <= 1, "More than one dimension has size of -1 (", negativeDimsCount, ")"); + + const auto& inputShapeVolume = shape_size(inputShape); + if (negativeDimsCount == 1) { + const auto& outputShapeVolume = std::abs(std::accumulate( + outputDimensionsValues.cbegin(), + outputDimensionsValues.cend(), + static_cast(1), + std::multiplies())); //shape_size(outputDimensionsValues); + NODE_VALIDATION_CHECK(this, inputShapeVolume % outputShapeVolume == 0, "StaticShapeReshape (", get_friendly_name(), ") ", + "output shape volume does not evenly divide the input shape volume: input shape volume = ", inputShapeVolume, " output shape ", + "volume = ", outputShapeVolume); + NODE_VALIDATION_CHECK(this, outputShapeVolume != 0, "StaticShapeReshape (", get_friendly_name(), ") ", + "output shape volume is equal to 0"); + + const auto actualValue = inputShapeVolume / outputShapeVolume; + NODE_VALIDATION_CHECK(this, actualValue <= static_cast(std::numeric_limits::max()), + "StaticShapeReshape (", get_friendly_name(), ") out of range output shape dimension value: ", actualValue); + std::replace(outputDimensionsValues.begin(), outputDimensionsValues.end(), + static_cast(-1), static_cast(actualValue)); + } + + const auto& outputShape = ngraph::PartialShape(outputDimensionsValues); + NODE_VALIDATION_CHECK(this, inputShapeVolume == shape_size(outputDimensionsValues), "Requested output shape (upper-bound) ", outputShape, + " is incompatible with input shape ", get_input_shape(0), " (upper-bound)"); + + set_output_type(0, get_input_element_type(0), outputShape); +} + +} // namespace op +} // namespace vpu +} // namespace ngraph diff --git a/inference-engine/src/vpu/common/src/ngraph/transformations/dynamic_to_static_shape.cpp b/inference-engine/src/vpu/common/src/ngraph/transformations/dynamic_to_static_shape.cpp index 9c7aa5f..1813bab 100644 --- a/inference-engine/src/vpu/common/src/ngraph/transformations/dynamic_to_static_shape.cpp +++ b/inference-engine/src/vpu/common/src/ngraph/transformations/dynamic_to_static_shape.cpp @@ -13,7 +13,6 @@ #include "vpu/ngraph/transformations/dynamic_to_static_shape_reduce.hpp" #include "vpu/ngraph/transformations/dynamic_to_static_shape_reshape.hpp" #include "vpu/ngraph/transformations/dynamic_to_static_shape_roialign.hpp" -#include "vpu/ngraph/transformations/dynamic_to_static_shape_shapeof.hpp" #include "vpu/ngraph/transformations/dynamic_to_static_shape_squeeze.hpp" #include "vpu/ngraph/transformations/dynamic_to_static_shape_strided_slice.hpp" #include "vpu/ngraph/transformations/dynamic_to_static_shape_topk.hpp" @@ -131,9 +130,6 @@ void DynamicToStaticShape::transform(std::shared_ptr function) transformation->second(operation); } - // Should be executed after all dynamic-to-static transformations - DynamicToStaticShapeShapeOf().run_on_function(function); - function->validate_nodes_and_infer_types(); validateStaticShapes(*function); } diff --git a/inference-engine/src/vpu/common/src/ngraph/transformations/dynamic_to_static_shape_reshape.cpp b/inference-engine/src/vpu/common/src/ngraph/transformations/dynamic_to_static_shape_reshape.cpp index cb144bb..e6255b2 100644 --- a/inference-engine/src/vpu/common/src/ngraph/transformations/dynamic_to_static_shape_reshape.cpp +++ b/inference-engine/src/vpu/common/src/ngraph/transformations/dynamic_to_static_shape_reshape.cpp @@ -12,6 +12,7 @@ #include "ngraph/opsets/opset3.hpp" #include +#include namespace vpu { @@ -21,19 +22,17 @@ void dynamicToStaticShapeReshape(std::shared_ptr target) { "DynamicToStaticShape transformation for {} of type {} expects {} as input with index {}", target->get_friendly_name(), target->get_type_info(), ngraph::vpu::op::DynamicShapeResolver::type_info, 0); - const auto outShapeDescriptor = target->get_argument(1); - VPU_THROW_UNLESS(ngraph::as_type_ptr(outShapeDescriptor), - "DynamicToStaticShape transformation for {} of type {} expects {} as input with index {}", - target->get_friendly_name(), target->get_type_info(), ngraph::opset3::Constant::type_info, 1); - const auto reshape = std::dynamic_pointer_cast(target); - const auto copied = reshape->clone_with_new_inputs(target->input_values()); - const auto inDataShape = dsr->input(1).get_source_output(); + const auto outShapeDescriptor = reshape->get_argument(1); - const auto outShapeOfReshape = std::make_shared( - inDataShape, outShapeDescriptor, reshape->get_special_zero()); + const auto replacement = ngraph::as_type_ptr(outShapeDescriptor) + ? reshape->clone_with_new_inputs(reshape->input_values()) + : std::make_shared(reshape); + + const auto inDataShape = dsr->input(1).get_source_output(); + const auto outShapeOfReshape = std::make_shared(inDataShape, outShapeDescriptor, reshape->get_special_zero()); - auto outDSR = std::make_shared(copied, outShapeOfReshape); + auto outDSR = std::make_shared(replacement, outShapeOfReshape); outDSR->set_friendly_name(reshape->get_friendly_name()); ngraph::replace_node(std::move(target), std::move(outDSR)); } diff --git a/inference-engine/src/vpu/common/src/ngraph/transformations/dynamic_to_static_shape_shapeof.cpp b/inference-engine/src/vpu/common/src/ngraph/transformations/eliminate_shapeof_after_dsr.cpp similarity index 86% rename from inference-engine/src/vpu/common/src/ngraph/transformations/dynamic_to_static_shape_shapeof.cpp rename to inference-engine/src/vpu/common/src/ngraph/transformations/eliminate_shapeof_after_dsr.cpp index e77eda8..5b8841d 100644 --- a/inference-engine/src/vpu/common/src/ngraph/transformations/dynamic_to_static_shape_shapeof.cpp +++ b/inference-engine/src/vpu/common/src/ngraph/transformations/eliminate_shapeof_after_dsr.cpp @@ -2,7 +2,7 @@ // SPDX-License-Identifier: Apache-2.0 // -#include "vpu/ngraph/transformations/dynamic_to_static_shape_shapeof.hpp" +#include "vpu/ngraph/transformations/eliminate_shapeof_after_dsr.hpp" #include #include @@ -11,7 +11,7 @@ namespace vpu { -DynamicToStaticShapeShapeOf::DynamicToStaticShapeShapeOf() : GraphRewrite() { +EliminateShapeOfAfterDSR::EliminateShapeOfAfterDSR() : GraphRewrite() { // We don't set strict_mode when use pattern Matcher, // so we can set any type and shape for input. auto inputWithAnyTypeAndShape = std::make_shared( @@ -33,7 +33,7 @@ DynamicToStaticShapeShapeOf::DynamicToStaticShapeShapeOf() : GraphRewrite() { return true; }; - auto m = std::make_shared(shapeOfPattern, "DynamicToStaticShapeShapeOf"); + auto m = std::make_shared(shapeOfPattern, "EliminateShapeOfAfterDSR"); this->add_matcher(m, callback, ngraph::pass::PassProperty::CHANGE_DYNAMIC_STATE); } diff --git a/inference-engine/src/vpu/common/src/ngraph/utilities.cpp b/inference-engine/src/vpu/common/src/ngraph/utilities.cpp new file mode 100644 index 0000000..d0a84b9 --- /dev/null +++ b/inference-engine/src/vpu/common/src/ngraph/utilities.cpp @@ -0,0 +1,62 @@ +// Copyright (C) 2020 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "vpu/ngraph/utilities.hpp" + +#include "ngraph/opsets/opset3.hpp" +#include "ngraph/evaluator.hpp" + +ngraph::HostTensorVector evaluateShapeOf(ngraph::Node* node, const ngraph::HostTensorVector&) { + auto shapeOf = ngraph::as_type(node); + const auto inputValue = shapeOf->input_value(0); + const auto outputValue = shapeOf->output(0); + const auto inputTensors = + ngraph::HostTensorVector{std::make_shared(inputValue)}; + const auto outputTensors = + ngraph::HostTensorVector{std::make_shared(outputValue)}; + + shapeOf->evaluate(outputTensors, inputTensors); + return outputTensors; +} + +ngraph::HostTensorVector evaluateConstant(ngraph::Node* node, const ngraph::HostTensorVector&) { + const auto constantNode = ngraph::as_type(node); + const auto constant = std::make_shared(*constantNode); + + const auto outputTensor = std::make_shared(constant); + + return {outputTensor}; +} + +ngraph::HostTensorVector evaluateOp(ngraph::Node* node, const ngraph::HostTensorVector& inputTensors) { + ngraph::HostTensorVector outputTensors; + for (const auto& output : node->outputs()) { + outputTensors.push_back(std::make_shared(output)); + } + + node->evaluate(outputTensors, inputTensors); + return outputTensors; +} + +std::vector evaluateTargetShape(const ngraph::Output& value) { + static ngraph::Evaluator::op_handler_map handlers = { + {ngraph::opset3::ShapeOf::type_info, evaluateShapeOf}, + {ngraph::opset3::Constant::type_info, evaluateConstant}, + {ngraph::opset3::Gather::type_info, evaluateOp}, + {ngraph::opset3::Concat::type_info, evaluateOp}, + {ngraph::opset3::Reshape::type_info, evaluateOp}, + {ngraph::opset3::Multiply::type_info, evaluateOp}, + {ngraph::opset3::Squeeze::type_info, evaluateOp}, + }; + ngraph::Evaluator::value_map value_map; + ngraph::Evaluator evaluator(handlers, value_map); + + const auto shapeTensor = evaluator.evaluate(value); + if (!shapeTensor || !shapeTensor->get_is_allocated()) { + return {}; + } + + const auto shapeConstNode = std::make_shared(shapeTensor); + return {shapeConstNode->cast_vector()}; +} diff --git a/inference-engine/src/vpu/graph_transformer/src/frontend/frontend.cpp b/inference-engine/src/vpu/graph_transformer/src/frontend/frontend.cpp index 743524e..beeb422 100644 --- a/inference-engine/src/vpu/graph_transformer/src/frontend/frontend.cpp +++ b/inference-engine/src/vpu/graph_transformer/src/frontend/frontend.cpp @@ -115,6 +115,7 @@ FrontEnd::FrontEnd(StageBuilder::Ptr stageBuilder) {"OutShapeOfReshape", LAYER_PARSER(parseOutShapeOfReshape)}, {"StaticShapeBroadcast", LAYER_PARSER(parseBroadcast)}, {"StaticShapeNonMaxSuppression", LAYER_PARSER(parseStaticShapeNMS)}, + {"StaticShapeReshape", LAYER_PARSER(parseReshape)}, }} {} ModelPtr FrontEnd::buildInitialModel(ie::ICNNNetwork& network) { diff --git a/inference-engine/src/vpu/graph_transformer/src/stages/reshape.cpp b/inference-engine/src/vpu/graph_transformer/src/stages/reshape.cpp index ebb5ae6..8af2637 100644 --- a/inference-engine/src/vpu/graph_transformer/src/stages/reshape.cpp +++ b/inference-engine/src/vpu/graph_transformer/src/stages/reshape.cpp @@ -61,9 +61,8 @@ private: } // namespace void FrontEnd::parseReshape(const Model& model, const ie::CNNLayerPtr& layer, const DataVector& inputs, const DataVector& outputs) const { - VPU_THROW_UNLESS(inputs.size() == 1 || (inputs.size() == 2 && inputs[1]->usage() == DataUsage::Const), + VPU_THROW_UNLESS(inputs.size() == 1 || inputs.size() == 2, "%v of type %v is not supported with dynamic shape", layer->name, layer->type); - IE_ASSERT((inputs.size() == 1) || (inputs.size() == 2 && inputs[1]->usage() == DataUsage::Const)); IE_ASSERT(outputs.size() == 1); _stageBuilder->addReshapeStage(model, layer->name, layer, inputs[0], outputs[0]); } diff --git a/inference-engine/src/vpu/myriad_plugin/myriad_plugin.cpp b/inference-engine/src/vpu/myriad_plugin/myriad_plugin.cpp index 2b1f7b9..bc756d7 100644 --- a/inference-engine/src/vpu/myriad_plugin/myriad_plugin.cpp +++ b/inference-engine/src/vpu/myriad_plugin/myriad_plugin.cpp @@ -19,6 +19,8 @@ #include #include "vpu/ngraph/transformations/dynamic_to_static_shape.hpp" +#include "vpu/ngraph/transformations/eliminate_shapeof_after_dsr.hpp" + #include "generic_ie.hpp" #include "myriad_plugin.h" @@ -41,6 +43,7 @@ ExecutableNetworkInternal::Ptr Engine::LoadExeNetworkImpl( ngraph::op::GenericIE::DisableReshape noReshape(function); ngraph::pass::CommonOptimizations().run_on_function(function); vpu::DynamicToStaticShape().transform(function); + vpu::EliminateShapeOfAfterDSR().run_on_function(function); } return std::make_shared(*clonedNetwork, _mvnc, _devicePool, parsedConfigCopy); diff --git a/inference-engine/tests/functional/plugin/myriad/ngraph/transformations/dynamic_to_static_shape_reshape.cpp b/inference-engine/tests/functional/plugin/myriad/ngraph/transformations/dynamic_to_static_shape_reshape.cpp index ee14731..5eb10c6 100644 --- a/inference-engine/tests/functional/plugin/myriad/ngraph/transformations/dynamic_to_static_shape_reshape.cpp +++ b/inference-engine/tests/functional/plugin/myriad/ngraph/transformations/dynamic_to_static_shape_reshape.cpp @@ -3,111 +3,132 @@ // #include "vpu/ngraph/transformations/dynamic_to_static_shape_reshape.hpp" +#include "vpu/ngraph/transformations/dynamic_to_static_shape.hpp" #include "vpu/ngraph/operations/out_shape_of_reshape.hpp" #include "vpu/ngraph/operations/dynamic_shape_resolver.hpp" -#include -#include -#include +#include "ngraph_functions/utils/ngraph_helpers.hpp" +#include "ngraph/opsets/opset3.hpp" +#include "ngraph/function.hpp" -#include -#include +#include "common_test_utils/test_common.hpp" +#include "gtest/gtest.h" #include #include #include #include -#include +#include namespace { -using DataType = ngraph::element::Type; using DataShape = ngraph::Shape; -using TestParams = std::tuple; +using DataType = ngraph::element::Type; +using ReshapePatternGenerator = std::function(std::shared_ptr)>; -class DynamicToStaticShapeReshapeTests - : public CommonTestUtils::TestsCommon, - public testing::WithParamInterface { +using TestParams = std::tuple; + +class DynamicToStaticShapeReshapeTests : public CommonTestUtils::TestsCommon, public testing::WithParamInterface { public: void SetUp() override { const auto& parameters = GetParam(); - const auto& inDataShape = std::get<0>(parameters); - const auto& inDataType = std::get<1>(parameters); + m_inDataShape = std::get<0>(parameters); + m_inDataType = std::get<1>(parameters); + m_patternGenerator = std::get<2>(parameters); - ngraph::helpers::CompareFunctions( - *transform(inDataType, inDataShape), - *reference(inDataType, inDataShape)); + const auto& actual = transform(); + const auto& expected = reference(); + ngraph::helpers::CompareFunctions(*actual, *expected); } protected: - std::shared_ptr transform( - const ngraph::element::Type& inDataType, - const ngraph::Shape& inDataShape) const { - const auto inDataParam = std::make_shared( - inDataType, inDataShape); - const auto inDataDimsParam = std::make_shared( - ngraph::element::i64, ngraph::Shape{inDataShape.size()}); - const auto outShapeDescriptorParam = std::make_shared( - ngraph::element::i64, ngraph::Shape{inDataShape.size()}, inDataShape); - - const auto dsr = std::make_shared( - inDataParam, inDataDimsParam); - const auto reshape = std::make_shared( - dsr, outShapeDescriptorParam, true); - - auto function = std::make_shared( - ngraph::NodeVector{reshape}, - ngraph::ParameterVector{inDataParam, inDataDimsParam}, - "Actual"); - reshape->set_output_type(0, dsr->get_input_element_type(0), ngraph::PartialShape::dynamic( - outShapeDescriptorParam->get_output_partial_shape(0).rank())); - - const auto transformations = vpu::Transformations{{ - ngraph::op::v1::Reshape::type_info, vpu::dynamicToStaticShapeReshape}}; + std::shared_ptr transform() const { + const auto dsr = generateInputSubgraph(); + + const auto outShapeDescriptorParam = m_patternGenerator(dsr); + const auto reshape = std::make_shared(dsr, outShapeDescriptorParam, true); + + const auto function = generateFunction(reshape, *dsr, "Actual"); + reshape->set_output_type( + 0, + dsr->get_input_element_type(0), + ngraph::PartialShape::dynamic(outShapeDescriptorParam->get_output_partial_shape(0).rank())); + + const auto transformations = vpu::Transformations{{ngraph::op::v1::Reshape::type_info, vpu::dynamicToStaticShapeReshape}}; vpu::DynamicToStaticShape(transformations).transform(function); return function; } - std::shared_ptr reference( - const ngraph::element::Type& inDataType, - const ngraph::Shape& inDataShape) const { - const auto inDataParam = std::make_shared( - inDataType, inDataShape); - const auto inDataDimsParam = std::make_shared( - ngraph::element::i64, ngraph::Shape{inDataShape.size()}); - const auto outShapeDescriptorParam = std::make_shared( - ngraph::element::i64, ngraph::Shape{inDataShape.size()}, inDataShape); - - const auto dsr0 = std::make_shared( - inDataParam, inDataDimsParam); - const auto reshape = std::make_shared( - dsr0, outShapeDescriptorParam, true); - - const auto outShapeOfReshape = std::make_shared( - inDataDimsParam, outShapeDescriptorParam, true); - const auto dsr1 = std::make_shared( - reshape, outShapeOfReshape); + std::shared_ptr reference() const { + const auto dsr0 = generateInputSubgraph(); + + const auto outShapeDescriptorParam = m_patternGenerator(dsr0); + const auto reshape = ngraph::as_type_ptr(outShapeDescriptorParam) + ? std::make_shared(dsr0, outShapeDescriptorParam, true) + : std::make_shared(dsr0, outShapeDescriptorParam, true); + + const auto outShapeOfReshape = std::make_shared(dsr0->input_value(1), outShapeDescriptorParam, true); + const auto dsr1 = std::make_shared(reshape, outShapeOfReshape); + return generateFunction(dsr1, *dsr0, "Expected"); + } + +private: + std::shared_ptr generateInputSubgraph() const { + const auto inDataParam = std::make_shared(m_inDataType, m_inDataShape); + const auto inDataDimsParam = std::make_shared(ngraph::element::i64, ngraph::Shape{m_inDataShape.size()}); + return std::make_shared(inDataParam, inDataDimsParam); + } + + static std::shared_ptr generateFunction(std::shared_ptr result, const ngraph::op::Op& input, const std::string& name) { return std::make_shared( - ngraph::NodeVector{dsr1}, - ngraph::ParameterVector{inDataParam, inDataDimsParam}, - "Expected"); + ngraph::NodeVector{std::move(result)}, + ngraph::ParameterVector{ + std::dynamic_pointer_cast(input.get_input_node_shared_ptr(0)), + std::dynamic_pointer_cast(input.get_input_node_shared_ptr(1)) + }, + name); } + +private: + DataShape m_inDataShape; + DataType m_inDataType; + ReshapePatternGenerator m_patternGenerator; }; -TEST_P(DynamicToStaticShapeReshapeTests, compareFunctions) { +TEST_P(DynamicToStaticShapeReshapeTests, compareFunctions) {} + +std::shared_ptr generateStaticReshapePattern(std::shared_ptr dsr) { + const auto& inDataShape = dsr->input_value(0).get_shape(); + std::vector pattern(inDataShape.rbegin(), inDataShape.rend()); + return std::make_shared(ngraph::element::i64, ngraph::Shape{pattern.size()}, pattern); +} + +std::shared_ptr generateDynamicReshapePattern(std::shared_ptr dsr) { + const auto shapeOf = std::make_shared(std::move(dsr)); + const auto axis = ngraph::opset3::Constant::create(ngraph::element::i64, {1}, {0}); + const auto indices = ngraph::opset3::Constant::create(ngraph::element::i64, {1}, {0}); + return std::make_shared( + ngraph::OutputVector{ + std::make_shared(shapeOf, indices, axis), + ngraph::opset3::Constant::create(ngraph::element::i64, {1}, {-1})}, + 0); } INSTANTIATE_TEST_CASE_P(NGraph, DynamicToStaticShapeReshapeTests, testing::Combine( - testing::Values( - DataShape{4, 1000}, - DataShape{3, 128, 256}, - DataShape{2, 3, 128, 256}), - testing::Values( - ngraph::element::f16, - ngraph::element::f32, - ngraph::element::i32, - ngraph::element::i64, - ngraph::element::u8) + testing::Values( + DataShape{4, 1000}, + DataShape{3, 128, 256}, + DataShape{2, 3, 128, 256}, + DataShape{1000, 256, 7, 7}), + testing::Values( + ngraph::element::f16, + ngraph::element::f32, + ngraph::element::i32, + ngraph::element::i64, + ngraph::element::u8), + testing::Values( + generateStaticReshapePattern, + generateDynamicReshapePattern) )); } // namespace diff --git a/inference-engine/tests/functional/plugin/myriad/ngraph/transformations/dynamic_to_static_shape_shapeof.cpp b/inference-engine/tests/functional/plugin/myriad/ngraph/transformations/eliminate_shapeof_after_dsr.cpp similarity index 84% rename from inference-engine/tests/functional/plugin/myriad/ngraph/transformations/dynamic_to_static_shape_shapeof.cpp rename to inference-engine/tests/functional/plugin/myriad/ngraph/transformations/eliminate_shapeof_after_dsr.cpp index 4c28ef7..028b89f 100644 --- a/inference-engine/tests/functional/plugin/myriad/ngraph/transformations/dynamic_to_static_shape_shapeof.cpp +++ b/inference-engine/tests/functional/plugin/myriad/ngraph/transformations/eliminate_shapeof_after_dsr.cpp @@ -2,7 +2,7 @@ // SPDX-License-Identifier: Apache-2.0 // -#include "vpu/ngraph/transformations/dynamic_to_static_shape_shapeof.hpp" +#include "vpu/ngraph/transformations/eliminate_shapeof_after_dsr.hpp" #include #include @@ -20,8 +20,8 @@ namespace { using TensorType = ngraph::element::Type_t; using TensorShape = ngraph::Shape; -class DynamicToStaticShapeShapeOfRemoveDSR : public CommonTestUtils::TestsCommon, - public testing::WithParamInterface> { +class EliminateShapeOfAfterDSRTest : public CommonTestUtils::TestsCommon, + public testing::WithParamInterface> { public: void SetUp() override { const auto& parameters = GetParam(); @@ -47,7 +47,7 @@ protected: ngraph::ParameterVector{data, shape}, "Actual"); - vpu::DynamicToStaticShapeShapeOf().run_on_function(function); + vpu::EliminateShapeOfAfterDSR().run_on_function(function); return function; } @@ -63,10 +63,10 @@ protected: } }; -TEST_P(DynamicToStaticShapeShapeOfRemoveDSR, CompareFunctions) { +TEST_P(EliminateShapeOfAfterDSRTest, CompareFunctions) { } -INSTANTIATE_TEST_CASE_P(NGraph, DynamicToStaticShapeShapeOfRemoveDSR, testing::Combine( +INSTANTIATE_TEST_CASE_P(NGraph, EliminateShapeOfAfterDSRTest, testing::Combine( testing::Values( ngraph::element::f16, ngraph::element::f32, @@ -80,8 +80,8 @@ INSTANTIATE_TEST_CASE_P(NGraph, DynamicToStaticShapeShapeOfRemoveDSR, testing::C TensorShape{2, 3, 128, 256}) )); -class DynamicToStaticShapeShapeOfWithOutRemoveDSR : public CommonTestUtils::TestsCommon, - public testing::WithParamInterface> { +class EliminateShapeOfAfterDSRWithoutOutputDSR : public CommonTestUtils::TestsCommon, + public testing::WithParamInterface> { public: void SetUp() override { const auto& parameters = GetParam(); @@ -108,7 +108,7 @@ protected: ngraph::ParameterVector{data, shape}, "Actual"); - vpu::DynamicToStaticShapeShapeOf().run_on_function(function); + vpu::EliminateShapeOfAfterDSR().run_on_function(function); return function; } @@ -126,10 +126,10 @@ protected: } }; -TEST_P(DynamicToStaticShapeShapeOfWithOutRemoveDSR, CompareFunctions) { +TEST_P(EliminateShapeOfAfterDSRWithoutOutputDSR, CompareFunctions) { } -INSTANTIATE_TEST_CASE_P(NGraph, DynamicToStaticShapeShapeOfWithOutRemoveDSR, testing::Combine( +INSTANTIATE_TEST_CASE_P(NGraph, EliminateShapeOfAfterDSRWithoutOutputDSR, testing::Combine( testing::Values( ngraph::element::f16, ngraph::element::f32, @@ -143,8 +143,8 @@ INSTANTIATE_TEST_CASE_P(NGraph, DynamicToStaticShapeShapeOfWithOutRemoveDSR, tes TensorShape{2, 3, 128, 256}) )); -class DynamicToStaticShapeShapeOfKeepDSR : public CommonTestUtils::TestsCommon, - public testing::WithParamInterface> { +class EliminateShapeOfAfterDSRKeepDSR : public CommonTestUtils::TestsCommon, + public testing::WithParamInterface> { public: void SetUp() override { const auto& parameters = GetParam(); @@ -172,7 +172,7 @@ protected: ngraph::ParameterVector{data, shape}, "Actual"); - vpu::DynamicToStaticShapeShapeOf().run_on_function(function); + vpu::EliminateShapeOfAfterDSR().run_on_function(function); return function; } @@ -193,10 +193,10 @@ protected: } }; -TEST_P(DynamicToStaticShapeShapeOfKeepDSR, CompareFunctions) { +TEST_P(EliminateShapeOfAfterDSRKeepDSR, CompareFunctions) { } -INSTANTIATE_TEST_CASE_P(NGraph, DynamicToStaticShapeShapeOfKeepDSR, testing::Combine( +INSTANTIATE_TEST_CASE_P(NGraph, EliminateShapeOfAfterDSRKeepDSR, testing::Combine( testing::Values( ngraph::element::f16, ngraph::element::f32, diff --git a/ngraph/src/ngraph/op/reshape.hpp b/ngraph/src/ngraph/op/reshape.hpp index a68f439..fa1e970 100644 --- a/ngraph/src/ngraph/op/reshape.hpp +++ b/ngraph/src/ngraph/op/reshape.hpp @@ -151,7 +151,7 @@ namespace ngraph bool evaluate(const HostTensorVector& outputs, const HostTensorVector& inputs) override; - private: + protected: bool m_special_zero; }; } diff --git a/ngraph/src/ngraph/partial_shape.cpp b/ngraph/src/ngraph/partial_shape.cpp index b3a049a..e31afb0 100644 --- a/ngraph/src/ngraph/partial_shape.cpp +++ b/ngraph/src/ngraph/partial_shape.cpp @@ -23,6 +23,15 @@ using namespace ngraph; +PartialShape::PartialShape(const std::vector& dimensions) + : m_rank_is_static(true) +{ + std::transform(dimensions.cbegin(), + dimensions.cend(), + std::back_inserter(m_dimensions), + [](const Dimension::value_type& dimension) { return dimension; }); +} + PartialShape::PartialShape(const Shape& shape) : PartialShape(true, {}) { diff --git a/ngraph/src/ngraph/partial_shape.hpp b/ngraph/src/ngraph/partial_shape.hpp index 78f34d6..e42b4f7 100644 --- a/ngraph/src/ngraph/partial_shape.hpp +++ b/ngraph/src/ngraph/partial_shape.hpp @@ -68,6 +68,10 @@ namespace ngraph { } + /// \brief Constructs a PartialShape with static rank from a vector of dimensions values. + /// \param dimensions The Dimension values for the constructed shape. + PartialShape(const std::vector& dimensions); + /// \brief Constructs a static PartialShape with zero rank (the shape of a scalar). PartialShape() : PartialShape(std::initializer_list{})