| OpenVINO™ Version | Actual Operations Set |
| :---------------- | :------------------------------- |
+| 2021.1 | [opset4](opset4.md) |
| 2020.4 | [opset3](opset3.md) |
| 2020.3 | [opset2](opset2.md) |
| 2020.2 | [opset2](opset2.md) |
--- /dev/null
+# Operation Set `opset4` Specification
+
+This specification document describes `opset4` operation set supported in OpenVINO.
+Support for each particular operation from the list below depends on the capabilities available in a inference plugin
+and may vary among different hardware platforms and devices. Examples of operation instances are expressed as IR V10 xml
+snippets. Such IR is generated by the Model Optimizer. The semantics match corresponding nGraph operation classes
+declared in `namespace opset4`.
+
+
+## Table of Contents <a name="toc"></a>
+
+* [Abs](arithmetic/Abs_1.md)
+* [Acos](arithmetic/Acos_1.md)
+* [Acosh](arithmetic/Acosh_1.md)
+* [Add](arithmetic/Add_1.md)
+* [Asin](arithmetic/Asin_1.md)
+* [Asinh](arithmetic/Asinh_1.md)
+* [Assign](infrastructure/Assign_3.md)
+* [Atan](arithmetic/Atan_1.md)
+* [Atanh](arithmetic/Atanh_1.md)
+* [AvgPool](pooling/AvgPool_1.md)
+* [BatchNormInference](normalization/BatchNormInference_1.md)
+* [BatchToSpace](movement/BatchToSpace_2.md)
+* [BinaryConvolution](convolution/BinaryConvolution_1.md)
+* [Broadcast](movement/Broadcast_3.md)
+* [Bucketize](condition/Bucketize_3.md)
+* [CTCGreedyDecoder](sequence/CTCGreedyDecoder_1.md)
+* [Ceiling](arithmetic/Ceiling_1.md)
+* [Clamp](activation/Clamp_1.md)
+* [Concat](movement/Concat_1.md)
+* [Constant](infrastructure/Constant_1.md)
+* [Convert](type/Convert_1.md)
+* [ConvertLike](type/ConvertLike_1.md)
+* [Convolution](convolution/Convolution_1.md)
+* [ConvolutionBackpropData](convolution/ConvolutionBackpropData_1.md)
+* [Cos](arithmetic/Cos_1.md)
+* [Cosh](arithmetic/Cosh_1.md)
+* [CumSum](arithmetic/CumSum_3.md)
+* [DeformableConvolution](convolution/DeformableConvolution_1.md)
+* [DeformablePSROIPooling](detection/DeformablePSROIPooling_1.md)
+* [DepthToSpace](movement/DepthToSpace_1.md)
+* [DetectionOutput](detection/DetectionOutput_1.md)
+* [Divide](arithmetic/Divide_1.md)
+* [Elu](activation/Elu_1.md)
+* [EmbeddingBagOffsetsSum](sparse/EmbeddingBagOffsetsSum_3.md)
+* [EmbeddingBagPackedSum](sparse/EmbeddingBagPackedSum_3.md)
+* [EmbeddingSegmentsSum](sparse/EmbeddingSegmentsSum_3.md)
+* [Equal](comparison/Equal_1.md)
+* [Erf](arithmetic/Erf_1.md)
+* [Exp](activation/Exp_1.md)
+* [ExtractImagePatches](movement/ExtractImagePatches_3.md)
+* [FakeQuantize](quantization/FakeQuantize_1.md)
+* [Floor](arithmetic/Floor_1.md)
+* [FloorMod](arithmetic/FloorMod_1.md)
+* [Gather](movement/Gather_1.md)
+* [GatherTree](movement/GatherTree_1.md)
+* [Gelu](activation/GELU_2.md)
+* [Greater](comparison/Greater_1.md)
+* [GreaterEqual](comparison/GreaterEqual_1.md)
+* [GRN](normalization/GRN_1.md)
+* [GroupConvolution](convolution/GroupConvolution_1.md)
+* [GroupConvolutionBackpropData](convolution/GroupConvolutionBackpropData_1.md)
+* [GRUCell](sequence/GRUCell_3.md)
+* [HardSigmoid](activation/HardSigmoid_1.md)
+* [Interpolate](image/Interpolate_1.md)
+* [Less](comparison/Less_1.md)
+* [LessEqual](comparison/LessEqual_1.md)
+* [Log](arithmetic/Log_1.md)
+* [LogicalAnd](logical/LogicalAnd_1.md)
+* [LogicalNot](logical/LogicalNot_1.md)
+* [LogicalOr](logical/LogicalOr_1.md)
+* [LogicalXor](logical/LogicalXor_1.md)
+* [LRN](normalization/LRN_1.md)
+* [LSTMCell](sequence/LSTMCell_1.md)
+* [LSTMSequence](sequence/LSTMSequence_1.md)
+* [MatMul](matrix/MatMul_1.md)
+* [MaxPool](pooling/MaxPool_1.md)
+* [Maximum](arithmetic/Maximum_1.md)
+* [Minimum](arithmetic/Minimum_1.md)
+* [Mod](arithmetic/Mod_1.md)
+* [MVN](normalization/MVN_1.md)
+* [Multiply](arithmetic/Multiply_1.md)
+* [Negative](arithmetic/Negative_1.md)
+* [NonMaxSuppression](sort/NonMaxSuppression_4.md)
+* [NonZero](condition/NonZero_3.md)
+* [NormalizeL2](normalization/NormalizeL2_1.md)
+* [NotEqual](comparison/NotEqual_1.md)
+* [OneHot](sequence/OneHot_1.md)
+* [Pad](movement/Pad_1.md)
+* [Parameter](infrastructure/Parameter_1.md)
+* [Power](arithmetic/Power_1.md)
+* [PReLU](activation/PReLU_1.md)
+* [PriorBoxClustered](detection/PriorBoxClustered_1.md)
+* [PriorBox](detection/PriorBox_1.md)
+* [Proposal](detection/Proposal_1.md)
+* [PSROIPooling](detection/PSROIPooling_1.md)
+* [Range](generation/Range_1.md)
+* [ReLU](activation/ReLU_1.md)
+* [ReadValue](infrastructure/ReadValue_3.md)
+* [ReduceLogicalAnd](reduction/ReduceLogicalAnd_1.md)
+* [ReduceLogicalOr](reduction/ReduceLogicalOr_1.md)
+* [ReduceMax](reduction/ReduceMax_1.md)
+* [ReduceMean](reduction/ReduceMean_1.md)
+* [ReduceMin](reduction/ReduceMin_1.md)
+* [ReduceProd](reduction/ReduceProd_1.md)
+* [ReduceSum](reduction/ReduceSum_1.md)
+* [RegionYolo](detection/RegionYolo_1.md)
+* [ReorgYolo](detection/ReorgYolo_1.md)
+* [Reshape](shape/Reshape_1.md)
+* [Result](infrastructure/Result_1.md)
+* [Reverse](movement/Reverse_1.md)
+* [ReverseSequence](movement/ReverseSequence_1.md)
+* [RNNCell](sequence/RNNCell_3.md)
+* [ROIAlign](detection/ROIAlign_3.md)
+* [ROIPooling](detection/ROIPooling_1.md)
+* [ScatterElementsUpdate](movement/ScatterElementsUpdate_3.md)
+* [ScatterNDUpdate](movement/ScatterNDUpdate_3.md)
+* [ScatterUpdate](movement/ScatterUpdate_3.md)
+* [Select](condition/Select_1.md)
+* [Selu](arithmetic/Selu_1.md)
+* [ShapeOf](shape/ShapeOf_3.md)
+* [ShuffleChannels](movement/ShuffleChannels_1.md)
+* [Sigmoid](activation/Sigmoid_1.md)
+* [Sign](arithmetic/Sign_1.md)
+* [Sin](arithmetic/Sin_1.md)
+* [Sinh](arithmetic/Sinh_1.md)
+* [SoftMax](activation/SoftMax_1.md)
+* [SpaceToBatch](movement/SpaceToBatch_2.md)
+* [SpaceToDepth](movement/SpaceToDepth_1.md)
+* [Split](movement/Split_1.md)
+* [Sqrt](arithmetic/Sqrt_1.md)
+* [SquaredDifference](arithmetic/SquaredDifference_1.md)
+* [Squeeze](shape/Squeeze_1.md)
+* [StridedSlice](movement/StridedSlice_1.md)
+* [Subtract](arithmetic/Subtract_1.md)
+* [Tan](arithmetic/Tan_1.md)
+* [Tanh](arithmetic/Tanh_1.md)
+* [TensorIterator](infrastructure/TensorIterator_1.md)
+* [Tile](movement/Tile_1.md)
+* [TopK](sort/TopK_3.md)
+* [Transpose](movement/Transpose_1.md)
+* [Unsqueeze](shape/Unsqueeze_1.md)
+* [VariadicSplit](movement/VariadicSplit_1.md)
--- /dev/null
+## NonMaxSuppression<a name="NonMaxSuppression"></a>
+
+**Versioned name**: *NonMaxSuppression-4*
+
+**Category**: *Sorting and maximization*
+
+**Short description**: *NonMaxSuppression* performs non maximum suppression of the boxes with predicted scores.
+
+**Detailed description**: *NonMaxSuppression* performs non maximum suppression algorithm as described below:
+
+1. Take the box with highest score. If the score is less than `score_threshold` then stop. Otherwise add the box to the
+output and continue to the next step.
+2. For each input box, calculate the IOU (intersection over union) with the box added during the previous step. If the
+value is greater than the `iou_threshold` threshold then remove the input box from further consideration.
+3. Return to step 1.
+
+This algorithm is applied independently to each class of each batch element. The total number of output boxes for each
+class must not exceed `max_output_boxes_per_class`.
+
+**Attributes**:
+
+* *box_encoding*
+
+ * **Description**: *box_encoding* specifies the format of boxes data encoding.
+ * **Range of values**: "corner" or "center"
+ * *corner* - the box data is supplied as `[y1, x1, y2, x2]` where `(y1, x1)` and `(y2, x2)` are the coordinates of any diagonal pair of box corners.
+ * *center* - the box data is supplied as `[x_center, y_center, width, height]`.
+ * **Type**: string
+ * **Default value**: "corner"
+ * **Required**: *no*
+
+* *sort_result_descending*
+
+ * **Description**: *sort_result_descending* is a flag that specifies whenever it is necessary to sort selected boxes across batches or not.
+ * **Range of values**: True of False
+ * *True* - sort selected boxes across batches.
+ * *False* - do not sort selected boxes across batches (boxes are sorted per class).
+ * **Type**: boolean
+ * **Default value**: True
+ * **Required**: *no*
+
+* *output_type*
+
+ * **Description**: the output tensor type
+ * **Range of values**: "i64" or "i32"
+ * **Type**: string
+ * **Default value**: "i64"
+ * **Required**: *No*
+
+**Inputs**:
+
+* **1**: `boxes` - tensor of type *T* and shape `[num_batches, num_boxes, 4]` with box coordinates. Required.
+
+* **2**: `scores` - tensor of type *T* and shape `[num_batches, num_classes, num_boxes]` with box scores. Required.
+
+* **3**: `max_output_boxes_per_class` - scalar tensor of type *T_MAX_BOXES* specifying maximum number of boxes to be selected per class. Optional with default value 0 meaning select no boxes.
+
+* **4**: `iou_threshold` - scalar tensor of type *T_THRESHOLDS* specifying intersection over union threshold. Optional with default value 0 meaning keep all boxes.
+
+* **5**: `score_threshold` - scalar tensor of type *T_THRESHOLDS* specifying minimum score to consider box for the processing. Optional with default value 0.
+
+**Outputs**:
+
+* **1**: `selected_indices` - tensor of type *T_IND* and shape `[min(num_boxes, max_output_boxes_per_class) * num_batches * num_classes, 3]` containing information about selected boxes as triplets `[batch_index, class_index, box_index]`.
+The output tensor is filled with 0s for output tensor elements if the total number of selected boxes is less than the output tensor size.
+
+**Types**
+
+* *T*: floating point type.
+
+* *T_MAX_BOXES*: integer type.
+
+* *T_THRESHOLDS*: floating point type.
+
+* *T_IND*: `int64` or `int32`.
+
+**Example**
+
+```xml
+<layer ... type="NonMaxSuppression" ... >
+ <data box_encoding="corner" sort_result_descending="1" output_type="i64"/>
+ <input>
+ <port id="0">
+ <dim>3</dim>
+ <dim>100</dim>
+ <dim>4</dim>
+ </port>
+ <port id="1">
+ <dim>3</dim>
+ <dim>5</dim>
+ <dim>100</dim>
+ </port>
+ <port id="2"/> <!-- 10 -->
+ <port id="3"/>
+ <port id="4"/>
+ </input>
+ <output>
+ <port id="5" precision="I64">
+ <dim>150</dim> <!-- min(100, 10) * 3 * 5 -->
+ <dim>3</dim>
+ </port>
+ </output>
+</layer>
+```
opsetNames.insert("opset1");
opsetNames.insert("opset2");
opsetNames.insert("opset3");
+ opsetNames.insert("opset4");
}
Core::Impl::~Impl() {}
<< " should be replaced by constant during constant folding.";
return nullptr;
});
+
+ addSpecificCreator({"NonMaxSuppressionIE"}, [](const std::shared_ptr<::ngraph::Node>& node,
+ const std::map<std::string, std::string> params) -> CNNLayerPtr {
+ LayerParams attrs = {node->get_friendly_name(), "NonMaxSuppression", details::convertPrecision(node->get_output_element_type(0))};
+ auto res = std::make_shared<InferenceEngine::NonMaxSuppressionLayer>(attrs);
+ res->params = params;
+ return res;
+ });
}
CNNLayerPtr InferenceEngine::details::CNNLayerCreator::create() {
std::make_shared<Builder::NodeConverter<::ngraph::op::v1::Minimum>>(),
std::make_shared<Builder::NodeConverter<::ngraph::op::v1::Multiply>>(),
std::make_shared<Builder::NodeConverter<::ngraph::op::v1::NonMaxSuppression>>(),
- std::make_shared<Builder::NodeConverter<::ngraph::op::NonMaxSuppressionIE>>(),
std::make_shared<Builder::NodeConverter<::ngraph::op::NormalizeL2>>(),
std::make_shared<Builder::NodeConverter<::ngraph::op::NormalizeIE>>(),
std::make_shared<Builder::NodeConverter<::ngraph::op::OneHotIE>>(),
THROW_IE_EXCEPTION << "NonMaxSuppression operation must be converted to NonMaxSuppressionIE operation.";
}
-template <>
-CNNLayer::Ptr NodeConverter<ngraph::op::NonMaxSuppressionIE>::createLayer(const std::shared_ptr<ngraph::Node>& layer) const {
- LayerParams params = {layer->get_friendly_name(), "NonMaxSuppression", Precision::I32};
-
- auto castedLayer = std::dynamic_pointer_cast<ngraph::op::NonMaxSuppressionIE>(layer);
- if (castedLayer == nullptr) THROW_IE_EXCEPTION << "Cannot get " << params.type << " layer " << params.name;
-
- auto res = std::make_shared<InferenceEngine::NonMaxSuppressionLayer>(params);
- res->params["sort_result_descending"] = std::to_string(castedLayer->m_sort_result_descending);
- res->params["center_point_box"] = std::to_string(castedLayer->m_sort_result_descending);
- return res;
-}
-
} // namespace Builder
} // namespace InferenceEngine
opsets["opset1"] = ngraph::get_opset1();
opsets["opset2"] = ngraph::get_opset2();
opsets["opset3"] = ngraph::get_opset3();
+ opsets["opset4"] = ngraph::get_opset4();
// Load custom opsets
for (const auto& ext : exts) {
void validate_and_infer_types() override;
+ bool visit_attributes(AttributeVisitor& visitor) override;
+
+ std::shared_ptr<Node> clone_with_new_inputs(const OutputVector & new_args) const override;
+
+ int m_center_point_box;
+ bool m_sort_result_descending = true;
+};
+
+class TRANSFORMATIONS_API NonMaxSuppressionIE2 : public NonMaxSuppressionIE {
+public:
+ static constexpr NodeTypeInfo type_info{"NonMaxSuppressionIE", 2};
+ const NodeTypeInfo& get_type_info() const override { return type_info; }
+
+ NonMaxSuppressionIE2(const Output<Node>& boxes,
+ const Output<Node>& scores,
+ const Output<Node>& max_output_boxes_per_class,
+ const Output<Node>& iou_threshold,
+ const Output<Node>& score_threshold,
+ int center_point_box,
+ bool sort_result_descending);
+
+ void validate_and_infer_types() override;
+
std::shared_ptr<Node> clone_with_new_inputs(const OutputVector & new_args) const override;
int m_center_point_box;
// This pass must be called first in pipeline
NGRAPH_PASS(InitNodeInfo, ::ngraph::pass)
NGRAPH_PASS(RemoveFilteringBoxesBySize, ::ngraph::pass) // Resolves dynamism (replaces NonZero), CF needed
-NGRAPH_PASS(UpgradeNMS3ToNMS4, ::ngraph::pass) // replaces v3::NMS with v4::NMS(always dyn output shape) if function has opset3::NonZero operation
+NGRAPH_PASS(UpgradeNMS4ToNMSDynamic, ::ngraph::pass) // replaces v4::NMS with dynamic::NMS(always dyn output shape) if function has opset3::NonZero operation
NGRAPH_PASS(ConstantFolding, ::ngraph::pass)
NGRAPH_PASS(StridedSliceOptimization, ::ngraph::pass) // depends on CF
NGRAPH_PASS(NopElimination, ::ngraph::pass) // may introduce fake dynamism
+++ /dev/null
-// Copyright (C) 2018-2020 Intel Corporation
-// SPDX-License-Identifier: Apache-2.0
-//
-
-#pragma once
-
-#include <vector>
-#include <memory>
-
-#include <transformations_visibility.hpp>
-
-#include <ngraph/pass/graph_rewrite.hpp>
-
-namespace ngraph {
-namespace pass {
-
-class TRANSFORMATIONS_API UpgradeNMS3ToNMS4;
-
-} // namespace pass
-} // namespace ngraph
-
-/*
- * Description:
- * UpgradeNMS3ToNMS4 transformation upgrades NonMaxSuppression operations from v3 version to v4
- * in case function has at least one NonZero operation (dynamism marker).
- * NMS of version v4 always has dynamic output
- */
-
-
-class ngraph::pass::UpgradeNMS3ToNMS4: public ngraph::pass::FunctionPass {
-public:
- bool run_on_function(std::shared_ptr<ngraph::Function> f) override;
-};
-
--- /dev/null
+// Copyright (C) 2018-2020 Intel Corporation
+// SPDX-License-Identifier: Apache-2.0
+//
+
+#pragma once
+
+#include <vector>
+#include <memory>
+
+#include <transformations_visibility.hpp>
+
+#include <ngraph/pass/graph_rewrite.hpp>
+
+namespace ngraph {
+namespace pass {
+
+class TRANSFORMATIONS_API UpgradeNMS4ToNMSDynamic;
+
+} // namespace pass
+} // namespace ngraph
+
+/*
+ * Description:
+ * UpgradeNMS4ToNMSDynamic transformation upgrades NonMaxSuppression operations from v3 version to dynamic
+ * in case function has at least one NonZero operation (dynamism marker).
+ * NMS dynamic always has dynamic output
+ */
+
+
+class ngraph::pass::UpgradeNMS4ToNMSDynamic: public ngraph::pass::GraphRewrite {
+public:
+ UpgradeNMS4ToNMSDynamic() : GraphRewrite() {
+ upgrade_nms4_to_nms_dynamic();
+ }
+ bool run_on_function(std::shared_ptr<ngraph::Function> f) override;
+
+private:
+ void upgrade_nms4_to_nms_dynamic();
+};
+
--- /dev/null
+// Copyright (C) 2018-2020 Intel Corporation
+// SPDX-License-Identifier: Apache-2.0
+//
+
+#pragma once
+
+#include <vector>
+#include <memory>
+
+#include <transformations_visibility.hpp>
+
+#include <ngraph/pass/graph_rewrite.hpp>
+
+namespace ngraph {
+namespace pass {
+
+ class TRANSFORMATIONS_API ConvertNMS4ToLegacy;
+
+} // namespace pass
+} // namespace ngraph
+
+/*
+ * Description:
+ * Convert NMS-4 directly to legacy NMS because NMS-3 and NMS-1 have different shape infer function
+ */
+
+
+class ngraph::pass::ConvertNMS4ToLegacy: public ngraph::pass::GraphRewrite {
+public:
+ ConvertNMS4ToLegacy() : GraphRewrite() {
+ convert_nms4_to_legacy();
+ }
+private:
+ void convert_nms4_to_legacy();
+};
+
NGRAPH_PASS(ConvertTopKToTopKIE, ::ngraph::pass)
NGRAPH_PASS(ConvertNMSToNMSIE, ::ngraph::pass)
NGRAPH_PASS(ConstantFolding, ::ngraph::pass)
+NGRAPH_PASS(ConvertNMS4ToLegacy, ::ngraph::pass)
#include <memory>
#include <ngraph/opsets/opset1.hpp>
+#include <ngraph/opsets/opset4.hpp>
using namespace std;
using namespace ngraph;
const Output<Node> &score_threshold,
int center_point_box,
bool sort_result_descending)
- : Op({boxes, scores, max_output_boxes_per_class, iou_threshold, score_threshold})
- , m_center_point_box(center_point_box)
- , m_sort_result_descending(sort_result_descending) {
+ : Op({boxes, scores, max_output_boxes_per_class, iou_threshold, score_threshold}),
+ m_center_point_box(center_point_box), m_sort_result_descending(sort_result_descending) {
constructor_validate_and_infer_types();
}
-std::shared_ptr<Node> op::NonMaxSuppressionIE::clone_with_new_inputs(const ngraph::OutputVector & new_args) const {
+std::shared_ptr<Node> op::NonMaxSuppressionIE::clone_with_new_inputs(const ngraph::OutputVector &new_args) const {
check_new_args_count(this, new_args);
return make_shared<NonMaxSuppressionIE>(new_args.at(0), new_args.at(1), new_args.at(2), new_args.at(3),
new_args.at(4), m_center_point_box, m_sort_result_descending);
}
void op::NonMaxSuppressionIE::validate_and_infer_types() {
- auto squeeze_input = [](const Output<Node> & input) -> std::shared_ptr<Node> {
+ auto squeeze_input = [](const Output<Node> &input) -> std::shared_ptr<Node> {
return std::make_shared<opset1::Squeeze>(input, opset1::Constant::create(element::i64, Shape{1}, {0}));
};
// Calculate output shape using opset1::NonMaxSuppression
auto max_output_boxes_per_class = std::dynamic_pointer_cast<opset1::Constant>(input_value(2).get_node_shared_ptr());
- auto nms = std::make_shared<opset1::NonMaxSuppression>(input_value(0),
- input_value(1),
+ auto nms = std::make_shared<opset1::NonMaxSuppression>(input_value(0), input_value(1),
/* second input is used for output calculation and only if it's Constant output shape won't be dynamic */
- max_output_boxes_per_class ? opset1::Constant::create(element::i64, Shape{}, max_output_boxes_per_class->cast_vector<int64_t>()) :
- squeeze_input(input_value(2)),
- squeeze_input(input_value(3)),
- squeeze_input(input_value(4)));
+ max_output_boxes_per_class ? opset1::Constant::create(element::i64, Shape{},
+ max_output_boxes_per_class->cast_vector<int64_t>())
+ : squeeze_input(input_value(2)),
+ squeeze_input(input_value(3)), squeeze_input(input_value(4)));
+ set_output_type(0, nms->output(0).get_element_type(), nms->output(0).get_partial_shape());
+}
+
+bool op::NonMaxSuppressionIE::visit_attributes(AttributeVisitor& visitor) {
+ visitor.on_attribute("center_point_box", m_center_point_box);
+ visitor.on_attribute("sort_result_descending", m_sort_result_descending);
+ return true;
+}
+
+// The second version of the operation is different just in the shape infer function (uses v4::NMS)
+constexpr NodeTypeInfo op::NonMaxSuppressionIE2::type_info;
+
+op::NonMaxSuppressionIE2::NonMaxSuppressionIE2(const Output<Node> &boxes,
+ const Output<Node> &scores,
+ const Output<Node> &max_output_boxes_per_class,
+ const Output<Node> &iou_threshold,
+ const Output<Node> &score_threshold,
+ int center_point_box,
+ bool sort_result_descending)
+ : op::NonMaxSuppressionIE(boxes, scores, max_output_boxes_per_class, iou_threshold, score_threshold, center_point_box, sort_result_descending) {
+ constructor_validate_and_infer_types();
+}
+
+
+std::shared_ptr<Node> op::NonMaxSuppressionIE2::clone_with_new_inputs(const ngraph::OutputVector &new_args) const {
+ check_new_args_count(this, new_args);
+ return make_shared<NonMaxSuppressionIE2>(new_args.at(0), new_args.at(1), new_args.at(2), new_args.at(3),
+ new_args.at(4), m_center_point_box, m_sort_result_descending);
+}
+
+void op::NonMaxSuppressionIE2::validate_and_infer_types() {
+ auto squeeze_input = [](const Output<Node> &input) -> std::shared_ptr<Node> {
+ return std::make_shared<opset1::Squeeze>(input, opset1::Constant::create(element::i64, Shape{1}, {0}));
+ };
+
+ // Calculate output shape using opset4::NonMaxSuppression
+ auto max_output_boxes_per_class = std::dynamic_pointer_cast<opset1::Constant>(input_value(2).get_node_shared_ptr());
+ auto nms = std::make_shared<opset4::NonMaxSuppression>(input_value(0), input_value(1),
+ /* second input is used for output calculation and only if it's Constant output shape won't be dynamic */
+ max_output_boxes_per_class ? opset1::Constant::create(element::i64, Shape{},
+ max_output_boxes_per_class->cast_vector<int64_t>())
+ : squeeze_input(input_value(2)), squeeze_input(input_value(3)),
+ squeeze_input(input_value(4)));
set_output_type(0, nms->output(0).get_element_type(), nms->output(0).get_partial_shape());
}
#include "transformations/optimize_strided_slice.hpp"
#include "transformations/convert_scatter_elements_to_scatter.hpp"
#include "transformations/remove_filtering_boxes_by_size.hpp"
-#include "transformations/convert_nms_3_to_nms_v4.hpp"
+#include "transformations/convert_nms_4_to_nms_dynamic.hpp"
#include "transformations/init_node_info.hpp"
#include <ngraph/pass/manager.hpp>
+++ /dev/null
-// Copyright (C) 2018-2020 Intel Corporation
-// SPDX-License-Identifier: Apache-2.0
-//
-
-#include <memory>
-#include <vector>
-
-#include <ngraph/graph_util.hpp>
-#include <ngraph/opsets/opset3.hpp>
-#include <ngraph/rt_info.hpp>
-#include <transformations/utils/utils.hpp>
-
-#include "transformations/convert_nms_3_to_nms_v4.hpp"
-
-bool ngraph::pass::UpgradeNMS3ToNMS4::run_on_function(std::shared_ptr<ngraph::Function> f) {
- bool rewritten = false;
- if (!ngraph::op::util::has_op_with_type<ngraph::opset3::NonZero>(f)) {
- return rewritten;
- }
-
- for (auto &node : f->get_ops()) {
- auto nms_3 = ngraph::as_type_ptr<opset3::NonMaxSuppression>(node);
- if (!nms_3)
- continue;
-
- const auto box_encoding = static_cast<const op::v4::NonMaxSuppression::BoxEncodingType>(nms_3->get_box_encoding());
- const auto new_args = nms_3->input_values();
- NODE_VALIDATION_CHECK(nms_3.get(),
- new_args.size() >= 2 && new_args.size() <= 5,
- "Number of inputs must be 2, 3, 4 or 5");
- const auto& arg2 = new_args.size() > 2 ? new_args.at(2) : ngraph::opset3::Constant::create(element::i32, Shape{}, {0});
- const auto& arg3 = new_args.size() > 3 ? new_args.at(3) : ngraph::opset3::Constant::create(element::f32, Shape{}, {.0f});
- const auto& arg4 = new_args.size() > 4 ? new_args.at(4) : ngraph::opset3::Constant::create(element::f32, Shape{}, {.0f});
-
- const auto nms_4 = std::make_shared<op::v4::NonMaxSuppression>(
- new_args.at(0),
- new_args.at(1),
- arg2,
- arg3,
- arg4,
- box_encoding,
- nms_3->get_sort_result_descending(),
- nms_3->get_output_type());
-
- nms_4->set_friendly_name(nms_3->get_friendly_name());
- ngraph::copy_runtime_info(nms_3, nms_4);
- ngraph::replace_node(nms_3, nms_4);
- rewritten = true;
- }
- return rewritten;
-}
--- /dev/null
+// Copyright (C) 2018-2020 Intel Corporation
+// SPDX-License-Identifier: Apache-2.0
+//
+
+#include <memory>
+#include <vector>
+
+#include <ngraph/graph_util.hpp>
+#include <ngraph/opsets/opset4.hpp>
+#include <ngraph/rt_info.hpp>
+#include <transformations/utils/utils.hpp>
+
+#include "transformations/convert_nms_4_to_nms_dynamic.hpp"
+
+void ngraph::pass::UpgradeNMS4ToNMSDynamic::upgrade_nms4_to_nms_dynamic() {
+ auto boxes = std::make_shared<pattern::op::Label>(element::f32, Shape{1, 1000, 4});
+ auto scores = std::make_shared<pattern::op::Label>(element::f32, Shape{1, 1, 1000});
+ auto max_output_boxes_per_class = ngraph::opset4::Constant::create(element::i64, Shape{}, {10});
+ auto iou_threshold = ngraph::opset4::Constant::create(element::f32, Shape{}, {0.75});
+ auto score_threshold = ngraph::opset4::Constant::create(element::f32, Shape{}, {0.7});
+ auto nms = std::make_shared<ngraph::opset4::NonMaxSuppression>(boxes, scores, max_output_boxes_per_class,
+ iou_threshold, score_threshold);
+
+ ngraph::graph_rewrite_callback callback = [](pattern::Matcher &m) {
+ auto nms_4 = std::dynamic_pointer_cast<ngraph::opset4::NonMaxSuppression>(m.get_match_root());
+ if (!nms_4) {
+ return false;
+ }
+
+ const auto box_encoding = static_cast<const op::dynamic::NonMaxSuppression::BoxEncodingType>(nms_4->get_box_encoding());
+ const auto new_args = nms_4->input_values();
+ const auto& arg2 = new_args.size() > 2 ? new_args.at(2) : ngraph::opset4::Constant::create(element::i32, Shape{}, {0});
+ const auto& arg3 = new_args.size() > 3 ? new_args.at(3) : ngraph::opset4::Constant::create(element::f32, Shape{}, {.0f});
+ const auto& arg4 = new_args.size() > 4 ? new_args.at(4) : ngraph::opset4::Constant::create(element::f32, Shape{}, {.0f});
+
+ const auto nms_dynamic = std::make_shared<op::dynamic::NonMaxSuppression>(
+ new_args.at(0),
+ new_args.at(1),
+ arg2,
+ arg3,
+ arg4,
+ box_encoding,
+ nms_4->get_sort_result_descending(),
+ nms_4->get_output_type());
+
+ nms_dynamic->set_friendly_name(nms_4->get_friendly_name());
+ ngraph::copy_runtime_info(nms_4, nms_dynamic);
+ ngraph::replace_node(nms_4, nms_dynamic);
+ return true;
+ };
+
+ auto m = std::make_shared<ngraph::pattern::Matcher>(nms, "UpgradeNMS4ToDynamic");
+ this->add_matcher(m, callback, PassProperty::CHANGE_DYNAMIC_STATE);
+}
+
+bool ngraph::pass::UpgradeNMS4ToNMSDynamic::run_on_function(std::shared_ptr<ngraph::Function> f) {
+ if (ngraph::op::util::has_op_with_type<ngraph::opset4::NonZero>(f)) {
+ return GraphRewrite::run_on_function(f);
+ }
+ return false;
+}
--- /dev/null
+// Copyright (C) 2018-2020 Intel Corporation
+// SPDX-License-Identifier: Apache-2.0
+//
+
+#include <memory>
+#include <vector>
+
+#include <ngraph/graph_util.hpp>
+#include <ngraph/opsets/opset1.hpp>
+#include <ngraph/opsets/opset4.hpp>
+#include <ngraph_ops/nms_ie.hpp>
+#include <ngraph/rt_info.hpp>
+#include <transformations/utils/utils.hpp>
+
+#include "transformations/convert_opset1_to_legacy/convert_nms_4_to_legacy.hpp"
+
+void ngraph::pass::ConvertNMS4ToLegacy::convert_nms4_to_legacy() {
+ auto boxes = std::make_shared<pattern::op::Label>(element::f32, Shape{1, 1000, 4});
+ auto scores = std::make_shared<pattern::op::Label>(element::f32, Shape{1, 1, 1000});
+ auto max_output_boxes_per_class = ngraph::opset4::Constant::create(element::i64, Shape{}, {10});
+ auto iou_threshold = ngraph::opset4::Constant::create(element::f32, Shape{}, {0.75});
+ auto score_threshold = ngraph::opset4::Constant::create(element::f32, Shape{}, {0.7});
+ auto nms = std::make_shared<ngraph::opset4::NonMaxSuppression>(boxes, scores, max_output_boxes_per_class,
+ iou_threshold, score_threshold);
+
+ ngraph::graph_rewrite_callback callback = [](pattern::Matcher &m) {
+ auto nms_4 = std::dynamic_pointer_cast<ngraph::opset4::NonMaxSuppression>(m.get_match_root());
+ if (!nms_4) {
+ return false;
+ }
+
+ const auto new_args = nms_4->input_values();
+ const auto& arg2 = new_args.size() > 2 ? new_args.at(2) : ngraph::opset4::Constant::create(element::i32, Shape{}, {0});
+ const auto& arg3 = new_args.size() > 3 ? new_args.at(3) : ngraph::opset4::Constant::create(element::f32, Shape{}, {.0f});
+ const auto& arg4 = new_args.size() > 4 ? new_args.at(4) : ngraph::opset4::Constant::create(element::f32, Shape{}, {.0f});
+
+ const auto max_output_boxes_per_class_rank = arg2.get_partial_shape().rank();
+ const auto iou_threshold_rank = arg3.get_partial_shape().rank();
+ const auto score_threshold_rank = arg4.get_partial_shape().rank();
+
+ // Check that required ranks are not dynamic
+ if (max_output_boxes_per_class_rank.is_dynamic() ||
+ iou_threshold_rank.is_dynamic() ||
+ score_threshold_rank.is_dynamic()) {
+ return false;
+ }
+
+ if (max_output_boxes_per_class_rank.get_length() == 1 &&
+ iou_threshold_rank.get_length() == 1 &&
+ score_threshold_rank.get_length() == 1) {
+ return false;
+ }
+
+ // vector of new nGraph operations
+ NodeVector new_ops;
+
+ auto new_max_per_class = arg2;
+ if (max_output_boxes_per_class_rank.get_length() == 0) {
+ // WA: we need to create Constant manually because it requires by NMS shape inference
+ // otherwise we will get dynamic shape until first CF is executed. It can be resolved
+ // if CF will be executed right after transformation and before Validate pass.
+ if (auto new_max_per_class_const = std::dynamic_pointer_cast<opset1::Constant>(new_max_per_class.get_node_shared_ptr())) {
+ new_max_per_class = opset1::Constant::create(element::i64, Shape{1}, new_max_per_class_const->cast_vector<int64_t>());
+ } else {
+ new_max_per_class = std::make_shared<ngraph::op::Unsqueeze>(arg2, opset1::Constant::create(element::i64, Shape{1}, {0}));
+ new_ops.push_back(new_max_per_class.get_node_shared_ptr());
+ }
+ }
+ auto new_iou_threshold = arg3;
+ if (iou_threshold_rank.get_length() == 0) {
+ new_iou_threshold = std::make_shared<ngraph::op::Unsqueeze>(arg3, opset1::Constant::create(element::i64, Shape{1}, {0}));
+ new_ops.push_back(new_iou_threshold.get_node_shared_ptr());
+ }
+ auto new_score_threshold = arg4;
+ if (score_threshold_rank.get_length() == 0) {
+ new_score_threshold = std::make_shared<ngraph::op::Unsqueeze>(arg4, opset1::Constant::create(element::i64, Shape{1}, {0}));
+ new_ops.push_back(new_score_threshold.get_node_shared_ptr());
+ }
+
+ int center_point_box = 0;
+ switch (nms_4->get_box_encoding()) {
+ case ::ngraph::opset4::NonMaxSuppression::BoxEncodingType::CENTER:
+ center_point_box = 1;
+ break;
+ case ::ngraph::opset4::NonMaxSuppression::BoxEncodingType::CORNER:
+ center_point_box = 0;
+ break;
+ default:
+ throw ngraph_error("NonMaxSuppression layer " + nms_4->get_friendly_name() +
+ " has unsupported box encoding");
+ }
+ const auto nms_legacy = std::make_shared<op::NonMaxSuppressionIE2>(
+ new_args.at(0),
+ new_args.at(1),
+ new_max_per_class,
+ new_iou_threshold,
+ new_score_threshold,
+ center_point_box,
+ nms_4->get_sort_result_descending());
+ new_ops.push_back(nms_legacy);
+
+ Output<Node> last;
+ // if the output is the i32 then it matches behavior of the v1::NonMaxSuppression otherwise need to insert Convert
+ if (nms_4->get_output_type() == element::i32) {
+ last = nms_legacy;
+ } else {
+ last = std::make_shared<ngraph::opset4::Convert>(nms_legacy, nms_4->get_output_type());
+ new_ops.push_back(last.get_node_shared_ptr());
+ }
+
+ last.get_node_shared_ptr()->set_friendly_name(nms_4->get_friendly_name());
+ ngraph::copy_runtime_info(nms_4, new_ops);
+ ngraph::replace_node(nms_4, last.get_node_shared_ptr());
+ return true;
+ };
+
+ auto m = std::make_shared<ngraph::pattern::Matcher>(nms, "ConvertNMS4ToNMSLegacy");
+ this->add_matcher(m, callback, PassProperty::CHANGE_DYNAMIC_STATE);
+}
#include <transformations/convert_opset1_to_legacy/convert_mul_or_add_finally.hpp>
#include <transformations/convert_negative.hpp>
#include <transformations/convert_opset1_to_legacy/convert_nms_to_nms_ie.hpp>
+#include <transformations/convert_opset1_to_legacy/convert_nms_4_to_legacy.hpp>
#include <transformations/convert_opset1_to_legacy/convert_normalizel2_to_normalize_ie.hpp>
#include <transformations/convert_opset1_to_legacy/convert_one_hot_to_one_hot_ie.hpp>
#include <transformations/convert_opset1_to_legacy/convert_pad_to_pad_ie.hpp>
namespace ngraph { namespace vpu { namespace op {
-class StaticShapeNonMaxSuppression : public ngraph::op::v3::NonMaxSuppression {
+class StaticShapeNonMaxSuppression : public ngraph::op::v4::NonMaxSuppression {
public:
static constexpr NodeTypeInfo type_info{"StaticShapeStaticShapeNonMaxSuppression", 0};
const NodeTypeInfo& get_type_info() const override { return type_info; }
const StaticShapeNonMaxSuppression::BoxEncodingType box_encoding,
const bool sort_result_descending,
const element::Type& output_type)
- : ngraph::op::v3::NonMaxSuppression({
+ : ngraph::op::v4::NonMaxSuppression(
boxes, scores, max_output_boxes_per_class, iou_threshold, score_threshold,
- box_encoding, sort_result_descending, output_type}) {
+ box_encoding, sort_result_descending, output_type) {
constructor_validate_and_infer_types();
}
const StaticShapeNonMaxSuppression::BoxEncodingType box_encoding,
const bool sort_result_descending,
const element::Type& output_type)
- : ngraph::op::v3::NonMaxSuppression({boxes,
+ : ngraph::op::v4::NonMaxSuppression(boxes,
scores,
ngraph::opset3::Constant::create(element::i64, Shape{}, {0}),
ngraph::opset3::Constant::create(element::f32, Shape{}, {.0f}),
ngraph::opset3::Constant::create(element::f32, Shape{}, {.0f}),
- box_encoding, sort_result_descending, output_type}) {
+ box_encoding, sort_result_descending, output_type) {
constructor_validate_and_infer_types();
}
}
void StaticShapeNonMaxSuppression::validate_and_infer_types() {
- ngraph::op::v3::NonMaxSuppression::validate_and_infer_types();
+ ngraph::op::v4::NonMaxSuppression::validate_and_infer_types();
const auto out_shape = this->get_output_partial_shape(0);
NODE_VALIDATION_CHECK(this, out_shape.is_static(),
{ngraph::opset3::Equal::type_info, dynamicToStaticShapeBinaryEltwise},
{ngraph::opset3::Greater::type_info, dynamicToStaticShapeBinaryEltwise},
{ngraph::opset3::Power::type_info, dynamicToStaticShapeBinaryEltwise},
- {ngraph::op::v4::NonMaxSuppression::type_info, dynamicToStaticNonMaxSuppression},
+ {ngraph::op::dynamic::NonMaxSuppression::type_info, dynamicToStaticNonMaxSuppression},
{ngraph::opset3::NonZero::type_info, dynamicToStaticShapeNonZero},
{ngraph::opset3::TopK::type_info, dynamicToStaticShapeTopK},
{ngraph::opset3::Transpose::type_info, dynamicToStaticShapeTranspose},
namespace vpu {
void dynamicToStaticNonMaxSuppression(std::shared_ptr<ngraph::Node> node) {
- auto nms_4 = std::dynamic_pointer_cast<ngraph::op::v4::NonMaxSuppression>(node);
- VPU_THROW_UNLESS(nms_4, "dynamicToStaticNonMaxSuppression transformation for {} of type {} expects {} as node for replacement",
- node->get_friendly_name(), node->get_type_info(), ngraph::op::v4::NonMaxSuppression::type_info);
+ auto nms_dynamic = std::dynamic_pointer_cast<ngraph::op::dynamic::NonMaxSuppression>(node);
+ VPU_THROW_UNLESS(nms_dynamic, "dynamicToStaticNonMaxSuppression transformation for {} of type {} expects {} as node for replacement",
+ node->get_friendly_name(), node->get_type_info(), ngraph::op::dynamic::NonMaxSuppression::type_info);
auto staticShapeNMS = std::make_shared<ngraph::vpu::op::StaticShapeNonMaxSuppression>(
- nms_4->input_value(0),
- nms_4->input_value(1),
- nms_4->input_value(2),
- nms_4->input_value(3),
- nms_4->input_value(4),
- nms_4->get_box_encoding(),
- nms_4->get_sort_result_descending(),
- nms_4->get_output_type());
+ nms_dynamic->input_value(0),
+ nms_dynamic->input_value(1),
+ nms_dynamic->input_value(2),
+ nms_dynamic->input_value(3),
+ nms_dynamic->input_value(4),
+ nms_dynamic->get_box_encoding(),
+ nms_dynamic->get_sort_result_descending(),
+ nms_dynamic->get_output_type());
auto dynamicShapeResolver = std::make_shared<ngraph::vpu::op::DynamicShapeResolver>(
staticShapeNMS->output(0), staticShapeNMS->output(1));
- dynamicShapeResolver->set_friendly_name(nms_4->get_friendly_name());
+ dynamicShapeResolver->set_friendly_name(nms_dynamic->get_friendly_name());
- ngraph::replace_node(std::move(nms_4), std::move(dynamicShapeResolver));
+ ngraph::replace_node(std::move(nms_dynamic), std::move(dynamicShapeResolver));
}
} // namespace vpu
--- /dev/null
+// Copyright (C) 2020 Intel Corporation
+// SPDX-License-Identifier: Apache-2.0
+//
+
+#include <gtest/gtest.h>
+
+#include <string>
+#include <memory>
+
+#include <ngraph/function.hpp>
+#include <ngraph/opsets/opset4.hpp>
+#include <ngraph_ops/nms_ie.hpp>
+#include <transformations/convert_opset1_to_legacy/convert_nms_4_to_legacy.hpp>
+#include <transformations/init_node_info.hpp>
+#include <transformations/utils/utils.hpp>
+
+#include "ngraph_test_utils.hpp"
+
+using namespace testing;
+using namespace ngraph;
+
+TEST(TransformationTests, ConvertNMS4ToNMSIEStatic) {
+ std::shared_ptr<Function> f(nullptr), f_ref(nullptr);
+ {
+ auto boxes = std::make_shared<opset4::Parameter>(element::f32, Shape{1, 1000, 4});
+ auto scores = std::make_shared<opset4::Parameter>(element::f32, Shape{1, 1, 1000});
+ auto max_output_boxes_per_class = opset4::Constant::create(element::i64, Shape{}, {10});
+ auto iou_threshold = opset4::Constant::create(element::f32, Shape{}, {0.75});
+ auto score_threshold = opset4::Constant::create(element::f32, Shape{}, {0.7});
+ auto nms = std::make_shared<opset4::NonMaxSuppression>(boxes, scores, max_output_boxes_per_class, iou_threshold, score_threshold,
+ opset4::NonMaxSuppression::BoxEncodingType::CORNER, true);
+
+ f = std::make_shared<Function>(NodeVector{nms}, ParameterVector{boxes, scores});
+
+ const auto &orig_shape = f->get_output_partial_shape(0);
+ pass::InitNodeInfo().run_on_function(f);
+ pass::ConvertNMS4ToLegacy().run_on_function(f);
+ ASSERT_NO_THROW(check_rt_info(f));
+ ASSERT_TRUE(f->get_output_partial_shape(0).is_static()) << "Shape " << f->get_output_partial_shape(0) << " should be static";
+ }
+
+ {
+ auto boxes = std::make_shared<opset4::Parameter>(element::f32, Shape{1, 1000, 4});
+ auto scores = std::make_shared<opset4::Parameter>(element::f32, Shape{1, 1, 1000});
+ auto max_output_boxes_per_class = opset4::Constant::create(element::i64, Shape{1}, {10});
+ auto iou_threshold = opset4::Constant::create(element::f32, Shape{}, {0.75});
+ auto score_threshold = opset4::Constant::create(element::f32, Shape{}, {0.7});
+ auto nms = std::make_shared<op::NonMaxSuppressionIE2>(boxes, scores, max_output_boxes_per_class,
+ std::make_shared<opset4::Unsqueeze>(iou_threshold,
+ opset4::Constant::create(element::i64, Shape{1}, {0})),
+ std::make_shared<opset4::Unsqueeze>(score_threshold,
+ opset4::Constant::create(element::i64, Shape{1}, {0})),
+ 0, true);
+ auto convert = std::make_shared<ngraph::opset4::Convert>(nms, element::i64);
+ convert->set_friendly_name("nms");
+
+ f_ref = std::make_shared<Function>(NodeVector{convert}, ParameterVector{boxes, scores});
+ ASSERT_TRUE(f_ref->get_output_partial_shape(0).is_static()) << "Shape " << f_ref->get_output_partial_shape(0) << " should be static";
+ }
+
+ auto res = compare_functions(f, f_ref);
+ ASSERT_TRUE(res.first) << res.second;
+}
+
+TEST(TransformationTests, ConvertNMS4ToNMSIEDynamic1) {
+ std::shared_ptr<Function> f(nullptr), f_ref(nullptr);
+ {
+ auto boxes = std::make_shared<opset4::Parameter>(element::f32, PartialShape::dynamic());
+ auto scores = std::make_shared<opset4::Parameter>(element::f32, PartialShape::dynamic());
+ auto max_output_boxes_per_class = opset4::Constant::create(element::i64, Shape{}, {10});
+ auto iou_threshold = opset4::Constant::create(element::f32, Shape{}, {0.75});
+ auto score_threshold = opset4::Constant::create(element::f32, Shape{}, {0.7});
+ auto nms = std::make_shared<opset4::NonMaxSuppression>(boxes, scores, max_output_boxes_per_class, iou_threshold, score_threshold,
+ opset4::NonMaxSuppression::BoxEncodingType::CORNER, true, element::i32);
+
+ f = std::make_shared<Function>(NodeVector{nms}, ParameterVector{boxes, scores});
+
+ pass::InitNodeInfo().run_on_function(f);
+ pass::ConvertNMS4ToLegacy().run_on_function(f);
+ f->validate_nodes_and_infer_types();
+ ASSERT_NO_THROW(check_rt_info(f));
+ }
+
+ {
+ auto boxes = std::make_shared<opset4::Parameter>(element::f32, PartialShape::dynamic());
+ auto scores = std::make_shared<opset4::Parameter>(element::f32, PartialShape::dynamic());
+ auto max_output_boxes_per_class = opset4::Constant::create(element::i64, Shape{1}, {10});
+ auto iou_threshold = opset4::Constant::create(element::f32, Shape{}, {0.75});
+ auto score_threshold = opset4::Constant::create(element::f32, Shape{}, {0.7});
+ auto nms = std::make_shared<op::NonMaxSuppressionIE2>(boxes, scores, max_output_boxes_per_class,
+ std::make_shared<opset4::Unsqueeze>(iou_threshold,
+ opset4::Constant::create(element::i64, Shape{1}, {0})),
+ std::make_shared<opset4::Unsqueeze>(score_threshold,
+ opset4::Constant::create(element::i64, Shape{1}, {0})),
+ 0, true);
+
+ f_ref = std::make_shared<Function>(NodeVector{nms}, ParameterVector{boxes, scores});
+ }
+
+ auto res = compare_functions(f, f_ref);
+ ASSERT_TRUE(res.first) << res.second;
+}
+
+TEST(TransformationTests, ConvertNMS4ToNMSIEDynamic2) {
+ std::shared_ptr<Function> f(nullptr), f_ref(nullptr);
+ {
+ auto boxes = std::make_shared<opset4::Parameter>(element::f32, PartialShape{DYN, 1000, 4});
+ auto scores = std::make_shared<opset4::Parameter>(element::f32, PartialShape{DYN, 1, 1000});
+ auto max_output_boxes_per_class = opset4::Constant::create(element::i64, Shape{}, {10});
+ auto iou_threshold = opset4::Constant::create(element::f32, Shape{}, {0.75});
+ auto score_threshold = opset4::Constant::create(element::f32, Shape{}, {0.7});
+ auto nms = std::make_shared<opset4::NonMaxSuppression>(boxes, scores, max_output_boxes_per_class, iou_threshold, score_threshold,
+ opset4::NonMaxSuppression::BoxEncodingType::CORNER, true, element::i32);
+
+ f = std::make_shared<Function>(NodeVector{nms}, ParameterVector{boxes, scores});
+
+ pass::InitNodeInfo().run_on_function(f);
+ pass::ConvertNMS4ToLegacy().run_on_function(f);
+ f->validate_nodes_and_infer_types();
+ ASSERT_NO_THROW(check_rt_info(f));
+ }
+
+ {
+ auto boxes = std::make_shared<opset4::Parameter>(element::f32, PartialShape{DYN, 1000, 4});
+ auto scores = std::make_shared<opset4::Parameter>(element::f32, PartialShape{DYN, 1, 1000});
+ auto max_output_boxes_per_class = opset4::Constant::create(element::i64, Shape{1}, {10});
+ auto iou_threshold = opset4::Constant::create(element::f32, Shape{}, {0.75});
+ auto score_threshold = opset4::Constant::create(element::f32, Shape{}, {0.7});
+ auto nms = std::make_shared<op::NonMaxSuppressionIE2>(boxes, scores, max_output_boxes_per_class,
+ std::make_shared<opset4::Unsqueeze>(iou_threshold,
+ opset4::Constant::create(element::i64, Shape{1}, {0})),
+ std::make_shared<opset4::Unsqueeze>(score_threshold,
+ opset4::Constant::create(element::i64, Shape{1}, {0})),
+ 0, true);
+
+ f_ref = std::make_shared<Function>(NodeVector{nms}, ParameterVector{boxes, scores});
+ }
+
+ auto res = compare_functions(f, f_ref);
+ ASSERT_TRUE(res.first) << res.second;
+}
const auto dims = std::make_shared<ngraph::opset3::Parameter>(ngraph::element::i64, ngraph::Shape{3});
const auto dsr = std::make_shared<ngraph::vpu::op::DynamicShapeResolver>(scores, dims);
- const auto node = std::make_shared<ngraph::op::v4::NonMaxSuppression>(
+ const auto node = std::make_shared<ngraph::op::dynamic::NonMaxSuppression>(
boxes, dsr, max_output_boxes_per_class, iou_threshold, score_threshold);
auto outputShape = node->get_output_partial_shape(0);
const auto dims = std::make_shared<ngraph::opset3::Parameter>(ngraph::element::i64, ngraph::Shape{3});
const auto dsr = std::make_shared<ngraph::vpu::op::DynamicShapeResolver>(scores, dims);
- const auto node = std::make_shared<ngraph::op::v4::NonMaxSuppression>(
+ const auto node = std::make_shared<ngraph::op::dynamic::NonMaxSuppression>(
boxes, dsr, max_output_boxes_per_class, iou_threshold, score_threshold);
const auto result = std::make_shared<ngraph::opset3::Result>(node);
function = std::make_shared<ngraph::Function>(ngraph::ResultVector{result},
- ngraph::ParameterVector{boxes, scores, dims}, "DSR-v4::NMS");
+ ngraph::ParameterVector{boxes, scores, dims}, "DSR-dynamic::NMS");
}
};
def __init__(self, graph: Graph, attrs: dict):
mandatory_props = {
- 'type': __class__.op,
- 'op': __class__.op,
- 'version': 'opset3',
- 'infer': __class__.infer,
+ 'type': self.op,
+ 'op': self.op,
+ 'version': 'opset4',
+ 'infer': self.infer,
'output_type': np.int64,
'center_point_box': 0,
'box_encoding': 'corner',
}
super().__init__(graph, mandatory_props, attrs)
- def supported_attrs(self):
- if self.ir_version < 10:
- return ['center_point_box']
+ def backend_attrs(self):
+ version = self.get_opset()
+ if version in ['opset3', 'opset4']:
+ return ['sort_result_descending', 'box_encoding',
+ ('output_type', lambda node: np_data_type_to_destination_type(node.output_type))]
+ elif version == 'opset1':
+ return ['sort_result_descending', 'box_encoding']
else:
- version = self.get_opset()
- if version == 'opset3':
- return ['sort_result_descending', 'box_encoding',
- ('output_type', lambda node: np_data_type_to_destination_type(node.output_type))]
- elif version == 'opset1':
- return ['sort_result_descending', 'box_encoding']
- else:
- raise Error('Unsupported operation opset version "{}"'.format(version))
+ raise Error('Unsupported operation opset version "{}"'.format(version))
@staticmethod
def infer(node: Node):
num_input_boxes = boxes_shape[1]
assert scores_shape[2] == num_input_boxes, 'Number of boxes mismatch'
- max_number_of_boxes = min(num_input_boxes, boxes_shape[0] * max_output_boxes_per_class * num_classes)
+ if node.get_opset() == 'opset4':
+ max_number_of_boxes = min(num_input_boxes, max_output_boxes_per_class) * boxes_shape[0] * num_classes
+ else:
+ max_number_of_boxes = min(num_input_boxes, boxes_shape[0] * max_output_boxes_per_class * num_classes)
node.out_port(0).data.set_shape(int64_array([max_number_of_boxes, 3]))
@staticmethod
def type_infer(node):
- if node.get_opset() == 'opset3':
+ if node.get_opset() in ['opset3', 'opset4']:
node.out_port(0).set_data_type(node.output_type)
else:
node.out_port(0).set_data_type(np.int64)
from extensions.ops.non_max_suppression import NonMaxSuppression
from mo.front.common.partial_infer.utils import int64_array
from mo.graph.graph import Node
-from mo.utils.unittest.graph import build_graph, regular_op_with_shaped_data, valued_const_with_data, result, \
- connect, FakeAttr
+from mo.utils.unittest.graph import build_graph, regular_op_with_shaped_data, valued_const_with_data, result, connect
class TestNonMaxSuppressionInfer(unittest.TestCase):
nodes = {
**regular_op_with_shaped_data('boxes', [10, 100, 4], {'type': 'Parameter'}),
**regular_op_with_shaped_data('scores', [10, 5, 100], {'type': 'Parameter'}),
- **valued_const_with_data('max_output_per_class', int64_array(10)),
+ **valued_const_with_data('max_output_per_class', int64_array(7)),
**regular_op_with_shaped_data('nms', None, {'op': 'NonMaxSuppression', 'type': 'NonMaxSuppression',
'name': 'nms'}),
**result('output'),
*connect('nms', 'output'),
], nodes_with_edges_only=True)
- def test_nms_infer_v10_opset1(self):
- self.graph.graph['cmd_params'] = FakeAttr(ir_version=10)
-
+ def test_nms_infer_opset1(self):
nms_node = Node(self.graph, 'nms')
nms_node['version'] = 'opset1'
NonMaxSuppression.infer(nms_node)
self.assertTrue(np.array_equal(nms_node.out_port(0).data.get_shape(), [100, 3]))
self.assertTrue(nms_node.out_port(0).get_data_type() == np.int64)
- def test_nms_infer_v10_i64_opset3(self):
- self.graph.graph['cmd_params'] = FakeAttr(ir_version=10)
-
+ def test_nms_infer_i64_opset3(self):
nms_node = Node(self.graph, 'nms')
nms_node['version'] = 'opset3'
nms_node['output_type'] = np.int64
self.assertTrue(np.array_equal(nms_node.out_port(0).data.get_shape(), [100, 3]))
self.assertTrue(nms_node.out_port(0).get_data_type() == np.int64)
- def test_nms_infer_v10_i32_opset3(self):
- self.graph.graph['cmd_params'] = FakeAttr(ir_version=10)
-
+ def test_nms_infer_i32_opset3(self):
nms_node = Node(self.graph, 'nms')
nms_node['version'] = 'opset3'
nms_node['output_type'] = np.int32
self.assertTrue(np.array_equal(nms_node.out_port(0).data.get_shape(), [100, 3]))
self.assertTrue(nms_node.out_port(0).get_data_type() == np.int32)
+
+ def test_nms_infer_i32_opset4(self):
+ nms_node = Node(self.graph, 'nms')
+ nms_node['version'] = 'opset4'
+ nms_node['output_type'] = np.int32
+ NonMaxSuppression.infer(nms_node)
+ NonMaxSuppression.type_infer(nms_node)
+
+ self.assertTrue(np.array_equal(nms_node.out_port(0).data.get_shape(), [10 * 5 * 7, 3]))
+ self.assertTrue(nms_node.out_port(0).get_data_type() == np.int32)
+
+ def test_nms_infer_i64_opset4(self):
+ nms_node = Node(self.graph, 'nms')
+ nms_node['version'] = 'opset4'
+ nms_node['output_type'] = np.int64
+ NonMaxSuppression.infer(nms_node)
+ NonMaxSuppression.type_infer(nms_node)
+
+ self.assertTrue(np.array_equal(nms_node.out_port(0).data.get_shape(), [10 * 5 * 7, 3]))
+ self.assertTrue(nms_node.out_port(0).get_data_type() == np.int64)
from _pyngraph import NodeFactory as _NodeFactory
from ngraph.impl import Node
-DEFAULT_OPSET = "opset3"
+DEFAULT_OPSET = "opset4"
class NodeFactory(object):
{"opset1", OpsetFunction(ngraph::get_opset1)},
{"opset2", OpsetFunction(ngraph::get_opset2)},
{"opset3", OpsetFunction(ngraph::get_opset3)},
+ {"opset4", OpsetFunction(ngraph::get_opset4)},
};
auto it = s_opsets.find(opset_ver);
out_shape[0] = std::min(num_boxes, max_output_boxes_per_class * num_classes);
}
- set_output_size(1);
set_output_type(0, output_element_type, out_shape);
}
return true;
}
-void op::v3::NonMaxSuppression::validate_and_infer_types()
+void op::v3::NonMaxSuppression::validate()
{
const auto boxes_ps = get_input_partial_shape(0);
const auto scores_ps = get_input_partial_shape(1);
m_output_type == element::i64 || m_output_type == element::i32,
"Output type must be i32 or i64");
- // NonMaxSuppression produces triplets
- // that have the following format: [batch_index, class_index, box_index]
- PartialShape out_shape = {Dimension::dynamic(), 3};
-
if (boxes_ps.is_dynamic() || scores_ps.is_dynamic())
{
- set_output_type(0, m_output_type, out_shape);
return;
}
boxes_ps[2].is_static() && boxes_ps[2].get_length() == 4u,
"The last dimension of the 'boxes' input must be equal to 4. Got:",
boxes_ps[2]);
+}
- const auto max_output_boxes_per_class = input_value(2).get_node_shared_ptr();
- if (num_boxes_boxes.is_static() && scores_ps[1].is_static() &&
- max_output_boxes_per_class->is_constant())
- {
- const auto num_boxes = num_boxes_boxes.get_length();
- const auto max_output_boxes_per_class = max_boxes_output_from_input();
- const auto num_classes = scores_ps[1].get_length();
+void op::v3::NonMaxSuppression::validate_and_infer_types()
+{
+ const auto boxes_ps = get_input_partial_shape(0);
+ const auto scores_ps = get_input_partial_shape(1);
- out_shape[0] = std::min(num_boxes, max_output_boxes_per_class * num_classes);
+ // NonMaxSuppression produces triplets
+ // that have the following format: [batch_index, class_index, box_index]
+ PartialShape out_shape = {Dimension::dynamic(), 3};
+
+ validate();
+
+ if (boxes_ps.rank().is_static() && scores_ps.rank().is_static())
+ {
+ const auto num_boxes_boxes = boxes_ps[1];
+ const auto max_output_boxes_per_class_node = input_value(2).get_node_shared_ptr();
+ if (num_boxes_boxes.is_static() && scores_ps[1].is_static() &&
+ max_output_boxes_per_class_node->is_constant())
+ {
+ const auto num_boxes = num_boxes_boxes.get_length();
+ const auto num_classes = scores_ps[1].get_length();
+ const auto max_output_boxes_per_class = max_boxes_output_from_input();
+
+ out_shape[0] = std::min(num_boxes, max_output_boxes_per_class * num_classes);
+ }
}
set_output_type(0, m_output_type, out_shape);
}
const op::v4::NonMaxSuppression::BoxEncodingType box_encoding,
const bool sort_result_descending,
const element::Type& output_type)
- : op::v3::NonMaxSuppression({boxes,
- scores,
- max_output_boxes_per_class,
- iou_threshold,
- score_threshold,
- box_encoding,
- sort_result_descending,
- output_type})
+ : op::v3::NonMaxSuppression(boxes,
+ scores,
+ max_output_boxes_per_class,
+ iou_threshold,
+ score_threshold,
+ box_encoding,
+ sort_result_descending,
+ output_type)
{
constructor_validate_and_infer_types();
}
const op::v4::NonMaxSuppression::BoxEncodingType box_encoding,
const bool sort_result_descending,
const element::Type& output_type)
- : op::v3::NonMaxSuppression({boxes,
- scores,
- op::Constant::create(element::i64, Shape{}, {0}),
- op::Constant::create(element::f32, Shape{}, {.0f}),
- op::Constant::create(element::f32, Shape{}, {.0f}),
- box_encoding,
- sort_result_descending,
- output_type})
+ : op::v3::NonMaxSuppression(boxes,
+ scores,
+ op::Constant::create(element::i64, Shape{}, {0}),
+ op::Constant::create(element::f32, Shape{}, {.0f}),
+ op::Constant::create(element::f32, Shape{}, {.0f}),
+ box_encoding,
+ sort_result_descending,
+ output_type)
{
constructor_validate_and_infer_types();
}
}
void op::v4::NonMaxSuppression::validate_and_infer_types()
+{
+ const auto boxes_ps = get_input_partial_shape(0);
+ const auto scores_ps = get_input_partial_shape(1);
+
+ // NonMaxSuppression produces triplets
+ // that have the following format: [batch_index, class_index, box_index]
+ PartialShape out_shape = {Dimension::dynamic(), 3};
+
+ op::v3::NonMaxSuppression::validate();
+
+ if (boxes_ps.rank().is_static() && scores_ps.rank().is_static())
+ {
+ const auto num_boxes_boxes = boxes_ps[1];
+ const auto max_output_boxes_per_class_node = input_value(2).get_node_shared_ptr();
+ if (num_boxes_boxes.is_static() && scores_ps[0].is_static() && scores_ps[1].is_static() &&
+ max_output_boxes_per_class_node->is_constant())
+ {
+ const auto num_boxes = num_boxes_boxes.get_length();
+ const auto num_classes = scores_ps[1].get_length();
+ const auto max_output_boxes_per_class = max_boxes_output_from_input();
+
+ out_shape[0] = std::min(num_boxes, max_output_boxes_per_class) * num_classes *
+ scores_ps[0].get_length();
+ }
+ }
+ set_output_type(0, m_output_type, out_shape);
+}
+
+// ------------------------------ dynamic ------------------------------
+
+constexpr NodeTypeInfo op::dynamic::NonMaxSuppression::type_info;
+
+op::dynamic::NonMaxSuppression::NonMaxSuppression(
+ const Output<Node>& boxes,
+ const Output<Node>& scores,
+ const Output<Node>& max_output_boxes_per_class,
+ const Output<Node>& iou_threshold,
+ const Output<Node>& score_threshold,
+ const op::dynamic::NonMaxSuppression::BoxEncodingType box_encoding,
+ const bool sort_result_descending,
+ const element::Type& output_type)
+ : op::v3::NonMaxSuppression(boxes,
+ scores,
+ max_output_boxes_per_class,
+ iou_threshold,
+ score_threshold,
+ box_encoding,
+ sort_result_descending,
+ output_type)
+{
+ constructor_validate_and_infer_types();
+}
+
+op::dynamic::NonMaxSuppression::NonMaxSuppression(
+ const Output<Node>& boxes,
+ const Output<Node>& scores,
+ const op::dynamic::NonMaxSuppression::BoxEncodingType box_encoding,
+ const bool sort_result_descending,
+ const element::Type& output_type)
+ : op::v3::NonMaxSuppression(boxes,
+ scores,
+ op::Constant::create(element::i64, Shape{}, {0}),
+ op::Constant::create(element::f32, Shape{}, {.0f}),
+ op::Constant::create(element::f32, Shape{}, {.0f}),
+ box_encoding,
+ sort_result_descending,
+ output_type)
+{
+ constructor_validate_and_infer_types();
+}
+
+shared_ptr<Node>
+ op::dynamic::NonMaxSuppression::clone_with_new_inputs(const OutputVector& new_args) const
+{
+ check_new_args_count(this, new_args);
+ NODE_VALIDATION_CHECK(this,
+ new_args.size() >= 2 && new_args.size() <= 5,
+ "Number of inputs must be 2, 3, 4 or 5");
+
+ const auto& arg2 = new_args.size() > 2
+ ? new_args.at(2)
+ : ngraph::op::Constant::create(element::i32, Shape{}, {0});
+ const auto& arg3 = new_args.size() > 3
+ ? new_args.at(3)
+ : ngraph::op::Constant::create(element::f32, Shape{}, {.0f});
+ const auto& arg4 = new_args.size() > 4
+ ? new_args.at(4)
+ : ngraph::op::Constant::create(element::f32, Shape{}, {.0f});
+
+ return std::make_shared<op::dynamic::NonMaxSuppression>(new_args.at(0),
+ new_args.at(1),
+ arg2,
+ arg3,
+ arg4,
+ m_box_encoding,
+ m_sort_result_descending,
+ m_output_type);
+}
+
+void op::dynamic::NonMaxSuppression::validate_and_infer_types()
{
op::v3::NonMaxSuppression::validate_and_infer_types();
BoxEncodingType m_box_encoding = BoxEncodingType::CORNER;
bool m_sort_result_descending = true;
ngraph::element::Type m_output_type = ngraph::element::i64;
-
- private:
+ void validate();
int64_t max_boxes_output_from_input() const;
};
} // namespace v3
clone_with_new_inputs(const OutputVector& new_args) const override;
};
} // namespace v4
+
+ namespace dynamic
+ {
+ /// \brief NonMaxSuppression operation
+ ///
+ class NGRAPH_API NonMaxSuppression : public op::v3::NonMaxSuppression
+ {
+ public:
+ static constexpr NodeTypeInfo type_info{"NonMaxSuppression", 99};
+ const NodeTypeInfo& get_type_info() const override { return type_info; }
+ NonMaxSuppression() = default;
+
+ /// \brief Constructs a NonMaxSuppression operation.
+ ///
+ /// \param boxes Node producing the box coordinates
+ /// \param scores Node producing the box scores
+ /// \param max_output_boxes_per_class Node producing maximum number of boxes to be
+ /// selected per class
+ /// \param iou_threshold Node producing intersection over union threshold
+ /// \param score_threshold Node producing minimum score threshold
+ /// \param box_encoding Specifies the format of boxes data encoding
+ /// \param sort_result_descending Specifies whether it is necessary to sort selected
+ /// boxes across batches
+ /// \param output_type Specifies the output tensor type
+ NonMaxSuppression(const Output<Node>& boxes,
+ const Output<Node>& scores,
+ const Output<Node>& max_output_boxes_per_class,
+ const Output<Node>& iou_threshold,
+ const Output<Node>& score_threshold,
+ const BoxEncodingType box_encoding = BoxEncodingType::CORNER,
+ const bool sort_result_descending = true,
+ const ngraph::element::Type& output_type = ngraph::element::i64);
+
+ /// \brief Constructs a NonMaxSuppression operation with default values for the last
+ /// 3 inputs
+ ///
+ /// \param boxes Node producing the box coordinates
+ /// \param scores Node producing the box coordinates
+ /// \param box_encoding Specifies the format of boxes data encoding
+ /// \param sort_result_descending Specifies whether it is necessary to sort selected
+ /// boxes across batches
+ /// \param output_type Specifies the output tensor type
+ NonMaxSuppression(const Output<Node>& boxes,
+ const Output<Node>& scores,
+ const BoxEncodingType box_encoding = BoxEncodingType::CORNER,
+ const bool sort_result_descending = true,
+ const ngraph::element::Type& output_type = ngraph::element::i64);
+
+ void validate_and_infer_types() override;
+
+ std::shared_ptr<Node>
+ clone_with_new_inputs(const OutputVector& new_args) const override;
+ };
+ } // namespace dynamic
} // namespace op
NGRAPH_API
return opset;
}
+const ngraph::OpSet& ngraph::get_opset4()
+{
+ static std::mutex init_mutex;
+ static bool opset_is_initialized = false;
+ static OpSet opset;
+ if (!opset_is_initialized)
+ {
+ std::lock_guard<std::mutex> guard(init_mutex);
+ if (!opset_is_initialized)
+ {
+#define NGRAPH_OP(NAME, NAMESPACE) opset.insert<NAMESPACE::NAME>();
+#include "ngraph/opsets/opset4_tbl.hpp"
+#undef NGRAPH_OP
+ opset_is_initialized = true;
+ }
+ }
+ return opset;
+}
+
const ngraph::OpSet& ngraph::get_ie_opset()
{
static std::mutex init_mutex;
#include "ngraph/opsets/opset1_tbl.hpp"
#include "ngraph/opsets/opset2_tbl.hpp"
#include "ngraph/opsets/opset3_tbl.hpp"
+#include "ngraph/opsets/opset4_tbl.hpp"
#undef NGRAPH_OP
opset_is_initialized = true;
}
const NGRAPH_API OpSet& get_opset1();
const NGRAPH_API OpSet& get_opset2();
const NGRAPH_API OpSet& get_opset3();
+ const NGRAPH_API OpSet& get_opset4();
// Every op after opset0
const NGRAPH_API OpSet& get_ie_opset();
}
--- /dev/null
+//*****************************************************************************
+// Copyright 2017-2020 Intel Corporation
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//*****************************************************************************
+
+#pragma once
+
+#include "ngraph/ops.hpp"
+
+namespace ngraph
+{
+ namespace opset4
+ {
+#define NGRAPH_OP(a, b) using b::a;
+#include "ngraph/opsets/opset4_tbl.hpp"
+#undef NGRAPH_OP
+ }
+}
--- /dev/null
+//*****************************************************************************
+// Copyright 2017-2020 Intel Corporation
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//*****************************************************************************
+
+#ifndef NGRAPH_OP
+#warning "NGRAPH_OP not defined"
+#define NGRAPH_OP(x, y)
+#endif
+
+NGRAPH_OP(Abs, ngraph::op::v0)
+NGRAPH_OP(Acos, ngraph::op::v0)
+NGRAPH_OP(Add, ngraph::op::v1)
+NGRAPH_OP(Asin, ngraph::op::v0)
+NGRAPH_OP(Atan, ngraph::op::v0)
+NGRAPH_OP(AvgPool, ngraph::op::v1)
+NGRAPH_OP(BatchNormInference, ngraph::op::v0)
+NGRAPH_OP(BinaryConvolution, ngraph::op::v1)
+NGRAPH_OP(Broadcast, ngraph::op::v3)
+NGRAPH_OP(Bucketize, ngraph::op::v3)
+NGRAPH_OP(CTCGreedyDecoder, ngraph::op::v0)
+NGRAPH_OP(Ceiling, ngraph::op::v0)
+NGRAPH_OP(Clamp, ngraph::op::v0)
+NGRAPH_OP(Concat, ngraph::op::v0)
+NGRAPH_OP(Constant, ngraph::op)
+NGRAPH_OP(Convert, ngraph::op::v0)
+NGRAPH_OP(ConvertLike, ngraph::op::v1)
+NGRAPH_OP(Convolution, ngraph::op::v1)
+NGRAPH_OP(ConvolutionBackpropData, ngraph::op::v1)
+NGRAPH_OP(Cos, ngraph::op::v0)
+NGRAPH_OP(Cosh, ngraph::op::v0)
+NGRAPH_OP(CumSum, ngraph::op::v0)
+NGRAPH_OP(DeformableConvolution, ngraph::op::v1)
+NGRAPH_OP(DeformablePSROIPooling, ngraph::op::v1)
+NGRAPH_OP(DepthToSpace, ngraph::op::v0)
+NGRAPH_OP(DetectionOutput, ngraph::op::v0)
+NGRAPH_OP(Divide, ngraph::op::v1)
+NGRAPH_OP(Elu, ngraph::op::v0)
+NGRAPH_OP(Erf, ngraph::op::v0)
+NGRAPH_OP(Equal, ngraph::op::v1)
+NGRAPH_OP(Exp, ngraph::op::v0)
+NGRAPH_OP(ExtractImagePatches, ngraph::op::v3)
+NGRAPH_OP(FakeQuantize, ngraph::op::v0)
+NGRAPH_OP(Floor, ngraph::op::v0)
+NGRAPH_OP(FloorMod, ngraph::op::v1)
+NGRAPH_OP(Gather, ngraph::op::v1)
+NGRAPH_OP(GatherTree, ngraph::op::v1)
+NGRAPH_OP(Greater, ngraph::op::v1)
+NGRAPH_OP(GreaterEqual, ngraph::op::v1)
+NGRAPH_OP(GroupConvolution, ngraph::op::v1)
+NGRAPH_OP(GroupConvolutionBackpropData, ngraph::op::v1)
+NGRAPH_OP(GRN, ngraph::op::v0)
+NGRAPH_OP(HardSigmoid, ngraph::op::v0)
+NGRAPH_OP(Interpolate, ngraph::op::v0)
+NGRAPH_OP(Less, ngraph::op::v1)
+NGRAPH_OP(LessEqual, ngraph::op::v1)
+NGRAPH_OP(Log, ngraph::op::v0)
+NGRAPH_OP(LogicalAnd, ngraph::op::v1)
+NGRAPH_OP(LogicalNot, ngraph::op::v1)
+NGRAPH_OP(LogicalOr, ngraph::op::v1)
+NGRAPH_OP(LogicalXor, ngraph::op::v1)
+NGRAPH_OP(LRN, ngraph::op::v0)
+NGRAPH_OP(LSTMCell, ngraph::op::v0)
+NGRAPH_OP(LSTMSequence, ngraph::op::v0)
+NGRAPH_OP(MatMul, ngraph::op::v0)
+NGRAPH_OP(MaxPool, ngraph::op::v1)
+NGRAPH_OP(Maximum, ngraph::op::v1)
+NGRAPH_OP(Minimum, ngraph::op::v1)
+NGRAPH_OP(Mod, ngraph::op::v1)
+NGRAPH_OP(Multiply, ngraph::op::v1)
+NGRAPH_OP(MVN, ngraph::op::v0)
+NGRAPH_OP(Negative, ngraph::op::v0)
+NGRAPH_OP(NormalizeL2, ngraph::op::v0)
+NGRAPH_OP(NotEqual, ngraph::op::v1)
+NGRAPH_OP(OneHot, ngraph::op::v1)
+NGRAPH_OP(PRelu, ngraph::op::v0)
+NGRAPH_OP(PSROIPooling, ngraph::op::v0)
+NGRAPH_OP(Pad, ngraph::op::v1)
+NGRAPH_OP(Parameter, ngraph::op::v0)
+NGRAPH_OP(Power, ngraph::op::v1)
+NGRAPH_OP(PriorBox, ngraph::op::v0)
+NGRAPH_OP(PriorBoxClustered, ngraph::op::v0)
+NGRAPH_OP(Proposal, ngraph::op::v0)
+NGRAPH_OP(Range, ngraph::op::v0)
+NGRAPH_OP(Relu, ngraph::op::v0)
+NGRAPH_OP(ReduceMax, ngraph::op::v1)
+NGRAPH_OP(ReduceLogicalAnd, ngraph::op::v1)
+NGRAPH_OP(ReduceLogicalOr, ngraph::op::v1)
+NGRAPH_OP(ReduceMean, ngraph::op::v1)
+NGRAPH_OP(ReduceMin, ngraph::op::v1)
+NGRAPH_OP(ReduceProd, ngraph::op::v1)
+NGRAPH_OP(ReduceSum, ngraph::op::v1)
+NGRAPH_OP(RegionYolo, ngraph::op::v0)
+NGRAPH_OP(ReorgYolo, ngraph::op::v0)
+NGRAPH_OP(Reshape, ngraph::op::v1)
+NGRAPH_OP(Result, ngraph::op::v0)
+NGRAPH_OP(ReverseSequence, ngraph::op::v0)
+NGRAPH_OP(ROIPooling, ngraph::op::v0)
+NGRAPH_OP(Select, ngraph::op::v1)
+NGRAPH_OP(Selu, ngraph::op::v0)
+NGRAPH_OP(Sign, ngraph::op::v0)
+NGRAPH_OP(Sigmoid, ngraph::op::v0)
+NGRAPH_OP(Sin, ngraph::op::v0)
+NGRAPH_OP(Sinh, ngraph::op::v0)
+NGRAPH_OP(Softmax, ngraph::op::v1)
+NGRAPH_OP(Sqrt, ngraph::op::v0)
+NGRAPH_OP(SpaceToDepth, ngraph::op::v0)
+NGRAPH_OP(Split, ngraph::op::v1)
+NGRAPH_OP(SquaredDifference, ngraph::op::v0)
+NGRAPH_OP(Squeeze, ngraph::op::v0)
+NGRAPH_OP(StridedSlice, ngraph::op::v1)
+NGRAPH_OP(Subtract, ngraph::op::v1)
+NGRAPH_OP(Tan, ngraph::op::v0)
+NGRAPH_OP(Tanh, ngraph::op::v0)
+NGRAPH_OP(TensorIterator, ngraph::op::v0)
+NGRAPH_OP(Tile, ngraph::op::v0)
+NGRAPH_OP(Transpose, ngraph::op::v1)
+NGRAPH_OP(Unsqueeze, ngraph::op::v0)
+NGRAPH_OP(VariadicSplit, ngraph::op::v1)
+
+// New operations added in opset2
+NGRAPH_OP(Gelu, ngraph::op::v0)
+NGRAPH_OP(BatchToSpace, ngraph::op::v1)
+NGRAPH_OP(SpaceToBatch, ngraph::op::v1)
+
+// New operations added in opset3
+NGRAPH_OP(EmbeddingBagPackedSum, ngraph::op::v3)
+NGRAPH_OP(EmbeddingSegmentsSum, ngraph::op::v3)
+NGRAPH_OP(EmbeddingBagOffsetsSum, ngraph::op::v3)
+NGRAPH_OP(GRUCell, ngraph::op::v3)
+NGRAPH_OP(NonZero, ngraph::op::v3)
+NGRAPH_OP(RNNCell, ngraph::op::v0)
+NGRAPH_OP(ROIAlign, ngraph::op::v3)
+NGRAPH_OP(ScatterElementsUpdate, ngraph::op::v3)
+NGRAPH_OP(ScatterUpdate, ngraph::op::v3)
+NGRAPH_OP(ShuffleChannels, ngraph::op::v0)
+NGRAPH_OP(ShapeOf, ngraph::op::v3)
+NGRAPH_OP(Assign, ngraph::op::v3)
+NGRAPH_OP(ReadValue, ngraph::op::v3)
+NGRAPH_OP(TopK, ngraph::op::v3)
+
+// New operations added in opset4
+NGRAPH_OP(NonMaxSuppression, ngraph::op::v4)
ASSERT_TRUE(
nms->get_output_partial_shape(0).same_scheme(PartialShape{Dimension::dynamic(), 3}));
}
+
+// ------------------------------ V4 ------------------------------
+
+TEST(type_prop, nms_v4_incorrect_boxes_rank)
+{
+ try
+ {
+ const auto boxes = make_shared<op::Parameter>(element::f32, Shape{1, 2, 3, 4});
+ const auto scores = make_shared<op::Parameter>(element::f32, Shape{1, 2, 3});
+
+ make_shared<op::v4::NonMaxSuppression>(boxes, scores);
+ }
+ catch (const NodeValidationFailure& error)
+ {
+ EXPECT_HAS_SUBSTRING(error.what(), "Expected a 3D tensor for the 'boxes' input");
+ }
+}
+
+TEST(type_prop, nms_v4_incorrect_scores_rank)
+{
+ try
+ {
+ const auto boxes = make_shared<op::Parameter>(element::f32, Shape{1, 2, 3});
+ const auto scores = make_shared<op::Parameter>(element::f32, Shape{1, 2});
+
+ make_shared<op::v4::NonMaxSuppression>(boxes, scores);
+ }
+ catch (const NodeValidationFailure& error)
+ {
+ EXPECT_HAS_SUBSTRING(error.what(), "Expected a 3D tensor for the 'scores' input");
+ }
+}
+
+TEST(type_prop, nms_v4_incorrect_scheme_num_batches)
+{
+ try
+ {
+ const auto boxes = make_shared<op::Parameter>(element::f32, Shape{1, 2, 3});
+ const auto scores = make_shared<op::Parameter>(element::f32, Shape{2, 2, 3});
+
+ make_shared<op::v4::NonMaxSuppression>(boxes, scores);
+ }
+ catch (const NodeValidationFailure& error)
+ {
+ EXPECT_HAS_SUBSTRING(error.what(),
+ "The first dimension of both 'boxes' and 'scores' must match");
+ }
+}
+
+TEST(type_prop, nms_v4_incorrect_scheme_num_boxes)
+{
+ try
+ {
+ const auto boxes = make_shared<op::Parameter>(element::f32, Shape{1, 2, 3});
+ const auto scores = make_shared<op::Parameter>(element::f32, Shape{1, 2, 3});
+
+ make_shared<op::v4::NonMaxSuppression>(boxes, scores);
+ }
+ catch (const NodeValidationFailure& error)
+ {
+ EXPECT_HAS_SUBSTRING(error.what(),
+ "'boxes' and 'scores' input shapes must match at the second and third "
+ "dimension respectively");
+ }
+}
+
+TEST(type_prop, nms_v4_scalar_inputs_check)
+{
+ const auto boxes = make_shared<op::Parameter>(element::f32, Shape{1, 2, 4});
+ const auto scores = make_shared<op::Parameter>(element::f32, Shape{1, 2, 2});
+
+ const auto scalar = make_shared<op::Parameter>(element::f32, Shape{});
+ const auto non_scalar = make_shared<op::Parameter>(element::f32, Shape{1});
+
+ try
+ {
+ make_shared<op::v4::NonMaxSuppression>(boxes, scores, non_scalar, scalar, scalar);
+ }
+ catch (const NodeValidationFailure& error)
+ {
+ EXPECT_HAS_SUBSTRING(error.what(),
+ "Expected a scalar for the 'max_output_boxes_per_class' input");
+ }
+
+ try
+ {
+ make_shared<op::v4::NonMaxSuppression>(boxes, scores, scalar, non_scalar, scalar);
+ }
+ catch (const NodeValidationFailure& error)
+ {
+ EXPECT_HAS_SUBSTRING(error.what(), "Expected a scalar for the 'iou_threshold' input");
+ }
+
+ try
+ {
+ make_shared<op::v4::NonMaxSuppression>(boxes, scores, scalar, scalar, non_scalar);
+ }
+ catch (const NodeValidationFailure& error)
+ {
+ EXPECT_HAS_SUBSTRING(error.what(), "Expected a scalar for the 'score_threshold' input");
+ }
+}
+
+TEST(type_prop, nms_v4_output_shape)
+{
+ const auto boxes = make_shared<op::Parameter>(element::f32, Shape{5, 2, 4});
+ const auto scores = make_shared<op::Parameter>(element::f32, Shape{5, 3, 2});
+
+ const auto nms = make_shared<op::v4::NonMaxSuppression>(boxes, scores);
+ const auto nms_out_ps = nms->get_output_partial_shape(0);
+
+ EXPECT_TRUE(nms_out_ps.rank().is_static());
+ EXPECT_EQ(nms_out_ps.rank().get_length(), 2);
+ EXPECT_EQ(nms->get_shape(), (Shape{0, 3}));
+}
+
+TEST(type_prop, nms_v4_output_shape_2)
+{
+ const auto boxes = make_shared<op::Parameter>(element::f32, Shape{2, 7, 4});
+ const auto scores = make_shared<op::Parameter>(element::f32, Shape{2, 5, 7});
+ const auto max_output_boxes_per_class = op::Constant::create(element::i32, Shape{}, {3});
+ const auto iou_threshold = make_shared<op::Parameter>(element::f32, Shape{});
+ const auto score_threshold = make_shared<op::Parameter>(element::f32, Shape{});
+
+ const auto nms = make_shared<op::v4::NonMaxSuppression>(
+ boxes, scores, max_output_boxes_per_class, iou_threshold, score_threshold);
+
+ ASSERT_EQ(nms->get_element_type(), element::i64);
+ ASSERT_EQ(nms->get_shape(), (Shape{2 * 5 * 3, 3}));
+}
+
+TEST(type_prop, nms_v4_output_shape_3)
+{
+ const auto boxes = make_shared<op::Parameter>(element::f32, Shape{2, 7, 4});
+ const auto scores = make_shared<op::Parameter>(element::f32, Shape{2, 5, 7});
+ const auto max_output_boxes_per_class = op::Constant::create(element::i16, Shape{}, {1000});
+ const auto iou_threshold = make_shared<op::Parameter>(element::f32, Shape{});
+ const auto score_threshold = make_shared<op::Parameter>(element::f32, Shape{});
+
+ const auto nms = make_shared<op::v4::NonMaxSuppression>(
+ boxes, scores, max_output_boxes_per_class, iou_threshold, score_threshold);
+
+ ASSERT_EQ(nms->get_element_type(), element::i64);
+ ASSERT_EQ(nms->get_shape(), (Shape{2 * 5 * 7, 3}));
+}
+
+TEST(type_prop, nms_v4_output_shape_i32)
+{
+ const auto boxes = make_shared<op::Parameter>(element::f32, Shape{2, 7, 4});
+ const auto scores = make_shared<op::Parameter>(element::f32, Shape{2, 5, 7});
+ const auto max_output_boxes_per_class = op::Constant::create(element::i16, Shape{}, {3});
+ const auto iou_threshold = make_shared<op::Parameter>(element::f32, Shape{});
+ const auto score_threshold = make_shared<op::Parameter>(element::f32, Shape{});
+
+ const auto nms =
+ make_shared<op::v4::NonMaxSuppression>(boxes,
+ scores,
+ max_output_boxes_per_class,
+ iou_threshold,
+ score_threshold,
+ op::v3::NonMaxSuppression::BoxEncodingType::CORNER,
+ true,
+ element::i32);
+
+ ASSERT_EQ(nms->get_element_type(), element::i32);
+ ASSERT_EQ(nms->get_shape(), (Shape{30, 3}));
+}
+
+TEST(type_prop, nms_v4_dynamic_boxes_and_scores)
+{
+ const auto boxes = make_shared<op::Parameter>(element::f32, PartialShape::dynamic());
+ const auto scores = make_shared<op::Parameter>(element::f32, PartialShape::dynamic());
+ const auto max_output_boxes_per_class = op::Constant::create(element::i16, Shape{}, {3});
+ const auto iou_threshold = make_shared<op::Parameter>(element::f32, Shape{});
+ const auto score_threshold = make_shared<op::Parameter>(element::f32, Shape{});
+
+ const auto nms = make_shared<op::v4::NonMaxSuppression>(
+ boxes, scores, max_output_boxes_per_class, iou_threshold, score_threshold);
+
+ ASSERT_EQ(nms->get_element_type(), element::i64);
+ ASSERT_TRUE(
+ nms->get_output_partial_shape(0).same_scheme(PartialShape{Dimension::dynamic(), 3}));
+}
projects / platform / upstream / dldt.git / commitdiff