From c28a48777779dc8db52ca69e8bb640de64d64643 Mon Sep 17 00:00:00 2001 From: Alexander Efimov/AI Tools Lab /SRR/Engineer/Samsung Electronics Date: Thu, 21 Nov 2019 23:13:13 +0300 Subject: [PATCH] [docs] Added Unit Test Report (#9080) Added UTR for DR2 Signed-off-by: Alexander Efimov --- docs/project/19_NN_Compiler_and_Runtime_UTR.rst | 370 +++++++++++++++++++++ .../19_NN_Compiler_and_Runtime_UTR.rst.cover | 7 + docs/project/Makefile | 2 +- 3 files changed, 378 insertions(+), 1 deletion(-) create mode 100644 docs/project/19_NN_Compiler_and_Runtime_UTR.rst create mode 100644 docs/project/19_NN_Compiler_and_Runtime_UTR.rst.cover diff --git a/docs/project/19_NN_Compiler_and_Runtime_UTR.rst b/docs/project/19_NN_Compiler_and_Runtime_UTR.rst new file mode 100644 index 0000000..c3124fa --- /dev/null +++ b/docs/project/19_NN_Compiler_and_Runtime_UTR.rst @@ -0,0 +1,370 @@ +=================== +SW Unit Test Report +=================== + +.. contents:: + +.. section-numbering:: + +.. raw:: pdf + + PageBreak oneColumn + + +**Revision history** + ++-------+-------------+-----------------------------+--------------------------+---------------------+ +| Ver. | Date | Contents | Author | Approver | ++=======+=============+=============================+==========================+=====================+ +| 1.0 | 2019.11.2? | Initial version | Alexander Efimov | Sung-Jae Lee | ++-------+-------------+-----------------------------+--------------------------+---------------------+ + +Test Status +=========== + +Summary +------- + +NNAS Version 1.0 (11.2019) +`````````````````````````` + +.. list-table:: + :widths: 4 4 3 3 2 2 3 5 + :header-rows: 1 + + * - Start date + - End date + - Total + - Pass + - Fail + - N/A + - Pass Rate + - Characteristic of TC + + * - 11/??/2019 + - 11/??/2019 + - 75 + - 75 + - 0 + - 0 + - 100% + - tests for Model IR based onnx frontend, mir to loco IR converter + + +Details +------- + +NNAS Version 1.0 (11.2019) +`````````````````````````` + +These unit tests are from commit 5ae3371d478a0cc3267f8202bc42affccf0a1696. + +Tests descritpion +''''''''''''''''' + +Unit tests could be found in: + +- compiler/nnc/unittests +- compiler/mir/unittests +- compiler/mir2loco/src + + +.. list-table:: + :widths: 5 3 6 + :header-rows: 1 + + * - Test Suite Name + - Number of Tests + - Tests Scope + + * - tests_for_optimizations + - 10 + - Verify correctness of high level optimizations: (transpose elimiation, arithmetic optimizations) + + * - nnc_transformations_test + - 5 + - Verify correctness of graph transformations: (operation layout switching) + + * - mir_test + - 27 + - Verify graph interfaces, shape inference and other IR helper functions + + * - mir2loco_test + - 15 + - verify correctness of Model IR to Canonical IR converter + + * - test_neurun + - 18 + - verify correctness of Scheduler and Executors in NN Runtime + + +Tests output +'''''''''''' + +**tests_for_optimizations** + +:: + + [==========] Running 10 tests from 1 test case. + [----------] Global test environment set-up. + [----------] 10 tests from OptPass + [ RUN ] OptPass.eliminateTransposesLinear + [ OK ] OptPass.eliminateTransposesLinear (0 ms) + [ RUN ] OptPass.combineTransposesLinear + [ OK ] OptPass.combineTransposesLinear (0 ms) + [ RUN ] OptPass.combineTransposesBush + [ OK ] OptPass.combineTransposesBush (0 ms) + [ RUN ] OptPass.combineTransposesOpOrder + [ OK ] OptPass.combineTransposesOpOrder (0 ms) + [ RUN ] OptPass.sinkTrReLU + [ OK ] OptPass.sinkTrReLU (1 ms) + [ RUN ] OptPass.sinkTrConcat + [ OK ] OptPass.sinkTrConcat (0 ms) + [ RUN ] OptPass.sinkReluConcat + [ OK ] OptPass.sinkReluConcat (0 ms) + [ RUN ] OptPass.sinkPoolReLU + [ OK ] OptPass.sinkPoolReLU (0 ms) + [ RUN ] OptPass.removeDeadEndConstants + [ OK ] OptPass.removeDeadEndConstants (0 ms) + [ RUN ] OptPass.fuseConvBiasScaleScaleBias + [ OK ] OptPass.fuseConvBiasScaleScaleBias (2 ms) + [----------] 10 tests from OptPass (3 ms total) + + [----------] Global test environment tear-down + [==========] 10 tests from 1 test case ran. (4 ms total) + [ PASSED ] 10 tests. + +**nnc_transformations_test** + +:: + + [==========] Running 5 tests from 1 test case. + [----------] Global test environment set-up. + [----------] 5 tests from TRANSFORMATIONS + [ RUN ] TRANSFORMATIONS.Switcher_Conv2D_NCHW2NHWC + [ OK ] TRANSFORMATIONS.Switcher_Conv2D_NCHW2NHWC (0 ms) + [ RUN ] TRANSFORMATIONS.Switcher_DWConv2D_NHWC2NCHW + [ OK ] TRANSFORMATIONS.Switcher_DWConv2D_NHWC2NCHW (1 ms) + [ RUN ] TRANSFORMATIONS.Switcher_DeConv2D_NHWC2NCHW + [ OK ] TRANSFORMATIONS.Switcher_DeConv2D_NHWC2NCHW (0 ms) + [ RUN ] TRANSFORMATIONS.Switcher_AvgPool2D_NHWC2NCHW + [ OK ] TRANSFORMATIONS.Switcher_AvgPool2D_NHWC2NCHW (0 ms) + [ RUN ] TRANSFORMATIONS.Switcher_MaxPool2D_NCHW2NHWC + [ OK ] TRANSFORMATIONS.Switcher_MaxPool2D_NCHW2NHWC (0 ms) + [----------] 5 tests from TRANSFORMATIONS (1 ms total) + + [----------] Global test environment tear-down + [==========] 5 tests from 1 test case ran. (1 ms total) + [ PASSED ] 5 tests. + +**mir_test** + +:: + + [==========] Running 27 tests from 8 test cases. + [----------] Global test environment set-up. + [----------] 5 tests from Operation + [ RUN ] Operation.ConnectionTest + [ OK ] Operation.ConnectionTest (0 ms) + [ RUN ] Operation.InputOutputShapeTest + [ OK ] Operation.InputOutputShapeTest (0 ms) + [ RUN ] Operation.SoftmaxAxisTest + [ OK ] Operation.SoftmaxAxisTest (0 ms) + [ RUN ] Operation.ConcatAxisTest + [ OK ] Operation.ConcatAxisTest (0 ms) + [ RUN ] Operation.OpNameTest + [ OK ] Operation.OpNameTest (0 ms) + [----------] 5 tests from Operation (0 ms total) + + [----------] 1 test from Shape + [ RUN ] Shape.Base + [ OK ] Shape.Base (0 ms) + [----------] 1 test from Shape (0 ms total) + + [----------] 1 test from Index + [ RUN ] Index.Base + [ OK ] Index.Base (0 ms) + [----------] 1 test from Index (0 ms total) + + [----------] 12 tests from ShapeInferenceTest + [ RUN ] ShapeInferenceTest.BidirectionalBroadcast + [ OK ] ShapeInferenceTest.BidirectionalBroadcast (0 ms) + [ RUN ] ShapeInferenceTest.ReshapeAutoDimension + [ OK ] ShapeInferenceTest.ReshapeAutoDimension (0 ms) + [ RUN ] ShapeInferenceTest.ResizeWithShape + [ OK ] ShapeInferenceTest.ResizeWithShape (0 ms) + [ RUN ] ShapeInferenceTest.ResizeWithScale + [ OK ] ShapeInferenceTest.ResizeWithScale (0 ms) + [ RUN ] ShapeInferenceTest.ReduceChangeRank + [ OK ] ShapeInferenceTest.ReduceChangeRank (0 ms) + [ RUN ] ShapeInferenceTest.ReshapeAutoDimensionShrink + [ OK ] ShapeInferenceTest.ReshapeAutoDimensionShrink (0 ms) + [ RUN ] ShapeInferenceTest.ReshapeAutoDimensionExpand + [ OK ] ShapeInferenceTest.ReshapeAutoDimensionExpand (0 ms) + [ RUN ] ShapeInferenceTest.ReshapeAutoDimensionUnsqueeze + [ OK ] ShapeInferenceTest.ReshapeAutoDimensionUnsqueeze (0 ms) + [ RUN ] ShapeInferenceTest.SqueezeTestAllDims + [ OK ] ShapeInferenceTest.SqueezeTestAllDims (0 ms) + [ RUN ] ShapeInferenceTest.ElementwiseBC + [ OK ] ShapeInferenceTest.ElementwiseBC (0 ms) + [ RUN ] ShapeInferenceTest.SqueezeTestSpecificDims + [ OK ] ShapeInferenceTest.SqueezeTestSpecificDims (0 ms) + [ RUN ] ShapeInferenceTest.SqueezeTestScalarResult + [ OK ] ShapeInferenceTest.SqueezeTestScalarResult (0 ms) + [----------] 12 tests from ShapeInferenceTest (0 ms total) + + [----------] 1 test from ShapeRange + [ RUN ] ShapeRange.Contains + [ OK ] ShapeRange.Contains (0 ms) + [----------] 1 test from ShapeRange (0 ms total) + + [----------] 2 tests from TensorVariant + [ RUN ] TensorVariant.BasicTest + [ OK ] TensorVariant.BasicTest (0 ms) + [ RUN ] TensorVariant.ElementSizeDeductionTest + [ OK ] TensorVariant.ElementSizeDeductionTest (0 ms) + [----------] 2 tests from TensorVariant (0 ms total) + + [----------] 1 test from NodeMutatorTest + [ RUN ] NodeMutatorTest.SimpleChainTest + [ OK ] NodeMutatorTest.SimpleChainTest (0 ms) + [----------] 1 test from NodeMutatorTest (0 ms total) + + [----------] 4 tests from SimpleInput/ShapeIteratorTest + [ RUN ] SimpleInput/ShapeIteratorTest.ElementCount/0 + [ OK ] SimpleInput/ShapeIteratorTest.ElementCount/0 (0 ms) + [ RUN ] SimpleInput/ShapeIteratorTest.ElementCount/1 + [ OK ] SimpleInput/ShapeIteratorTest.ElementCount/1 (0 ms) + [ RUN ] SimpleInput/ShapeIteratorTest.ElementCount/2 + [ OK ] SimpleInput/ShapeIteratorTest.ElementCount/2 (0 ms) + [ RUN ] SimpleInput/ShapeIteratorTest.ElementCount/3 + [ OK ] SimpleInput/ShapeIteratorTest.ElementCount/3 (0 ms) + [----------] 4 tests from SimpleInput/ShapeIteratorTest (1 ms total) + + [----------] Global test environment tear-down + [==========] 27 tests from 8 test cases ran. (1 ms total) + [ PASSED ] 27 tests. + +**mir2loco_test** + +:: + + [==========] Running 15 tests from 1 test case. + [----------] Global test environment set-up. + [----------] 15 tests from TestTransformer_mir2loco + [ RUN ] TestTransformer_mir2loco.Input_Output_Test + [ OK ] TestTransformer_mir2loco.Input_Output_Test (0 ms) + [ RUN ] TestTransformer_mir2loco.Relu_Test + [ OK ] TestTransformer_mir2loco.Relu_Test (0 ms) + [ RUN ] TestTransformer_mir2loco.Avg_Pool_Test + [ OK ] TestTransformer_mir2loco.Avg_Pool_Test (1 ms) + [ RUN ] TestTransformer_mir2loco.Max_Pool_Test + [ OK ] TestTransformer_mir2loco.Max_Pool_Test (0 ms) + [ RUN ] TestTransformer_mir2loco.Concat_Test + [ OK ] TestTransformer_mir2loco.Concat_Test (0 ms) + [ RUN ] TestTransformer_mir2loco.Reshape_Test + [ OK ] TestTransformer_mir2loco.Reshape_Test (0 ms) + [ RUN ] TestTransformer_mir2loco.Const_Float_Test + [ OK ] TestTransformer_mir2loco.Const_Float_Test (0 ms) + [ RUN ] TestTransformer_mir2loco.Add_Test + [ OK ] TestTransformer_mir2loco.Add_Test (0 ms) + [ RUN ] TestTransformer_mir2loco.Conv2D_Test + [ OK ] TestTransformer_mir2loco.Conv2D_Test (1 ms) + [ RUN ] TestTransformer_mir2loco.Softmax_Test + [ OK ] TestTransformer_mir2loco.Softmax_Test (0 ms) + [ RUN ] TestTransformer_mir2loco.Mul_Test + [ OK ] TestTransformer_mir2loco.Mul_Test (0 ms) + [ RUN ] TestTransformer_mir2loco.DepthwiseConv2D_Test + [ OK ] TestTransformer_mir2loco.DepthwiseConv2D_Test (0 ms) + [ RUN ] TestTransformer_mir2loco.DeConv2D_Test + [ OK ] TestTransformer_mir2loco.DeConv2D_Test (0 ms) + [ RUN ] TestTransformer_mir2loco.FullyConnected_Test + [ OK ] TestTransformer_mir2loco.FullyConnected_Test (1 ms) + [ RUN ] TestTransformer_mir2loco.Transpose_Test + [ OK ] TestTransformer_mir2loco.Transpose_Test (0 ms) + [----------] 15 tests from TestTransformer_mir2loco (3 ms total) + + [----------] Global test environment tear-down + [==========] 15 tests from 1 test case ran. (3 ms total) + [ PASSED ] 15 tests. + +**test_neurun --gtest_filter=ShapeInference.*:AllExecutors*:SchedulerTest*:ExecTime*** + +:: + + [==========] Running 18 tests from 4 test cases. + [----------] Global test environment set-up. + [----------] 2 tests from ExecTime + [ RUN ] ExecTime.roundtrip_ok + [ OK ] ExecTime.roundtrip_ok (4 ms) + [ RUN ] ExecTime.structure + [ OK ] ExecTime.structure (2 ms) + [----------] 2 tests from ExecTime (6 ms total) + + [----------] 1 test from SchedulerTest + [ RUN ] SchedulerTest.branched_graph_profiling_mode + [ OK ] SchedulerTest.branched_graph_profiling_mode (7 ms) + [----------] 1 test from SchedulerTest (7 ms total) + + [----------] 9 tests from ShapeInference + [ RUN ] ShapeInference.Elementwise + [ OK ] ShapeInference.Elementwise (0 ms) + [ RUN ] ShapeInference.IncorrectElementwise + [ OK ] ShapeInference.IncorrectElementwise (0 ms) + [ RUN ] ShapeInference.Pool2DNodeSame + [ OK ] ShapeInference.Pool2DNodeSame (0 ms) + [ RUN ] ShapeInference.Pool2DNodeValid + [ OK ] ShapeInference.Pool2DNodeValid (0 ms) + [ RUN ] ShapeInference.Pool2DNodeExplicit + [ OK ] ShapeInference.Pool2DNodeExplicit (0 ms) + [ RUN ] ShapeInference.Conv2D + [ OK ] ShapeInference.Conv2D (0 ms) + [ RUN ] ShapeInference.DepthwiseConv2D + [ OK ] ShapeInference.DepthwiseConv2D (0 ms) + [ RUN ] ShapeInference.Concat + [ OK ] ShapeInference.Concat (0 ms) + [ RUN ] ShapeInference.FullyConnected + [ OK ] ShapeInference.FullyConnected (0 ms) + [----------] 9 tests from ShapeInference (2 ms total) + + [----------] 6 tests from AllExecutors/SchedulerTestWithExecutorParam + [ RUN ] AllExecutors/SchedulerTestWithExecutorParam. + straight_graph_known_exec_time/0 + [ OK ] AllExecutors/SchedulerTestWithExecutorParam. + straight_graph_known_exec_time/0 (9 ms) + [ RUN ] AllExecutors/SchedulerTestWithExecutorParam. + straight_graph_known_exec_time/1 + [ OK ] AllExecutors/SchedulerTestWithExecutorParam. + straight_graph_known_exec_time/1 (8 ms) + [ RUN ] AllExecutors/SchedulerTestWithExecutorParam. + straight_graph_known_exec_time/2 + [ OK ] AllExecutors/SchedulerTestWithExecutorParam. + straight_graph_known_exec_time/2 (9 ms) + [ RUN ] AllExecutors/SchedulerTestWithExecutorParam. + branched_graph_known_exec_time/0 + [ OK ] AllExecutors/SchedulerTestWithExecutorParam. + branched_graph_known_exec_time/0 (11 ms) + [ RUN ] AllExecutors/SchedulerTestWithExecutorParam. + branched_graph_known_exec_time/1 + [ OK ] AllExecutors/SchedulerTestWithExecutorParam. + branched_graph_known_exec_time/1 (9 ms) + [ RUN ] AllExecutors/SchedulerTestWithExecutorParam. + branched_graph_known_exec_time/2 + [ OK ] AllExecutors/SchedulerTestWithExecutorParam. + branched_graph_known_exec_time/2 (10 ms) + [----------] 6 tests from AllExecutors/ + SchedulerTestWithExecutorParam (56 ms total) + + [----------] Global test environment tear-down + [==========] 18 tests from 4 test cases ran. (71 ms total) + [ PASSED ] 18 tests. + + +Defect Status +------------- + +Summary +``````` + +No defects are detected + diff --git a/docs/project/19_NN_Compiler_and_Runtime_UTR.rst.cover b/docs/project/19_NN_Compiler_and_Runtime_UTR.rst.cover new file mode 100644 index 0000000..173d502 --- /dev/null +++ b/docs/project/19_NN_Compiler_and_Runtime_UTR.rst.cover @@ -0,0 +1,7 @@ +=================== +SW Unit Test Report +=================== + +------------------------- +SRR-RAJ0119ZZ-BERF-UTR001 +------------------------- diff --git a/docs/project/Makefile b/docs/project/Makefile index 87efddb..5848930 100644 --- a/docs/project/Makefile +++ b/docs/project/Makefile @@ -1,4 +1,4 @@ -DOC_SRC = 19_NN_Compiler_and_Runtime_SRS.rst 19_NN_Compiler_and_Runtime_SDD.rst 19_NN_Compiler_and_Runtime_STD.rst +DOC_SRC = 19_NN_Compiler_and_Runtime_SRS.rst 19_NN_Compiler_and_Runtime_SDD.rst 19_NN_Compiler_and_Runtime_STD.rst 19_NN_Compiler_and_Runtime_UTR.rst DOC_PDF = $(DOC_SRC:.rst=.pdf) -- 2.7.4