| 0.1 | 2018.04.12 | Initial version | Vostokov Sergey | Sung-Jae Lee |
| 0.2 | 2018.04.13 | SE member in-charge review | Aleksei Kondrashov | |
| 1.0 | 2018.04.17 | Final Execution DR version | Vostokov Sergey | Sung-Jae Lee |
+| 1.1 | 2018.06.20 | DR1 version | Vostokov Sergey | Sung-Jae Lee |
**Terminology and Abbreviation**
| GUI | The graphical user interface |
| CLI | The command-line interface |
+**References**
+
+\[1\] Vostokov Sergey, [SW Requirements Specification](requirements_specification.md)
+
## SW System Test Overview
### Purpose
customers. Purpose of testing is to detect software failures so that
defects may be discovered and corrected.
-Software system test are processes and methods used to ensure quality.
-Additional goals is to make sure the product follows regulations and
-meets the quality standards expected by the customer.
+Software system test procedure is a collection of processes and methods
+used to ensure quality. An additional goal is to make sure that the
+product follows regulations and meets the quality standards expected by
+the customer.
### Scope
bugs:
- Compiler crashes (also known as an ICE or Internal Compiler Error)
+
- Compiler hangs (kind of infinite loop in the compiler)
+
- Bad code generation (a result of incorrect compiler output):
- * Bad code generation that leads to a crash in the application
- * “Silent” bad code generation
+
+ - Bad code generation that leads to a crash in the application
+ - “Silent” bad code generation
+
- Compiler throughput issues (Issues that affect the amount of time
the compiler takes to compile code )
+
- Code quality issues (Issues that affect the performance of the
compiled application)
+
- Compiler feature correctness issues (This class of bugs involves the
compiler generating correct code, but not doing what a particular
feature specifies should be
### Functions to be tested
-| Feature | Test Item ID | Test Item description |
-| ---------------------------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
-| RF-1, RIF-3 - RIF-7 | TST-1 | Test suite checks import of NN ops of Tensorflow Lite format: check loading NN model that consists of 1 NN ops. One test for 1 NN ops. |
-| RF-2, RIF-3 - RIF-7 | TST-2 | Test suite checks import of NN ops of Caffe format: check loading NN model that consists of 1 NN ops. One test for 1 NN ops. |
-| RF-3, RIF-3 - RIF-7 | TST-3 | Test suite checks import of NN ops of Caffe2 format: check loading NN model that consists of 1 NN ops. One test for 1 NN ops. |
-| RF-5, RIF-3 - RIF-7 | TST-4 | The test should verify successful loading the Inception V3 NN model |
-| RF-6, RIF-3 - RIF-7 | TST-5 | The test should verify successful loading the MobileNet NN model |
-| RF-4 | TST-6 | The test suite should automatically verify the completeness of information which was read from the raw data using comparison them with serialized from Model IR raw data |
-| RF-7, RF-19, RIF-13 | TST-7 | The unit test should automatically verify successful execution of binary on target ARM CPU |
-| RF-8, RF-18, RIF-14, RIF-15 | TST-8 | The unit test should automatically verify successful execution of calculation on GPU |
-| RF-10, RNF-1, RIF-17, RIF-18 | TST-9 | Unit test should verify the existence and format of binary (shared or static) in according to specified options. |
-| RF-11 | TST-10 | Unit test should verify that compiler produce the compiled artefact for the Inception V3 NN model (Validity of compiled artefact is checked by other tests) |
-| RF-12 | TST-11 | Unit test should verify that compiler produce the compiled artefact for the MobileNet NN model (Validity of compiled artefact is checked by other tests) |
-| RF-13, RF-14, RF-15, RNF-6, RNF-7, RNF-8 | TST-12 | The test suite should verify correctness of configuration object by unit testing |
-| RF-16, RNF-1 | TST-13 | The test suite is to verify the correctness of calculations by comparison the result of original NN model and result of compiled artefact on the same input date |
-| RF-17 | TST-14 | Unit test should verify that the incorrect input data are processing with the error message without unexpected application termination |
-| RNF-4, RNF-5, RIF-8 | TST-15 | The Linux-based OS should be used while the test environment are built. |
-| RIF-16 | TST-16 | The unit test should verify the existence and validity of generated C/C++ header for compiled artefact |
+| Feature | Test Item ID | Test Item description |
+| ---------------------------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| RF-1, RIF-3 - RIF-7 | TST-1 | Test suite checks NN ops import from Tensorflow Lite format by loading NN model that consists of a single NN op. One test for each NN op. |
+| RF-2, RIF-3 - RIF-7 | TST-2 | Test suite checks NN ops import from Caffe format by loading NN model that consists of a single NN op. One test for each NN op. |
+| RF-3, RIF-3 - RIF-7 | TST-3 | Test suite checks NN ops import from Caffe2 format by loading NN model that consists of a single NN op. One test for each NN op. |
+| RF-5, RIF-3 - RIF-7 | TST-4 | The test should verify successful loading the Inception V3 NN model |
+| RF-6, RIF-3 - RIF-7 | TST-5 | The test should verify successful loading the MobileNet NN model |
+| RF-4 | TST-6 | The test suite should automatically verify the completeness of information that was read from the raw data by comparing it with serialized raw data from Model IR |
+| RF-7, RF-18, RIF-13 | TST-7 | The unit test should automatically verify successful execution of binary on target ARM CPU |
+| RF-8, RF-17, RIF-14, RIF-15 | TST-8 | The unit test should automatically verify successful execution of calculation on GPU |
+| RF-9, RNF-1, RIF-17, RIF-18 | TST-9 | Unit test should verify the existence and format of binary (shared or static) in accordance to specified options |
+| RF-10 | TST-10 | Unit test should verify that compiler produces a compiled artefact for the Inception V3 NN model (Validity of compiled artefact is checked by other tests) |
+| RF-11 | TST-11 | Unit test should verify that compiler produces a compiled artefact for the MobileNet NN model (Validity of compiled artefact is checked by other tests) |
+| RF-12, RF-13, RF-14, RNF-6, RNF-7, RNF-8 | TST-12 | The test suite should verify correctness of configuration object by unit testing |
+| RF-15, RNF-1 | TST-13 | The test suite is to verify the correctness of calculations by comparing the result of original NN model and the result of compiled artefact on the same input data |
+| RF-16 | TST-14 | Unit test should verify that the incorrect input data is processed with an error message without unexpected termination of the application |
+| RNF-4, RNF-5, RIF-8 | TST-15 | A Linux-based OS should be used while the test environment are built. |
+| RIF-16 | TST-16 | The unit test should verify the existence and validity of generated C/C++ header for compiled artefact |
Table 2-1. Test Item
#### Directory Structure
-| Directory | Description |
-| ------------------------ | ------------------------------------------------------------------ |
-| / | source codes of the build system, main README file |
-| /contrib | Incubating projects |
-| /doc | Contains the documentation of the project |
-| /libs | Contains the source of the libraries which are used by the nncc |
-| /libs/core | Contains the source code of the core library of nncc |
-| /libs/frontend | Contains the source code of supported frontend's plugins |
-| /libs/frontend/caffe | The source code for the Caffe frontend |
-| /libs/frontend/tflite | The source code for the Tensorflow Lite frontend |
-| /libs/backend | Contains the source code of supported backend’ plugins |
-| /libs/backend/cpu | Contains the source code of CPU backend |
-| /libs/backend/gpu | Contains the source code of GPU backend |
-| /libs/backend/3rd\_party | Contains the source code of backend to utilize 3rd party libraries |
-| /scripts | Various scripts for building and testing the nncc |
-| /tools | The source code of the executables |
+| Directory | Description |
+| ------------------------ | -------------------------------------------------------------------- |
+| / | source codes of the build system, main README file |
+| /contrib | Incubating projects |
+| /doc | Contains the documentation of the project |
+| /doc/project | Contains project management documents (SRS, SDD, STD, HLD, DLD, etc) |
+| /libs | Contains the source of the libraries which are used by the nncc |
+| /libs/core | Contains the source code of the core library of nncc |
+| /libs/frontend | Contains the source code of supported frontend's plugins |
+| /libs/frontend/caffe | The source code for the Caffe frontend |
+| /libs/frontend/caffe2 | The source code for the Caffe2 frontend |
+| /libs/frontend/tflite | The source code for the Tensorflow Lite frontend |
+| /libs/backend | Contains the source code of supported backend plugins |
+| /libs/backend/cpu | Contains the source code of CPU backend |
+| /libs/backend/gpu | Contains the source code of GPU backend |
+| /libs/backend/3rd\_party | Contains the source code of backend to utilize 3rd party libraries |
+| /scripts | Various scripts for building and testing the nncc |
+| /tools | The source code of the executables |
Table 4-7. Directory Structure
| Round 2 | The nncc v1.0 | SRR-RAJ0118ZZ-BWRF-STD002, SRR-RAJ0118ZZ-BWRF-UTR002 | 01.12.2018 |
Table 4-8. Baselines
+
+## SW System Test Case
+
+| TestItem ID | Testcase ID | Test Procedures | Expected Results |
+| ----------- | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| TST-1 | TST-1-1 | Import a NN consisting of a single Tensorflow Lite ADD operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-2 | Import a NN consisting of a single Tensorflow Lite AVERAGE\_POOL\_2D operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-3 | Import a NN consisting of a single Tensorflow Lite CONCATENATION operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-4 | Import a NN consisting of a single Tensorflow Lite CONV\_2D operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-5 | Import a NN consisting of a single Tensorflow Lite DEPTHWISE\_CONV\_2D operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-6 | Import a NN consisting of a single Tensorflow Lite DEQUANTIZE operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-7 | Import a NN consisting of a single Tensorflow Lite EMBEDDING\_LOOKUP operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-8 | Import a NN consisting of a single Tensorflow Lite FULLY\_CONNECTED operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-9 | Import a NN consisting of a single Tensorflow Lite HASHTABLE\_LOOKUP operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-10 | Import a NN consisting of a single Tensorflow Lite L2\_NORMALIZATION operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-11 | Import a NN consisting of a single Tensorflow Lite L2\_POOL\_2D operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-12 | Import a NN consisting of a single Tensorflow Lite LOCAL\_RESPONSE\_NORMALIZATION operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-13 | Import a NN consisting of a single Tensorflow Lite LOGISTIC operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-14 | Import a NN consisting of a single Tensorflow Lite LSH\_PROJECTION operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-15 | Import a NN consisting of a single Tensorflow Lite LSTM operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-16 | Import a NN consisting of a single Tensorflow Lite MAX\_POOL\_2D operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-17 | Import a NN consisting of a single Tensorflow Lite MUL operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-18 | Import a NN consisting of a single Tensorflow Lite RELU operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-19 | Import a NN consisting of a single Tensorflow Lite RELU\_N1\_TO\_1 operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-20 | Import a NN consisting of a single Tensorflow Lite RELU6 operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-21 | Import a NN consisting of a single Tensorflow Lite RESHAPE operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-22 | Import a NN consisting of a single Tensorflow Lite RESIZE\_BILINEAR operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-23 | Import a NN consisting of a single Tensorflow Lite RNN operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-24 | Import a NN consisting of a single Tensorflow Lite SOFTMAX operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-25 | Import a NN consisting of a single Tensorflow Lite SPACE\_TO\_DEPTH operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-26 | Import a NN consisting of a single Tensorflow Lite SVDF operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-27 | Import a NN consisting of a single Tensorflow Lite TANH operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-28 | Import a NN consisting of a single Tensorflow Lite CONCAT\_EMBEDDINGS operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-29 | Import a NN consisting of a single Tensorflow Lite SKIP\_GRAM operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-30 | Import a NN consisting of a single Tensorflow Lite CALL operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-31 | Import a NN consisting of a single Tensorflow Lite CUSTOM operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-32 | Import a NN consisting of a single Tensorflow Lite EMBEDDING\_LOOKUP\_SPARSE operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-33 | Import a NN consisting of a single Tensorflow Lite PAD operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-34 | Import a NN consisting of a single Tensorflow Lite UNIDIRECTIONAL\_SEQUENCE\_RNN operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-35 | Import a NN consisting of a single Tensorflow Lite GATHER operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-36 | Import a NN consisting of a single Tensorflow Lite BATCH\_TO\_SPACE\_ND operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-37 | Import a NN consisting of a single Tensorflow Lite SPACE\_TO\_BATCH\_ND operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-38 | Import a NN consisting of a single Tensorflow Lite TRANSPOSE operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-39 | Import a NN consisting of a single Tensorflow Lite MEAN operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-40 | Import a NN consisting of a single Tensorflow Lite SUB operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-41 | Import a NN consisting of a single Tensorflow Lite DIV operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-42 | Import a NN consisting of a single Tensorflow Lite SQUEEZE operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-43 | Import a NN consisting of a single Tensorflow Lite UNIDIRECTIONAL\_SEQUENCE\_LSTM operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-44 | Import a NN consisting of a single Tensorflow Lite STRIDED\_SLICE operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-45 | Import a NN consisting of a single Tensorflow Lite BIDIRECTIONAL\_SEQUENCE\_RNN operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-46 | Import a NN consisting of a single Tensorflow Lite EXP operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-47 | Import a NN consisting of a single Tensorflow Lite TOPK\_V2 operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-48 | Import a NN consisting of a single Tensorflow Lite SPLIT operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-49 | Import a NN consisting of a single Tensorflow Lite LOG\_SOFTMAX operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-50 | Import a NN consisting of a single Tensorflow Lite DELEGATE operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-51 | Import a NN consisting of a single Tensorflow Lite BIDIRECTIONAL\_SEQUENCE\_LSTM operation | During import no crashes or error messages occurred |
+| TST-1 | TST-1-52 | Import a NN consisting of a single Tensorflow Lite CAST operation | During import no crashes or error messages occurred |
+| TST-2 | TST-2-1 | Import a NN consisting of two Caffe layers - Input layer and Eltwise layer with parameter SUM | During import no crashes or error messages occurred |
+| TST-2 | TST-2-2 | Import a NN consisting of two Caffe layers - Input layer and Pooling layer with parameter AVE | During import no crashes or error messages occurred |
+| TST-2 | TST-2-3 | Import a NN consisting of two Caffe layers - Input layer and Concat layer | During import no crashes or error messages occurred |
+| TST-2 | TST-2-4 | Import a NN consisting of two Caffe layers - Input layer and Convolution layer | During import no crashes or error messages occurred |
+| TST-2 | TST-2-5 | Import a NN consisting of two Caffe layers - Input layer and InnerProduct layer | During import no crashes or error messages occurred |
+| TST-2 | TST-2-6 | Import a NN consisting of two Caffe layers - Input layer and LRN layer | During import no crashes or error messages occurred |
+| TST-2 | TST-2-7 | Import a NN consisting of two Caffe layers - Input layer and Sigmoid layer | During import no crashes or error messages occurred |
+| TST-2 | TST-2-8 | Import a NN consisting of two Caffe layers - Input layer and Recurrent layer | During import no crashes or error messages occurred |
+| TST-2 | TST-2-9 | Import a NN consisting of two Caffe layers - Input layer and Pooling layer with parameter MAX | During import no crashes or error messages occurred |
+| TST-2 | TST-2-10 | Import a NN consisting of two Caffe layers - Input layer and Eltwise layer with parameter PROD | During import no crashes or error messages occurred |
+| TST-2 | TST-2-11 | Import a NN consisting of two Caffe layers - Input layer and ReLU layer | During import no crashes or error messages occurred |
+| TST-2 | TST-2-12 | Import a NN consisting of two Caffe layers - Input layer and Reshape layer | During import no crashes or error messages occurred |
+| TST-2 | TST-2-13 | Import a NN consisting of two Caffe layers - Input layer and Softmax layer | During import no crashes or error messages occurred |
+| TST-2 | TST-2-14 | Import a NN consisting of two Caffe layers - Input layer and TanH layer | During import no crashes or error messages occurred |
+| TST-2 | TST-2-15 | Import a NN consisting of two Caffe layers - Input layer and Python layer | During import no crashes or error messages occurred |
+| TST-2 | TST-2-16 | Import a NN consisting of two Caffe layers - Input layer and Reduction layer with parameter MEAN | During import no crashes or error messages occurred |
+| TST-2 | TST-2-17 | Import a NN consisting of two Caffe layers - Input layer and Exp layer | During import no crashes or error messages occurred |
+| TST-2 | TST-2-18 | Import a NN consisting of two Caffe layers - Input layer and Split layer | During import no crashes or error messages occurred |
+| TST-3 | TST-3-1 | Import a NN consisting of a single Caffe2 Add operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-2 | Import a NN consisting of a single Caffe2 AveragePool2D operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-3 | Import a NN consisting of a single Caffe2 Concat operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-4 | Import a NN consisting of a single Caffe2 Conv2D operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-5 | Import a NN consisting of a single Caffe2 FC operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-6 | Import a NN consisting of a single Caffe2 LRN operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-7 | Import a NN consisting of a single Caffe2 Sigmoid operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-8 | Import a NN consisting of a single Caffe2 MaxPool2D operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-9 | Import a NN consisting of a single Caffe2 Mul operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-10 | Import a NN consisting of a single Caffe2 Relu operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-11 | Import a NN consisting of a single Caffe2 Reshape operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-12 | Import a NN consisting of a single Caffe2 Softmax operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-13 | Import a NN consisting of a single Caffe2 Tanh operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-14 | Import a NN consisting of a single Caffe2 PadImage operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-15 | Import a NN consisting of a single Caffe2 BatchToSpace operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-16 | Import a NN consisting of a single Caffe2 SpaceToBatch operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-17 | Import a NN consisting of a single Caffe2 Transpose operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-18 | Import a NN consisting of a single Caffe2 Mean operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-19 | Import a NN consisting of a single Caffe2 Sub operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-20 | Import a NN consisting of a single Caffe2 Div operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-21 | Import a NN consisting of a single Caffe2 Squeeze operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-22 | Import a NN consisting of a single Caffe2 Exp operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-23 | Import a NN consisting of a single Caffe2 TopK operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-24 | Import a NN consisting of a single Caffe2 Split operation | During import no crashes or error messages occurred |
+| TST-3 | TST-3-25 | Import a NN consisting of a single Caffe2 Cast operation | During import no crashes or error messages occurred |
+| TST-4 | TST-4-1 | Import Inception V3 NN model | During import no crashes or error messages occurred |
+| TST-5 | TST-5-1 | Import MobileNet NN model | During import no crashes or error messages occurred |
+| TST-6 | TST-6-1 | Import Inception V3 NN model, serialize all model weights, compare serialized data with the initial NN model | Test executed successfully, serialized weights are equal to initial model weights |
+| TST-6 | TST-6-2 | Import MobileNet NN model, serialize all model weigths, compare serialized data with the initial NN model | Test executed successfully, serialized weights are equal to initial model weights |
+| TST-7 | TST-7-1 | Generate binary for the Inception V3 NN model and run its inference on a device with ARM CPU | Test executed successfully, no crashes occurred, inference result was output, amount and format of the outputs corresponds to the expected NN model outputs |
+| TST-7 | TST-7-2 | Generate binary for the MobileNet NN model and run its inference on a device with ARM CPU | Test executed successfully, no crashes occurred, inference result was output, amount and format of the outputs corresponds to the expected NN model outputs |
+| TST-8 | TST-8-1 | Generate binary for the Inception V3 NN model and run its inference on a GPU-enabled device | Test executed successfully, no crashes occurred, inference result was output, amount and format of the outputs corresponds to the expected NN model outputs |
+| TST-8 | TST-8-2 | Generate binary for the MobileNet V3 NN model and run its inference on a GPU-enabled device | Test executed successfully, no crashes occurred, inference result was output, amount and format of the outputs corresponds to the expected NN model outputs |
+| TST-9 | TST-9-1 | Provide correct NN model, compile it as a static library, then check that corresponding binary exists and it is a static library | Test executed successfully |
+| TST-9 | TST-9-2 | Provide correct NN model, compile it as a shared library, then check that corresponding binary exists and it is a shared library | Test executed successfully |
+| TST-9 | TST-9-3 | Provide incorrect model, compile it as a static library, then check that no compiled artifact is produced | Test executed successfully |
+| TST-9 | TST-9-4 | Provide incorrect model, compile it as a shared library, then check that no compiled artifact is produced | Test executed successfully |
+| TST-10 | TST-10-1 | Check that a static library is provided after compiling Inception V3 as a static library | Test executed successfully |
+| TST-10 | TST-10-2 | Check that a shared library is provided after compiling Inception V3 as a shared library | Test executed successfully |
+| TST-11 | TST-11-1 | Check that a static library is provided after compiling MobileNet as a static library | Test executed successfully |
+| TST-11 | TST-11-2 | Check that a shared library is provided after compiling MobileNet as a shared library | Test executed successfully |
+| TST-12 | TST-12-1 | Check that configuration object is constructed correctly when getting configuration parameters from command line | Test executed successfully |
+| TST-12 | TST-12-2 | Check that configuration object is constructed correctly when getting configuration parameters from config file | Test executed successfully |
+| TST-12 | TST-12-3 | Check that configuration object is constructed correctly when getting configuration parameters from environment variables | Test executed successfully |
+| TST-13 | TST-13-1 | Compile Inception V3 as static library for CPU, provide it and the original model with same correct input data, then compare the result from original model with the result from compiled artifact | Test executed successfully, results are comparable |
+| TST-13 | TST-13-2 | Compile Inception V3 as shared library for CPU, provide it and the original model with same correct input data, then compare the result from original model with the result from compiled artifact | Test executed successfully, results are comparable |
+| TST-13 | TST-13-3 | Compile Inception V3 as static library for GPU, provide it and the original model with same correct input data, then compare the result from original model with the result from compiled artifact | Test executed successfully, results are comparable |
+| TST-13 | TST-13-4 | Compile Inception V3 as shared library for GPU, provide it and the original model with same correct input data, then compare the result from original model with the result from compiled artifact | Test executed successfully, results are comparable |
+| TST-13 | TST-13-5 | Compile MobileNet as static library for CPU, provide it and the original model with same correct input data, then compare the result from original model with the result from compiled artifact | Test executed successfully, results are comparable |
+| TST-13 | TST-13-6 | Compile MobileNet as shared library for CPU, provide it and the original model with same correct input data, then compare the result from original model with the result from compiled artifact | Test executed successfully, results are comparable |
+| TST-13 | TST-13-7 | Compile MobileNet as static library for GPU, provide it and the original model with same correct input data, then compare the result from original model with the result from compiled artifact | Test executed successfully, results are comparable |
+| TST-13 | TST-13-8 | Compile MobileNet as shared library for GPU, provide it and the original model with same correct input data, then compare the result from original model with the result from compiled artifact | Test executed successfully, results are comparable |
+| TST-14 | TST-14-1 | Provide compiled Inception V3 artifact with invalid input, check that no unexpected termination occurs | Test executed successfully |
+| TST-14 | TST-14-2 | Provide compiled Inception V3 artifact with invalid input, check that an error message is provided | Test executed successfully |
+| TST-14 | TST-14-3 | Provide compiled MobileNet artifact with invalid input, check that no unexpected termination occurs | Test executed successfully |
+| TST-14 | TST-14-4 | Provide compiled MobileNet artifact with invalid input, check that an error message is provided | Test executed successfully |
+| TST-15 | TST-15-1 | Check that the OS used during test environment build is Linux-based | Test executed successfully |
+| TST-16 | TST-16-1 | Compile a valid NN model, then check that C/C++ header corresponding to compiled artifact exists | Test executed successfully |
+| TST-16 | TST-16-2 | Compile a valid NN model, then if C/C++ header corresponding to compiled artifact exists, verify its validity | Test executed successfully |
+
+Table 5-1. System Test case
projects / platform / core / ml / nnfw.git / commitdiff