From: Sergey Vostokov/AI Tools Lab /SRR/Staff Engineer/삼성전자 Date: Wed, 4 Jul 2018 07:30:09 +0000 (+0900) Subject: SW Detailed-Level Design Document v1.0 (#357) X-Git-Tag: nncc_backup~2496 X-Git-Url: http://review.tizen.org/git/?a=commitdiff_plain;h=d110379c5d91ecc1763c17e154b58820b5f441a0;p=platform%2Fcore%2Fml%2Fnnfw.git SW Detailed-Level Design Document v1.0 (#357) * Add Detailed-Level Design Document This commit adds the SW Detailed-Level Design Document Signed-off-by: Sergey Vostokov * Remove "Apply optimizations" section * Update DLD template * Add content of "SW Detailed Structure Design" section * Remove structure image. Update section "Generate the code" and "Interface" --- diff --git a/doc/project/detailed_level_design.md b/doc/project/detailed_level_design.md new file mode 100644 index 0000000..3662510 --- /dev/null +++ b/doc/project/detailed_level_design.md @@ -0,0 +1,329 @@ +# SW Detailed Level Design + +**Revision history** + +| Ver. | Date | Contents | Author | Approver | +| ---- | ---------- | ----------------- | ----------------- | ------------ | +| 0.1 | 2018.06.20 | Initial version | Vostokov Sergey | Sung-Jae Lee | +| 0.2 | 2018.06.21 | SE member review | Alexey Kondrashov | | +| 1.0 | 2018.06.22 | Final DR1 version | Vostokov Sergey | Sung-Jae Lee | + +**Terminology and Abbreviation** + +| | | +| ------------ | ------------------------------------------------------------- | +| OS | Operating System | +| OS API | Application interface of OS | +| HW | Hardware | +| SW | Software | +| NN | Neural Network | +| NN model | Neural network model (Instance of NN built with ML framework) | +| NN compiler | The compiler for neural network | +| ML framework | The machine learning framework | +| TF/TF Lite | Tensorflow/Tensorflow Lite ML framework | +| IR | Intermediate representation | +| CI/CI system | Continuous integration system | +| UI | The user interface | +| GUI | The graphical user interface | +| CLI | The command-line interface | + +**References** + +\[1\] Vostokov Sergey, [SW Requirements Specification](requirements_specification.md) + +\[2\] Vostokov Sergey, [SW High-Level Design](high_level_design.md) + +## Overview + +### Scope + +The main goal of the project is to develop a compiler for neural +networks to produce executable artefact for specified SW and HW +platform. + +The development scope includes the following components: + + - Develop importer module to parse, verify and represent NN model for + further optimization and compilation + - Develop code emitters to produce executable binary for CPU and GPU + + +**2018 year goals:** + + - Support TensorFlow Lite NN model format + - Support Caffe NN model format + - Support Caffe2 NN model format (Optional) + - Support compilation of MobileNet NN + - Support compilation of Inception v3 NN + - Support ARM CPU + - Support ARM GPU (Mali) + - Support Tizen OS + - Support SmartMachine OS +(Optional) + +| Product | Target Model Name | Comment | +| ------------------- | ------------------------------ | ---------------- | +| Tizen phone | Tizen TM2 | Reference device | +| Tizen device | Odroid XU4 | Reference board | +| SmartMachine target | Microvision mv8890, exynos8890 | Reference device | + +Table 1-1. Target Model + +### Design Consideration + +Deep learning software demands reliability and performance. The common +approach which comes from the history is to develop a SW framework +(machine learning framework) which would compute each step of the neural +network inference process using supported hardware. This approach is +used in many popular solutions like Google Tensorflow/Tensorflow Lite, +Caffe/2, etc. Traditionally, neural network developers build a +computation graph and then an appropriate machine learning framework +interprets it. The latest discoveries in AI field show that the +node-visitor method of execution is inefficient. As a result, a second +approach has been worked out by the industry, which is a neural network +compiler that executes code more efficiently. + +This document presents the design of the *nncc*, a neural network +compiler collection. The design should provide the easiest way to extend +the functionality of the *nncc* by adding new modules with the following +features: + + - Support neural networks produced by various machine learning + frameworks; + - Produce an artefact taking advantages of various hardware + including specialized processors like NPU; + - Apply new domain specific optimization techniques over given NN. + +Non-functional requirements to the developed software are well-described +in the SW Requirements Specification, such requirements are not shown +here to avoid duplication. + +### Constraints + +See constraints in SW Requirements Specification. + + + +++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ItemAssumptions, Dependencies and the ConstraintsReference
Tizen SW Platform
+
The following items should be provided:
+
    +
  • Tizen API
    • +
  • Tizen kernel
    • +
  • Tizen FW
    • +
  • Tizen SDK
    • +
  • Tizen naming convention
    • +
+
+
- www.tizen.org
- wiki.tizen.org
- developer.tizen.org
SmartMachine OS Platform
+
The following items should be provided:
+
    +
  • SmartMachine API
    • +
  • SmartMachine kernel
    • +
  • SmartMachine FW
    • +
  • SmartMachine SDK
    • +
  • SmartMachine naming convention
    • +
+
+
- Platform confluence
- Github
- Functional Safety confluence
Host OSLinux-based OS (Ubuntu, Archlinux, etc)- Ubuntu site
- Archlinux site
Tizen target HWThe reference device should be provided: Tizen TM2
SmartMachine target HWThe reference device should be provided
+Table 1-2. Assumptions, Dependecies and the Constraints + +## SW Detailed Structure Design + +### SW Block Structure + +Top-Level Components of the nncc descriped in HLD. More detailed +structure and class diagram will be available after development +completion. + +### SW Block Feature + +1. Initialization: configure all internal modules (see + [{Initialization} Detailed Design](#initialization-detailed-design)) +2. Frontend: Import NN model (see [{Import NN model} Detailed + Design](#import-nn-model-detailed-design)) + - *Caffe frontend*: includes the parser of Caffe NN model format, + verifier to ensure that parsed data is valid and consentient, + and Caffe-specific IR converter + - *Caffe2 frontend*: includes the parser of Caffe2 NN model + format, verifier to ensure that parsed data is valid and + consentient, and Caffe2-specific IR converter to Model IR + - *Tensorflow Lite frontend*: includes the parser of Tensorflow NN + model format with automatic version recognition feature, + verifier to ensure that parsed data is valid and consentient, + and Tensorflow Lite-specific IR converter to Model IR +3. Backend: Generate the code (see [{Generate the code} Detailed + Design](#generate-the-code-detailed-design)) + - *Interpreter:* As it was described in SW High-Level Document + imported NN model may proceed through three step of Intermediate + representation: Model IR, Coarse-Grained IR, Fine-Grained IR. + The Interpreter backed uses each this IR to do inference of + given NN model. As the output, the user gets the resulting + calculation of all NN ops included into original computation + graph. + - *Binary*:This type refers to generating binary code that can be + executed on the target device. NN compiler can generate code + that is either executed solely on CPU or takes advantage of the + GPU when possible if the corresponding target was specified. The + user may want to incorporate 3rd party libraries included into + target firmware or delivered with the application package. In + this case, the compiler prepares the data following EABI + convention and embeds an invocation of high-level functions by + appropriate symbol. + - *Soft*: Resulting program is a generated source code in + high-level programming language C or C++. Here there are two + options: the first one is to generate the source code that does + not depend on libraries outside of itself, with the exception of + system libraries. The second one is to include the code to + invoke high-level functions from 3rd party libraries. For + example, it may be an invocation of matrix multiplication from + GEMM library. + +## SW Detailed Operation Design + +### {Initialization} Detailed Design + +#### Major Function + +To provide a valid configuration session for all modules of *nncc* using +user input from the command line/config file/environment variables. + +#### Operation Sequence + +Initialization of the *nncc* includes command line option processing, +configuration of its subsystems as well as any error checking possible +at this stage. It consists of the following steps: + +1. Collect all command line options and verify their format for + validity (no syntax errors etc.) + +2. Check for validity and then process general options + +3. Load subsystem modules + +4. For each one of them: + + - Configure + - Pass command line options + - Check command line options for validity (for example, check + that every required option is present) + +At the end of this process each subsystem is configured and has access +to all data needed for its operation. + +### {Import NN model} Detailed Design + +#### Major Function + +To convert given NN model from framework-specific IR to Model IR for +further processing. + +#### Operation Sequence + +As you may see on the diagram, neural network import is the main +function of the compiler front-end part. The result of this operation is +a computation graph which is presented as Model IR. + +![image](../images/nncc_idef0_a12.png) + +The import process consists of three parts: + +1. NN model parsing +2. Verification of the result from the previous step +3. Converting the model to the Model IR + +During the first step, file or files containing the model are read and +represented in some format specific to each NN framework. + +Verification step is included to ensure that: + + - None of the files constituting the model are damaged + - Model format corresponds to the specified one + - Version of the model format corresponds to the specified one + +The most important step is accurately converting the model from the +framework-specific representation to the Model IR. This conversion +includes: + + - *Translation of the NN model computation graph to the Model IR + computation graph.* During the translation new nodes may be + introduced - for example, a high-level NN operation may be split + into a few smaller ones. + - *NN model parameter layout conversion.* The way parameters (also + known as weights) of a model are layed out in each specific NN + framework may differ, and it is necessary to convert such layout + into a unified format. + - *NN operation parameter conversion.* Each NN operation has a set + of its own parameters describing the way this operation should be + performed, and these parameters also differ between frameworks. + +Resulting Model IR is equivalent to the initial NN model in terms of how +NN model inputs would be transformed into its outputs if all the +operations in the Model IR were executed. + +### {Generate the code} Detailed Design + +Development in progress. Will be described on Completion DR. + +## Interface Design + +Development in progress. Will be described on DR2. + +## SW Code Structure + +| Directory | Description | +| ------------------------ | -------------------------------------------------------------------- | +| / | source codes of the build system, main README file | +| /contrib | Incubating projects | +| /doc | Contains the documentation of the project | +| /doc/project | Contains project management documents (SRS, SDD, STD, HLD, DLD, etc) | +| /libs | Contains the source of the libraries which are used by the nncc | +| /libs/core | Contains the source code of the core library of nncc | +| /libs/frontend | Contains the source code of supported frontend's plugins | +| /libs/frontend/caffe | The source code for the Caffe frontend | +| /libs/frontend/caffe2 | The source code for the Caffe2 frontend | +| /libs/frontend/tflite | The source code for the Tensorflow Lite frontend | +| /libs/backend | Contains the source code of supported backend’ plugins | +| /libs/backend/cpu | Contains the source code of CPU backend | +| /libs/backend/gpu | Contains the source code of GPU backend | +| /libs/backend/3rd\_party | Contains the source code of backend to utilize 3rd party libraries | +| /scripts | Various scripts for building and testing the nncc | +| /tools | The source code of the executables |