From ec136902b0d43e1a8a355a02e9a6eb7d4da0094d Mon Sep 17 00:00:00 2001 From: "Efimov Alexander/AI Tools Lab/./Samsung Electronics" Date: Mon, 29 Jul 2019 09:24:50 +0900 Subject: [PATCH] Dr1 development document (#5646) * Updated SDD document for DR1 - Added more tasks - Changed formatiing of task table - Responsible person is changed * change development plan, fix responcibilities, remove NNCC repo mentions * add control flow related item * remove redundant tizen requirements * relax control flow requirements * fix naming of components Signed-off-by: Efimov Alexander --- docs/project/19_NN_Compiler_and_Runtime_SDD.rst | 629 +++++++++++++++++++++ .../19_NN_Compiler_and_Runtime_SDD.rst.cover | 7 + docs/project/Makefile | 2 +- docs/project/project_purpose_and_scope.rst | 15 +- docs/project/project_sw_hw_constraints.rst | 33 ++ docs/project/project_target_model.rst | 14 + 6 files changed, 691 insertions(+), 9 deletions(-) create mode 100644 docs/project/19_NN_Compiler_and_Runtime_SDD.rst create mode 100644 docs/project/19_NN_Compiler_and_Runtime_SDD.rst.cover create mode 100644 docs/project/project_sw_hw_constraints.rst create mode 100644 docs/project/project_target_model.rst diff --git a/docs/project/19_NN_Compiler_and_Runtime_SDD.rst b/docs/project/19_NN_Compiler_and_Runtime_SDD.rst new file mode 100644 index 0000000..cc005e6 --- /dev/null +++ b/docs/project/19_NN_Compiler_and_Runtime_SDD.rst @@ -0,0 +1,629 @@ +======================= +SW Development Document +======================= + +.. contents:: + +.. section-numbering:: + +.. raw:: pdf + + PageBreak oneColumn + + +**Revision history** + ++-------+-------------+-----------------------------+--------------------------+---------------------+ +| Ver. | Date | Contents | Author | Approver | ++=======+=============+=============================+==========================+=====================+ +| 0.1 | 2019.04.22 | Initial version | Rusyaev Roman | Sung-Jae Lee | ++-------+-------------+-----------------------------+--------------------------+---------------------+ +| 0.2 | 2019.04.23 | SE member in-charge review | Aleksei Kondrashov | | ++-------+-------------+-----------------------------+--------------------------+---------------------+ +| 1.0 | 2019.04.24 | Final Execution DR version | Rusyaev Roman | Sung-Jae Lee | ++-------+-------------+-----------------------------+--------------------------+---------------------+ +| 1.1 | 2019.07.24 | DR1 version | Efimov Alexander | Sung-Jae Lee | ++-------+-------------+-----------------------------+--------------------------+---------------------+ + +.. include:: project_terms_and_abbreviation.rst + +Project Overview +================ + +Purpose and Scope +----------------- + + +.. include:: project_purpose_and_scope.rst +.. include:: project_target_model.rst + +Assumptions, Dependencies and Constraints +----------------------------------------- + +.. include:: project_sw_hw_constraints.rst + +Development Plan And Result +=========================== + +Development Schedule +-------------------- + +.. list-table:: Table 2-1. SW Development Schedule + :widths: 10 10 5 5 4 4 6 + :header-rows: 1 + + * - Task + - Deliverable + - Plan start + - Plan end + - Result start + - Result end + - Responsibility + + * - Prepare SW requirements + - SRS v1.0 + - 04.2019 + - 05.2019 + - + - + - R\. Rusyaev + + * - Prepare SW System Test Plan + - STD v1.0 + - 04.2019 + - 05.2019 + - + - + - R\. Rusyaev + + * - Prepare Initial Project Plan + - SDD v1.0 + - 04.2019 + - 05.2019 + - + - + - R\. Rusyaev + + * - Prepare SW System Cases + - STD v2.0 + - 04.2019 + - 06.2019 + - + - + - R\. Rusyaev + + * - Prepare software design documents + - HLD, DLD + - 05.2019 + - 08.2019 + - + - + - A\. Efimov + + * - Implement proof of concept for custom operations support in `NN Runtime` + - Proof of concept for custom operations support in `NN Runtime` + - 05.2019 + - 08.2019 + - + - + - A\. Efimov + + * - Develop initial version of ONXX importer + - Initial version of ONNX importer, capable for one operation networks import + - 05.2019 + - 08.2019 + - + - + - A\. Efimov + + * - Develop `Common IR` for `NN Compiler` and `NN Runtime` + - `Common IR` for `NN Compiler` and `NN Runtime` + - 04.2019 + - 09.2019 + - + - + - A\. Efimov + + * - Support heterogeneous execution + - Heterogeneous execution for CPU and GPU backends + - 04.2019 + - 09.2019 + - + - + - A\. Efimov + + * - Implement TF Importer + - TensorFlow NN Model Importer tested and capable to import all target models + - 05.2019 + - 09.2019 + - + - + - A\. Efimov + + * - Support custom operations in `NN Runtime` + - Custom operations support in `NN Runtime` and provide internal interface for custom operations registration + - 05.2019 + - 09.2019 + - + - + - A\. Efimov + + * - Develop mature version of ONXX importer + - Initial version of ONNX importer, capable for inception-v3 network import + - 05.2019 + - 10.2019 + - + - + - A\. Efimov + + * - Prepare proof of concept for control flow operations in `NN Runtime` + - Proof of concept for control flow operations in `NN Runtime` + - 07.2019 + - 11.2019 + - + - + - A\. Efimov + + * - Implement final version of ONNX Importer + - ONNX NN Model Importer tested and capable to import all target models + - 05.2019 + - 11.2019 + - + - + - A\. Efimov + + * - Prepare test result + - STD v2.0, UTR v1.0 + - 04.2019 + - 11.2019 + - + - + - A\. Efimov + + * - Prepare project completion documents + - STD v2.0, Technology Items completion reports + - 05.2019 + - 12.2019 + - + - + - A\. Efimov + + +SW Metrics +---------- + +.. list-table:: Table 2-2. SW Metrics + :widths: 5 10 5 7 5 4 8 + :header-rows: 1 + + * - Category + - Metric + - Collection Method + - Collection Period + - Planned + - Actual + - Responsibility + + * - Quality + - Test pass rate + - GTest + - 22.02.2019 - 31.12.2019 + - 100% + - + - A\. Efimov + + * - Quality + - Defects density + - Manual (GTest/Scripts) + - 22.02.2019 - 31.12.2019 + - <= 1 defect/KLOC + - + - A\. Efimov + + * - Quality + - Defects removal rate + - Manual (GTest/Scripts) + - 22.02.2019 - 31.12.2019 + - 100% + - + - A\. Efimov + + * - Quality + - Critical defects + - SVACE + - 22.02.2019 - 31.12.2019 + - 0 + - + - A\. Efimov + + * - Quality + - Major defects + - SVACE + - 22.02.2019 - 31.12.2019 + - 0 + - + - A\. Efimov + + * - Quality + - Code review issue removal + - Samsung Research GitHub + - 22.02.2019 - 31.12.2019 + - 100% + - + - A\. Efimov + + * - Quality + - Comments Rate + - ``cloc`` tool + - 22.02.2019 - 31.12.2019 + - Exceed 20% + - + - A\. Efimov + + * - Quality + - Cyclomatic Complexity (per file) + - SonarQube + - 22.02.2019 - 31.12.2019 + - < 50 + - + - A\. Efimov + + * - Quality + - Unused Items (Unused Files, Unused Functions, Unused Global Variables) + - gcc/g++ + - 22.02.2019 - 31.12.2019 + - 0 + - + - A\. Efimov + + * - Process + - Project On-time Completion Rate + - PLM + - 22.02.2019 - 31.12.2019 + - 100% + - + - A\. Efimov + + * - Process + - Milestone On-time Completion Rate + - PLM + - 22.02.2019 - 31.12.2019 + - 100% + - + - A\. Efimov + + * - Process + - Process compliance + - Audit + - 22.02.2019 - 31.12.2019 + - 100% + - + - A\. Kondrashov + + +SW Configurations Management +---------------------------- + +Document +```````` + + +.. list-table:: Table 2-3. Document Configuration Item + :widths: 2 20 5 8 + :header-rows: 1 + + * - No + - Configuration Item + - Location + - Submitter + + * - 1 + - SW Requirement Specification + - PLM, GitHub + - A\. Efimov + + * - 2 + - SW Development Document + - PLM, GitHub + - A\. Efimov + + * - 3 + - SW High Level Document + - PLM, GitHub + - A\. Efimov + + + * - 4 + - SW Detailed Level Document + - PLM, GitHub + - A\. Efimov + + * - 5 + - SW System Test Document + - PLM, GitHub + - A\. Efimov + + * - 6 + - SW Unit Test Report + - PLM, GitHub + - A\. Efimov + + +SW Source Code +`````````````` + + +SW Repositories: https://github.sec.samsung.net/STAR/nnfw + +:: + + git clone https://github.sec.samsung.net/STAR/nnfw.git + + + +Baseline +```````` + +.. list-table:: Table 2-4. Baseline Establishment + :widths: 5 5 8 + :header-rows: 1 + + * - Phase + - Baseline Name + - SW Configuration Item + + * - 04.2019 Plan + - Execution DR + - SW Requirement Specification, SW Development Document, System Test Document initial version + + * - 06.2019 Execution + - DR1 + - System Test Document + + * - 08.2019 Execution + - Design document + - SW High Level Document, SW Detailed Design Document + + * - 09.2019 Execution + - DR2 + - Update design documents, UTR (1st round), STD (1st round) + + * - 10.2019 Execution + - Test report + - SW System Test Document (result), SW Unit Test Report + + * - 12.2019 Completion + - Project Completion + - Project Completion Report + +SW Quality Verification +======================= + +SW Verification +--------------- + + +.. list-table:: Table 3-1. SW Verification Plan and Result + :widths: 4 11 11 8 8 10 7 6 + :header-rows: 1 + + * - No + - Verification Item + - Quality Goal + - Tool + - Phase + - Development Team Member in Charge + - Result + - Note + + * - 1 + - Open source License Verification + - Clear violations of open source obligation + - ProtexIP + - Execution + - Efimov Alexander + - + - + + * - 2 + - Potential Defect + - Fix all defects + - Svace + - Test + - Efimov Alexander + - + - + + * - 3 + - System Defect + - Fix Critical/ Major defects + - GitHub + - Test + - Efimov Alexander + - + - + +Static Analysis +--------------- + +.. list-table:: Table 3-2. Static Analysis Verification + :widths: 5 20 10 7 20 + :header-rows: 1 + + * - No + - Activity + - Schedule + - Result + - Comment + + * - 1 + - SA Verification I (SVACE) + - 28.09.2019 + - + - + + * - 2 + - SA Verification II (SVACE) + - 30.11.2019 + - + - + + * - 2 + - SA Verification III (SVACE) + - 31.12.2019 + - + - + +Coding Standard +--------------- + +.. list-table:: Table 3-3. Coding Standard Verification + :widths: 5 20 10 7 20 + :header-rows: 1 + + * - No + - Activity + - Schedule + - Result + - Comment + + * - 1 + - Coding standard enforcement with ``clang-format`` tool. + - Regular + - + - + + +Convergence (integration testing) +--------------------------------- + +Out of scope since the integration with other SW is not required by SW Requirement Specification. + +Dynamic Analysis +---------------- + +Dynamic analysis for NN Compiler memory leaks investigation performed with Valgrind/AdressSanitizer/LeakSanitizer tools. + +.. list-table:: Table 3-4. Dynamic Analysis Verification + :widths: 5 20 10 7 20 + :header-rows: 1 + + * - No + - Activity + - Schedule + - Result + - Comment + + * - 1 + - DA Verification I + - 28.09.2019 + - + - + + * - 2 + - DA Verification II + - 30.11.2019 + - + - + + * - 2 + - DA Verification III + - 31.12.2019 + - + - + +Architecture Analysis +--------------------- + +SW architecture verification is managed by HQ. + + +SW Security +----------- + +Out of the project scope since the project is not related to SW security. + + +Code Review +----------- + + +.. list-table:: + :widths: 5 20 10 7 20 + :header-rows: 1 + + * - No + - Activity + - Schedule + - Result + - Comment + + * - 1 + - Code review via GitHub pull requests + - Regular + - + - + +Risk Management +=============== + + +.. list-table:: Table 4-1. SW Risk Management + :widths: 7 12 12 10 12 12 + :header-rows: 1 + + * - Priority + - Risk Description + - Risk Reduction Solution + - Schedule + - Result + - Responsibility + + * - 1 + - Project scope is changed due extra HQ request + - Discuss the new requirements via email and messenger, update SRS + - 02.2019 - 12.2019 + - + - A\. Efimov + + * - 2 + - Unavoidable technical difficulties during requirements implementation + - Submit requirements changes and get confirmation from HQ + - 02.2019 - 12.2019 + - + - A\. Efimov + + * - 3 + - Not enough HR + - Hire team members as soon as possible, request assistance from other teams + - 02.2019 - 12.2019 + - + - A\. Efimov + + * - 4 + - Use of GPL code + - Minimize usage of GPL code, wrap GPL modules with well-defined interfaces + so they can be easily replaced. + - 02.2019 - 12.2019 + - + - A\. Efimov + + * - 5 + - Requirements would change due external or internal circumstances, e.g. new technology or product launch + - Discuss project changes and make corrections + - 02.2019 - 12.2019 + - + - A\. Efimov + + +**References** + +Competitors: + +.. [#] `NNVM: Open Compiler for AI Frameworks `_. +.. [#] `Latte.jl: A high-performance DSL for deep neural networks in Julia `_. +.. [#] `DNNDK: Full-stack solution for deep learning development & deployment `_. +.. [#] `nngraph: Neural Network Graph Package `_. +.. [#] `Richard Wei, Lane Schwartz, Vikram Adve, DLVM: A modern compiler infrastructure for deep learning systems, arXiv:1711.03016, 2017 `_. +.. [#] `XLA (Accelerated Linear Algebra): a domain-specific compiler for linear algebra that optimizes TensorFlow computations `_. +.. [#] `nGraph: an open source C++ library, compiler and runtime for Deep Learning frameworks `_. diff --git a/docs/project/19_NN_Compiler_and_Runtime_SDD.rst.cover b/docs/project/19_NN_Compiler_and_Runtime_SDD.rst.cover new file mode 100644 index 0000000..d584d03 --- /dev/null +++ b/docs/project/19_NN_Compiler_and_Runtime_SDD.rst.cover @@ -0,0 +1,7 @@ +======================= +SW Development Document +======================= + +------------------------- +SRR-RAJ0119ZZ-BERF-SDD001 +------------------------- diff --git a/docs/project/Makefile b/docs/project/Makefile index 5a58c8c..fe3a0c6 100644 --- a/docs/project/Makefile +++ b/docs/project/Makefile @@ -1,4 +1,4 @@ -DOC_SRC = 19_NN_Compiler_and_Runtime_SRS.rst +DOC_SRC = 19_NN_Compiler_and_Runtime_SRS.rst 19_NN_Compiler_and_Runtime_SDD.rst DOC_PDF = $(DOC_SRC:.rst=.pdf) diff --git a/docs/project/project_purpose_and_scope.rst b/docs/project/project_purpose_and_scope.rst index b153326..c6f7eac 100644 --- a/docs/project/project_purpose_and_scope.rst +++ b/docs/project/project_purpose_and_scope.rst @@ -3,11 +3,10 @@ The main goals of the project are to develop compiler and runtime for neural net | | **2019 year goals:** -- Support heterogeneous computing in NN compiler and runtime -- Support TensorFlow NN model format -- Support ONNX NN model format -- Implement target-independent compiler optimizations -- Improve inference performance in NN runtime -- Develop common IR for both NN compiler and runtime -- Support RNN in NN runtime -- Support custom operators in NN runtime +- Support heterogeneous computing in NN Runtime +- Support TensorFlow NN Model format +- Support ONNX NN Model format +- Improve inference performance in NN Runtime +- Develop common IR for both NN Compiler and NN Runtime +- Support RNN in NN Runtime +- Support custom operators in NN Runtime diff --git a/docs/project/project_sw_hw_constraints.rst b/docs/project/project_sw_hw_constraints.rst new file mode 100644 index 0000000..5c602b4 --- /dev/null +++ b/docs/project/project_sw_hw_constraints.rst @@ -0,0 +1,33 @@ +| + +.. list-table:: Table 1-2. Assumptions, Dependencies and the Constraints + :widths: 23 40 23 + :header-rows: 1 + + * - Item + - Assumptions, Dependencies and the Constraints + - Reference + + * - Tizen SW Platform + - The following items should be provided: + - Tizen API + - Tizen kernel + - Tizen FW + - Tizen SDK + - Tizen naming convention + - + - `www.tizen.org `_ + - `wiki.tizen.org `_ + - `developer.tizen.org `_ + + * - Host OS + - Tizen, Linux-based OS (Ubuntu, Archlinux, etc) + - + - `Ubuntu site `_ + - `Archlinux site `_ + - `www.tizen.org `_ + + * - Target HW + - The reference device should be provided: Odroid-XU4 + - + diff --git a/docs/project/project_target_model.rst b/docs/project/project_target_model.rst new file mode 100644 index 0000000..e953c62 --- /dev/null +++ b/docs/project/project_target_model.rst @@ -0,0 +1,14 @@ +| + +.. list-table:: Table 1-1. Target Model + :widths: 23 50 20 + :header-rows: 1 + + * - Product + - Target Model Name + - Comment + + * - Linux-based device + - Odroid XU4 + - Reference board + -- 2.7.4