README.md

   1 # NNtrainer
   2
   3 [![Code Coverage](http://nnstreamer.mooo.com/nntrainer/ci/badge/codecoverage.svg)](http://nnstreamer.mooo.com/nntrainer/ci/gcov_html/index.html)
   4 ![GitHub repo size](https://img.shields.io/github/repo-size/nnstreamer/nntrainer)
   5 ![GitHub issues](https://img.shields.io/github/issues/nnstreamer/nntrainer)
   6 ![GitHub pull requests](https://img.shields.io/github/issues-pr/nnstreamer/nntrainer)
   7 <a href="https://scan.coverity.com/projects/nnstreamer-nntrainer">
   8   <img alt="Coverity Scan Build Status"
   9        src="https://scan.coverity.com/projects/22512/badge.svg"/>
  10 </a>
  11 [![DailyBuild](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/badge/daily_build_test_result_badge.svg)](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/build_result/)
  12
  13 NNtrainer is a Software Framework for training Neural Network models on devices.
  14
  15 ## Overview
  16
  17 NNtrainer is an Open Source Project. The aim of the NNtrainer is to develop a Software Framework to train neural network models on embedded devices which have relatively limited resources. Rather than training whole layers of a network, NNtrainer trains only one or a few layers of the layers added after a feature extractor.
  18
  19 Even though NNTrainer can be used to train sub-models, it requires implementation of additional functionalities to train models obtained from other machine learning and deep learning libraries. In the current version, various machine learning algorithms such as k-Nearest Neighbor (k-NN), Neural Networks, Logistic Regression and Reinforcement Learning algorithms are implemented. We also provide examples for various tasks such as transfer learning of models. In some of these examples, deep learning models such as Mobilenet V2 trained with Tensorflow-lite, are used as feature extractors. All of these were tested on Galaxy S8 with Android and PC (Ubuntu 16.04).
  20
  21 ## Official Releases
  22
  23 |     | [Tizen](http://download.tizen.org/snapshots/tizen/unified/latest/repos/standard/packages/) | [Ubuntu](https://launchpad.net/~nnstreamer/+archive/ubuntu/ppa) | Android/NDK Build |
  24 | :-- | :--: | :--: | :--: |
  25 |     | 6.0M2 and later | 18.04 | 9/P |
  26 | arm | [![armv7l badge](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/badge/tizen.armv7l_result_badge.svg)](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/build_result/) | Available  | Ready |
  27 | arm64 |  [![aarch64 badge](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/badge/tizen.aarch64_result_badge.svg)](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/build_result/) | Available  | [![android badge](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/badge/arm64_v8a_android_result_badge.svg)](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/build_result/) |
  28 | x64 | [![x64 badge](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/badge/tizen.x86_64_result_badge.svg)](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/build_result/)  | [![ubuntu badge](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/badge/ubuntu_result_badge.svg)](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/build_result/)  | Ready  |
  29 | x86 | [![x86 badge](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/badge/tizen.i586_result_badge.svg)](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/build_result/)  | N/A  | N/A  |
  30 | Publish | [Tizen Repo](http://download.tizen.org/snapshots/tizen/unified/latest/repos/standard/packages/) | [PPA](https://launchpad.net/~nnstreamer/+archive/ubuntu/ppa) |   |
  31 | API | C (Official) | C/C++ | C/C++  |
  32
  33 - Ready: CI system ensures build-ability and unit-testing. Users may easily build and execute. However, we do not have automated release & deployment system for this instance.
  34 - Available: binary packages are released and deployed automatically and periodically along with CI tests.
  35 - [Daily Release](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/build_result/)
  36 - SDK Support: Tizen Studio (6.0 M2+)
  37
  38 ## Maintainer
  39 * [Jijoong Moon](https://github.com/jijoongmoon)
  40 * [MyungJoo Ham](https://github.com/myungjoo)
  41 * [Geunsik Lim](https://github.com/leemgs)
  42
  43 ## Reviewers
  44 * [Sangjung Woo](https://github.com/again4you)
  45 * [Wook Song](https://github.com/wooksong)
  46 * [Jaeyun Jung](https://github.com/jaeyun-jung)
  47 * [Hyoungjoo Ahn](https://github.com/helloahn)
  48 * [Parichay Kapoor](https://github.com/kparichay)
  49 * [Dongju Chae](https://github.com/dongju-chae)
  50 * [Gichan Jang](https://github.com/gichan-jang)
  51 * [Yongjoo Ahn](https://github.com/anyj0527)
  52 * [Jihoon Lee](https://github.com/zhoonit)
  53 * [Hyeonseok Lee](https://github.com/lhs8928)
  54 * [Mete Ozay](https://github.com/meteozay)
  55
  56 ## Components
  57
  58 ### Supported Layers
  59
  60 This component defines layers which consist of a neural network model. Layers have their own properties to be set.
  61
  62  | Keyword | Layer Class Name | Description |
  63  |:-------:|:---:|:---|
  64  | conv1d | Conv1DLayer | Convolution 1-Dimentional Layer |
  65  | conv2d | Conv2DLayer |Convolution 2-Dimentional Layer |
  66  | pooling2d | Pooling2DLayer |Pooling 2-Dimentional Layer. Support average / max / global average / global max pooling |
  67  | flatten | FlattenLayer | Flatten layer |
  68  | fully_connected | FullyConnectedLayer | Fully connected layer |
  69  | input | InputLayer | Input Layer.  This is not always required. |
  70  | batch_normalization | BatchNormalizationLayer | Batch normalization layer |
  71  | activation | ActivaitonLayer | Set by layer property |
  72  | addition | AdditionLayer | Add input input layers |
  73  | attention | AttentionLayer | Attenstion layer |
  74  | centroid_knn | CentroidKNN | Centroid K-nearest neighbor layer |
  75  | concat | ConcatLayer | Concatenate input layers |
  76  | multiout | MultiOutLayer | Multi-Output Layer |
  77  | backobne_nnstreamer | NNStreamerLayer | Encapsulate NNStreamer layer |
  78  | backbone_tflite | TfLiteLayer | Encapsulate tflite as an layer |
  79  | permute | PermuteLayer | Permute layer for transpose |
  80  | preprocess_flip | PreprocessFlipLayer | Preprocess random flip layer |
  81  | preprocess_l2norm | PreprocessL2NormLayer | Preprocess simple l2norm layer to normalize |
  82  | preprocess_translate | PreprocessTranslateLayer | Preprocess translate layer |
  83  | reshape | ReshapeLayer | Reshape tensor dimension layer |
  84  | split | SplitLayer | Split layer |
  85  | dropout | DropOutLayer | Dropout Layer |
  86  | embedding | EmbeddingLayer | Embedding Layer |
  87  | rnn | RNNLayer | Recurrent Layer |
  88  | gru | GRULayer | Gated Recurrent Unit Layer |
  89  | lstm | LSTMLayer | Long Short-Term Memory Layer |
  90  | lstmcell | LSTMCellLayer | Long Short-Term Memory Cell Layer |
  91  | time_dist | TimeDistLayer | Time distributed Layer |
  92
  93 ### Supported Optimizers
  94
  95 NNTrainer Provides
  96
  97  | Keyword | Optimizer Name | Description |
  98  |:-------:|:---:|:---:|
  99  | sgd | Stochastic Gradient Decent | - |
 100  | adam | Adaptive Moment Estimation | - |
 101
 102 ### Supported Loss Functions
 103
 104 NNTrainer provides
 105
 106  | Keyword | Class Name | Description |
 107  |:-------:|:---:|:---:|
 108  | cross_sigmoid | CrossEntropySigmoidLossLayer | Cross entropy sigmoid loss layer |
 109  | cross_softmax | CrossEntropySoftmaxLossLayer | Cross entropy softmax loss layer |
 110  | constant_derivative | ConstantDerivativeLossLayer | Constant derivative loss layer |
 111  | mse | MSELossLayer | Mean square error loss layer |
 112
 113 ### Supported Activation Functions
 114
 115 NNTrainer provides
 116
 117  | Keyword | Loss Name | Description |
 118  |:-------:|:---:|:---|
 119  | tanh | tanh function | set as layer property |
 120  | sigmoid | sigmoid function | set as layer property |
 121  | relu | relu function | set as layer propery |
 122  | softmax | softmax function | set as layer propery |
 123  | weight_initializer | Weight Initialization | Xavier(Normal/Uniform), LeCun(Normal/Uniform),  HE(Normal/Unifor) |
 124  | weight_regularizer | weight decay ( L2Norm only ) | needs set weight_regularizer_param & type |
 125  | learnig_rate_decay | learning rate decay | need to set step |
 126
 127 ### Tensor
 128
 129 Tensor is responsible for calculation of a layer. It executes several operations such as addition, division, multiplication, dot production, data averaging and so on. In order to accelerate  calculation speed, CBLAS (C-Basic Linear Algebra: CPU) and CUBLAS (CUDA: Basic Linear Algebra) for PC (Especially NVIDIA GPU) are implemented for some of the operations. Later, these calculations will be optimized.
 130 Currently, we supports lazy calculation mode to reduce complexity for copying tensors during calculations.
 131
 132  | Keyword | Description |
 133  |:-------:|:---:|
 134  | 4D Tensor | B, C, H, W|
 135  | Add/sub/mul/div | - |
 136  | sum, average, argmax | - |
 137  | Dot, Transpose | - |
 138  | normalization, standardization | - |
 139  | save, read | - |
 140
 141 ### Others
 142
 143 NNTrainer provides
 144
 145  | Keyword | Loss Name | Description |
 146  |:-------:|:---:|:---|
 147  | weight_initializer | Weight Initialization | Xavier(Normal/Uniform), LeCun(Normal/Uniform),  HE(Normal/Unifor) |
 148  | weight_regularizer | weight decay ( L2Norm only ) | needs set weight_regularizer_constant & type |
 149  | learnig_rate_decay | learning rate decay | need to set step |
 150
 151 ### APIs
 152 Currently, we provide [C APIs](https://github.com/nnstreamer/nntrainer/blob/master/api/capi/include/nntrainer.h) for Tizen. C++ API will be also provided soon.
 153
 154
 155 ### Examples for NNTrainer
 156
 157 #### [Custom Shortcut Application](https://github.com/nnstreamer/nntrainer/tree/main/Applications/Tizen_native/CustomShortcut)
 158
 159
 160 A demo application which enable user defined custom shortcut on galaxy watch.
 161
 162 #### [MNIST Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/MNIST)
 163
 164 An example to train mnist dataset. It consists two convolution 2d layer, 2 pooling 2d layer, flatten layer and fully connected layer.
 165
 166 #### [Reinforcement Learning Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/ReinforcementLearning/DeepQ)
 167
 168 A reinforcement learning example with cartpole game. It is using DeepQ algorithm.
 169
 170 #### [Transfer Learning Examples](https://github.com/nnstreamer/nntrainer/tree/main/Applications/TransferLearning)
 171
 172 Transfer learning examples with for image classification using the Cifar 10 dataset and for OCR. TFlite is used for feature extractor and modify last layer (fully connected layer) of network.
 173
 174 #### [ResNet Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/Resnet)
 175
 176 An example to train resnet18 network.
 177
 178 #### [VGG Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/VGG)
 179
 180 An example to train vgg16 network.
 181
 182 #### [ProductRating Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/ProductRatings)
 183
 184 This application contains a simple embedding-based model that predicts ratings given a user and a product.
 185
 186 #### [SimpleShot Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/SimpleShot)
 187
 188 An example to demonstrate few-shot learning : SimpleShot
 189
 190 #### [Custom Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/Custom)
 191
 192 An example to demonstrate how to create custom layers, optimizers or other supported objects.
 193
 194 #### ~Tizen CAPI Example~
 195
 196 An example to demonstrate c api for Tizen. It is same transfer learing but written with tizen c api.~
 197 Deleted instead moved to a [test](https://github.com/nnstreamer/nntrainer/blob/master/test/tizen_capi/unittest_tizen_capi.cpp)
 198
 199 #### [KNN Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/KNN)
 200
 201 A transfer learning example with for image classification using the Cifar 10 dataset. TFlite is used for feature extractor and compared with KNN.
 202
 203 #### [Logistic Regression Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/LogisticRegression)
 204
 205 A logistic regression example using NNTrainer.
 206
 207 ## [Getting Started](https://github.com/nnstreamer/nntrainer/blob/main/docs/getting-started.md)
 208
 209 Instructions for installing NNTrainer.
 210
 211 ### [Running Examples](https://github.com/nnstreamer/nntrainer/blob/main/docs/how-to-run-examples.md)
 212
 213 Instructions for preparing NNTrainer for execution
 214
 215 ## Open Source License
 216
 217 The nntrainer is an open source project released under the terms of the Apache License version 2.0.
 218
 219 ## Contributing
 220
 221 Contributions are welcome! Please see our [Contributing](https://github.com/nnstreamer/nntrainer/blob/main/docs/contributing.md) Guide for more details.
 222
 223 [![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/0)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/0)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/1)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/1)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/2)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/2)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/3)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/3)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/4)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/4)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/5)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/5)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/6)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/6)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/7)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/7)