README.md

   1 # NNtrainer
   2
   3 [![Code Coverage](http://nnstreamer.mooo.com/nntrainer/ci/badge/codecoverage.svg)](http://nnstreamer.mooo.com/nntrainer/ci/gcov_html/index.html)
   4 ![GitHub repo size](https://img.shields.io/github/repo-size/nnstreamer/nntrainer)
   5 ![GitHub issues](https://img.shields.io/github/issues/nnstreamer/nntrainer)
   6 ![GitHub pull requests](https://img.shields.io/github/issues-pr/nnstreamer/nntrainer)
   7 <a href="https://scan.coverity.com/projects/nnstreamer-nntrainer">
   8   <img alt="Coverity Scan Build Status"
   9        src="https://scan.coverity.com/projects/22512/badge.svg"/>
  10 </a>
  11 [![DailyBuild](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/badge/daily_build_test_result_badge.svg)](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/build_result/)
  12
  13 NNtrainer is a Software Framework for training Neural Network models on devices.
  14
  15 ## Overview
  16
  17 NNtrainer is an Open Source Project. The aim of the NNtrainer is to develop a Software Framework to train neural network models on embedded devices which have relatively limited resources. Rather than training whole layers of a network from the scratch, NNtrainer finetunes the neural network model on device with user data for the personalization.
  18
  19 Even if NNtariner runs on device, it provides full functionalities to train models and also utilizes limited device resources efficiently. NNTrainer is able to train various machine learning algorithms such as k-Nearest Neighbor (k-NN), Neural Networks, Logistic Regression, Reinforcement Learning algorithms, Recurrent network and more. We also provide examples for various tasks such as Few-shot learning, ResNet, VGG, Product Rating and more will be added. All of these were tested on Samsung Galaxy smart phone with Android and PC (Ubuntu 18.04/20.04).
  20
  21 [ NNTrainer: Towards the on-device learning for personalization ](https://www.youtube.com/watch?v=HWiV7WbIM3E), Samsung Software Developer Conference 2021 (Korean) <br />
  22 [ NNTrainer: Personalize neural networks on devices! ](https://www.youtube.com/watch?v=HKKowY78P1A), Samsung Developer Conference 2021 <br />
  23 [ NNTrainer: "On-device learning" ](https://www.youtube.com/embed/Jy_auavraKg?start=4035&end=4080), Samsung AI Forum 2021
  24
  25 ## Official Releases
  26
  27 |     | [Tizen](http://download.tizen.org/snapshots/tizen/unified/latest/repos/standard/packages/) | [Ubuntu](https://launchpad.net/~nnstreamer/+archive/ubuntu/ppa) | Android/NDK Build |
  28 | :-- | :--: | :--: | :--: |
  29 |     | 6.0M2 and later | 18.04 | 9/P |
  30 | arm | [![armv7l badge](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/badge/tizen.armv7l_result_badge.svg)](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/build_result/) | Available  | Ready |
  31 | arm64 |  [![aarch64 badge](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/badge/tizen.aarch64_result_badge.svg)](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/build_result/) | Available  | [![android badge](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/badge/arm64_v8a_android_result_badge.svg)](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/build_result/) |
  32 | x64 | [![x64 badge](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/badge/tizen.x86_64_result_badge.svg)](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/build_result/)  | [![ubuntu badge](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/badge/ubuntu_result_badge.svg)](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/build_result/)  | Ready  |
  33 | x86 | [![x86 badge](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/badge/tizen.i586_result_badge.svg)](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/build_result/)  | N/A  | N/A  |
  34 | Publish | [Tizen Repo](http://download.tizen.org/snapshots/tizen/unified/latest/repos/standard/packages/) | [PPA](https://launchpad.net/~nnstreamer/+archive/ubuntu/ppa) |   |
  35 | API | C (Official) | C/C++ | C/C++  |
  36
  37 - Ready: CI system ensures build-ability and unit-testing. Users may easily build and execute. However, we do not have automated release & deployment system for this instance.
  38 - Available: binary packages are released and deployed automatically and periodically along with CI tests.
  39 - [Daily Release](http://nnstreamer.mooo.com/nntrainer/ci/daily-build/build_result/)
  40 - SDK Support: Tizen Studio (6.0 M2+)
  41
  42 ## Maintainer
  43 * [Jijoong Moon](https://github.com/jijoongmoon)
  44 * [MyungJoo Ham](https://github.com/myungjoo)
  45 * [Geunsik Lim](https://github.com/leemgs)
  46
  47 ## Reviewers
  48 * [Sangjung Woo](https://github.com/again4you)
  49 * [Wook Song](https://github.com/wooksong)
  50 * [Jaeyun Jung](https://github.com/jaeyun-jung)
  51 * [Hyoungjoo Ahn](https://github.com/helloahn)
  52 * [Parichay Kapoor](https://github.com/kparichay)
  53 * [Dongju Chae](https://github.com/dongju-chae)
  54 * [Gichan Jang](https://github.com/gichan-jang)
  55 * [Yongjoo Ahn](https://github.com/anyj0527)
  56 * [Jihoon Lee](https://github.com/zhoonit)
  57 * [Hyeonseok Lee](https://github.com/lhs8928)
  58 * [Mete Ozay](https://github.com/meteozay)
  59 * [Hyunil Park](https://github.com/songgot)
  60 * [Jiho Chu](https://github.com/jihochu)
  61
  62 ## Components
  63
  64 ### Supported Layers
  65
  66 This component defines layers which consist of a neural network model. Layers have their own properties to be set.
  67
  68  | Keyword | Layer Class Name | Description |
  69  |:-------:|:---:|:---|
  70  | conv1d | Conv1DLayer | Convolution 1-Dimentional Layer |
  71  | conv2d | Conv2DLayer |Convolution 2-Dimentional Layer |
  72  | pooling2d | Pooling2DLayer |Pooling 2-Dimentional Layer. Support average / max / global average / global max pooling |
  73  | flatten | FlattenLayer | Flatten layer |
  74  | fully_connected | FullyConnectedLayer | Fully connected layer |
  75  | input | InputLayer | Input Layer.  This is not always required. |
  76  | batch_normalization | BatchNormalizationLayer | Batch normalization layer |
  77  | activation | ActivaitonLayer | Set by layer property |
  78  | addition | AdditionLayer | Add input input layers |
  79  | attention | AttentionLayer | Attenstion layer |
  80  | centroid_knn | CentroidKNN | Centroid K-nearest neighbor layer |
  81  | concat | ConcatLayer | Concatenate input layers |
  82  | multiout | MultiOutLayer | Multi-Output Layer |
  83  | backbone_nnstreamer | NNStreamerLayer | Encapsulate NNStreamer layer |
  84  | backbone_tflite | TfLiteLayer | Encapsulate tflite as an layer |
  85  | permute | PermuteLayer | Permute layer for transpose |
  86  | preprocess_flip | PreprocessFlipLayer | Preprocess random flip layer |
  87  | preprocess_l2norm | PreprocessL2NormLayer | Preprocess simple l2norm layer to normalize |
  88  | preprocess_translate | PreprocessTranslateLayer | Preprocess translate layer |
  89  | reshape | ReshapeLayer | Reshape tensor dimension layer |
  90  | split | SplitLayer | Split layer |
  91  | dropout | DropOutLayer | Dropout Layer |
  92  | embedding | EmbeddingLayer | Embedding Layer |
  93  | rnn | RNNLayer | Recurrent Layer |
  94  | gru | GRULayer | Gated Recurrent Unit Layer |
  95  | lstm | LSTMLayer | Long Short-Term Memory Layer |
  96  | lstmcell | LSTMCellLayer | Long Short-Term Memory Cell Layer |
  97  | time_dist | TimeDistLayer | Time distributed Layer |
  98
  99 ### Supported Optimizers
 100
 101 NNTrainer Provides
 102
 103  | Keyword | Optimizer Name | Description |
 104  |:-------:|:---:|:---:|
 105  | sgd | Stochastic Gradient Decent | - |
 106  | adam | Adaptive Moment Estimation | - |
 107
 108 ### Supported Loss Functions
 109
 110 NNTrainer provides
 111
 112  | Keyword | Class Name | Description |
 113  |:-------:|:---:|:---:|
 114  | cross_sigmoid | CrossEntropySigmoidLossLayer | Cross entropy sigmoid loss layer |
 115  | cross_softmax | CrossEntropySoftmaxLossLayer | Cross entropy softmax loss layer |
 116  | constant_derivative | ConstantDerivativeLossLayer | Constant derivative loss layer |
 117  | mse | MSELossLayer | Mean square error loss layer |
 118
 119 ### Supported Activation Functions
 120
 121 NNTrainer provides
 122
 123  | Keyword | Loss Name | Description |
 124  |:-------:|:---:|:---|
 125  | tanh | tanh function | set as layer property |
 126  | sigmoid | sigmoid function | set as layer property |
 127  | relu | relu function | set as layer propery |
 128  | softmax | softmax function | set as layer propery |
 129  | weight_initializer | Weight Initialization | Xavier(Normal/Uniform), LeCun(Normal/Uniform),  HE(Normal/Unifor) |
 130  | weight_regularizer | weight decay ( L2Norm only ) | needs set weight_regularizer_param & type |
 131  | learnig_rate_decay | learning rate decay | need to set step |
 132
 133 ### Tensor
 134
 135 Tensor is responsible for calculation of a layer. It executes several operations such as addition, division, multiplication, dot production, data averaging and so on. In order to accelerate  calculation speed, CBLAS (C-Basic Linear Algebra: CPU) and CUBLAS (CUDA: Basic Linear Algebra) for PC (Especially NVIDIA GPU) are implemented for some of the operations. Later, these calculations will be optimized.
 136 Currently, we supports lazy calculation mode to reduce complexity for copying tensors during calculations.
 137
 138  | Keyword | Description |
 139  |:-------:|:---:|
 140  | 4D Tensor | B, C, H, W|
 141  | Add/sub/mul/div | - |
 142  | sum, average, argmax | - |
 143  | Dot, Transpose | - |
 144  | normalization, standardization | - |
 145  | save, read | - |
 146
 147 ### Others
 148
 149 NNTrainer provides
 150
 151  | Keyword | Loss Name | Description |
 152  |:-------:|:---:|:---|
 153  | weight_initializer | Weight Initialization | Xavier(Normal/Uniform), LeCun(Normal/Uniform),  HE(Normal/Unifor) |
 154  | weight_regularizer | weight decay ( L2Norm only ) | needs set weight_regularizer_constant & type |
 155  | learnig_rate_decay | learning rate decay | need to set step |
 156
 157 ### APIs
 158 Currently, we provide [C APIs](https://github.com/nnstreamer/nntrainer/blob/master/api/capi/include/nntrainer.h) for Tizen. [C++ APIs](https://github.com/nnstreamer/nntrainer/blob/master/api/ccapi/include) are also provided for other platform. Java & C# APIs will be provided soon.
 159
 160
 161 ### Examples for NNTrainer
 162
 163 #### [Custom Shortcut Application](https://github.com/nnstreamer/nntrainer/tree/main/Applications/Tizen_native/CustomShortcut)
 164
 165
 166 A demo application which enable user defined custom shortcut on galaxy watch.
 167
 168 #### [MNIST Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/MNIST)
 169
 170 An example to train mnist dataset. It consists two convolution 2d layer, 2 pooling 2d layer, flatten layer and fully connected layer.
 171
 172 #### [Reinforcement Learning Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/ReinforcementLearning/DeepQ)
 173
 174 A reinforcement learning example with cartpole game. It is using DeepQ algorithm.
 175
 176 #### [Transfer Learning Examples](https://github.com/nnstreamer/nntrainer/tree/main/Applications/TransferLearning)
 177
 178 Transfer learning examples with for image classification using the Cifar 10 dataset and for OCR. TFlite is used for feature extractor and modify last layer (fully connected layer) of network.
 179
 180 #### [ResNet Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/Resnet)
 181
 182 An example to train resnet18 network.
 183
 184 #### [VGG Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/VGG)
 185
 186 An example to train vgg16 network.
 187
 188 #### [ProductRating Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/ProductRatings)
 189
 190 This application contains a simple embedding-based model that predicts ratings given a user and a product.
 191
 192 #### [SimpleShot Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/SimpleShot)
 193
 194 An example to demonstrate few-shot learning : SimpleShot
 195
 196 #### [Custom Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/Custom)
 197
 198 An example to demonstrate how to create custom layers, optimizers or other supported objects.
 199
 200 <!-- #### Tizen CAPI Example -->
 201
 202 <!-- An example to demonstrate c api for Tizen. It is same transfer learing but written with tizen c api.~ -->
 203 <!-- Deleted instead moved to a [test](https://github.com/nnstreamer/nntrainer/blob/master/test/tizen_capi/unittest_tizen_capi.cpp) -->
 204
 205 #### [KNN Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/KNN)
 206
 207 A transfer learning example with for image classification using the Cifar 10 dataset. TFlite is used for feature extractor and compared with KNN.
 208
 209 #### [Logistic Regression Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/LogisticRegression)
 210
 211 A logistic regression example using NNTrainer.
 212
 213 ## [Getting Started](https://github.com/nnstreamer/nntrainer/blob/main/docs/getting-started.md)
 214
 215 Instructions for installing NNTrainer.
 216
 217 ### [Running Examples](https://github.com/nnstreamer/nntrainer/blob/main/docs/how-to-run-examples.md)
 218
 219 Instructions for preparing NNTrainer for execution
 220
 221 ## Open Source License
 222
 223 The nntrainer is an open source project released under the terms of the Apache License version 2.0.
 224
 225 ## Contributing
 226
 227 Contributions are welcome! Please see our [Contributing](https://github.com/nnstreamer/nntrainer/blob/main/docs/contributing.md) Guide for more details.
 228
 229 [![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/0)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/0)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/1)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/1)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/2)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/2)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/3)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/3)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/4)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/4)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/5)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/5)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/6)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/6)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/7)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/7)