README.md

   1 # NNtrainer
   2
   3 [![Code Coverage](http://ci.nnstreamer.ai/nntrainer/ci/badge/codecoverage.svg)](http://ci.nnstreamer.ai/nntrainer/ci/gcov_html/index.html)
   4 ![GitHub repo size](https://img.shields.io/github/repo-size/nnstreamer/nntrainer)
   5 ![GitHub issues](https://img.shields.io/github/issues/nnstreamer/nntrainer)
   6 ![GitHub pull requests](https://img.shields.io/github/issues-pr/nnstreamer/nntrainer)
   7 <a href="https://scan.coverity.com/projects/nnstreamer-nntrainer">
   8   <img alt="Coverity Scan Build Status"
   9        src="https://scan.coverity.com/projects/22512/badge.svg"/>
  10 </a>
  11 [![DailyBuild](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/badge/daily_build_test_result_badge.svg)](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/)
  12
  13 NNtrainer is a Software Framework for training Neural Network models on devices.
  14
  15 ## Overview
  16
  17 NNtrainer is an Open Source Project. The aim of the NNtrainer is to develop a Software Framework to train neural network models on embedded devices which have relatively limited resources. Rather than training whole layers of a network from the scratch, NNtrainer finetunes the neural network model on device with user data for the personalization.
  18
  19 Even if NNtariner runs on device, it provides full functionalities to train models and also utilizes limited device resources efficiently. NNTrainer is able to train various machine learning algorithms such as k-Nearest Neighbor (k-NN), Neural Networks, Logistic Regression, Reinforcement Learning algorithms, Recurrent network and more. We also provide examples for various tasks such as Few-shot learning, ResNet, VGG, Product Rating and more will be added. All of these were tested on Samsung Galaxy smart phone with Android and PC (Ubuntu 18.04/20.04).
  20
  21 [ NNTrainer: Light-Weight On-Device Training Framework ](https://arxiv.org/pdf/2206.04688.pdf), arXiv, 2022 <br />
  22 [ NNTrainer: Towards the on-device learning for personalization ](https://www.youtube.com/watch?v=HWiV7WbIM3E), Samsung Software Developer Conference 2021 (Korean) <br />
  23 [ NNTrainer: Personalize neural networks on devices! ](https://www.youtube.com/watch?v=HKKowY78P1A), Samsung Developer Conference 2021 <br />
  24 [ NNTrainer: "On-device learning" ](https://www.youtube.com/embed/Jy_auavraKg?start=4035&end=4080), Samsung AI Forum 2021
  25
  26 ## Official Releases
  27
  28 |     | [Tizen](http://download.tizen.org/snapshots/tizen/unified/latest/repos/standard/packages/) | [Ubuntu](https://launchpad.net/~nnstreamer/+archive/ubuntu/ppa) | Android/NDK Build |
  29 | :-- | :--: | :--: | :--: |
  30 |     | 6.0M2 and later | 18.04 | 9/P |
  31 | arm | [![armv7l badge](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/badge/tizen.armv7l_result_badge.svg)](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/) | Available  | Ready |
  32 | arm64 |  [![aarch64 badge](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/badge/tizen.aarch64_result_badge.svg)](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/) | Available  | [![android badge](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/badge/arm64_v8a_android_result_badge.svg)](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/) |
  33 | x64 | [![x64 badge](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/badge/tizen.x86_64_result_badge.svg)](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/)  | [![ubuntu badge](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/badge/ubuntu_result_badge.svg)](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/)  | Ready  |
  34 | x86 | [![x86 badge](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/badge/tizen.i586_result_badge.svg)](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/)  | N/A  | N/A  |
  35 | Publish | [Tizen Repo](http://download.tizen.org/snapshots/tizen/unified/latest/repos/standard/packages/) | [PPA](https://launchpad.net/~nnstreamer/+archive/ubuntu/ppa) |   |
  36 | API | C (Official) | C/C++ | C/C++  |
  37
  38 - Ready: CI system ensures build-ability and unit-testing. Users may easily build and execute. However, we do not have automated release & deployment system for this instance.
  39 - Available: binary packages are released and deployed automatically and periodically along with CI tests.
  40 - [Daily Release](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/)
  41 - SDK Support: Tizen Studio (6.0 M2+)
  42
  43 ## Maintainer
  44 * [Jijoong Moon](https://github.com/jijoongmoon)
  45 * [MyungJoo Ham](https://github.com/myungjoo)
  46 * [Geunsik Lim](https://github.com/leemgs)
  47
  48 ## Reviewers
  49 * [Sangjung Woo](https://github.com/again4you)
  50 * [Wook Song](https://github.com/wooksong)
  51 * [Jaeyun Jung](https://github.com/jaeyun-jung)
  52 * [Hyoungjoo Ahn](https://github.com/helloahn)
  53 * [Parichay Kapoor](https://github.com/kparichay)
  54 * [Dongju Chae](https://github.com/dongju-chae)
  55 * [Gichan Jang](https://github.com/gichan-jang)
  56 * [Yongjoo Ahn](https://github.com/anyj0527)
  57 * [Jihoon Lee](https://github.com/zhoonit)
  58 * [Hyeonseok Lee](https://github.com/lhs8928)
  59 * [Mete Ozay](https://github.com/meteozay)
  60 * [Hyunil Park](https://github.com/songgot)
  61 * [Jiho Chu](https://github.com/jihochu)
  62 * [Yelin Jeong](https://github.com/niley7464)
  63 * [Donghak Park](https://github.com/DonghakPark)
  64 * [Hyungjun Seo](https://github.com/SeoHyungjun)
  65 * [Seungbaek Hong](https://github.com/baek2sm)
  66
  67
  68 ## Components
  69
  70 ### Supported Layers
  71
  72 This component defines layers which consist of a neural network model. Layers have their own properties to be set.
  73
  74  | Keyword | Layer Class Name | Description |
  75  |:-------:|:---:|:---|
  76  | conv1d | Conv1DLayer | Convolution 1-Dimentional Layer |
  77  | conv2d | Conv2DLayer |Convolution 2-Dimentional Layer |
  78  | pooling2d | Pooling2DLayer |Pooling 2-Dimentional Layer. Support average / max / global average / global max pooling |
  79  | flatten | FlattenLayer | Flatten layer |
  80  | fully_connected | FullyConnectedLayer | Fully connected layer |
  81  | input | InputLayer | Input Layer.  This is not always required. |
  82  | batch_normalization | BatchNormalizationLayer | Batch normalization layer |
  83  | layer_normalization | LayerNormalizationLayer | Layer normalization layer |
  84  | activation | ActivationLayer | Set by layer property |
  85  | addition | AdditionLayer | Add input input layers |
  86  | attention | AttentionLayer | Attenstion layer |
  87  | centroid_knn | CentroidKNN | Centroid K-nearest neighbor layer |
  88  | concat | ConcatLayer | Concatenate input layers |
  89  | multiout | MultiOutLayer | Multi-Output Layer |
  90  | backbone_nnstreamer | NNStreamerLayer | Encapsulate NNStreamer layer |
  91  | backbone_tflite | TfLiteLayer | Encapsulate tflite as a layer |
  92  | permute | PermuteLayer | Permute layer for transpose |
  93  | preprocess_flip | PreprocessFlipLayer | Preprocess random flip layer |
  94  | preprocess_l2norm | PreprocessL2NormLayer | Preprocess simple l2norm layer to normalize |
  95  | preprocess_translate | PreprocessTranslateLayer | Preprocess translate layer |
  96  | reshape | ReshapeLayer | Reshape tensor dimension layer |
  97  | split | SplitLayer | Split layer |
  98  | dropout | DropOutLayer | Dropout Layer |
  99  | embedding | EmbeddingLayer | Embedding Layer |
 100  | positional_encoding | PositionalEncodingLayer | Positional Encoding Layer |
 101  | rnn | RNNLayer | Recurrent Layer |
 102  | rnncell | RNNCellLayer | Recurrent Cell Layer |
 103  | gru | GRULayer | Gated Recurrent Unit Layer |
 104  | grucell | GRUCellLayer | Gated Recurrent Unit Cell Layer |
 105  | lstm | LSTMLayer | Long Short-Term Memory Layer |
 106  | lstmcell | LSTMCellLayer | Long Short-Term Memory Cell Layer |
 107  | zoneoutlstmcell | ZoneoutLSTMCellLayer | Zoneout Long Short-Term Memory Cell Layer |
 108  | time_dist | TimeDistLayer | Time distributed Layer |
 109  | multi_head_attention | MultiHeadAttentionLayer | Multi Head Attention Layer |
 110
 111
 112 ### Supported Optimizers
 113
 114 NNTrainer Provides
 115
 116  | Keyword | Optimizer Name | Description |
 117  |:-------:|:---:|:---:|
 118  | sgd | Stochastic Gradient Decent | - |
 119  | adam | Adaptive Moment Estimation | - |
 120
 121  | Keyword | Leanring Rate | Description |
 122  |:-------:|:---:|:---:|
 123  | exponential | exponential learning rate decay | - |
 124  | constant | constant learning rate | - |
 125  | step | step learning rate | - |
 126
 127 ### Supported Loss Functions
 128
 129 NNTrainer provides
 130
 131  | Keyword | Class Name | Description |
 132  |:-------:|:---:|:---:|
 133  | cross_sigmoid | CrossEntropySigmoidLossLayer | Cross entropy sigmoid loss layer |
 134  | cross_softmax | CrossEntropySoftmaxLossLayer | Cross entropy softmax loss layer |
 135  | constant_derivative | ConstantDerivativeLossLayer | Constant derivative loss layer |
 136  | mse | MSELossLayer | Mean square error loss layer |
 137  | kld | KLDLossLayer | Kullback-Leibler Divergence loss layer |
 138
 139 ### Supported Activation Functions
 140
 141 NNTrainer provides
 142
 143  | Keyword | Loss Name | Description |
 144  |:-------:|:---:|:---|
 145  | tanh | tanh function | set as layer property |
 146  | sigmoid | sigmoid function | set as layer property |
 147  | relu | relu function | set as layer property |
 148  | softmax | softmax function | set as layer property |
 149
 150 ### Tensor
 151
 152 Tensor is responsible for calculation of a layer. It executes several operations such as addition, division, multiplication, dot production, data averaging and so on. In order to accelerate  calculation speed, CBLAS (C-Basic Linear Algebra: CPU) and CUBLAS (CUDA: Basic Linear Algebra) for PC (Especially NVIDIA GPU) are implemented for some of the operations. Later, these calculations will be optimized.
 153 Currently, we support lazy calculation mode to reduce complexity for copying tensors during calculations.
 154
 155  | Keyword | Description |
 156  |:-------:|:---:|
 157  | 4D Tensor | B, C, H, W|
 158  | Add/sub/mul/div | - |
 159  | sum, average, argmax | - |
 160  | Dot, Transpose | - |
 161  | normalization, standardization | - |
 162  | save, read | - |
 163
 164 ### Others
 165
 166 NNTrainer provides
 167
 168  | Keyword | Loss Name | Description |
 169  |:-------:|:---:|:---|
 170  | weight_initializer | Weight Initialization | Xavier(Normal/Uniform), LeCun(Normal/Uniform),  HE(Normal/Uniform) |
 171  | weight_regularizer | weight decay ( L2Norm only ) | needs set weight_regularizer_param & type |
 172
 173 ### APIs
 174 Currently, we provide [C APIs](https://github.com/nnstreamer/nntrainer/blob/master/api/capi/include/nntrainer.h) for Tizen. [C++ APIs](https://github.com/nnstreamer/nntrainer/blob/master/api/ccapi/include) are also provided for other platform. Java & C# APIs will be provided soon.
 175
 176
 177 ## [Getting Started](https://github.com/nnstreamer/nntrainer/blob/main/docs/getting-started.md)
 178
 179 Instructions for installing NNTrainer.
 180
 181 ### [Running Examples](https://github.com/nnstreamer/nntrainer/blob/main/docs/how-to-run-examples.md)
 182
 183 Instructions for preparing NNTrainer for execution
 184
 185 ### [Examples for NNTrainer](https://github.com/nnstreamer/nntrainer/tree/main/Applications)
 186
 187 NNTrainer example for a variety of networks
 188
 189 ## Open Source License
 190
 191 The NNtrainer is an open source project released under the terms of the Apache License version 2.0.
 192
 193 ## Contributing
 194
 195 Contributions are welcome! Please see our [Contributing](https://github.com/nnstreamer/nntrainer/blob/main/docs/contributing.md) Guide for more details.
 196
 197 [![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/0)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/0)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/1)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/1)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/2)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/2)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/3)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/3)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/4)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/4)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/5)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/5)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/6)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/6)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/images/7)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/7)