# NNtrainer
-[](http://nnsuite.mooo.com/nntrainer/ci/gcov_html/index.html)
+[](http://ci.nnstreamer.ai/nntrainer/ci/gcov_html/index.html)
<img alt="Coverity Scan Build Status"
src="https://scan.coverity.com/projects/22512/badge.svg"/>
</a>
-[](http://nnsuite.mooo.com/nntrainer/ci/daily-build/build_result/)
+[](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/)
NNtrainer is a Software Framework for training Neural Network models on devices.
## Overview
-NNtrainer is an Open Source Project. The aim of the NNtrainer is to develop a Software Framework to train neural network models on embedded devices which have relatively limited resources. Rather than training whole layers of a network, NNtrainer trains only one or a few layers of the layers added after a feature extractor.
+NNtrainer is an Open Source Project. The aim of the NNtrainer is to develop a Software Framework to train neural network models on embedded devices which have relatively limited resources. Rather than training whole layers of a network from the scratch, NNtrainer finetunes the neural network model on device with user data for the personalization.
-Even though NNTrainer can be used to train sub-models, it requires implementation of additional functionalities to train models obtained from other machine learning and deep learning libraries. In the current version, various machine learning algorithms such as k-Nearest Neighbor (k-NN), Neural Networks, Logistic Regression and Reinforcement Learning algorithms are implemented. We also provide examples for various tasks such as transfer learning of models. In some of these examples, deep learning models such as Mobilenet V2 trained with Tensorflow-lite, are used as feature extractors. All of these were tested on Galaxy S8 with Android and PC (Ubuntu 16.04).
+Even if NNtariner runs on device, it provides full functionalities to train models and also utilizes limited device resources efficiently. NNTrainer is able to train various machine learning algorithms such as k-Nearest Neighbor (k-NN), Neural Networks, Logistic Regression, Reinforcement Learning algorithms, Recurrent network and more. We also provide examples for various tasks such as Few-shot learning, ResNet, VGG, Product Rating and more will be added. All of these were tested on Samsung Galaxy smart phone with Android and PC (Ubuntu 18.04/20.04).
+
+[ NNTrainer: Light-Weight On-Device Training Framework ](https://arxiv.org/pdf/2206.04688.pdf), arXiv, 2022 <br />
+[ NNTrainer: Towards the on-device learning for personalization ](https://www.youtube.com/watch?v=HWiV7WbIM3E), Samsung Software Developer Conference 2021 (Korean) <br />
+[ NNTrainer: Personalize neural networks on devices! ](https://www.youtube.com/watch?v=HKKowY78P1A), Samsung Developer Conference 2021 <br />
+[ NNTrainer: "On-device learning" ](https://www.youtube.com/embed/Jy_auavraKg?start=4035&end=4080), Samsung AI Forum 2021
## Official Releases
| | [Tizen](http://download.tizen.org/snapshots/tizen/unified/latest/repos/standard/packages/) | [Ubuntu](https://launchpad.net/~nnstreamer/+archive/ubuntu/ppa) | Android/NDK Build |
| :-- | :--: | :--: | :--: |
| | 6.0M2 and later | 18.04 | 9/P |
-| arm | [](http://nnsuite.mooo.com/nntrainer/ci/daily-build/build_result/) | Available | [](http://nnsuite.mooo.com/nntrainer/ci/daily-build/build_result/) |
-| arm64 | [](http://nnsuite.mooo.com/nntrainer/ci/daily-build/build_result/) | Available | [](http://nnsuite.mooo.com/nntrainer/ci/daily-build/build_result/) |
-| x64 | [](http://nnsuite.mooo.com/nntrainer/ci/daily-build/build_result/) | [](http://nnsuite.mooo.com/nntrainer/ci/daily-build/build_result/) | Ready |
-| x86 | [](http://nnsuite.mooo.com/nntrainer/ci/daily-build/build_result/) | N/A | N/A |
+| arm | [](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/) | Available | Ready |
+| arm64 | [](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/) | Available | [](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/) |
+| x64 | [](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/) | [](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/) | Ready |
+| x86 | [](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/) | N/A | N/A |
| Publish | [Tizen Repo](http://download.tizen.org/snapshots/tizen/unified/latest/repos/standard/packages/) | [PPA](https://launchpad.net/~nnstreamer/+archive/ubuntu/ppa) | |
| API | C (Official) | C/C++ | C/C++ |
- Ready: CI system ensures build-ability and unit-testing. Users may easily build and execute. However, we do not have automated release & deployment system for this instance.
- Available: binary packages are released and deployed automatically and periodically along with CI tests.
-- [Daily Release](http://nnsuite.mooo.com/nntrainer/ci/daily-build/build_result/)
+- [Daily Release](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/)
- SDK Support: Tizen Studio (6.0 M2+)
## Maintainer
* [Jihoon Lee](https://github.com/zhoonit)
* [Hyeonseok Lee](https://github.com/lhs8928)
* [Mete Ozay](https://github.com/meteozay)
+* [Hyunil Park](https://github.com/songgot)
+* [Jiho Chu](https://github.com/jihochu)
+* [Yelin Jeong](https://github.com/niley7464)
+* [Donghak Park](https://github.com/DonghakPark)
+
## Components
This component defines layers which consist of a neural network model. Layers have their own properties to be set.
- | Keyword | Layer Name | Description |
+ | Keyword | Layer Class Name | Description |
|:-------:|:---:|:---|
- | conv2d | Convolution 2D |Convolution 2-Dimentional Layer |
- | pooling2d | Pooling 2D |Pooling 2-Dimentional Layer. Support average / max / global average / global max pooling |
- | flatten | Flatten | Flatten Layer |
- | fully_connected | Fully Connected | Fully Connected Layer |
- | input | Input | Input Layer. This is not always required. |
- | batch_normalization | Batch Normalization Layer | Batch Normalization Layer. |
- | loss layer | loss layer | hidden from users |
- | activation | activaiton layer | set by layer property |
+ | conv1d | Conv1DLayer | Convolution 1-Dimentional Layer |
+ | conv2d | Conv2DLayer |Convolution 2-Dimentional Layer |
+ | pooling2d | Pooling2DLayer |Pooling 2-Dimentional Layer. Support average / max / global average / global max pooling |
+ | flatten | FlattenLayer | Flatten layer |
+ | fully_connected | FullyConnectedLayer | Fully connected layer |
+ | pooling2D | Pooling2DLayer | Pooling 2D layer |
+ | input | InputLayer | Input Layer. This is not always required. |
+ | batch_normalization | BatchNormalizationLayer | Batch normalization layer |
+ | layer_normalization | LayerNormalizationLayer | Layer normalization layer |
+ | activation | ActivaitonLayer | Set by layer property |
+ | addition | AdditionLayer | Add input input layers |
+ | attention | AttentionLayer | Attenstion layer |
+ | centroid_knn | CentroidKNN | Centroid K-nearest neighbor layer |
+ | concat | ConcatLayer | Concatenate input layers |
+ | multiout | MultiOutLayer | Multi-Output Layer |
+ | backbone_nnstreamer | NNStreamerLayer | Encapsulate NNStreamer layer |
+ | backbone_tflite | TfLiteLayer | Encapsulate tflite as a layer |
+ | permute | PermuteLayer | Permute layer for transpose |
+ | preprocess_flip | PreprocessFlipLayer | Preprocess random flip layer |
+ | preprocess_l2norm | PreprocessL2NormLayer | Preprocess simple l2norm layer to normalize |
+ | preprocess_translate | PreprocessTranslateLayer | Preprocess translate layer |
+ | reshape | ReshapeLayer | Reshape tensor dimension layer |
+ | split | SplitLayer | Split layer |
+ | dropout | DropOutLayer | Dropout Layer |
+ | embedding | EmbeddingLayer | Embedding Layer |
+ | positional_encoding | PositionalEncodingLayer | Positional Encoding Layer |
+ | rnn | RNNLayer | Recurrent Layer |
+ | rnncell | RNNCellLayer | Recurrent Cell Layer |
+ | gru | GRULayer | Gated Recurrent Unit Layer |
+ | grucell | GRUCellLayer | Gated Recurrent Unit Cell Layer |
+ | lstm | LSTMLayer | Long Short-Term Memory Layer |
+ | lstmcell | LSTMCellLayer | Long Short-Term Memory Cell Layer |
+ | zoneoutlstmcell | ZoneoutLSTMCellLayer | Zoneout Long Short-Term Memory Cell Layer |
+ | time_dist | TimeDistLayer | Time distributed Layer |
+ | multi_head_attention | MultiHeadAttentionLayer | Multi Head Attention Layer |
+
### Supported Optimizers
| sgd | Stochastic Gradient Decent | - |
| adam | Adaptive Moment Estimation | - |
+ | Keyword | Leanring Rate | Description |
+ |:-------:|:---:|:---:|
+ | exponential | exponential learning rate decay | - |
+ | constant | constant learning rate | - |
+ | step | step learning rate | - |
+
### Supported Loss Functions
NNTrainer provides
- | Keyword | Loss Name | Description |
+ | Keyword | Class Name | Description |
|:-------:|:---:|:---:|
- | mse | Mean squared Error | - |
- | cross | Cross Entropy - sigmoid | if activation last layer is sigmoid |
- | cross | Cross Entropy - softmax | if activation last layer is softmax |
+ | cross_sigmoid | CrossEntropySigmoidLossLayer | Cross entropy sigmoid loss layer |
+ | cross_softmax | CrossEntropySoftmaxLossLayer | Cross entropy softmax loss layer |
+ | constant_derivative | ConstantDerivativeLossLayer | Constant derivative loss layer |
+ | mse | MSELossLayer | Mean square error loss layer |
+ | kld | KLDLossLayer | Kullback-Leibler Divergence loss layer |
### Supported Activation Functions
| sigmoid | sigmoid function | set as layer property |
| relu | relu function | set as layer propery |
| softmax | softmax function | set as layer propery |
- | weight_initializer | Weight Initialization | Xavier(Normal/Uniform), LeCun(Normal/Uniform), HE(Normal/Unifor) |
- | weight_regularizer | weight decay ( L2Norm only ) | needs set weight_regularizer_param & type |
- | learnig_rate_decay | learning rate decay | need to set step |
### Tensor
| Keyword | Loss Name | Description |
|:-------:|:---:|:---|
| weight_initializer | Weight Initialization | Xavier(Normal/Uniform), LeCun(Normal/Uniform), HE(Normal/Unifor) |
- | weight_regularizer | weight decay ( L2Norm only ) | needs set weight_regularizer_constant & type |
- | learnig_rate_decay | learning rate decay | need to set step |
+ | weight_regularizer | weight decay ( L2Norm only ) | needs set weight_regularizer_param & type |
### APIs
-Currently, we provide [C APIs](https://github.com/nnstreamer/nntrainer/blob/master/api/capi/include/nntrainer.h) for Tizen. C++ API will be also provided soon.
-
-
-### Examples for NNTrainer
-
-#### [Custom Shortcut Application](https://github.com/nnstreamer/nntrainer/tree/main/Applications/Tizen_native/CustomShortcut)
-
-
-A demo application which enable user defined custom shortcut on galaxy watch.
+Currently, we provide [C APIs](https://github.com/nnstreamer/nntrainer/blob/master/api/capi/include/nntrainer.h) for Tizen. [C++ APIs](https://github.com/nnstreamer/nntrainer/blob/master/api/ccapi/include) are also provided for other platform. Java & C# APIs will be provided soon.
-#### [MNIST Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/MNIST)
-
-An example to train mnist dataset. It consists two convolution 2d layer, 2 pooling 2d layer, flatten layer and fully connected layer.
-
-#### [Reinforcement Learning Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/ReinforcementLearning/DeepQ)
-
-A reinforcement learning example with cartpole game. It is using DeepQ algorithm.
-
-#### [Transfer Learning Examples](https://github.com/nnstreamer/nntrainer/tree/main/Applications/TransferLearning)
-
-Transfer learning examples with for image classification using the Cifar 10 dataset and for OCR. TFlite is used for feature extractor and modify last layer (fully connected layer) of network.
-
-#### ~Tizen CAPI Example~
-
-An example to demonstrate c api for Tizen. It is same transfer learing but written with tizen c api.~
-Deleted instead moved to a [test](https://github.com/nnstreamer/nntrainer/blob/master/test/tizen_capi/unittest_tizen_capi.cpp)
-
-#### [KNN Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/KNN)
-
-A transfer learning example with for image classification using the Cifar 10 dataset. TFlite is used for feature extractor and compared with KNN.
-
-#### [Logistic Regression Example](https://github.com/nnstreamer/nntrainer/tree/main/Applications/LogisticRegression)
-
-A logistic regression example using NNTrainer.
## [Getting Started](https://github.com/nnstreamer/nntrainer/blob/main/docs/getting-started.md)
Instructions for preparing NNTrainer for execution
+### [Examples for NNTrainer](https://github.com/nnstreamer/nntrainer/tree/main/Applications)
+
+NNTrainer example for a variety of networks
+
## Open Source License
-The nntrainer is an open source project released under the terms of the Apache License version 2.0.
+The NNtrainer is an open source project released under the terms of the Apache License version 2.0.
## Contributing
projects / platform / core / ml / nntrainer.git / blobdiff