# NNtrainer
-[](http://ec2-54-180-96-14.ap-northeast-2.compute.amazonaws.com/nntrainer/ci/gcov_html/index.html)
-
-NNtrainer is Software Framework for Training Neural Network Models on Devices.
+[](http://ci.nnstreamer.ai/nntrainer/ci/gcov_html/index.html)
+
+
+
+<a href="https://scan.coverity.com/projects/nnstreamer-nntrainer">
+ <img alt="Coverity Scan Build Status"
+ src="https://scan.coverity.com/projects/22512/badge.svg"/>
+</a>
+[](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/)
+
+NNtrainer is a Software Framework for training Neural Network models on devices.
## Overview
-NNtrainer is an Open Source Project. The aim of the NNtrainer is to develop Software Framework to train neural network model on embedded devices which has relatively limited resources. Rather than training the whole layers, NNtrainer trains only one or a few layers added after the feature extractor.
+NNtrainer is an Open Source Project. The aim of the NNtrainer is to develop a Software Framework to train neural network models on embedded devices which have relatively limited resources. Rather than training whole layers of a network from the scratch, NNtrainer finetunes the neural network model on device with user data for the personalization.
+
+Even if NNtariner runs on device, it provides full functionalities to train models and also utilizes limited device resources efficiently. NNTrainer is able to train various machine learning algorithms such as k-Nearest Neighbor (k-NN), Neural Networks, Logistic Regression, Reinforcement Learning algorithms, Recurrent network and more. We also provide examples for various tasks such as Few-shot learning, ResNet, VGG, Product Rating and more will be added. All of these were tested on Samsung Galaxy smart phone with Android and PC (Ubuntu 18.04/20.04).
+
+[ NNTrainer: Light-Weight On-Device Training Framework ](https://arxiv.org/pdf/2206.04688.pdf), arXiv, 2022 <br />
+[ NNTrainer: Towards the on-device learning for personalization ](https://www.youtube.com/watch?v=HWiV7WbIM3E), Samsung Software Developer Conference 2021 (Korean) <br />
+[ NNTrainer: Personalize neural networks on devices! ](https://www.youtube.com/watch?v=HKKowY78P1A), Samsung Developer Conference 2021 <br />
+[ NNTrainer: "On-device learning" ](https://www.youtube.com/embed/Jy_auavraKg?start=4035&end=4080), Samsung AI Forum 2021
+
+## Official Releases
-Even though it trains part of the neural network models, NNtrainer requires quite a lot of functionalities to train from common neural network frameworks. By implementing them, it is good enough to run several examples which can help to understand how it works. There are KNN, Neural Network, Logistic Regression and Reinforcement Learning with CartPole in Applications directory and some of them use Mobilenet V2 with tensorflow-lite as a feature extractor. All of them tested on Galaxy S8 with Android and PC (Ubuntu 16.04).
+| | [Tizen](http://download.tizen.org/snapshots/tizen/unified/latest/repos/standard/packages/) | [Ubuntu](https://launchpad.net/~nnstreamer/+archive/ubuntu/ppa) | Android/NDK Build |
+| :-- | :--: | :--: | :--: |
+| | 6.0M2 and later | 18.04 | 9/P |
+| arm | [](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/) | Available | Ready |
+| arm64 | [](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/) | Available | [](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/) |
+| x64 | [](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/) | [](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/) | Ready |
+| x86 | [](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/) | N/A | N/A |
+| Publish | [Tizen Repo](http://download.tizen.org/snapshots/tizen/unified/latest/repos/standard/packages/) | [PPA](https://launchpad.net/~nnstreamer/+archive/ubuntu/ppa) | |
+| API | C (Official) | C/C++ | C/C++ |
+
+- Ready: CI system ensures build-ability and unit-testing. Users may easily build and execute. However, we do not have automated release & deployment system for this instance.
+- Available: binary packages are released and deployed automatically and periodically along with CI tests.
+- [Daily Release](http://ci.nnstreamer.ai/nntrainer/ci/daily-build/build_result/)
+- SDK Support: Tizen Studio (6.0 M2+)
## Maintainer
* [Jijoong Moon](https://github.com/jijoongmoon)
* [Dongju Chae](https://github.com/dongju-chae)
* [Gichan Jang](https://github.com/gichan-jang)
* [Yongjoo Ahn](https://github.com/anyj0527)
+* [Jihoon Lee](https://github.com/zhoonit)
+* [Hyeonseok Lee](https://github.com/lhs8928)
+* [Mete Ozay](https://github.com/meteozay)
+* [Hyunil Park](https://github.com/songgot)
+* [Jiho Chu](https://github.com/jihochu)
+* [Yelin Jeong](https://github.com/niley7464)
+* [Donghak Park](https://github.com/DonghakPark)
-## Components
-
-### NeuralNetwork
-
-This is the component which controls neural network layers. Read the configuration file ([Iniparser](https://github.com/ndevilla/iniparser) is used to parse the configuration file.) and constructs Layers including Input and Output Layer, according to configured information by the user.
-The most important role of this component is to activate forward / backward propagation. It activates inferencing and training of each layer while handling the data properly among them. There are properties to describe network model as below:
-- **_Type:_** Network Type - Regression, KNN, NeuralNetwork
-- **_Layers:_** Name of Layers of Network Model
-- **_Learning\_rate:_** Learning rate which is used for all Layers
-- **_Decay\_rate:_** Rate for Exponentially Decayed Learning Rate
-- **_Epoch:_** Max Number of Training Iteration.
-- **_Optimizer:_** Optimizer for the Network Model - sgd, adam
-- **_Activation:_** Activation Function - sigmoid , tanh
-- **_Cost:_** Cost Function -
- msr(mean square root error), categorical (for logistic regression), cross (cross entropy)
-- **_Model:_** Name of Model. Weight Data is saved in the name of this.
-- **_minibach:_** mini batch size
-- **_beta1,beta2,epsilon:_** hyper parameters for the adam optimizer
-
-
-### Layers
-
-This component defines Layers which consist of Neural Network Model. Every neural network model must have one Input & Output Layer and other layers such as Fully Connected or Convolution can be added between them. (For now, supports Input & Output Layer, Fully Connected Layer.)
-
-- **_Type:_** InputLayer, OutputLayer, FullyConnectedLayer
-- **_Id:_** Index of Layer
-- **_Height:_** Height of Weight Data (Input Dimension)
-- **_Width:_** Width of Weight Data ( Hidden Layer Dimension)
-- **_Bias\_zero:_** Boolean for Enable/Disable Bias
+## Components
+### Supported Layers
+
+This component defines layers which consist of a neural network model. Layers have their own properties to be set.
+
+ | Keyword | Layer Class Name | Description |
+ |:-------:|:---:|:---|
+ | conv1d | Conv1DLayer | Convolution 1-Dimentional Layer |
+ | conv2d | Conv2DLayer |Convolution 2-Dimentional Layer |
+ | pooling2d | Pooling2DLayer |Pooling 2-Dimentional Layer. Support average / max / global average / global max pooling |
+ | flatten | FlattenLayer | Flatten layer |
+ | fully_connected | FullyConnectedLayer | Fully connected layer |
+ | pooling2D | Pooling2DLayer | Pooling 2D layer |
+ | input | InputLayer | Input Layer. This is not always required. |
+ | batch_normalization | BatchNormalizationLayer | Batch normalization layer |
+ | layer_normalization | LayerNormalizationLayer | Layer normalization layer |
+ | activation | ActivaitonLayer | Set by layer property |
+ | addition | AdditionLayer | Add input input layers |
+ | attention | AttentionLayer | Attenstion layer |
+ | centroid_knn | CentroidKNN | Centroid K-nearest neighbor layer |
+ | concat | ConcatLayer | Concatenate input layers |
+ | multiout | MultiOutLayer | Multi-Output Layer |
+ | backbone_nnstreamer | NNStreamerLayer | Encapsulate NNStreamer layer |
+ | backbone_tflite | TfLiteLayer | Encapsulate tflite as a layer |
+ | permute | PermuteLayer | Permute layer for transpose |
+ | preprocess_flip | PreprocessFlipLayer | Preprocess random flip layer |
+ | preprocess_l2norm | PreprocessL2NormLayer | Preprocess simple l2norm layer to normalize |
+ | preprocess_translate | PreprocessTranslateLayer | Preprocess translate layer |
+ | reshape | ReshapeLayer | Reshape tensor dimension layer |
+ | split | SplitLayer | Split layer |
+ | dropout | DropOutLayer | Dropout Layer |
+ | embedding | EmbeddingLayer | Embedding Layer |
+ | positional_encoding | PositionalEncodingLayer | Positional Encoding Layer |
+ | rnn | RNNLayer | Recurrent Layer |
+ | rnncell | RNNCellLayer | Recurrent Cell Layer |
+ | gru | GRULayer | Gated Recurrent Unit Layer |
+ | grucell | GRUCellLayer | Gated Recurrent Unit Cell Layer |
+ | lstm | LSTMLayer | Long Short-Term Memory Layer |
+ | lstmcell | LSTMCellLayer | Long Short-Term Memory Cell Layer |
+ | zoneoutlstmcell | ZoneoutLSTMCellLayer | Zoneout Long Short-Term Memory Cell Layer |
+ | time_dist | TimeDistLayer | Time distributed Layer |
+ | multi_head_attention | MultiHeadAttentionLayer | Multi Head Attention Layer |
+
+
+### Supported Optimizers
+
+NNTrainer Provides
+
+ | Keyword | Optimizer Name | Description |
+ |:-------:|:---:|:---:|
+ | sgd | Stochastic Gradient Decent | - |
+ | adam | Adaptive Moment Estimation | - |
+
+ | Keyword | Leanring Rate | Description |
+ |:-------:|:---:|:---:|
+ | exponential | exponential learning rate decay | - |
+ | constant | constant learning rate | - |
+ | step | step learning rate | - |
+
+### Supported Loss Functions
+
+NNTrainer provides
+
+ | Keyword | Class Name | Description |
+ |:-------:|:---:|:---:|
+ | cross_sigmoid | CrossEntropySigmoidLossLayer | Cross entropy sigmoid loss layer |
+ | cross_softmax | CrossEntropySoftmaxLossLayer | Cross entropy softmax loss layer |
+ | constant_derivative | ConstantDerivativeLossLayer | Constant derivative loss layer |
+ | mse | MSELossLayer | Mean square error loss layer |
+ | kld | KLDLossLayer | Kullback-Leibler Divergence loss layer |
+
+### Supported Activation Functions
+
+NNTrainer provides
+
+ | Keyword | Loss Name | Description |
+ |:-------:|:---:|:---|
+ | tanh | tanh function | set as layer property |
+ | sigmoid | sigmoid function | set as layer property |
+ | relu | relu function | set as layer propery |
+ | softmax | softmax function | set as layer propery |
### Tensor
-Tensor is responsible for the calculation of Layer. It executes the addition, division, multiplication, dot production, averaging of Data and so on. In order to accelerate the calculation speed, CBLAS (C-Basic Linear Algebra: CPU) and CUBLAS (CUDA: Basic Linear Algebra) for PC (Especially NVIDIA GPU) for some of the operation. Later, these calculations will be optimized.
+Tensor is responsible for calculation of a layer. It executes several operations such as addition, division, multiplication, dot production, data averaging and so on. In order to accelerate calculation speed, CBLAS (C-Basic Linear Algebra: CPU) and CUBLAS (CUDA: Basic Linear Algebra) for PC (Especially NVIDIA GPU) are implemented for some of the operations. Later, these calculations will be optimized.
+Currently, we supports lazy calculation mode to reduce complexity for copying tensors during calculations.
-## Getting Started
+ | Keyword | Description |
+ |:-------:|:---:|
+ | 4D Tensor | B, C, H, W|
+ | Add/sub/mul/div | - |
+ | sum, average, argmax | - |
+ | Dot, Transpose | - |
+ | normalization, standardization | - |
+ | save, read | - |
-### Prerequisites
+### Others
-The following dependencies are needed to compile / build / run.
+NNTrainer provides
-* gcc/g++
-* CMake ( >= 2.8.3)
-* blas library ( CBLAS ) (for CPU Acceleration)
-* cuda, cudart, cublas (should match the version) (GPU Acceleration on PC)
-* tensorflow-lite (>=1.4.0)
-* jsoncpp ( >=0.6.0) (openAI Environment on PC)
-* libcurl3 (>= 7.47 ) (openAI Environment on PC)
+ | Keyword | Loss Name | Description |
+ |:-------:|:---:|:---|
+ | weight_initializer | Weight Initialization | Xavier(Normal/Uniform), LeCun(Normal/Uniform), HE(Normal/Unifor) |
+ | weight_regularizer | weight decay ( L2Norm only ) | needs set weight_regularizer_param & type |
-### How to Build
+### APIs
+Currently, we provide [C APIs](https://github.com/nnstreamer/nntrainer/blob/master/api/capi/include/nntrainer.h) for Tizen. [C++ APIs](https://github.com/nnstreamer/nntrainer/blob/master/api/ccapi/include) are also provided for other platform. Java & C# APIs will be provided soon.
-Download the source file by clone the github repository.
-```bash
-$ git clone https://github.com/nnsuite/nntrainer
-```
+## [Getting Started](https://github.com/nnstreamer/nntrainer/blob/main/docs/getting-started.md)
-After completing download the sources, you can find the several directories as below.
+Instructions for installing NNTrainer.
-``` bash
-$ ls
-Applications CMakeLists.txt external include jni LICENSE package.pc.in
-```
+### [Running Examples](https://github.com/nnstreamer/nntrainer/blob/main/docs/how-to-run-examples.md)
-There are four applications tested on the Android and Ubuntu (16.04). All of them include the code in NeuralNet directory and has their own CMake file to compile. This is the draft version of the code and need more tailoring.
-Just for the example, let\'s compile Training application. Once it is compiled and installed you will find the libnntrainer.so and related .h in /usr/local/lib and /usr/local/include directories.
+Instructions for preparing NNTrainer for execution
-``` bash
-$ mkdir build
-$ cd build
-$ cmake ..
-$ make
-$ sudo make install
-```
+### [Examples for NNTrainer](https://github.com/nnstreamer/nntrainer/tree/main/Applications)
-And you could test the nntrainer with Examples in Applications. There are several examples you could try and you can compile with cmake(Ubuntu) and ndk-build (Android). Tensorflow-lite is pre-requite for this example, so you have to install it by your self. If you are trying to run on the Android, it will automatically download tensorflow (1.9.0) and compile as static library.
-For the Training Examples, you can do like this:
+NNTrainer example for a variety of networks
-```bash
-$ cd Application/Training/jni
-$ mkdir build
-$ cd build
-$ cmake ..
-$ make
-$ ls
-CMakeCache.txt CMakeFiles cmake_install.cmake Makefile TransferLearning
-$ ./Transfer_learning ../../res/Training.ini ../../res/
-../../res/happy/happy1.bmp
-../../res/happy/happy2.bmp
-../../res/happy/happy3.bmp
-../../res/happy/happy4.bmp
-../../res/happy/happy5.bmp
-../../res/sad/sad1.bmp
-../../res/sad/sad2.bmp
+## Open Source License
-...
+The NNtrainer is an open source project released under the terms of the Apache License version 2.0.
-```
+## Contributing
-## Open Source License
+Contributions are welcome! Please see our [Contributing](https://github.com/nnstreamer/nntrainer/blob/main/docs/contributing.md) Guide for more details.
-The nntrainer is an open source project released under the terms of the Apache License version 2.0.
+[](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/0)[](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/1)[](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/2)[](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/3)[](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/4)[](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/5)[](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/6)[](https://sourcerer.io/fame/dongju-chae/nnstreamer/nntrainer/links/7)
projects / platform / core / ml / nntrainer.git / blobdiff