Split unittest for util func to diffrent source. (#61)

 Split util_func unittest to `unittest_util_func.cpp`.

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [ ]Passed [X]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

Add Reviewer

Add Reviewer in README.md

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[README] Add some explanation

 * Update prerequisite
 * Propose GIAG Build Environment
 * Make example section

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

Split files for DataBufferFromFile & DataBufferformCallback

data_buffer.h & cpp is too big to manage. Its inherited clases are
saved in seperate files.

- data_buffer_func.h & data_buffer_func.cpp
- data_buffer_file.h & data_buffer_file.cpp

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Move images to docs/images

 This PR moves images to docs/images to clean up docs folder for later
use.

Self evaluation:

Build test: [ ]Passed [ ]Failed [X]Skipped
Run test: [ ]Passed [ ]Failed [X]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

Introduce DataBufferFromCallback Class

With this class, it is possible to train with user specific data
generation callback funciton. This is the default to get training
data unless there is [DataSet] key in configuration file.

- in Application/Classification/jni/main_func.cpp
 NN.train(getMiniBatch_train, getMiniBatch_val, getMiniBatch_train)

Then, data buffer thread call these functions to get newest data with
size of mini batch.

The format this function should be :
/*
 * @brief Callback function to get user specific data
 * @param[in] X data  3D float vector type
 * @param[in] Y label 3D float vector type
 * @param[out] status status for error handle
 * @retval true / false generate all data for this epoch
 */
bool  func(std::vector<std::vector<std::vector<float>>>& X,
      std::vector<std::vector<std::vector<float>>>& Y, int &status)

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Tensor unit tests

Add Tensor unit tests

**Self evaluation:**
1. Build test:   [X]Passed [ ]Failed [ ]Skipped
2. Run test:     [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Unit Test Cases for math utilities function

Add unit test cases for math utilities functions

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Modify softmax calculation to handle large values.

Currently it is not numerically stable.
it is more stable with this pr.

Signed-off-by: jijoongmoon <jijoong.moon@samsung.com>

Throw error exception from thread

thow error exceptions from data buffer thread to main.
By this pr, program stops normally with proper error handling.

Signed-off-by: jijoongmoon <jijoong.moon@samsung.com>

Introduce DataBufferFromDataFile Class

In order to handle various input generation cases, make DataBuffer
Class is base class and inherit this, several children classes is
introduced.
. DataBufferFromDataFile
. DataBufferFromCallback : NYI
. DataBufferFromFramework : NYI
. others. : NYI

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add train function in NeuralNetwork Class.

Add train member function of NeuralNetwork Class.
- NeuralNetwork Class has DataBuffer instance to handle datas.
- New Keywords are introduced for specify data set
  . TrainData, ValidData, TestData, LabelData
- DataBuffer instance is initialized with parameters from Neural
Network instance.
- In train function,
   . Data Buffer instance is running ( collecting or generating data )
   . get the data from Data Buffer instance.
   . Backwarding
   . Display progress ( Data Buffer member funciton )
   . Validate if it is enabled.
- Add Classification Example with train member function

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Initialize data buffer when neural network initializes.

Initilaize data buffer and set parameters from neural network.

Signed-off-by: jijoongmoon <jijoong.moon@samsung.com>

[README] Update Getting Started

This patch updates Getting Started section to reflect the recent
changes.

Signed-off-by: Wook Song <wook16.song@samsung.com>

Add more badges

badges
- repo size
- issues
- pull requests

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Unit Test Cases for Neural Network Initialization

Nerual Network Initialization Unit Test including layer
initialization.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

lower case for layer type.

Instead of using camel type, snakes type is used.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Update clang-format

Update clang-format for better formatting

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Unit Test for Neural Network

- Neural Network Initialization
- set Data Buffer
- set Optimizer

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Coverage Test badge for nntrainer

Add Coverage Test badge for nntrainer in README.md

Signed-off-by: jijoongmoon <jijoong.moon@samsung.com>

README: Correct typos and grammar

This is a trivial patch that corrects typos and grammar.

Signed-off-by: Wook Song <wook16.song@samsung.com>

Add Unit Test Coverage

Add Unittest coverage

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Naming Rules & change namespace

- Naming Rules
  . namespace : lower-case (nntrianer)
  . type names ( struct, class .. etc. ) : Pascal (TypeNames)
  . variable names : snakes ( variable_names )
  . enum : Upper Case with underbar

- using namespace nntrainer
  . remove other namespace

- add #ifdef __cplusplus

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Unit Tests for the NNTrainer internal

Add Unit Test to evaluate NNTrainer interals

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Remove Duplicated Codes @UpdateData

Remove Duplicated Codes @UpdataData

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Remove Duplicated Code @WeightInitialization

Remove Duplicated Code @WeightInitialization Fuc.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Optimizer Class & separate math functions

- Add Optimizer Class and remove duplicated codes.
  . Hyperparameters related with Optimizer class placed Optimizer
  Class.
    . OptParam, WeightDecayParam, OptType, WeightDecayType

- Seperate math functions in util_func.h/cpp

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Remove global variables to parse the configuration file

There is no reason to be global because only one function use these
global variables.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Fix test bug

- Add test configuration file in test directory
- modify code to read the path correctly

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Eable Logging

Modify codes and buld for Logging

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[API] Add Logger for NNTrainer

In order to log, implement Logger class to use when there is no
logging system. If there is, we are going to use them. For example,
we are going to use tizen dlog for the tizen and android log system
for android with same interface such as nn_log[i,w,e,d].

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[API] Add Neural Network Model Construction with Configuration File

- Add Construction of Nerual Network Model with Configuration File.
 . ml_nnmodel_construct_with_conf()
 . Add one positive and one negative tests

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add testing during packaging

Add run test during build.

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[API] Prototype of C API & test for Neural Network Model

- Add C API to construct / destruct Neural Network Model object
. ml_nnmodel_construct()
. ml_nnmodel_destruct()

- Add test cases

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Remove gym-http-api files

Remove 'include/gym/gym.h'
Remove 'gym-bininding.cpp'

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Enabling Android build

Enable Android build including Applications build

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[SVACE] Check the index of test case if it is exceed the file size

The value of an arithmetic expression (I * this->input_size + I *
this->class_num) * sizeof(float) is subject to overflow due to a
failure to cast operands to a larger data type before perfoming arithmetic

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Modify CODEOWNERS

Add new members & fix id

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Debian packaging files

Add debian packaging files

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

meson: Add fallback mechanism to find iniparser

In case that there are no cmake files or no pkg-config files for
iniparser, this patch adds fallback mechanism to find it.

Signed-off-by: Wook Song <wook16.song@samsung.com>

Change Build Framework & add application package

- From now on, we are going to use meson as an build framework instead
of cmake.
- Add nntrainer-applications packages

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Fix compile warnning messages

Fix the compile warnning messages. Depending on compile options, it
could be errors.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

fix image url

add blob/master/

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Upload Pictures for README.md

Upload pictures for README.md at Application directories.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Modify applyFunction for Tensor data type

Currently only element by element calculation is available with
applyFunction. However, there are cases to deal with whole data of
tensor like softmax. Due to this reason, applyFunction is modifed.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Fix nntrainer home URL

Change nntrainer home URL to github.com/nnstreamer/nntrainer

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Remove Categorical enum in cost function

No Need Categorical Cost Fucntion. It is duplication of cross entropy.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Fix code for Clang format 4.0

fix code for clang format 4.0

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

fix .gitignore

remove newline error

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

add .gitignore

add .gitignore

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add CODEOWNERS

CODEOWNERS are

  @AIP/nntrainer @myungjoo-ham @jijoong-moon @geunsik-lim @wook16-song
  @helloahn @pk-kapoor @dongju-chae @sangjung-woo @gichan2-jang
  @jy1210-jung @yongjoo1-ahn

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

add TensorDim Class

Add TensorDim class to handle tensor demension.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Make Examples work properly

Modify Exampels for the Updated NNtrainer
- Classification
- KNN
- LogisticRegression
- ReinforcementLearning
- Training

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Modify LogisticRegression & Training Example for updated nntrainer

Make Logistic Regression and Training Example work properly with
current nntrainer.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Update Training & Classification Examples according to new format

CMakeLists.txt & configuration format is changed. Therefore, changes
are necessary.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add namespace Tensors for Tensor

for better readibility, using Tensors::Tensor

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add nntrainer.spec for tizen packaging

. add nntrainer.spec
. add nntrainer.manifest

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

fix error when calculate sum

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Remove iniparser directory.

It would be better to maintain package without iniparser srouce
directory. It will use iniparser installed in platform.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add using Cublas for matrix multiplication ( SGEMM )

Instead of CBlas, it is possble to use cublas for sgemm.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add test example for the assign activation function per layer

test case for assigning activation per layer

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add progress bar to trace training process

Add progress bar to trace processing.

Using Training set to validate training algorighm. It should be over
95% at least.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Let assign activation fuction per layer

make it possible to assign activation fuction per each layer.

Also, make softmax as an activation function and only softmax and
sigmoid is avaliable for the output layer.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Fix Weight Decay L2Norm

There is wrong implementation about weight decay l2norm.
In this pr, it is fixed

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Weight Decay (L2Norm)

Implement Weight Decay ( L2Norm Only )

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

add Weight initialization Method

Add Weight Initialization Method

    "lecun_normal"  : LeCun Normal Initialization
    "lecun_uniform"  : LeCun Uniform Initialization
    "xavier_normal"  : Xavier Normal Initialization
    "xavier_uniform"  : Xavier Uniform Initialization
    "he_normal"  : He Normal Initialization
    "he_uniform"  : He Uniform Initialization

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Standardization in InputLayer

Add Standardization in InputLayer

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add normalization at InputLayer

Add normalization option for inputlayer

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Make softmax optional

make softmax optional when the output is calculated

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Update Bias when the optimizer is adam

Until now, Adam optimizer updates weight only. So that, adds Bias
update accordingly.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Relu Activation Function

Add Relu Activation Function

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Modify databuffer to generate random input variables at first time

Modify databuffer to generate random input variables at first time
- Add Conditional variable to synchronize thread

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Fix the compilation Error for Batch Normalization

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Batch Normalization Draft

- This is unstable version of Batch normalization

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Keras & Tensorflow Example for transfer learning

 mobilenetv2 output (features) using cifar10 data
     : 10 Classes x 100 image each --> trainingSet.dat

        * trainingSet.dat                       * ValSet.dat
               62720                                 62720
          +--------------+                      +--------------+
          |              |                      |              |
  10x100  |              |                10x10 |              |
          |              |                      |              |
          |              |                      |              |
          +--------------+                      +--------------+

- Layers
  : InputFeatures(62720)->10 hidden FC + softmax -> 10 Classes

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Subtract with float value in tensor & add sum according to axis

Add Subtract(float value)
Add sum(int axis)
  : sum according to axis
   - 0 : batch
   - 1 : height
   - 2 : width

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Change Configuration format fot batch normalization

Change Configuration format
- Delete Width, Hieght
- Add HiddenSize

Add BatchNormalizationLayer Scaleton Code

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Softmax token in ini file

Softmax token is add to use to enable softmax at outputlayer
This is boolean and if it is set by true, then softmax is enabled.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Fix calculation of Logistics Regression

Fix calculation of Logistics Regression at outputlayer

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add DataBuffer for big data

- Add DataBuffer Class to read/write big data from file
- Multi-Threaded
- Automatic Updated

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Calculation of Validation Loss & Accuracy

Calculate Validation Loss & Accuracy

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Classification Example (cifar10 data set)

Classification Example is added. Data Set is cifar10.
The configuraion is in Classification.ini.
The base model for feature extractor is mobilenet v2 with pretrained
imagenet data. ( 244x244x3 ) Input feature size is 1280x7x7 = 62720.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Implement Cross Entropy Cost Function

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Using CBLAS for Tensor Calculation

Implement Tensor Calculation for Tensor Calculation
Can use with "-DUSE_BLAS"

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Decayed Learning Rate

Implement Decayed Learning Rate for better convergence

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Fix Maintainer in README.md

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add README.md

README includes Descriptions of NNtrainer, how-to's and Open Source License.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Use float data type instead of double.

For the efficiency, we are going to use float data type.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

add .gitmodules

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Change Class Name Matrix to Tensor

Tensor is more appropreate than Matrix.

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Code Refectoring

Code Refactoring
 - Move to applicatoins into Application Directory
 - Remove NeuralNet Directory
 - Make include & src directory for the neural network
 - Modifiy build configuration to build with libnntrainer.so
 - Add nntrainer install & add pc.in for pkgconfig
 - Add install path

**Changes proposed in this PR:**
- Added TOC generator for README.md

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Make nntrainer shared library

Change Directory structure and make nntrainer shared library,
nntrainer.so

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add License file

Add Apache 2.0 full license file

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Copyright for Appache 2.0 License

Add Copyright

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

add doxygen doc for DeepQ

add doxygen doc for DeepQ

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

add doxygen doc for KNN example

add doxygen doc for KNN example

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

add doxygen doc for Logistic regression

add doxygen doc for Logistic regression

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

add doxygen document for Training

add doxygen documents for Training

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Doxygen Documentation for Environment

Add Doxygen Doc. for Environment

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add doxygen Documentation for NeuralNet

Add doxygen Documentation format of NeuralNet Directory

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>