Add Coverage Test badge for nntrainer

Add Coverage Test badge for nntrainer in README.md

Signed-off-by: jijoongmoon <jijoong.moon@samsung.com>

README: Correct typos and grammar

This is a trivial patch that corrects typos and grammar.

Signed-off-by: Wook Song <wook16.song@samsung.com>

Add Unit Test Coverage

Add Unittest coverage

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Naming Rules & change namespace

- Naming Rules
  . namespace : lower-case (nntrianer)
  . type names ( struct, class .. etc. ) : Pascal (TypeNames)
  . variable names : snakes ( variable_names )
  . enum : Upper Case with underbar

- using namespace nntrainer
  . remove other namespace

- add #ifdef __cplusplus

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Unit Tests for the NNTrainer internal

Add Unit Test to evaluate NNTrainer interals

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Remove Duplicated Codes @UpdateData

Remove Duplicated Codes @UpdataData

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Remove Duplicated Code @WeightInitialization

Remove Duplicated Code @WeightInitialization Fuc.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Optimizer Class & separate math functions

- Add Optimizer Class and remove duplicated codes.
  . Hyperparameters related with Optimizer class placed Optimizer
  Class.
    . OptParam, WeightDecayParam, OptType, WeightDecayType

- Seperate math functions in util_func.h/cpp

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Remove global variables to parse the configuration file

There is no reason to be global because only one function use these
global variables.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Fix test bug

- Add test configuration file in test directory
- modify code to read the path correctly

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Eable Logging

Modify codes and buld for Logging

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[API] Add Logger for NNTrainer

In order to log, implement Logger class to use when there is no
logging system. If there is, we are going to use them. For example,
we are going to use tizen dlog for the tizen and android log system
for android with same interface such as nn_log[i,w,e,d].

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[API] Add Neural Network Model Construction with Configuration File

- Add Construction of Nerual Network Model with Configuration File.
 . ml_nnmodel_construct_with_conf()
 . Add one positive and one negative tests

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add testing during packaging

Add run test during build.

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[API] Prototype of C API & test for Neural Network Model

- Add C API to construct / destruct Neural Network Model object
. ml_nnmodel_construct()
. ml_nnmodel_destruct()

- Add test cases

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Remove gym-http-api files

Remove 'include/gym/gym.h'
Remove 'gym-bininding.cpp'

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Enabling Android build

Enable Android build including Applications build

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[SVACE] Check the index of test case if it is exceed the file size

The value of an arithmetic expression (I * this->input_size + I *
this->class_num) * sizeof(float) is subject to overflow due to a
failure to cast operands to a larger data type before perfoming arithmetic

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Modify CODEOWNERS

Add new members & fix id

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Debian packaging files

Add debian packaging files

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

meson: Add fallback mechanism to find iniparser

In case that there are no cmake files or no pkg-config files for
iniparser, this patch adds fallback mechanism to find it.

Signed-off-by: Wook Song <wook16.song@samsung.com>

Change Build Framework & add application package

- From now on, we are going to use meson as an build framework instead
of cmake.
- Add nntrainer-applications packages

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Fix compile warnning messages

Fix the compile warnning messages. Depending on compile options, it
could be errors.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

fix image url

add blob/master/

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Upload Pictures for README.md

Upload pictures for README.md at Application directories.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Modify applyFunction for Tensor data type

Currently only element by element calculation is available with
applyFunction. However, there are cases to deal with whole data of
tensor like softmax. Due to this reason, applyFunction is modifed.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Fix nntrainer home URL

Change nntrainer home URL to github.com/nnstreamer/nntrainer

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Remove Categorical enum in cost function

No Need Categorical Cost Fucntion. It is duplication of cross entropy.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Fix code for Clang format 4.0

fix code for clang format 4.0

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

fix .gitignore

remove newline error

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

add .gitignore

add .gitignore

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add CODEOWNERS

CODEOWNERS are

  @AIP/nntrainer @myungjoo-ham @jijoong-moon @geunsik-lim @wook16-song
  @helloahn @pk-kapoor @dongju-chae @sangjung-woo @gichan2-jang
  @jy1210-jung @yongjoo1-ahn

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

add TensorDim Class

Add TensorDim class to handle tensor demension.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Make Examples work properly

Modify Exampels for the Updated NNtrainer
- Classification
- KNN
- LogisticRegression
- ReinforcementLearning
- Training

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Modify LogisticRegression & Training Example for updated nntrainer

Make Logistic Regression and Training Example work properly with
current nntrainer.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Update Training & Classification Examples according to new format

CMakeLists.txt & configuration format is changed. Therefore, changes
are necessary.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add namespace Tensors for Tensor

for better readibility, using Tensors::Tensor

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add nntrainer.spec for tizen packaging

. add nntrainer.spec
. add nntrainer.manifest

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

fix error when calculate sum

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Remove iniparser directory.

It would be better to maintain package without iniparser srouce
directory. It will use iniparser installed in platform.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add using Cublas for matrix multiplication ( SGEMM )

Instead of CBlas, it is possble to use cublas for sgemm.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add test example for the assign activation function per layer

test case for assigning activation per layer

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add progress bar to trace training process

Add progress bar to trace processing.

Using Training set to validate training algorighm. It should be over
95% at least.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Let assign activation fuction per layer

make it possible to assign activation fuction per each layer.

Also, make softmax as an activation function and only softmax and
sigmoid is avaliable for the output layer.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Fix Weight Decay L2Norm

There is wrong implementation about weight decay l2norm.
In this pr, it is fixed

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Weight Decay (L2Norm)

Implement Weight Decay ( L2Norm Only )

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

add Weight initialization Method

Add Weight Initialization Method

    "lecun_normal"  : LeCun Normal Initialization
    "lecun_uniform"  : LeCun Uniform Initialization
    "xavier_normal"  : Xavier Normal Initialization
    "xavier_uniform"  : Xavier Uniform Initialization
    "he_normal"  : He Normal Initialization
    "he_uniform"  : He Uniform Initialization

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Standardization in InputLayer

Add Standardization in InputLayer

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add normalization at InputLayer

Add normalization option for inputlayer

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Make softmax optional

make softmax optional when the output is calculated

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Update Bias when the optimizer is adam

Until now, Adam optimizer updates weight only. So that, adds Bias
update accordingly.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Relu Activation Function

Add Relu Activation Function

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Modify databuffer to generate random input variables at first time

Modify databuffer to generate random input variables at first time
- Add Conditional variable to synchronize thread

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Fix the compilation Error for Batch Normalization

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Batch Normalization Draft

- This is unstable version of Batch normalization

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Keras & Tensorflow Example for transfer learning

 mobilenetv2 output (features) using cifar10 data
     : 10 Classes x 100 image each --> trainingSet.dat

        * trainingSet.dat                       * ValSet.dat
               62720                                 62720
          +--------------+                      +--------------+
          |              |                      |              |
  10x100  |              |                10x10 |              |
          |              |                      |              |
          |              |                      |              |
          +--------------+                      +--------------+

- Layers
  : InputFeatures(62720)->10 hidden FC + softmax -> 10 Classes

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Subtract with float value in tensor & add sum according to axis

Add Subtract(float value)
Add sum(int axis)
  : sum according to axis
   - 0 : batch
   - 1 : height
   - 2 : width

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Change Configuration format fot batch normalization

Change Configuration format
- Delete Width, Hieght
- Add HiddenSize

Add BatchNormalizationLayer Scaleton Code

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Softmax token in ini file

Softmax token is add to use to enable softmax at outputlayer
This is boolean and if it is set by true, then softmax is enabled.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Fix calculation of Logistics Regression

Fix calculation of Logistics Regression at outputlayer

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add DataBuffer for big data

- Add DataBuffer Class to read/write big data from file
- Multi-Threaded
- Automatic Updated

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Calculation of Validation Loss & Accuracy

Calculate Validation Loss & Accuracy

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Classification Example (cifar10 data set)

Classification Example is added. Data Set is cifar10.
The configuraion is in Classification.ini.
The base model for feature extractor is mobilenet v2 with pretrained
imagenet data. ( 244x244x3 ) Input feature size is 1280x7x7 = 62720.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Implement Cross Entropy Cost Function

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Using CBLAS for Tensor Calculation

Implement Tensor Calculation for Tensor Calculation
Can use with "-DUSE_BLAS"

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Decayed Learning Rate

Implement Decayed Learning Rate for better convergence

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Fix Maintainer in README.md

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add README.md

README includes Descriptions of NNtrainer, how-to's and Open Source License.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Use float data type instead of double.

For the efficiency, we are going to use float data type.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

add .gitmodules

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Change Class Name Matrix to Tensor

Tensor is more appropreate than Matrix.

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Code Refectoring

Code Refactoring
 - Move to applicatoins into Application Directory
 - Remove NeuralNet Directory
 - Make include & src directory for the neural network
 - Modifiy build configuration to build with libnntrainer.so
 - Add nntrainer install & add pc.in for pkgconfig
 - Add install path

**Changes proposed in this PR:**
- Added TOC generator for README.md

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Make nntrainer shared library

Change Directory structure and make nntrainer shared library,
nntrainer.so

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add License file

Add Apache 2.0 full license file

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Copyright for Appache 2.0 License

Add Copyright

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

add doxygen doc for DeepQ

add doxygen doc for DeepQ

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

add doxygen doc for KNN example

add doxygen doc for KNN example

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

add doxygen doc for Logistic regression

add doxygen doc for Logistic regression

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

add doxygen document for Training

add doxygen documents for Training

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Doxygen Documentation for Environment

Add Doxygen Doc. for Environment

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add doxygen Documentation for NeuralNet

Add doxygen Documentation format of NeuralNet Directory

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

fix to use epsilon from config file

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Logistic Regression

Add Logistic Regression with example

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

add network description for generality

- include iniparser
- add parse configuratio ini file to describe network

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add bias correction of adam optimizer

add bias correction according to the epoch.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add CMakeLists.txt for KNN & support ubuntu

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add README for DeepQ

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Layers::Layer classes

Add Layers
. Layers::Layer
. Layers::InputLayer : public Layer
. Layers::FullyConnectedLayer : public Layer
. Layers::OutputLayer : public Layer
  with forwarding, backwarding support

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Modify code for clang format

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Restructure neural network

Make NeuralNet directory for common usages.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Android.mk for android build

- Add Android.mk
- Remove Boost Dependency

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Custom Environment Cartpole

Depending the USING_CUSTOM_ENV definition, can choose custom
env (Env::CartPole) or Gym::Environment.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Custom Environment : CartPole

Implement Custom CartPole Environment with test example

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

add reference model.bin

Add Trained model for CartPole-v0 ( Terminate 200 Iteration Max )

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

add ADAM (Adaptive Momemt Estimation) to accelerate convering

- Add ADAM optimizer

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add Another Fullyconnected Layer

Add Fully Connnected Layer : ( Two FC )

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add tanh & minibach mode

- Add tanh activation
- Add minbatch mode

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Implement DeepQ Learning without minibatch

Implement DeepQ Learning for CartPole-v0

Currently It is not stable.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add initialize DeepQ with gym (CartPole-v0)

Add DeepQ for CartPole-v0

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Add NeuralNetwork Class

Add Network namespace & NeuralNetwork Class.
Add copy Matrix & NeuralNetwork

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>