[IniSerializer] Add `serialize` to ini

**Changes proposed in this PR:**
- implements ini interpreter::serialize
- add corresponding test (reference shoud be equal to
serialize->deserialized graph)
- Fix some const correctness

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

Handle Uninitialized pointer field issue

Initialize pointer
resolve: 1227126

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

Handle Uncaught exception issue

Added try catch statement

resolves: 1143556, 1144840

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

Handle NO_EFFECT issue

Delete not effective condition
The type of variable i is unsigned int so i < 0 always false

resolves: 461267, 461268

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

Handle GTEST.NO_ASSERTION issue

Added EXPECT_NO_THROW to handle No Assert Statement Found issue

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[Layer/props] Convert name to props.

**Changes proposed in this PR:**
- Convert name to props
- Add std::string specialization for property

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[TensorDim/Tensor] Update transpose relates

**Changes proposed in this PR:**
- Add tensorDim::transpose(out)
- Add tensor::transpose
- Add test about tensor::transpose

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[NDK] Add frtti to application build

As real time type information is used in nntrainer, frtti flag is turned
on to fix build issue

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Props] Implement load properties

This patch implement `load_properties` which helps loading properties
for a tuple of properties.

Please note that `getKeyVal` is taken from `parse_util` because there is
a build option issue with android build

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Doxygen] Add doxygen to tensordim

This patch adds doxygen to tensordim.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix] handle edge case for countNonTrainable

**Changes proposed in this PR:**
- Handle edge case when every layer is non trainable
for `countNonTrainableLayersAtBegin`
- Trivial code cleans on simpleshot task_runner / neuralnetwork

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[graph] Bug fix for duplicate output connections

Added bugfix for duplicate output connections

Resolves #1140

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Doxygen] Add missing function tags

This patch adds missing function tags detected from doxygen

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[FC/props] Wrap properties with properties

**Changes proposed in this PR:**
- Add base_properties for common use
- Change fc unit to property
- Add export test
- Add `uint_prop_tag` and it's handler

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ TIME DIST ] Implement Backwarding of TimeDistLayer

This commit includes
  - implementation of TimeDistLayer Backwarding

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ TIME DIST ] add example

Describe a commit content (Until 80 colums per line) in detail ASAP.

**Changes proposed in this PR:**
- Added TOC generator for README.md

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ TIME DIST ] Forwarding Implementation

This commit includes
 . Implementation of Forwarding Implementation of Time Dist Layer

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ TIME DIST ] Implement Initialization of Time Dist Layer

This commit includes,
  - Time Dist Layer initialization
  - Some modification of network graph

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Revert "[ TIME DIST ] Implement Initialization of Time Dist Layer"

This reverts commit a386b1113cacf8652ebc2bf3765bb470dcc6ea10.

Revert "[ TIME DIST ] Forwarding Implementation"

This reverts commit c2122d5a5256fa31ffc39d8f5be70f56ee77ff05.

Revert "[ TIME DIST ] add example"

This reverts commit 5c486995b44be42f1734a1b7423c932eee364aa4.

Revert "[ TIME DIST ] Implement Backwarding of TimeDistLayer"

This reverts commit 11d08996944f755af909f7a2bfce8996268d05b2.

[ TIME DIST ] Implement Backwarding of TimeDistLayer

This commit includes
  - implementation of TimeDistLayer Backwarding

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ TIME DIST ] add example

Describe a commit content (Until 80 colums per line) in detail ASAP.

**Changes proposed in this PR:**
- Added TOC generator for README.md

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ TIME DIST ] Forwarding Implementation

This commit includes
 . Implementation of Forwarding Implementation of Time Dist Layer

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ TIME DIST ] Implement Initialization of Time Dist Layer

This commit includes,
  - Time Dist Layer initialization
  - Some modification of network graph

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[Fix] Reorder error in layernode

This patch fixes trivial build warning which is treated as error.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Competibility] Change test reference value

This patch change tolerance for the `model->getValidationLoss` which
affects in i586

resolves #1125

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

Handle Undecieded issue

Init variables in constructor
resolve: 460230

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[doxygen] Var_Grad doxygen fix

Added doxygen fix for var_grad

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Exporter] Add exporter test to prop test

This patch adds exporter test to prop test to ensure that it is working
fine.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Exporter] Add string vector exporter

This patch adds string vector exporter implementation

Exporter will iterate through props (`iterate_props`) and will pass a
generic lambda to be called.

Later, We can retrieve the result by `get_result` if needed.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Docs] Fix doxygen

- Fix some doxygen errors
- Fix author email typo

Signed-off-by: Juyeong Lee <2jy22@naver.com>

[README] Replace nnsuite to nnstreamer

Since web page of TAOS-CI has been changed replace nnsuite to nnstreamer

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[test] Doxygen fix

Added doxygen fix for the unittests.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph] Update graph node

Update graph node to use the updated LayerNode

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Test/Props] Add basic property tests

This patch adds basic property test to demonstrate specification and
cover some test cases

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Props] Add base properties

Add base properties to be used for the property implementation

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Android/mk] Fix includes

This patch adds utils to the include header so to pass android build

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Serializer] Add exporter to export node

This patch adds exporter to export node. This exploits visitor pattern
because, exporting need templated function to be executed, but virtual
functions does not allow templated function.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Addition_layer] Set output to zero in forward

Before adding inputs output should be set zero

Self evaluation:

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

Fix indent in the SPDX identifier

Change tab indents to spaces in the SPDX identifier.
Resolves #1083

Signed-off-by: Juyeong Lee <2jy22@naver.com>

[Ini/Serialize] Implement logic to serialize

This patch implements general concept of serializing a node.

Below is the plan to implement this logic

0. Implement logic to serialize [*]
1. Add export api
2. Add exporter
3. Add export test
4. Implement properties
5. combine with export
6. Add extra test

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ TIME DIST ] Modify graph to recognize time distribute layer

if time distributed layer has activation, we have to add time
distributed layer for activation, and if it need to realize loss
layer, we also add the time distribute layer for loss.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[Ini] Remove IniTestWrapper

This patch removes initestwrapper in favor of nntrianer::iniwrapper

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Preprocess] Set default trainable to false

**Changes Proposed**
- Set default trainable to `false` for preprocess layer
- Register preprocess layer to the app context
- Add validate test

resolves #1129

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[Unittest] Added doxygen tag

1. Added missing doxygen tag
2. remove unused function parameter

Self evaluation:

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[Ini] Propose ini wrapper 2

Propose ini wrapper in nntrainer, most codes are from
`nntrainer_test_util.cpp`.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Android] Fix android build

**Changes proposed in this PR:**
- Add link option to android.mk to deal with gpu delegate added in
prebuilt tflite library
- Add doxygen, change defaulted ctor that was causing warning
- Add new warning flags regarding the error

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

Handle Uninitialized scalar variable issue

Set default recurrent_activation_type in parameter
resolves: 1224616, 1224617

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[Fix] Multiout layer calcDerivative initialize

This patch fixes multiout layer to initialize for calcDerivative

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[manager] disable inference_memory_opt

Enable inference_inout_memory_opt in inference could make a problem

Cause: If the networks are not flatten which means have a cycle, some of output of layer should be preserved.
For example, in resnet there is a residual output and if the opt is enabled this data could be corrupted.

See Also #1094

Self evaluation:

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[Ini] Propose ini wrapper

This patch adds `IniSection` which was often used inside unittest.

Most codes are taken from the test directly.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[graph] Add node the graph of layers

Create class for the graph of layers which will represent nodes
This class also represents the object which will be created with APIs from now.
The actual class object resides inside this layer node object.

Properties for this layer node will be intercepted by the layer node object,
and properties of the graph node will be set in the layer node.
Others will be passed down to the layer node.

See Aso #986

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Graph] Add interface for graph node

Add interface for the graph node.
Graph will use this interface.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Loader] Remove functions which has been moved

Remove functions that has been moved to `ini_interpreter`

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Loader] Use ini interpreter for the graph

This patch substitues graph parsing part to use interpreter.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[IniInterpreter] Add test setup

This patch add a test setup to test interpreter.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[IniInterpreter] Add ini backbone deserialization

**Changes proposed in this PR:**
- Add ini backbone deserialization support

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[IniInterpreter] Add section2layer

This patch add plain `section2layer`.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Ini] Implement ini deserializer

This patch implements ini deserializer as well as changing the interface
signature to get filepath.

Many NYI parts will be implemented soon.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ TIME DIST ] Implement initialization

This commit includes,
 - Implementation of Time Distribute Layer Initialization.

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ TIME DIST ] Add skeleton Code for Time Distribution Layer

This includes
 . Skeleton Code for Time Distribution Layer

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ TIME DIST ] Add skeleton Code for Time Distribution Layer

This includes
 . Skeleton Code for Time Distribution Layer

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ LSTM ] Layer forwarding implementation

This commit includes:
 . implementation of lstm forwarding implementation

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ RNN ] Fix the bug when getting input data

This commit fix:
     Using wrong tensor to get input vector.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ LSTM ] Layer Initialization

This commit includes
     . LSTM Layer initialization

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

Handle Missing break in switch issue

Added break which was omitted
resolve: 1224463

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

Handle Uninitialized scalar field issue

Added default constructor to initialize scalar field
resolves: 1216569, 1216584

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

Handle Unchecked return value issue

Added if statement to check return value
resolve: 1216589

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[Compiler] Alternate GraphRepresentation

This patch alternates graph representation and move interpreter to a new
file

v2.
Also change documentation setup

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Compiler] Add Graph Representation

This patch adds graph representation and it's interface

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Compiler] Declare interface

This patch declares interface of compiler / interpreter

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Compiler] Add concept of the compiler

This patch adds a concept of the compiler and class declaration.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Compiler/skeleton] Set up compiler source files

This patch setup compiler source files

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ RNN ] Backwarding Implementation

This commit includes calculate gradient and derivatives of RNN Layer.

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[Deps] Add iniparser dependency version

From SRI-D it was reported that iniparser 3.x is not compatible.

This patch adds a version check for the iniparser

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[c-api] c-api depends only on cc-api

Update c-api implementation to only depend on cc-api
Also add some minor additional functions in the cc-api
to support the c-api functionality.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[dataset] Update dataset cc-api

Update dataset cc-api to match with the c-api
Also remove extra redundant functionality

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[model/API] Sync model cc API with c API

Sync the cc-API of model with the c-API
Some unnecessary functions have been removed.
Some others have been renamed.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] Update layer cpp API

Update layer cpp API to match the c-api interface.
Move pluggableLayer related stuff to layer_internal.

See Also #986

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph] Remove graph edges after compile

This patch removes edges from the graph after compiling to
save the memory. This graph can be reconstructed by adding those edges
again from the information in the layers themselves.

See also #986

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph] Update setting output layers in graph + bug fix

Update setting output layers in graph to a unified location.
The graph is first made with just input layers.
Output layers are set at the end for the completeness at
one go than adding at each realization to avoid more errors.

Also, added a bug fix where the last layer in  the model
is not the last layer in the ini file. Now the last layer
is not the last layer in the adj but the layer without any
output_layers.

See also #986

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[model/ini] Update ini loader for model

Update ini loader for model to support more properties
than the ones which are hardcoded.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[ini] Add backward compatibility for optimizer

Added backward compatibility for optimizers.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[unittest] Added unittest for train without optimizer

Added unittest for training without optimizer and some
corresponding fixes.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[optimizer] Add optimizer section to INI

Add a new optimizer section to INI
This is essential to allow custom optimizers as the
properties of optimizers needs to be iterables.
Currently, optimizer needs to specified in the model, which
along being not the best design, limits the properties to
be set for the optimizer as the model loader has to know
which properties are for optimizer and which for model.

This patch separates the optimizer as a new section and updates
the unittests.

Note that this removes the default optimizer from adam. Now, it is
necessary for an optimizer to be defined in the model file, if to be used for training.

For inference, model can be defined without an optimizer.
Updated documentation for the changes made.

See also #986

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Test/Util] use exit in script

'return' in script is not available, call 'exit' instead.

Signed-off-by: Jaeyun <jy1210.jung@samsung.com>

[ LSTM ] Add Skeleton LSTM Layer Class

This PR includes skeleton code of lstm layer

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[Pooling2D] Change warning to error at init

For global_* pooling change warning to error for some parameter check

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Pooling2D] Rework global max pooling

**Changes proposed in this PR:**
- Rework global max pooling to work with lambda function
- Remove switch cases for the main loop in pooling

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Pooling2d] Rework Average Pooling

This patch refactors average pooling layer to not rely
on the pooling loop.

From this patch, max_idx is used to remember number of
effective elements used for the average calculation

Also add some checks on the `initailize`,
Also global average pooling is merged to average pooling

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Pooling2d] Rework pooling to avoid padding

This patch
1. set up scaffolding for the pooling to reuse some loop.
2. initialize index vector when setting batch (fixes a hidden
bug)
3. implements max pooling without additional memory allocation

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Pooling2D/Test] Add model tests

Add model test for various types of pooling2d

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

Handle BAD_OVERRIDE issue

Match type signatures with nntrainer::Optimizer by adding const
resolves: 458424, 458425

Self evaluation:
 1. Build test: [x]Passed [ ]Failed [ ]Skipped
 2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[rpm] Fix nnstreamer dependency

This patch resolves nnstreamer dependency problem. Thus making rpm to
enable plugin related test.

This patch is wating for nnstreamer/api#21

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[graph] Bug fix for no loss

Added bug fix for handling the scenario when there is no loss
specified. The issue was the difference in ordering of the loss
types in the parser and loss type enums.
This patch fixes it. Further, graph now simply returns if no loss found.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Handle Unchecked return value issue

Added return value of remove function
resolves: 1222078, 1223055

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[graph/ini] Support for default input_layers

This patch adds support for default input_layers
If no input_layers is specified, the layer above
the current layer in the ini file or the previously
adding layer in the model becomes the input layer for the current layer.
However, this connection is only made if the current layer
has no input layers.

Resolves #1046

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Conv2d/Test] Add conv2d multistride test

This patch enables conv2d uneven multistride test where some part of
images are dropped

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>