[context] Layer context creation scaffolding

Added todo list for the layer context creation
Added minor new interfaces for the layer context
Disable in-place optimization and inference memory optimization

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[addition] Bug fix addition layer calcDerivative

Bug fix for addition layer calcDerivative().
addition layer assign the same tensor derivative memory back to its
other layers. If the other layers connecting with addition layers are
inplace, then this can lead to wrong results.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Plugin/Test] Add preset plugin for general test

This extracts common test to be used multiple times.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Custom/Loss] Add scaffolding for loss example

This patch generates a skeleton code for mae custom loss example.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CC17] Update type to inline static

This patch update `type` to be inline static data member inside an
object for clarity

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ GRU ] Add GRU Unittest

This commit includes,
  . unittests of gru layer
  . add keras code to compare

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ Recurrent ] Fix activation type

using hidden_state_activation_type

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ GRU ] implement fowarding / backwarding of GRU Layer

This commit includes,
  . forwarding implementation
  . calGradient implementation
  . calDerivative implementation
  . gru_basic unittest

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[Props] Add concept of empty to property

This patch adds empty concept to property instead of having insensible
value inside a property as a default

before this patch,
```cpp
auto a = Property<int>();
EXPECT_EQ(a.get(), -1); // let's assume -1 is default value which might not be sensible.
```
after this patch

```cpp
auto a = Property<int>();
EXPECT_THROW(a.get(), std::invalid_argument);
EXPECT_TRUE(a.empty());
```

so underlying value remains always null or valid.

Now, it became natural to distinguish mandatory props and non-mandatory
props

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ Unit Test ] Fix LSTM / RNN Unittest

This commit includes,
  . fix wrong unittest cases of lstm & rnn

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ GRU ] Skeleton Code for GRU

This commit includes,
 . Skeleton Code for GRU Layer

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[docs] Update docs for tflite version

Updated docs with tensorflow-lite version to be 2.3.0 or higher
and added missing dependencies for local meson build as well.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Refactor] s/Layer/LayerV1

Change class name of layer to layerV1 to prepare migration
This should pass CI test.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[API/Devel] Retouch devel header

**Changes proposed in this PR:**
- Change function sinature to exploit forward declaration
- s/export_to/exportTo
- Change header guard to be unique

**Self evaluation:**
1. Build test: [ ]Passed [ ]Failed [X]Skipped
2. Run test: [ ]Passed [ ]Failed [X]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[neuralnet] Neural network bugfix

Two PRs merged have caused main branch to fail.
This patch provides temporary fix.
Proper fix  will be done with layer_v2.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[unittests] Update unittests for new gtest version

Remove gtest version check comments and update unittests for the new
gtest versions.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Loss] Apply requireLabel to feed label

Currently, loss layer is the only one that can accept label which was
prohibiting building a custom loss layer.

**Major Changes**
- Apply requireLabel to feed label
- Last layer is not assumed loss layer when feeding label(still it is in
other places, need to be dealt)

**Minor Changes**
- Feeding multiple label is now possible (spliting label is future work)
- [Fix] Count Trainable includes loss layer
- [Fix] Handle cases When all layer is not trainable for
neuralnet::backwarding

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[layer] Move name to LayerNode

Move layer identifier name to LayerNode.
Now each node in the graph has name as its direct identifier.

This patch also disable unittest_nntrianer_layers for some while
till layer refactoring is taking place. However, other unittests
and apptests must remain enabled to ensure that everything else is
working.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[split layer] Initialize class member bugfix

Split layer has class member leading_helper_dim
which remains uninitialized and results in bugs
when setBatchSize() is called before initialization of the layer.

This patch fixes the bug.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer context] Propose layer context and layer_devel

This patch proposes layer context and layer_devel.
The layer context provides basic necessities for each layer
like inputs, outputs, weights, tensors. Weights are the trainable tensors
required by the layers which can be requested with the context interfaces.
Correspondingly, any other tensor requirements by the layer can also be
requested with the context.

There are two types of context :
1. InitContext - provides context for initialization with input_dimensions
filled. This contains specifications for all the inputs/outputs/weights/tensors
which will be made available during executiong.
2. RunContext - provides allocated weights/inputs/outputs/tensors along with
some of the basic properties of the layer like trainable, etc.

This design allows layer devel to be stateless (derived layer can be stateful with their
own member objects), and behave as layer ops.
Further, this allows dependencies on weights/var_grads as context simplified the interface
with just tensors.
Also, this patch also removes dependency on the manager from layer_devel header.

layer context for each layer is owned by the corresponding LayerNode.
Many of the layer getters like getInputs/Outputs/Weights/etc can now
be moved to LayerNode which does not concern the custom layer writer,
and simplifies the layer operation code as well.

See also #986

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[C++17] Bump base version to c++17

This patch bumps baseline version to c++17.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[deb/pkg] Updates for debian packaging

Update tolerance for sqrt unittest for debian arm64 packaging
increase timeout for appdraw classification app for debian armhf packaging

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Props] Implement input_layers as a property

This patch implements input_layers as a property with following changes
below

**Changes proposed in this PR:**
- Add ConnectionSpec object for generic purpose
- Add InputSepc as a property
- Add following tests

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Test/models] Add addition test

This patch adds addition test, while add support for reading multiple
inputs and outputs inside `unittest_nntrainer_models.cpp`

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Retouch] unittest_models test cases

**Changes proposed in this PR:**
- Reduce scale of inputs
- Add iterations for recurrent layers (and delete _n) family tcs
- Clean duplicated codes in genModelTests.py

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[deb/pkg] Increase c-api unittest timeout

Increase c-api unittest timeout even further to
pass armhf build on launchpad.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Docs] Update custom documentation for loss layer

This patch updates documentation of custom for loss layer.
Also deletes redundant `pow.cpp` that sneaked in.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[deb] Update lower tolerance bug + increase timeouts

Inverse the check for lower error tolerance for debian build
Also increase timeouts for nntrainer capi/ccapi unittests
as they timeout for armhf architecture on launchpad.

See also #1251

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Trivial] Change return -> retval

When stating value in doxygen it is recommended to use @retval, this
patch reflects the issue

**Self evaluation:**
1. Build test: [ ]Passed [ ]Failed [X]Skipped
2. Run test: [ ]Passed [ ]Failed [X]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[interpreter] Updated export_to usage with LayerNode

Updated export_to function usage based on LayerNode

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layerNode] Refactor get/set number of inputs/outputs

Refactor get/set number of inputs/outputs.
This is now handled by layer_node.
Updating the input/output layers updates the count.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer/node] Move input/output layers to LayerNode

This patch moves input/output layers to LayerNode.
Also create an interface for accessing/setting input/output layers
from in the LayerNode.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[api] Added api dependency in pc files

Added API dependecy in pc files for the capi and ccapi.
The dependency is of the capi-ml-common which provides ml-api-common.h header.
Further, ml-api-common.h header from nntrainer repo is also removed.
Updated android build to download ml-api-common.h for nnstreamer daily builds.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[ RNN ] Add Multi-Layer RNN Unit Tests

This commit includes,
 . Multi-Layered RNN Unit Tests
 . Some Fixes to run

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[Trivial] Fix NDK warning -Wbraced-scalar-init

This patch fixes trivial NDK warning below

```
././../nntrainer/layers/lstm.cpp:315:25: warning: braces around scalar initializer [-Wbraced-scalar-init]
  Tensor dh_nx = Tensor({derivative_.width()});
                        ^~~~~~~~~~~~~~~~~~~~~
././../nntrainer/layers/lstm.cpp:316:25: warning: braces around scalar initializer [-Wbraced-scalar-init]
  Tensor dc_nx = Tensor({derivative_.width()});
                        ^~~~~~~~~~~~~~~~~~~~~
././../nntrainer/layers/lstm.cpp:352:26: warning: braces around scalar initializer [-Wbraced-scalar-init]
        hs_prev = Tensor({hs_t.width()});
                         ^~~~~~~~~~~~~~
././../nntrainer/layers/lstm.cpp:354:26: warning: braces around scalar initializer [-Wbraced-scalar-init]
        cs_prev = Tensor({cs_t.width()});
```

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[test/model] Cleanup unittest_nntrainer_models

Cleaned up unittest_nntrainer_models.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] Support inPlace() as an interface for the layer

Support inPlace() as an interface for the layer.
If returns true, manager can set its inputs/outputs to be
the same memory location, but not necessarily.
Corresponding graph code is also updated.

Resolves #867

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Layer] Add requireLabel() function

Add a simple function `Layer::requireLayer()`.
This function is used to detect if a certain layer requires a layer.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[APP] Add tensorflow based product rating

Added tensorflow based product rating.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[SplitLayer] Support flexible dimension

This patch adds support for the flexible dimension in split layer.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Cc: Parichay Kapoor <pk.kapoor@samsung.com>
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Permute] Add permute layer forward/backward

This patch implements forward/backward with a following test

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ RNN ] Add unittest case & return_sequence

This PR includes.
  . unittest of rnn layer
  . support of return sequence of rnn layer

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[Recorder] Add multiout equivalent in transLayer

When getting gradient from `gradientTape` it adds all derivatives of the
dependent layer. While this behavior is true, we need to measure to what
extent each layer contributed to the layer.

This patch adds a multiout layer which consists of some stublayer that
seperates derivatives one by one.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ LSTM ] Add Multi-Layerd LSTM Unittest

This commit includes,
  . Multi-Layerd LSTM Unit test.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[Permute] Implement peripheral functions

This patch implements initialize, export and setProperty of permute
layer

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[layer] Move distribute to LayerNode

move distribute to LayerNode from layer
Further, the distribute layer is now supposed to be used
as a wrapper of a layer maintained by the LayerNode,
and must not be maintained externally.
Rather, use getDistribute() to check if the node is distributed
and treat the layer as a regular layer.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Layer] Add permute layer scaffolding

This patch adds permute layer scaffolding

**Minor Fix**
- Add error handling for transpose tensordim

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ LSTM ] Add More Unit Test Cases

This Commit inclues,
 . Additional unittest
   - with return sequneces
   - with multiple batch size
   - with multiple batch + return sequences
   - with multiple batch + return sequences + multi-iteration
 . Fixed initialization bug during training
 . Remove check for tensor dot product to support multi-dimension
 tensor.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ LSTM ] Add Unit test for LSTM Layer

This commit includes,
  . unit test case for lstm layer
  . fixup of potential bug : tanh prime calculation
  .

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ LSTM ] Add return_sequence

This commit includes,
  . impelementation of return_sequence. If it is true, then it
  generate all the output of time iteration and if it false, then
  only last output is available.
  . add 'return_sequence' keyward

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ LSTM ] LSTM Unit Test Cases

This PR includes,
 . LSTM Unittest Case
 . python code in genModels

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ LSTM ] Fix implementation & add backwarding testcase

This commits includes:
  . fix implementation ( apply actvation )
  . add one unittest to check functionality of backwarding

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[Exporter] Implement saving fc node

**Changes proposed in this PR:**
- This patch add saving fullyconnected layer with node_exporter
- add dedeicated buffer to `TfNode`

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[app] reduce Logistic regression application epochs

Reduce logistic regression application epochs.
This application is to check working than verify accuracy and can
run with fewer epochs.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Recorder] Add Input map generation

There was a bug that layer input is directly from the previous layer.
If the model is not sequential, this assumption is not true, making
Addition test fail.

This patch build `model.recoder__input_maps` to correctly locate
 the input tensor with some minor clean ups

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[meson] Enable tflite-interpreter by default

Enable tflite-interpreter by default as tflite layer is
already enabled.
Move flatc dependency out to the main meson.build
Also add proper path for the generated tf_schema

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] Move flatten property to layerNode

Move flatten property out to layerNode
also prepare for moving other properties out to layerNode

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Refactor] Delegate tfNode creation to exporter

This patch enables delegating exporting logic to exporter.
Note that this is supposed to be 1:1 mapping, if it has to be m:n
mapping, it should be directly handled by the interpreter, which
actually can see the node.

Also s/`save_result`/`saveResult` and `get_result`/`getResult`

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
[Android] Add tflite interpreter option

This patch add tflite interpreter option to ndk build

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix] Initialize member variable

 - Initialize member variable sorted
 - Catch exception when create the tensor

resolves: 1231330, 1143556

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[deb/pkg] Reduce error tolerance for release build

Reduce error tolerance for release builds to succeed build on launchpad and tizen builds.
The default error tolerance is only reduced for all release builds for all tests that use defined tolerance.
However, the CI will run at original tolerance for all the builds and will ensure that
values are matched with original precision.

The tolerance has to be reduced because different os
have different versions and implementations of the blas
which results in minor difference in the results of the float
operations. However, when the difference is accumulated over
a range of operations, the unittests fail to match the
exact values.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Tflite] Separate `tfopnode`

This patch separates tfopnode to a file.

tfOpNode will be directly built from the exporter if it doesn't need to
be fused.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[tflite] Update dependency from tflite to tf2lite

Update dependency from tensorflow-lite to tensorflow2-lite.
This is done for Tizen and Debian build. It has already been done for Android.

Tensorflow2-lite header is dependent on flatbuffer, so flatbuffers-devel
is additionally added as a build-dependency.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[pkg/spec] Update spec packaging for tflite layer

Update spec packaging for tflite layer with tflite dependency

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[test] Remove run_unittests

Remove run_unittests

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[deb/pkg] Minor update to deb packaging

Update debian packaging:
- update the unittests to use meson tests
- update dh_missing to fail than give warnings

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[unittest] Update unittest tolerance for debian build

Update unittest tolerance to be 1e-5 for debian build to match
across platforms (bionic, focal) and architectures.
Similar issues have occured for i586 tizen as well.
So, this patch lowers the tolerance for all ccapi and capi test.

This issue occurs to os and library versioning issues.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[deb/pkg] Add nnstreamer filter and update testing

Add nnstreamer filter in the packaging
Also update the unittests to use the meson tests

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[ Release ] NNTrainer 0.2.0 Release

NNTrainer v0.2.0 is released.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[deb/pkg] Update debian install filenames

Update debian install filenames for the capi and ccapi.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Product Ratings] Fix inconsistent batch

This patch fixes inconcistent batch size between source file and model
configuration.

Also found a bug #1238, which will be dealt seperately.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[debian] Refactor debian packaging

Refactor debian package to make separate packages for capi and ccapi, along with their devels.
Other bug fixes/improvements are:
- Update the names of the packages in pkg-config files for the capi and ccapi
- Update the installed files based on the requirement by the devel headers
(this might be even expanded). This will be refactored again once layer_internal and neuralnet
cleanup is performed.
- Update the included headers

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[APP] Change embeding to product rating

This patch changes embedding application to product rating application
which is more practical use-case

**Changes proposed in this PR:**
- Update Readme
- Change directory and app name embedding->productratings
- tune model and sample dataset

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[tflite] implement build operations in subgraph

This patch implements build operation in the subgraph

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[context/optimizer] Enable plugged optimizer

Enable plugged optimizer with the app_context

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[optimizer] Update optimizer setProperty interface

Update optimizer setProperty interface based on the requirement
from the custom optimizer.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Example] Concat example

This patch adds concat example with embedding

The structure is

      (input)
     /       \
  (embed1)   (embed2)
    \         /
      (concat)
         |
      (flatten)
         |
        (fc)

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix] Prevent creating multiout for split

This patch make an exception for split layer to not create multiout
layer. While fixing trivial build error of sign compare

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[split/test] Added unittests for split layer

Added unittests for split layer covering initialization,
forwarding and backwarding. Corresponding bug fixes are
also added.
This has revealed an issue with concat layer #1226
which will be resolved soon.

Also add split layer to android build

Resolves #1208

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[split layer] Add forwarding and backwarding

Add forwarding and backwarding for the split layer

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[split layer] Added split layer scaffolding

Added split layer scaffolding with its properties and initialization

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[App] Make embedding golden data optional

This patch makes embedding golden data optional while changing return
status

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ LSTM ] Fix implementation & add backwarding testcase

This commits includes:
  . fix implementation ( apply actvation )
  . add one unittest to check functionality of backwarding

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ LSTM ] Backwarding Implementation

This commit includes
 . calDerivative & calGradient of LSTM Layer
 . Modification of Forwarding of LSTM Layer

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[VarGrad] Add clone Transpose

This patch adds clone transpose to `var_grad`, this function clones the
current `var_grad` but transpose and save

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Tflite] Add model input, output

This patch add model to track input and output

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix/graph] fix index after removing node

**Changes proposed in this PR:**
- After removing last node, adj node after the target node becomes not
correct, this patch fixes the issue

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Test/Model] Prepare addition model

This patch add addition model test

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix] Output layer setup correctly

There was a bug that output layer is not set correctly because of the
timing issue and overlapping checker issue.
This patch sets output layer correctly.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[LayerNode] Add print function

This patch adds simple print function to be used for debugging
Note that, input_layers are from `layer_internal` for now, but it's
private so it is commented out

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Test/models] prepare to support non-linear models

**Changes proposed in this PR:**
- Make `transLayer` from proxy to a legit layer
- Add output map so that in each layer can locate it's output from model
- Support non-linear output generation
- Add support for multi input, multi output

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[graph] Add non-const iterators

Add non-const iterators for the graph.
Most of the locations use const-iterators.
This will be updated when layer_node is updated to use its
properties than layer.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph] Graph cleanup getNodes()

Graph remove support for getNodes() but rather
use iterators to iterator over the graph.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph] Cleanup graph usage

Cleanup graph usage

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph] Migrate to graph core

Migrate neural network independent and generic sections of the graph
to graph core. This is not yet complete and will be done over a few commits.
This will cleanup NetworkGraph class and will allow optimizations to
be done on the GraphCore/NetworkGraph class easily, and simplify
the classes as well.

Now, graph related structures can move out from layer to layer node.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Addition] fix forwarding

Correct initialization of output

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[Tflite] Fix tf shape building

This patch fixes tf shape building, by adding `eff_dim_flag` and
`dyn_dim_flag`.

`eff_dim_flag` checks which dimension are actually used so that to be
squeezed.
`dyn_dim_flag` checks which dimension should be set to -1 (meaning it is
not fixed)

With those flag set, our tensor dim is able to convert to the dimension
representation that supports `-1` and not fixed size

Those fixes are apply to fc layer for now

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix] Add missing header

This patch adds missing header for logger when `enable_logging=false`

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Props] Add Name validator and unittest

**Changes proposed in this PR:**
- Add name validator
- Add source file for `common_properties`
- Change name of `unittest_properties.cpp` ->
`unittest_base_properties.cpp`
- Add `unittest_common_properties.cpp`

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Conv2d/Test] Add more test

This patch adds test suggested at #1042

Resolves #1054

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Conv2d] Clean up conv2d relates

This patch handles issue left at #1042

**Changes proposed in this PR:**
- [x] move commented code upward
- [x] remove channel argument
- [x] Add conv negative test
- [ ] Add more test

See also: #1042, #1054

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>