[spec/pkg] Bugfix for dependency

Some of the packages state the `requires` as wrong names (missing
package name as prefix). This does not show in build and unit/app tests
but fails when using these packages externally as those dependent
`requires` packaging dont exist.
For example, `nntrainer-devel-static` package depends on `devel`
package, and not on `nntrainer-devel`. This patch provides the
corresponding fix.

Resolves #1399

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Padding] Add padding compute function

This patch implements padding2d::compute with a test

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Add ml_train_model_get_input|output_dims

This commit contains capi proposal to
`ml_train_model_get_input_dimensions` and
`ml_train_model_get_output_dimensions`

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Resnet] Sync resnet app with validated model

This patch syncs resnet application with validated model.
Changes include:

1. s/setIteration/updateIteration
2. normalize input
3. fixed batchsize of 128 -> args received from the sysargs
4. output layer to copy
5. silent failed to read epoch idx and iteration

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Padding] Add padding verification

This patch implements `Padding2D::isValid` and tests respectively.

Padding2D property is valid when
1. string is "valid" or "same"
2. comma seperated, non-integer value of size 1, 2, 4

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix] File name sanitization

As colon (':') is not permitted in windows file system, having files
with 'colon' prohibits cloning our repo. this patch fixes the issue

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[docs] Add readme for resnet

This patch adds readme for resnet including citations.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Padding] Add padding2d prop header

This patch adds padding2d property header.

Padding2D prop will be saved as a string and compute the integer when it
is needed

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix] Weight initializer stddev calculation

The weight initializer was fitted to only fully connected layer case,
this patch extends weight initializer to properly work with other
weights as well.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Resnet] Connect the model with cifar100

**Changes proposed in this PR:**
- Implement cifar100dataloader
- Connect the data loader
- Display arguments/dataset information/time

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Test/Bn] Add conv2d model test

This patch adds conv2d bn test which was not properly tested

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[resnet] Implement test run to resnet

**Changes proposed**
- Add dataloader with interface `next()`.
- Add RandomDataLoader as an example
- Minor bug fixes to the model architecture regarding name
- Add routine to train the model

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ Recurrent ] Implement Dropout for Recurrent Net

In this commit, drop out for recurrent network is intrduced.
dropout property is introduced and if the element value of random
tensor is smaller than dropout rate, then it will be set zero.
The element which is not set zero, then it will scale with
1.0/(1.0-dropout).

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[Fix/trivial] Change rpm group description

**Changes proposed in this PR:**
- Change group description to machine learning/ML Framework

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Resnet] Create resnet model

This patch creates code to create resnet.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Resnet/skeleton] Add helper functions

**Changes proposed in this PR:**
- Add `withKey()`, `resnetBlock()`, `createResnet18()`

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Skeleton] Add resnet application skeleton

This patch adds resnet application skeleton

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix] coverity, svace issues

Coverity
1. Deleted noexception keyword where throw exception can occured.
2. Initialze member variable in constructor
3. Correct bitwise operation

resolves: 1238192, 1238193, 1238195, 1238196, 1238298

Svace
1. Catch unhandled throw

resolve: 464062

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[CustomLoss] Update example

**Changes proposed in this PR:**
- Update example using LayerClient
- Add LayerClient example to AppTest

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CustomLoss] Implement mae loss layer

This patch implements mae loss layer forward and backward, also moves
other functions to the sourcefile instead of inlinining it.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[layer_v2] Fixes to merge layer_v2 with main branch

Fixes to merge layer_v2 branch with main branch.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[LayerNode] Change flatten and distribute to prop

**Changes proposed in this PR:**
- Fix bug on `loadProperties`
- Add flatten and distribute to prop

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Optimizer] Implement getOptimizerVariableDim

- Implement getOptimizerVariableDim in optimizer
- Implement requestOptimizerVariable in manager

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[LayerNode] Add layer(devel) to the node

**Changes proposed in this PR:**
- Add `Layer` to the node
- Remove `node_exporter.h` from `layer_node.h`
- Change compile time switch to runtime switch of checking layer v1 and
layer v2

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[AppContext] Integrate layer-devel

This patch integrates layer devel to appcontext and plugin features.

From this patch, having it is able to include LayerV1 and Layer at the
same time.

Also adding a test

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[manager] Support initialization/allocation of weights

Support initialization and allocation of weights for LayerV2

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[network] Remove check for double activation

Remove the check for double activation, it is now allowed to have
two activation operations one after the other.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layercontext] Minor bugfix for layer context

Minor bugfix for layer context
Removes the extra trainable parameter from layer context

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[networkgraph] Network graph updated for Layer V2

Network graph updated to work with LayerV2
This involves settings input dimension in InitContext, and setting
up RunContext for each layer before their execution.

Further, some helper functions are also added in LayerNode.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Test/Prepare] Add layer semantics test

For layer_v2, there will be more structured way to access tests.
This patch adds a skeleton for layer semantics test.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Layer] Add trainable prop

This patch add trainable property to layer

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[manager] Add support for request Inputs/outputs

Added support for requesting inputs and outputs
Also update implementation to use a common function for easier
management

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer-internal] refactoring weight, input, output to layer_node

- Implement getter for variable and grad of weight, input, output
- Implement getNumWeights

Self evaluation:

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[graph] Support creating RunLayerContext

Implement the function to create RunLayerContext given the
InitLayerContext and update it in the given LayerNode.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[manager] Add support for request Inputs/outputs

Added support for requesting inputs and outputs
Also update implementation to use a common function for easier
management

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Properties] Add float and boolean cases

Add float and boolean case to be used later.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Props] Add concept of empty to property

This patch adds empty concept to property instead of having insensible
value inside a property as a default

before this patch,
```cpp
auto a = Property<int>();
EXPECT_EQ(a.get(), -1); // let's assume -1 is default value which might not be sensible.
```
after this patch

```cpp
auto a = Property<int>();
EXPECT_THROW(a.get(), std::invalid_argument);
EXPECT_TRUE(a.empty());
```

so underlying value remains always null or valid.

Now, it became natural to distinguish mandatory props and non-mandatory
props

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Layer] Scaffold layer impl

**Major changes**
- Create `LayerImpl`
- Change `LayerDevel` signature
  s/int initialize/void finalize
  s/int setProperty/void setProperty
- Move createLayer to layerImpl

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Layer] Add const to the layer::getName()

getName() is independent of const, so this patch adds const to
layer::getName();

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[layer-internal] refactoring getInputDimension getOutputDimension

implement getInputDimension, getOutputDimension in layer_node

Self evaluation:

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[neuralnet] Bug fix for graph usage

Bug fix patch to use constant iterators for the graph.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph] Remove non-const iterators

Remove non-const iterators for the graph.
Update the corresponding usages for the graph.

Note: the pattern for (auto const &node : graph) has been
removed as this can begin()/end() in the background.
Workaround for this is to use std::as_const(graph) but this
is only available from c++17, so this patch uses
the basic const iterator for loop pattern.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph] Graph iterator for sorted and unsorted

Updated graph iteartor to run over unsorted list of nodes
if the graph is not yet sorted. If the graph is sorted,
the iterator iterates over sorted nodes.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph] Update node_names to unordered_set

Update node_names storing all the names of the nodes
in the graph for uniqueness to use unordered_set.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph] Make adjacency list lazily

Update graph to make adjacenct list lazily when required
during the topological sort.
Until then, keep the node_list which contains simply all the nodes.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[manager] Support request Tensors and Weights

Added support for request Tensors and Weights
Moved spec of Tensors and Weights to VarGrad and Weights correspondingly
Also created constructors for weights and var_grad with their specs

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer_internal] refactoring read, save

implement read, save function in layer_node

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[Test/Scaffolding] Prepare test for the new layer

For now, layer test is too cluttered so that it is not agile enought to
make any change.

This PR mainly separates the tests into two parts to make the test
scallable itself.

1. Layer only test leaving inside `unittest_layers_{layer_name}`
2. Common tests like setProperty, golden_test in `layers_common_tests.h`

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Refactor] s/Layer/LayerV1

Change class name of layer to layerV1 to prepare migration
This should pass CI test.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[context] Layer context creation scaffolding

Added todo list for the layer context creation
Added minor new interfaces for the layer context
Disable in-place optimization and inference memory optimization

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[addition] Bug fix addition layer calcDerivative

Bug fix for addition layer calcDerivative().
addition layer assign the same tensor derivative memory back to its
other layers. If the other layers connecting with addition layers are
inplace, then this can lead to wrong results.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Plugin/Test] Add preset plugin for general test

This extracts common test to be used multiple times.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Custom/Loss] Add scaffolding for loss example

This patch generates a skeleton code for mae custom loss example.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CC17] Update type to inline static

This patch update `type` to be inline static data member inside an
object for clarity

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ GRU ] Add GRU Unittest

This commit includes,
  . unittests of gru layer
  . add keras code to compare

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ Recurrent ] Fix activation type

using hidden_state_activation_type

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ GRU ] implement fowarding / backwarding of GRU Layer

This commit includes,
  . forwarding implementation
  . calGradient implementation
  . calDerivative implementation
  . gru_basic unittest

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[Props] Add concept of empty to property

This patch adds empty concept to property instead of having insensible
value inside a property as a default

before this patch,
```cpp
auto a = Property<int>();
EXPECT_EQ(a.get(), -1); // let's assume -1 is default value which might not be sensible.
```
after this patch

```cpp
auto a = Property<int>();
EXPECT_THROW(a.get(), std::invalid_argument);
EXPECT_TRUE(a.empty());
```

so underlying value remains always null or valid.

Now, it became natural to distinguish mandatory props and non-mandatory
props

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ Unit Test ] Fix LSTM / RNN Unittest

This commit includes,
  . fix wrong unittest cases of lstm & rnn

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ GRU ] Skeleton Code for GRU

This commit includes,
 . Skeleton Code for GRU Layer

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[docs] Update docs for tflite version

Updated docs with tensorflow-lite version to be 2.3.0 or higher
and added missing dependencies for local meson build as well.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Refactor] s/Layer/LayerV1

Change class name of layer to layerV1 to prepare migration
This should pass CI test.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[API/Devel] Retouch devel header

**Changes proposed in this PR:**
- Change function sinature to exploit forward declaration
- s/export_to/exportTo
- Change header guard to be unique

**Self evaluation:**
1. Build test: [ ]Passed [ ]Failed [X]Skipped
2. Run test: [ ]Passed [ ]Failed [X]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[neuralnet] Neural network bugfix

Two PRs merged have caused main branch to fail.
This patch provides temporary fix.
Proper fix  will be done with layer_v2.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[unittests] Update unittests for new gtest version

Remove gtest version check comments and update unittests for the new
gtest versions.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Loss] Apply requireLabel to feed label

Currently, loss layer is the only one that can accept label which was
prohibiting building a custom loss layer.

**Major Changes**
- Apply requireLabel to feed label
- Last layer is not assumed loss layer when feeding label(still it is in
other places, need to be dealt)

**Minor Changes**
- Feeding multiple label is now possible (spliting label is future work)
- [Fix] Count Trainable includes loss layer
- [Fix] Handle cases When all layer is not trainable for
neuralnet::backwarding

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[layer] Move name to LayerNode

Move layer identifier name to LayerNode.
Now each node in the graph has name as its direct identifier.

This patch also disable unittest_nntrianer_layers for some while
till layer refactoring is taking place. However, other unittests
and apptests must remain enabled to ensure that everything else is
working.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[split layer] Initialize class member bugfix

Split layer has class member leading_helper_dim
which remains uninitialized and results in bugs
when setBatchSize() is called before initialization of the layer.

This patch fixes the bug.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer context] Propose layer context and layer_devel

This patch proposes layer context and layer_devel.
The layer context provides basic necessities for each layer
like inputs, outputs, weights, tensors. Weights are the trainable tensors
required by the layers which can be requested with the context interfaces.
Correspondingly, any other tensor requirements by the layer can also be
requested with the context.

There are two types of context :
1. InitContext - provides context for initialization with input_dimensions
filled. This contains specifications for all the inputs/outputs/weights/tensors
which will be made available during executiong.
2. RunContext - provides allocated weights/inputs/outputs/tensors along with
some of the basic properties of the layer like trainable, etc.

This design allows layer devel to be stateless (derived layer can be stateful with their
own member objects), and behave as layer ops.
Further, this allows dependencies on weights/var_grads as context simplified the interface
with just tensors.
Also, this patch also removes dependency on the manager from layer_devel header.

layer context for each layer is owned by the corresponding LayerNode.
Many of the layer getters like getInputs/Outputs/Weights/etc can now
be moved to LayerNode which does not concern the custom layer writer,
and simplifies the layer operation code as well.

See also #986

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[C++17] Bump base version to c++17

This patch bumps baseline version to c++17.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[deb/pkg] Updates for debian packaging

Update tolerance for sqrt unittest for debian arm64 packaging
increase timeout for appdraw classification app for debian armhf packaging

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Props] Implement input_layers as a property

This patch implements input_layers as a property with following changes
below

**Changes proposed in this PR:**
- Add ConnectionSpec object for generic purpose
- Add InputSepc as a property
- Add following tests

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Test/models] Add addition test

This patch adds addition test, while add support for reading multiple
inputs and outputs inside `unittest_nntrainer_models.cpp`

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Retouch] unittest_models test cases

**Changes proposed in this PR:**
- Reduce scale of inputs
- Add iterations for recurrent layers (and delete _n) family tcs
- Clean duplicated codes in genModelTests.py

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[deb/pkg] Increase c-api unittest timeout

Increase c-api unittest timeout even further to
pass armhf build on launchpad.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Docs] Update custom documentation for loss layer

This patch updates documentation of custom for loss layer.
Also deletes redundant `pow.cpp` that sneaked in.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[deb] Update lower tolerance bug + increase timeouts

Inverse the check for lower error tolerance for debian build
Also increase timeouts for nntrainer capi/ccapi unittests
as they timeout for armhf architecture on launchpad.

See also #1251

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Trivial] Change return -> retval

When stating value in doxygen it is recommended to use @retval, this
patch reflects the issue

**Self evaluation:**
1. Build test: [ ]Passed [ ]Failed [X]Skipped
2. Run test: [ ]Passed [ ]Failed [X]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[interpreter] Updated export_to usage with LayerNode

Updated export_to function usage based on LayerNode

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layerNode] Refactor get/set number of inputs/outputs

Refactor get/set number of inputs/outputs.
This is now handled by layer_node.
Updating the input/output layers updates the count.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer/node] Move input/output layers to LayerNode

This patch moves input/output layers to LayerNode.
Also create an interface for accessing/setting input/output layers
from in the LayerNode.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[api] Added api dependency in pc files

Added API dependecy in pc files for the capi and ccapi.
The dependency is of the capi-ml-common which provides ml-api-common.h header.
Further, ml-api-common.h header from nntrainer repo is also removed.
Updated android build to download ml-api-common.h for nnstreamer daily builds.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[ RNN ] Add Multi-Layer RNN Unit Tests

This commit includes,
 . Multi-Layered RNN Unit Tests
 . Some Fixes to run

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[Trivial] Fix NDK warning -Wbraced-scalar-init

This patch fixes trivial NDK warning below

```
././../nntrainer/layers/lstm.cpp:315:25: warning: braces around scalar initializer [-Wbraced-scalar-init]
  Tensor dh_nx = Tensor({derivative_.width()});
                        ^~~~~~~~~~~~~~~~~~~~~
././../nntrainer/layers/lstm.cpp:316:25: warning: braces around scalar initializer [-Wbraced-scalar-init]
  Tensor dc_nx = Tensor({derivative_.width()});
                        ^~~~~~~~~~~~~~~~~~~~~
././../nntrainer/layers/lstm.cpp:352:26: warning: braces around scalar initializer [-Wbraced-scalar-init]
        hs_prev = Tensor({hs_t.width()});
                         ^~~~~~~~~~~~~~
././../nntrainer/layers/lstm.cpp:354:26: warning: braces around scalar initializer [-Wbraced-scalar-init]
        cs_prev = Tensor({cs_t.width()});
```

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[test/model] Cleanup unittest_nntrainer_models

Cleaned up unittest_nntrainer_models.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] Support inPlace() as an interface for the layer

Support inPlace() as an interface for the layer.
If returns true, manager can set its inputs/outputs to be
the same memory location, but not necessarily.
Corresponding graph code is also updated.

Resolves #867

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Layer] Add requireLabel() function

Add a simple function `Layer::requireLayer()`.
This function is used to detect if a certain layer requires a layer.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[APP] Add tensorflow based product rating

Added tensorflow based product rating.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[SplitLayer] Support flexible dimension

This patch adds support for the flexible dimension in split layer.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Cc: Parichay Kapoor <pk.kapoor@samsung.com>
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Permute] Add permute layer forward/backward

This patch implements forward/backward with a following test

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ RNN ] Add unittest case & return_sequence

This PR includes.
  . unittest of rnn layer
  . support of return sequence of rnn layer

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[Recorder] Add multiout equivalent in transLayer

When getting gradient from `gradientTape` it adds all derivatives of the
dependent layer. While this behavior is true, we need to measure to what
extent each layer contributed to the layer.

This patch adds a multiout layer which consists of some stublayer that
seperates derivatives one by one.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ LSTM ] Add Multi-Layerd LSTM Unittest

This commit includes,
  . Multi-Layerd LSTM Unit test.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[Permute] Implement peripheral functions

This patch implements initialize, export and setProperty of permute
layer

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[layer] Move distribute to LayerNode

move distribute to LayerNode from layer
Further, the distribute layer is now supposed to be used
as a wrapper of a layer maintained by the LayerNode,
and must not be maintained externally.
Rather, use getDistribute() to check if the node is distributed
and treat the layer as a regular layer.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Layer] Add permute layer scaffolding

This patch adds permute layer scaffolding

**Minor Fix**
- Add error handling for transpose tensordim

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ LSTM ] Add More Unit Test Cases

This Commit inclues,
 . Additional unittest
   - with return sequneces
   - with multiple batch size
   - with multiple batch + return sequences
   - with multiple batch + return sequences + multi-iteration
 . Fixed initialization bug during training
 . Remove check for tensor dot product to support multi-dimension
 tensor.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ LSTM ] Add Unit test for LSTM Layer

This commit includes,
  . unit test case for lstm layer
  . fixup of potential bug : tanh prime calculation
  .

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ LSTM ] Add return_sequence

This commit includes,
  . impelementation of return_sequence. If it is true, then it
  generate all the output of time iteration and if it false, then
  only last output is available.
  . add 'return_sequence' keyward

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ LSTM ] LSTM Unit Test Cases

This PR includes,
 . LSTM Unittest Case
 . python code in genModels

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>