[Backbone/CAPI] Implement backbone layer creation

**Changes proposed in this PR:**
- Connect enum to nnstreamer backbone
- Change `model_file` property to `model_path` for generality
- Add handling for when model_file property is loaded from interpretation

Resolves #1072

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Cc: Inki Dae <inki.dae@samsung.com>
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Dataset] Implement samplewise producer for random

This patch implements samplewise producer for random and its
corresponding test abstraction

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Trivial] layer client test was not running

This patch changes test variable argument in order for layer client
tests to run

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[context/graph] Catch lifespan + 3-way execution order

This patch adds the setup to catch lifespan set by the layer developer
when requesting for tensors.
Further, the execution order now consists of 3 values - fowarding,
calcGradient and calcDerivative. The memory optimization will now be
done using the 3 functions to be called in order.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[manager] Set the default usage timestamp for tensors

Set default usage time index for all the tensors based on when the layer
is to be run in the sorted list.
The usage time indexes are stored in each GraphNode by the GraphCore.
When creating the sorted list, each layer is updated to store its
order location/index when executed in this sequence for forwarding as
well as backwarding.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layers] Make requested weight/tensor names unique

Update the layers to make the requested weights and tensors names to be
unique. This done using the layer name with the postfix of the
weights/tensors name making the combination unqiue as layer name is
unique.

InitLayerContext is updated to provide access to the layer name while
finalizing the layer.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph/neuralnet] Move manager from neuralnet to netgraph

Move manager from NeuralNetwork class to NetworkGraph class.
Some corresponding minor adjustments are also added.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Dataset] Add interface for samplewise producer

This patch adds interface to migrate batchwise producer to samplewise
producer

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Dataset] Add user_data set Property handler

**Changes proposed in this PR:**
- PtrType properties
- Add user_data set property handler for backward competibility

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Reflect privilege reviews

This patch update document to add privilege to `ml_train_model_save()`
and `ml_train_model_load()`

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[fix] ahub issues

 - Added try catch statement
 - Check value to avoid divide by zero
 - Added const keyword to match override function
 - Initialize member variable in constructor

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[graph/neuralnet] Move manager from neuralnet to netgraph

Move manager from NeuralNetwork class to NetworkGraph class.
Some corresponding minor adjustments are also added.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph] Update check for input node

Update the check for checking if a given node is input node.
The check uses in-degree for the node to be 0.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graphnode] Remove index from graph node

Currently each graph node has two identifiers - name and idx - where
both were unique. This patch removes the idx as the indentifier.
The corresponding functions are also removed.
In order to support fast fetching of nodes, graphcore has a map nodes.

This change revealed a bug with the names in the code. As ini does not
allow case sensitive names, the names from the ini are changed to lower
case. However, input_layers property names were allowed to be
case-sensitive. Further names set by the API were also case sensitive.
This patch forces all such names to be case-insensitive to lowercase.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph] Remove layerv1 initialize code

Remove layerv2 initialize code for network graph.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[CAPI] Assign values to the enum

This patch assigns values to enums in nntrainer-api-common.h

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Update regarding the review script

This patch updates nntrainer.h header with regard to the review script

- @a -> @c
- merge conflict with usage / mode

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Trivial] dataset usage enum rename

This patch renames enumeration as discussed ahead of time

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Add permission denied return value

As privilege is sometimes needed by dataset, add permission denied for
`ml_train_dataset_add_file()` and `ml_train_model_construct()`

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Change privilege to remark

This patch change privilege to remark as recommended

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Add privilege to documentation

This patch adds privilege to documentation as guided

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI/ACR] Fix typo and doxygen tag

This patch reflects review from #1371, #1446

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Trivial] Reflect acr review script

This patch reflect acr review scripts, which contains some trivial fixes
like adding a period at the end.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI/ACR] Add remark on tensor info getter

This patch adds remark on tensor info getter that info should not be
destroyed when fail to create

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[fixes] Add bug fixes from asan

**Changes proposed in this PR:**
- Fix issues from address sanitizer
- Enable deprecated-declaration warning but disable for some files

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[fixes] Add bug fixes found with address sanitizer

This patch adds bug fixes after adding fixes related to address
sanitzer:
- NNStreamerLayer wasnt freeing resources upon destruction
- accessing released model handle in capi unittest
- not releasing model handle before end of unittest

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Dataset] Remove unused part from refactoring

This patch removes parts that are no longer used from the refactoring

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Dataset] Adopt new datagetter to databuffer

**Changes Propsed in this PR**
1. Adopt new data getter to neuralnet / factory
2. Fix: bugs found in raw_file_data_producer
3. Fix: databuffer not feeding last iteration when error

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[tensor/test] Unittests for tensor allocate and initialize

Add unittests for tensor allocation and initialize the tensor memory in
various scenarios.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] Add support for initializer in requestTensor

This patch adds support for initializer in the requestTensor interface
of the layer context.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[weight] Weight cleanup related to initializer

This patch provides weight cleanup related to the initializer.
Weights dont take care of the initializer but rather let var_grad handle
it which in turn let tensor handle it.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[var_grad] Support initializer for var_grad

Support initializer for variables and gradient in var_grad with the
initializer of the tensor. This allows initialization to be done for
requested tensors as well, which will be updated in a later commit.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Tensor] Support initializer with tensor

Support initializer with tensor object natively.
Initialization will be done when the memory is allocated or can be
invoked manually as well.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[tensor] WeightInitializer refactor to Tensor::Initializer

Rename/cleanup tensor functions to adhere with STL like function names.
Further, weight initializer has officially become tensor initializer.
The usage will be updated soon.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[tensor/weight] Update WeightInitializer to TensorInitializer

Update weightInitializer to TensorInitializer.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Dataset] Change factory signature

Add properties to factory, given contructor argument, it is delegated to
the constructor

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Cc: Parichay Kapoor<pk.kapoor@samsung.com>
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[devel/pkg] Update the devel packaging

Update the devel packaging to remove unnecessary headers.
- weight/var_grad headers are removed
- app_context and other graph, network related headers are removed. They
can be added once network/neural network headers are refactored for
custom models.
- manager and dynamic_training_optimization header is removed
- databuffer headers are commented. They will be updated once dataset
refactoring is done.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[unittest] enable commented unittest

 - Using ValuesIn function instead of Values cause the arguments are more than 50
 - Handle unused variables

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[tensordim] move implementation to source file

 - header file remains without implementaion

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

Debian: you don't have debian-build meson option.

Remove unsupported meson build option from debian script.

Signed-off-by: MyungJoo Ham <myungjoo.ham@samsung.com>

[mesonbuild] update meson build

 - Make platform meson option
 - Check sysroot

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[optimizer] Rename initialize to finalize

Rename initialize to finalize.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] Update layer.h setProperty interface

Update the interface for the layer.h setProperty.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[optimizer] Cleanup optimizer and remove dependency on weight

Cleanup optimizer interface where some extra interfaces have been
removed, and arguments interface has been updated.
Further, the dependency on weight header has been removed, and added an
optimizer context interface which provides an interface to weight
related tensors required by the optimizer.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Dataset] Implement data getter

This patch implements data getter.
There are two types of data getter.

1. queue multiple iterations and get one by one
(`DataBuffer::startFetchWorker()`, `DataBuffer::fetch()`)

2. generate one iteration on the fly (`DataBuffer::batcher()`)

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[tensorfilter] Update tensor filter to use ccapi

Update nntrainer's tensor filter for nnstreamer to depend only on ccapi
and nntrainer directly and internal headers.

See Also #986

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[DataSet] Refactor to use properties

This patch enables properties in dataset while changing signature of
dataset

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Dataset] Add thread safe queue for the buffer

Currently, databuffer is not queuing any iteration but requesting buffer
on-demand basis.
This patch adds thread safe queue for the buffer to queue multiple
buffers

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[weight/var_grad] Remove exposure of weight/var_grad

This patch updates the usage of weight and var_grad headers in order to
hide them internally and not expose them in the devel headers.
The below changes are made to support this:
- weight and var_grad specs are declared separately in tensor wrapper
specs header.
- layer_context is made indepedent of the weight and var_grad
definition. The usages and implementation are moved to layer_context
souce file.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[utils] Split parse_utils into two for cleaner headers

Carve props_utils out of parse_utils to get cleaner headers
being shipped out with devel package.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[optimizer] Deprecate optimizer factory

Deprecate usage of optimizer factory and replace it with app_context.
Further, app_context throw code update from runtime_error to
invalid_argument when the requested type of object is not found.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[test] capi unittest fix

C-API unittest was checking for throw but instead should be checking for
return value. This patch fixes the test.
Further, the unittests tries to free unallocated tensordim in error
cases. This patch resolves it too.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[CAPI] Move save/load api to public

This patch moves save/load api to public

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[tensor_filter] Remove dependency on nntrainer::Tensor

Remove tensor filter implementations dependency on nntrianer::Tensor.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[tensor filter] Bugfix for nnstreamer's tensor filter

This patch adds bugfix for the nnstreamer's tensorfilter of nntrainer.
- include proper headers required by the tensor filter header
- use the correct offset for the multi-input scenario

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[c++/model] Add model inference to c++ API

Add model inference to the c++ API using the vector of float ptrs as
inputs and outputs.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[loss] BugFix for regularization loss

This patch adds bugfix for regularization loss calculation.
Regularization loss must be calculated before the weight is updated in
backwarding, but this was not being done.
This patch provides the corresponding fix.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[test] Enable c/cc diabled tests

This patch enables c/cc disabled tests.
Corresponding required updates in the tests are also added.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[dataset] Add file producer

This patch adds file producer which abstracts reading a raw file. Also,
this component makes sure that the file accepts various kind of input
shapes.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Implement ml_train_model_get_input|output_dims

 - Implement ml_train_model_get_input_dimensions
 - Implement ml_train_model_get_output_dimensions
 - Make a unittest for implemented apis

Self evaluation:

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[tensor_dim] package tensor_dim.h with ccapi

 - Package tensor_dim with ccapi
 - MAXDIM belongs to tensorDim class

Self evaluation:

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[CAPI] Expose layer enums

This patch contains layer enums to be exposed

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Change tensor dim -> information

This patch updates description of the tensors_info, while explicitly
stating that the information object is newly created

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Cc: Hyeonseok Lee<hs89.lee@samsung.com>
Cc: Parichay Kapoor<pk.kapoor@samsung.com>
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ Layer ] Change OutputLayer to MultiOutLayer

Currently the feature of OutputLayer is not match. It is more like
Multiout layer. The type name and others are already multiout.

This commit includes,
  . Class name changed : OutputLayer --> MultiOutLayer
  . File name changed : output_layer.x --> multiout_layer.x

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[CAPI] Add ml_train_model_get_layer to internal

This patch adds ml_train_model_get_layer to internal api

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Deprecate old dataset apis

This patch updates to deprecate old dataset apis

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Dataset] Add callback generator

This patch add callback producer to abstract generator

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ GRU ] Add dropout mask

This commit includes:
. Implementation of drop out mask

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[Dataset] Add test for the random dataproducer

**Changes proposed in this PR:**
- Add random data producer tests
- Add abstract test for more data producers

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ RNNT LOSS ] Add skeleton code for rnnt loss

This commit includes:
  . Sekeleton Code for RNNT Loss as an Custom Loss Layer
  . Sementic Unit Test

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[Android] Fix undefined definition

This patch defines ML_API_COMMON_ROOT previously undefined

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ LSTM ] Add dropout mask

In this commit,
. drop out mask is added.
. calculation of drop out is implemented.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[ RNN ] Add dropout mask

This commit includes :
  . implementation of mask for dropout

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[Android] Add missing dependency

This patch adds mssing dependency to the android application build

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ Fix ] Fix android build error with meson

This commit includes,
  . Fix android build issues
    : openblas link and download

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[Fix] Initialize member variable

 - Initialize std::function variable in constructor

Self evaluation:

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[ TEST ] fix compilation error

This commit includes,
  . compilation error fix : unused + wrong type comparison

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[unittest] multi output unittest

 - Modify unittest to test multi output case
 - Make setLabel function in neuralnet
   to set labels at all loss layers

Resolve #1367

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[network_graph] revise addlosslayer for multi output

 - reorder addlosslayer to run before topologicalsort
 - Replace node_list rather than delete when the loss is entropy

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[graph_core] Implement input_list, output_list for multi input, output

- Make input_list, output_list and its getter to support multi input, output

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[layer] Update dropout layer to V2

Update dropout layer to V2 design.

Signed-off-by: Parichay Kapoor <kparichay@gmail.com>

[coverity] Fixes realtd to coverity

Add fixes related to coverity.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[svace] Fix svace issues

Fix svace issues for layer_v2 branch

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer_v2] Merge commit for branch layer_v2

This commit forms the merge commit including minor updates
while rebase layer_v2 with main branch so as to apply it.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[LayerV1] Delete for LayerV1

This patch deletes LayerV1 headers and its implementations.
Some of the relevant codes from LayerV1 are moved to either LayerNode or
to LayerDevel.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layernode] Add print functionality to LayerNode

Add print functionality from LayerV1 to LayerNode.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] Update NNStreamer layer to V2

Update NNStreamer layer to V2 design.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] Enable permute layer for V2

Enable permute layer for V2 design.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[unittest] Enable backbone unittests

Enable backbone unittests for modelfile

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] Enable time dist layer for V2

Enable time dist layer for LayerV2 design.
This patch tries to simulate the InitContext and the RunContext inside
the time dist layer so that proper shapes of variables can be passed to
the internal layer. Further, context changing function calls are
replicated on the actual InitContext/RunContext by detecting changes and
making those function calls again.

LayerNode was updated to ensure that a layer is not getting distributed
again and again.
Some more getter APIs were added for TensorDim and LayerContext.

Unittests related to distribute have also been enabled with this patch.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layernode] Remove LayerNode dependence on LayerV1

Remove LayerNode dependence on LayerV1.
Further, NetworkGraph dependence on LayerV1 is also removed.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[weight] Add missing initialization in constructor

Add missing initialization in constructor for the weight class.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] Initialize missing variables for LayerImpl

Add initialization for the missing variables for LayerImpl class.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[application] Add common tests for simpleshot layers

Add common standalone tests for simpleshot layers tests.
This allows basic testing semantics test for simpleshot layers.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[application] SimpleShot application update for V2

SimpleShot application update for layer V2 design.
Further, task_runner has been updated to use only 1 loss kind of layer.

Unittests of layers have not been enabled, they will be enabled with unittest_layers.

Signed-off-by: Parichay Kapoor <kparichay@gmail.com>

[application] Update centroid KNN layer for V2

Update centroid knn layer for V2 design.

Signed-off-by: Parichay Kapoor <kparichay@gmail.com>

[application] Update l2norm layer for V2

Update l2norm layer for V2 design.

Signed-off-by: Parichay Kapoor <kparichay@gmail.com>

[application] Update centering layer for V2

Update centering layer for V2 design.

Signed-off-by: Parichay Kapoor <kparichay@gmail.com>

[test] Added common unittest for custom layers

This patch add common unittests for custom layers.
This demonstrates how layer developers can use the unittests made by
nntrainer developers to test their own layers.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[application] Enable custom application

Enable custom application for v2 layer design.
This also includes updating the power layer and mae loss layer implementation
with layerv2 and updating their corresponding unittests.

Signed-off-by: Parichay Kapoor <kparichay@gmail.com>