[layernode] Update getNumInputs/Outputs

Separate getNumInputs/Outputs semantics for inputs/outputs
and for connections as a node. Number of inputs/outputs will always be
more than 1, but number of input/output connections can be 0 for
input/output nodes of the graph.
This patch separates the two concepts, and its usage.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[unittest] Enable models unittests

Enable models unittest for layerv2
Corresponding bugfixes are also added

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[manager] Support input/output tensor allocation

Support input/output tensor allocation for layerv2 design

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[tests] Enable more modelfile unittests

Enable more unittests for modelfile which depended on activation layers.
Also updated some of the unittests with the change in behavior of the
network graphs.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] Update Activation layer to LayerV2

Update activation layer implementation with layerV2 design.
Also add minor updates to input layer.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[test] Enable modelfile unittest

Enable modelfile unittests with newly update LayerV2 to test
building and initialization of the models.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] Cleanup layer_factory

Cleanup layer_factory and use app_context at all places to create layer.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[api/network] Update api/network for new losses

Update API/network including app_context to work with new losses:
- register all the losses
- update API implementation, and add other loss layers
- deprecate layer factory
- also set other elements in RunLayerContext properly

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[losslayer] Update loss layer with LayerV2 design

This patch update loss layer with LayerV2 design.
In order to limit the interface to the updated Layer interface,
different type of loss layers are split into individual classes
extending LossLayer class. LossLayer class is an abstract class
providing some common utility functions to all the loss functions.

Also added loss to RunLayerContext, which allows setting and getting
loss. However, note that although this loss value can be set/get by any
layer for themselves, it will only be used by the model for loss layers.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
[loss layer v2] loss layer v2

loss layer v2

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layernode] Maintain input_dim with LayerContext

Input dimensions has been removed from layer node.
Input dimensions are now directly managed by InitLayerContext.
This has minor overhead (at load if dimensions are edited many times but
is rare) but is less confusing and less prones to bug.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[inputlayer] Update input layer with LayerV2 design

Update input layer to upgrade to LayerV2 design

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[var_grad] Remove cloneTransposeVariableOnly interface

Remove cloneTransposeVariableOnly interface which is not used anymore.
Also add todo in layer_context.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layerV2] Update build for Layerv2

Update the build for Layerv2
This includes turning off model related unittests, creating Layer
objects instead of Layerv1, and certain implementations missing in LayerNode.
There are some temporary codes with this patch just to make the CI/build
pass. They will be updated as other layers are converted to Layerv2.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer/factory] Update layer factory for LayerV2

Update layer factory for LayerV2 for creation of layers of signature
LayerV2.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer/node] Update layer and node for LayerV2

Update node_exporter to export weights from run_context
Add corresponding const getters and setters in layer_context
Updated some fixes in var_grad

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[fc] Update fc layer for LayerV2

Update fc layer for LayerV2

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer/optimizer] Reduce usage of getObject() for optimizer

Reduce the usage of getObject() for optimizer.
This patch changes optimizer interface to work on each weight
individually and adds a getWeightObject() for the LayerContext.
This allows removal of getObject() from the neural network class for
backwarding.

This patch also disable dynamic_training_optimization.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[network/neuralnet] Reduce dependence on LayerV1

Reduce dependence on LayerNode()->getObject() which returns LayerV1
object.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[backbone] Remove support for scalesize

Remove support for scalesize for the backbone.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layernode] Update throw to retval in setProperty

Update throw to retval in setProperty when the property
is not set by LayerNode. Illegal properties vs non-captured properties
were being mixed in this case.
Now it has been separated by return false. If setProperty() throws, it
must propagate upwards as that error shows the error in the property
given.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[executionMode] Added mode of execution

Added mode of execution. This would be passed to the layers
as well as used internally than just keeping bool training to denote
the current mode of execution.
The header can be kept separate so as to be able to import it whereever
needed without being header heavy.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[var_grad] Update trainable to need_gradient

Update trainable logic to need_gradient for var_grad for verbosity.
If need_gradient is set, var_grad will have gradient set for it.
If it is a weight, with need_gradient set, gradients will be applied
every iteration.
If need_gradient is not, gradient will be a null tensor.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[backbone] Update default backbone to be trainable

Update default backbone to be trainable by default.
This will handle the case of tflite_layer and nnstreamer_layer
separately when handling trainable for layers like activation etc later.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] Update getTrainable with supportBackwarding

This patch introduces supportBackwarding.
Currently getTrainable does two jobs -
1. check if the layer is trainable
2. check if the layer can do backwarding

Now, the two is separated.
Further, the trainable property has been moved to LayerNode.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layernode] Move actiovation to LayerNode

Move activation from Layer internal to LayerNode
Added bug fix to avoid looping of distribute properties

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[LayerImpl] Add weight/bias properties to LayerImpl

Add weights/bias related properties such as initializer, regularizer,
etc to LayerImpl. This creates a differentiating factor of LayerImpl
from LayerDevel as LayerImpl provides a base class for layers with
weights/bias properties loaded.

Added description to LayerImpl and LayerNode for the properties they
support, and the work they do/must do.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layernode] Move loss to layer node

Move loss to layer node for the registered weights.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer context] Add interfaces for setBatch

Add interfaces to layer context required by layer developer
when they override setBatch()

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layernode] Layer interfaces support in LayerNode

Add support of Layer (layer devel) interfaces in LayerNode.
LayerNode interfaces are without arguments, and LayerNode will fill in
the corresponding context needed by the Layer.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[manager] Memory allocation for non-weight tensors

Added memory allocation for non-weight tensors has been added
including inputs, outputs and tensors.
For now, this does include any optimization except the gradient
based optimization.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Dataset] Add random producer

Add onehot random producer which generatese a fake data

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Dataset] Introduce data producer

This patch introduces data producer which abstracts a single iteration
creator.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix] Logical expression

 - Fix logical expression
 - Initialize member variable

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[ Layer ] implementation of DropOut Layer

In this commit,
  . implementation of DropOutLayer

Resolves:

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[Dataset/CAPI] implement dataset ctor

**Changes proposed in this PR:**
- Implement `ml_train_dataset_create()`
- Implement `ml_train_add_generator()`
- Implement `ml_train_add_file()`
- Add test as per the implementation.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[application] MNIST application

Update MNIST application benchmark values based on the updated weight
initialization.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[activation] Update implementation for in-place

Update implementation to handle in-place and non-inplace scenarios.
With new memory scheme coming in, in-place activation optimization and
derivative optimization is disabled. This patch updates the activation
function implementations to work for in-place and non-in-place
scenarios.
RNN, LSTM and GRU use activation functions internally with always
in-place settings. So, both the modes are supported.
This patch specifically convers the activation function scenario.
The generic support of in-place will be done when manager is updated.

This resolves the MNIST training bug.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[application] Disable MNIST unittest

MNIST unittest was not running as patch c87963c5433514bccdf0d13f733dd15de4356105
added some filecheck but returned 0 upon failure.
This patch removes returning on failure with just a warning.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[capi] Implement dataset_set_property_for_usage

This patch implements `ml_train_dataset_set_property_for_usage` and it's
test

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[dataset/cleanup] Remove usage from dataset impl

As each dataset now has one usage(train, valid or test).
This patch removes usage handles from dataset impl.

**Major Changes**
1. merge train/test/val members to one
2. clarify some variable names.
3. move global variables to members of an instance.
4. remove databuffer_utils.h as it is no longer used.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[devel] Add layer_devel to devel

Add layer_devel to devel, without this, build fails with upstream/main
build

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[dataset/cleanup] Remove type from dataset

This patch removes `datasetUsageType` from dataset interface

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[dataset] split train / val / test databuffer

This patch splits train / val / test dataset.

It is also possible to set dataset separately from the model.

**Major Changes**
1. `auto dataset = createDataset(train_cb, val_cb, test_cb)` -> `auto
dataset_train = createDataset(train_cb)`
1. `NN.setDataset(dataset);` -> `NN.setDataset(DATA_TRAIN,
dataset_train)`

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[dataset] Clean up dataset enums

`nntrainer::DataType` and `nntrainer::BufferType` is duplicated from
ccapi which was adding complication. This patch simply alternate those
types ccapi `DatasetType` / `DatasetDataUsageType`

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[dataset] Remove label data

Before this patch, label.dat was only used to get number of classes,
unfortunately, it was clashing with how databuffer_generator is getting
the number of classes and creating some inconsistency. This patch fixes
the issue by deprecating label data.

* Checked TCT and there is no critical issue that makes tests fail with
this PR.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Propose save/load api

**Motivations**
1. Fine grained api required to save and load

**Changes proposed in this PR:**
- Add function to model.h  / `save`, `load`
- Mark `readModel()` deprecated (later remove...)
- Mark `saveModel()` to change `exportTo`
- Add `ModelSaveLoadFormat`
- Create capi-machine-learning-training-tizen-internal for internal api
support

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Propose dataset api sets

**Changes proposed in this PR:**
- `ml_train_dataset_create(ml_train_dataset_h *dataset);
- `ml_train_dataset_add_generator(dataset, usage, callback, user_data)`
- `ml_train_dataset_add_file(dataset, usage, file)`
- `ml_train_dataset_set_property_for_usage(dataset, usage)`

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Pooling] Apply padding property

This patch applies padding property with tests

Also deletes `enum class PaddingType`

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Conv2D] Apply padding props to conv2d

This patch applies padding property to conv2d

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[spec/pkg] Bugfix for dependency

Some of the packages state the `requires` as wrong names (missing
package name as prefix). This does not show in build and unit/app tests
but fails when using these packages externally as those dependent
`requires` packaging dont exist.
For example, `nntrainer-devel-static` package depends on `devel`
package, and not on `nntrainer-devel`. This patch provides the
corresponding fix.

Resolves #1399

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Padding] Add padding compute function

This patch implements padding2d::compute with a test

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Add ml_train_model_get_input|output_dims

This commit contains capi proposal to
`ml_train_model_get_input_dimensions` and
`ml_train_model_get_output_dimensions`

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Resnet] Sync resnet app with validated model

This patch syncs resnet application with validated model.
Changes include:

1. s/setIteration/updateIteration
2. normalize input
3. fixed batchsize of 128 -> args received from the sysargs
4. output layer to copy
5. silent failed to read epoch idx and iteration

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Padding] Add padding verification

This patch implements `Padding2D::isValid` and tests respectively.

Padding2D property is valid when
1. string is "valid" or "same"
2. comma seperated, non-integer value of size 1, 2, 4

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix] File name sanitization

As colon (':') is not permitted in windows file system, having files
with 'colon' prohibits cloning our repo. this patch fixes the issue

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[docs] Add readme for resnet

This patch adds readme for resnet including citations.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Padding] Add padding2d prop header

This patch adds padding2d property header.

Padding2D prop will be saved as a string and compute the integer when it
is needed

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix] Weight initializer stddev calculation

The weight initializer was fitted to only fully connected layer case,
this patch extends weight initializer to properly work with other
weights as well.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Resnet] Connect the model with cifar100

**Changes proposed in this PR:**
- Implement cifar100dataloader
- Connect the data loader
- Display arguments/dataset information/time

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Test/Bn] Add conv2d model test

This patch adds conv2d bn test which was not properly tested

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[resnet] Implement test run to resnet

**Changes proposed**
- Add dataloader with interface `next()`.
- Add RandomDataLoader as an example
- Minor bug fixes to the model architecture regarding name
- Add routine to train the model

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ Recurrent ] Implement Dropout for Recurrent Net

In this commit, drop out for recurrent network is intrduced.
dropout property is introduced and if the element value of random
tensor is smaller than dropout rate, then it will be set zero.
The element which is not set zero, then it will scale with
1.0/(1.0-dropout).

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[Fix/trivial] Change rpm group description

**Changes proposed in this PR:**
- Change group description to machine learning/ML Framework

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Resnet] Create resnet model

This patch creates code to create resnet.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Resnet/skeleton] Add helper functions

**Changes proposed in this PR:**
- Add `withKey()`, `resnetBlock()`, `createResnet18()`

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Skeleton] Add resnet application skeleton

This patch adds resnet application skeleton

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix] coverity, svace issues

Coverity
1. Deleted noexception keyword where throw exception can occured.
2. Initialze member variable in constructor
3. Correct bitwise operation

resolves: 1238192, 1238193, 1238195, 1238196, 1238298

Svace
1. Catch unhandled throw

resolve: 464062

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[CustomLoss] Update example

**Changes proposed in this PR:**
- Update example using LayerClient
- Add LayerClient example to AppTest

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CustomLoss] Implement mae loss layer

This patch implements mae loss layer forward and backward, also moves
other functions to the sourcefile instead of inlinining it.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[layer_v2] Fixes to merge layer_v2 with main branch

Fixes to merge layer_v2 branch with main branch.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[LayerNode] Change flatten and distribute to prop

**Changes proposed in this PR:**
- Fix bug on `loadProperties`
- Add flatten and distribute to prop

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Optimizer] Implement getOptimizerVariableDim

- Implement getOptimizerVariableDim in optimizer
- Implement requestOptimizerVariable in manager

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[LayerNode] Add layer(devel) to the node

**Changes proposed in this PR:**
- Add `Layer` to the node
- Remove `node_exporter.h` from `layer_node.h`
- Change compile time switch to runtime switch of checking layer v1 and
layer v2

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[AppContext] Integrate layer-devel

This patch integrates layer devel to appcontext and plugin features.

From this patch, having it is able to include LayerV1 and Layer at the
same time.

Also adding a test

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[manager] Support initialization/allocation of weights

Support initialization and allocation of weights for LayerV2

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[network] Remove check for double activation

Remove the check for double activation, it is now allowed to have
two activation operations one after the other.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layercontext] Minor bugfix for layer context

Minor bugfix for layer context
Removes the extra trainable parameter from layer context

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[networkgraph] Network graph updated for Layer V2

Network graph updated to work with LayerV2
This involves settings input dimension in InitContext, and setting
up RunContext for each layer before their execution.

Further, some helper functions are also added in LayerNode.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Test/Prepare] Add layer semantics test

For layer_v2, there will be more structured way to access tests.
This patch adds a skeleton for layer semantics test.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Layer] Add trainable prop

This patch add trainable property to layer

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[manager] Add support for request Inputs/outputs

Added support for requesting inputs and outputs
Also update implementation to use a common function for easier
management

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer-internal] refactoring weight, input, output to layer_node

- Implement getter for variable and grad of weight, input, output
- Implement getNumWeights

Self evaluation:

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[graph] Support creating RunLayerContext

Implement the function to create RunLayerContext given the
InitLayerContext and update it in the given LayerNode.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[manager] Add support for request Inputs/outputs

Added support for requesting inputs and outputs
Also update implementation to use a common function for easier
management

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Properties] Add float and boolean cases

Add float and boolean case to be used later.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Props] Add concept of empty to property

This patch adds empty concept to property instead of having insensible
value inside a property as a default

before this patch,
```cpp
auto a = Property<int>();
EXPECT_EQ(a.get(), -1); // let's assume -1 is default value which might not be sensible.
```
after this patch

```cpp
auto a = Property<int>();
EXPECT_THROW(a.get(), std::invalid_argument);
EXPECT_TRUE(a.empty());
```

so underlying value remains always null or valid.

Now, it became natural to distinguish mandatory props and non-mandatory
props

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Layer] Scaffold layer impl

**Major changes**
- Create `LayerImpl`
- Change `LayerDevel` signature
  s/int initialize/void finalize
  s/int setProperty/void setProperty
- Move createLayer to layerImpl

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Layer] Add const to the layer::getName()

getName() is independent of const, so this patch adds const to
layer::getName();

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[layer-internal] refactoring getInputDimension getOutputDimension

implement getInputDimension, getOutputDimension in layer_node

Self evaluation:

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[neuralnet] Bug fix for graph usage

Bug fix patch to use constant iterators for the graph.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph] Remove non-const iterators

Remove non-const iterators for the graph.
Update the corresponding usages for the graph.

Note: the pattern for (auto const &node : graph) has been
removed as this can begin()/end() in the background.
Workaround for this is to use std::as_const(graph) but this
is only available from c++17, so this patch uses
the basic const iterator for loop pattern.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph] Graph iterator for sorted and unsorted

Updated graph iteartor to run over unsorted list of nodes
if the graph is not yet sorted. If the graph is sorted,
the iterator iterates over sorted nodes.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph] Update node_names to unordered_set

Update node_names storing all the names of the nodes
in the graph for uniqueness to use unordered_set.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph] Make adjacency list lazily

Update graph to make adjacenct list lazily when required
during the topological sort.
Until then, keep the node_list which contains simply all the nodes.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[manager] Support request Tensors and Weights

Added support for request Tensors and Weights
Moved spec of Tensors and Weights to VarGrad and Weights correspondingly
Also created constructors for weights and var_grad with their specs

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer_internal] refactoring read, save

implement read, save function in layer_node

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[Test/Scaffolding] Prepare test for the new layer

For now, layer test is too cluttered so that it is not agile enought to
make any change.

This PR mainly separates the tests into two parts to make the test
scallable itself.

1. Layer only test leaving inside `unittest_layers_{layer_name}`
2. Common tests like setProperty, golden_test in `layers_common_tests.h`

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Refactor] s/Layer/LayerV1

Change class name of layer to layerV1 to prepare migration
This should pass CI test.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[context] Layer context creation scaffolding

Added todo list for the layer context creation
Added minor new interfaces for the layer context
Disable in-place optimization and inference memory optimization

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[addition] Bug fix addition layer calcDerivative

Bug fix for addition layer calcDerivative().
addition layer assign the same tensor derivative memory back to its
other layers. If the other layers connecting with addition layers are
inplace, then this can lead to wrong results.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>