[test/layers] enable tests when only available

This patch enables tests when only available. for example,
unittest_layers_nnstreamer.cpp cannot be run on tizen so excluded.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[gbs fix] Enable test not running with main lib

**Changes proposed in this PR:**
- Enable test not running with main lib
- Enable memory test
- Enable fix for the props checking bug

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix] Fix semantic tests were not running

This patch fixes semantic tests were not running

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Spec] Exclude generated coverage items

This patch excludes generated coverage items from daily coverage report

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[LayerImpl] Maintain LayerImpl property with props

 - All the layerImpl property will be maintain with props

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[layer] clean up parse_util header

 - parse_util header is no longer needed cause setProperty function is deleted
   so clean up header

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[bn layer] Maintain bn layer property with props

 - All the property will be maintain with props

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[rnn lstm gru layer] Maintain rnn, lstm, gru layer property with props

 - All rnn, lstm, gru layer property will be maintain with props
 - Allow dropout to zero

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[pooling2d layer] Maintain pooling2d layer property with props

 - All pooling2d layer property will be maintain with props

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[preprocess flip layer] Maintain layer property with props

 - All the preprocess flip layer property will be maintain with props
 - Remove default flip direction value which was horizontal_and_vertical

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[conv layer] Maintain conv layer property with props

 - All the conv layer property will be maintain with props

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[preprocess translate layer] Maintain layer property with props

 - All the preprocess translate layer property will be maintain with props

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[split layer] Maintain split layer property with props

 - All the property of split layer will be maintain with props

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[embedding layer] Maintain embedding layer property with props

 - All the property of embedding layer will be maintain with props

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[input layer] Maintain input layer property with props

 - All the property of input layer will be maintain with props
 - Remove unused setProperty function of tflite_layer

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[Test] Add save load test

Add save load test while adding options to model test

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ini] Implement dataset save/loader

This patch implements dataset saver/loader.
This patch propose sections "train_set", "valid_set", "test_set". Just
like optimizer.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Props] Implement activation property

This patch implements activation property by enum

**Additional changes**
- TimeDistribute property is now handled at finalize()

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Bug] Fix support inplace for distribute Layer

This patch fixes support inplace for distirbute layer

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Cc: Parichay Kapoor <pk.kapoor@samsung.com>
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[bnlayer] Batch norm layer bug fix

This patch provides bug fix to the initialization of the batch norm
values.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[neuralnet] Fix validation reporting

Fix validation values reporting.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[resnet] Fix data loader for resnet application

This patch fixes the data loader for resnet application after the
dataset loader has been changed to load data element by element than the
whole batch.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[batchnorm] Bug fix for batch norm layer

Bug fix for requested variable for batch norm layer.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[props] Support enum properties

This patch aims to support enum based properties, please refer to the
test to see how it works.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Ini Save] input shape property

This patch includes input shape property generation

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[LayerNode] Change context to be RAII

This patch refactors context to be RAII to make layer node properties
dumb.

**Changes**
- init context is removed
   - layer node instead created and returned from layerNode::finalize(input_dims)
   - remove dependency to input dimension before initialize
   - layerNode now has input_shapes instead of setting directly from initContext
   - runcontext is used to query things
   - fix multiple bugs regarding the validity of layer node

- networkGraph::updateRunContext() -> finalizeContext() now
finalizes inside this function for brevity
- layerNode::updateRunContext() -> configureContext() to make it RAIIer.
- minor code cleans in networkGraph::initialize

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[model] implement save best acc logic

This patch implements saving best accuracy when training

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ini] Implement input layers property

This patch implements input layers property while decoupling input
dimensions from input layers as much as possible
to make the property dumb.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ini/save] Save Optimizer Section

This patch saves optimizer section to ini.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Optimizer] Refactor to use props

This patch refactors optimizer families to use properties while removing
some unused functions and constructors

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[IniWrapper] Add factory method

This patch adds a function to create ini section directly from
exportable which implements .exportTo to reduce overlapping codes

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Trivial] Fix gcc warning (sign compare)

This patch fixes gcc warning of sign compare from tests

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ini] Implement model save

This patch implements model section save in ini format

**Side Changes proposed in this PR:**
- Move `THROW_STATUS` and `RETURN_STATUS` to util_func.h

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ini] Migrate to properties for model

This patch migrate to properties for neuralnet

**Semantic Changes**
- NeuralNetwork::batch_size does not change when actual batchsize change
from neuralnetwork instead, model graph keeps batch size separately.

**Changes proposed in this PR:**
- Move members to props in model
- Move inline declaration to declaration for some functions
- Remove unused member/functions from parse_util / neuralnet

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ini] Prepare model properties

This patch prepare model properties in `model_common_properties`

**Additional Changes proposed in this PR:**
- Add PositiveIntegerProperty base class
- Fix copy assignment/ctor in base_properties

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[NeuralNet] Fix testset run

This patch fixes testset running while cleaning up the code to reuse
repeating logics with callback

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[test/planner] Unittests for memory planner

This patch adds unittests for the memory planner.
The unittests covers some of the extreme scenarios for memory planning
and ensures that the constraints are satisfied.
Note that these tests do not test the efficient of the planning but the
validity of the planned layout.

Corresponding bug fixes are also added.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[planner] Support basic memory planner

This patch adds support for the basic memory planner.
The basic memory planner does not provide any memory sharing, and
results in all tensors being allocated in separate memories.
This forms the baseline for memory allocation.
Other memory planners will be enhancement over this.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Trivial] delete no meaningful save/load test

This patch delete no menaingful save/load test which was drawing
attention of static anaysis tool

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Implement model save load

this patch implements model save load in capi.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Dataset] Reflect final reviews

**Changes proposed in this PR:**
- rename request* / requestSlots()
- change raw_file_data_producer

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Cleanup] Cleanup unused parts from datasets

**Changes proposed in this PR:**
- batch_queue.[h|cpp] -> iteration_queue.[h|cpp]
- delete Class BatchcQueue
- delete Producer::finalize, Producer::size,
databuffer::startFetchWorker, databuffer::fetch and following patch
- delete DataProducer::Iteration, DataProducer::Sample as we have
dedicated classes instead.
- remove *_sample suffix. (eg) DataProducer::finalize_sample becomes
DataProducer::finalize)

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Dataset] Change and apply callback handler

This patch change and apply callback handler from batchwise ->
samplewise

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Dataset] DataBuffer Handling

This patch adds DataBuffer handling in case of the producer throws.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[dataset] Migrate file to dataset sample

This patch apply new dataset to file producer.

There was a test number changes.

reason:

1. Now, partial batch drop is determinisitic
2. Reshuffling is now called after partial batch is filled, (previously
it was before partial batch is filled)

Verified numbers without shuffling with
https://github.com/nnstreamer/nntrainer/pull/1416/files#r685695542

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Dataset] Implement databuffer getter

This patch implements databuffer getter and corresponding tests

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Dataset] Handle queue edge cases

Handle queue edge cases where ScopedView runs into an exception

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[itq] Add multiple mt itq test

This patch adds multiple test based on scenarios

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Dataset] Add mt support to iteration Queue

This path adds async support to iteration queue and ending mechanism

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Dataset] Allow iteration to have partial batch

This patch updates the semantics of batch to have distance between
endsample iteration and beginsample iterator to emplace partial batch
semantics.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Ini] change setProperty signature of model

This patch changes setProperty signautre of model to return void instead
of int

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Save] Prepare to save ini

This patch adds a skeleton to save into ini format

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Add permission denied return

This patch add permission denied return when failed to read/write to a
file.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CCAPI] Implement basic load save

This patch adds basic load save from CCAPI.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[layer_node] handle deprecated property

 - Handle deprecated property num_inputs, weight_initialize, bias_initialize

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[Dataset] Add iteration queue tests

**Changes proposed in this PR:**
- Add iteration queue tests within sync (async will be followed)
- Add `slots` and `batch` query method to batch_queue
- Fix some trivial issues like uninitialized variables

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[raw_file_data_producer] handle divide by zero

 - Check sample_size to avoid divide by zero

Self evaluation:

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[Dataset] Change generator_sample signature

This patch changes generator sample signature from std::vector<Tensor *>
to std::vector<Tensor>

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Add enum for ini saving

This patch adds enum for ini save and load.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Dataset] Prepare q for the samplewise iteration

This patch prepares queue (`IterationQueue` to avoid naming conflict)
for the samplewise data feeding.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Dataset] Add Iteration Object

This patch adds a Iteration class and Sample class which allocate a tensor memory.
This object will be used in the queue.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Expose l2norm layer

**Changes proposed in this PR:**
- Change name of l2norm layer to PreprocessL2Norm Layer
- Move l2norm layer to the main code base

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[tensor_filter] Fix tensorfilter fail

This patch fixes tensor filter fail due to two critical bugs

- readModel was broken
- tensor getSharedDataTensor was broken

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[layer_node] bugfix/validate init_context

 - Validate init_context before using the init_context in print function
 - Reorder adding default layer object to appcontext based on LayerType enum
 - Correct typo

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[Dataset] Change finalize signature

This patch changes finalize signature to contain user_data,
user_data will be needed to replicate the current worker(with diffrent
arguments) in case of
MT_safty is not provided by the producer. Please note that this does not
change any current behavior but the preparation for the future
enhancements.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Dataset] Rework func dataset to samplewise

**Changes proposed in this PR:**
- Fix bugs in Tensor::Map
- Add shortcut for the Tensor::updateBatch
- Add func_data_producer with sample

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[unittest] backbone property name update

Update backbone property name for the layers unittests.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[CAPI] Open centroid KNN

**Changes proposed in this PR:**
- Move centroid KNN to layer, delete centroid knn from the Application

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Cc: Inki Dae <inki.dae@samsung.com>
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[memorypool] Introducing memory pool

This patch introduces memory pool for tensors. All the tensors requiring
memory can ask memory pool for their memory. The memory pool collects
all the requirements, uses the provided memory planner to plan a memory
layout, allocates the required memory, and then finally returns the
allocated memory to the requesters.

There are certain limitations to this memory pool:
1. All the memory must be requested before layout planning and
allocation.
2. There is no support for freeing selectively allocated tensors
3. In order to request support for more tensors and their memories,
all previously allocated memory has to be freed.
4. MemoryPool for now allocates all its required memory in a single
block. Support for breaking the required memory into smaller blocks is
left a TODO for later (this can be useful for devices without IOMMU).

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[memoryplanner] Introducing memory planner

This patch introduces memory planner interface which will provide a
layout plan given the various memories required by the model.
Different memory allocations plans/strategies will be implemented by
extending this class with the given interface.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[manager] Weight/gradient allocation simplify

Simplify weight and gradient allocation out to the most basic strategy
so that the memory optimization can be applied on it.
With this patch, the memory for gradient is not shared anymore till the
memory optimization is implemented.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Dataset] Implement samplewise for file

**Changes proposed in this PR:**
- implements samplewise size(), finalize() for file
- add corresponding test

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Update description of ml_train_datagen_cb

This patch updates ml_train_datagen_cb with newly added api sets and
clarify the parameter @a last means.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Reflect feature review

This patch reflects feature review.

**Changes Proposed**
- Adding Not Supported enum

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Backbone/CAPI] Implement backbone layer creation

**Changes proposed in this PR:**
- Connect enum to nnstreamer backbone
- Change `model_file` property to `model_path` for generality
- Add handling for when model_file property is loaded from interpretation

Resolves #1072

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Cc: Inki Dae <inki.dae@samsung.com>
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Dataset] Implement samplewise producer for random

This patch implements samplewise producer for random and its
corresponding test abstraction

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Trivial] layer client test was not running

This patch changes test variable argument in order for layer client
tests to run

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[context/graph] Catch lifespan + 3-way execution order

This patch adds the setup to catch lifespan set by the layer developer
when requesting for tensors.
Further, the execution order now consists of 3 values - fowarding,
calcGradient and calcDerivative. The memory optimization will now be
done using the 3 functions to be called in order.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[manager] Set the default usage timestamp for tensors

Set default usage time index for all the tensors based on when the layer
is to be run in the sorted list.
The usage time indexes are stored in each GraphNode by the GraphCore.
When creating the sorted list, each layer is updated to store its
order location/index when executed in this sequence for forwarding as
well as backwarding.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layers] Make requested weight/tensor names unique

Update the layers to make the requested weights and tensors names to be
unique. This done using the layer name with the postfix of the
weights/tensors name making the combination unqiue as layer name is
unique.

InitLayerContext is updated to provide access to the layer name while
finalizing the layer.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph/neuralnet] Move manager from neuralnet to netgraph

Move manager from NeuralNetwork class to NetworkGraph class.
Some corresponding minor adjustments are also added.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Dataset] Add interface for samplewise producer

This patch adds interface to migrate batchwise producer to samplewise
producer

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Dataset] Add user_data set Property handler

**Changes proposed in this PR:**
- PtrType properties
- Add user_data set property handler for backward competibility

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Reflect privilege reviews

This patch update document to add privilege to `ml_train_model_save()`
and `ml_train_model_load()`

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[fix] ahub issues

 - Added try catch statement
 - Check value to avoid divide by zero
 - Added const keyword to match override function
 - Initialize member variable in constructor

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[graph/neuralnet] Move manager from neuralnet to netgraph

Move manager from NeuralNetwork class to NetworkGraph class.
Some corresponding minor adjustments are also added.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph] Update check for input node

Update the check for checking if a given node is input node.
The check uses in-degree for the node to be 0.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graphnode] Remove index from graph node

Currently each graph node has two identifiers - name and idx - where
both were unique. This patch removes the idx as the indentifier.
The corresponding functions are also removed.
In order to support fast fetching of nodes, graphcore has a map nodes.

This change revealed a bug with the names in the code. As ini does not
allow case sensitive names, the names from the ini are changed to lower
case. However, input_layers property names were allowed to be
case-sensitive. Further names set by the API were also case sensitive.
This patch forces all such names to be case-insensitive to lowercase.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph] Remove layerv1 initialize code

Remove layerv2 initialize code for network graph.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[CAPI] Assign values to the enum

This patch assigns values to enums in nntrainer-api-common.h

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Update regarding the review script

This patch updates nntrainer.h header with regard to the review script

- @a -> @c
- merge conflict with usage / mode

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Trivial] dataset usage enum rename

This patch renames enumeration as discussed ahead of time

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Add permission denied return value

As privilege is sometimes needed by dataset, add permission denied for
`ml_train_dataset_add_file()` and `ml_train_model_construct()`

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Change privilege to remark

This patch change privilege to remark as recommended

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI] Add privilege to documentation

This patch adds privilege to documentation as guided

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI/ACR] Fix typo and doxygen tag

This patch reflects review from #1371, #1446

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Trivial] Reflect acr review script

This patch reflect acr review scripts, which contains some trivial fixes
like adding a period at the end.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[CAPI/ACR] Add remark on tensor info getter

This patch adds remark on tensor info getter that info should not be
destroyed when fail to create

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[fixes] Add bug fixes from asan

**Changes proposed in this PR:**
- Fix issues from address sanitizer
- Enable deprecated-declaration warning but disable for some files

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[fixes] Add bug fixes found with address sanitizer

This patch adds bug fixes after adding fixes related to address
sanitzer:
- NNStreamerLayer wasnt freeing resources upon destruction
- accessing released model handle in capi unittest
- not releasing model handle before end of unittest

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>