[Tp] rename externallyAllocated -> placeholder

This patch renames externallyAllocated -> placeholder

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[KNN] Fix querying label when not training

This patch fix querying label when not training, which is not feasible

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[KNN] Implement knn saving

As knn was not saving it's props, this patch implements saving property

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Test] Add ml singleshot example

This patch adds integrated test to show and test ml singleshot.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[tp] Implement reidentifySource()

This patch implements reidentifySource(). Test will be followed

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Tp] Add dimension check to extend

This patch add dimension check to extend

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Tp] rename create -> request

As per request, create has been renamed to request

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Trivial] Use if instead of template

This patch changes template to if constexpr. There seems to be false
positive with cppcheck with previous implementation, but do not need to
bother by a code used only in very small places.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

Policy based tensor request for tensor pool.

This commit specifically implement createOrExtend().

From this commit, basic tensor pool request will be possible except
`reidentifySource()`.
`reidentifySource()` will come as a separate PR to get better review.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[TP] Implement extend()

This patch implements extend() and corresponding tests

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Tpool] Remove updateExternalTensor

Instead of requesting externally to update external tensor, now syncing
dependency done upon request and renamed to `fillPlaceholder` to align
with incoming changes

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Clean] Separate src spec from dep spec

As source specification and dep specification is being deviated, this
patch prepares for the further code changes with minor clearance.

**Changes proposed in this PR:**
- RequestSpec now has tensor, details
- Detail is `std::variant<SourceDetails, DependencyDetails>`

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Trivial] ZERO_LIFESPAN -> UNMANAGED

This patch renames zero_lifespan to unmanaged.

zero_lifespan suggests it should be extendable but it's precisely not.
So changed name for clarity.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[tpool] Implement view()

**Changes proposed in this PR:**
- merge updateExternalTensor()
- enable offset for view/placeholder
- corresponding tests
- s/requestSpec/RequestSpec

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[TPool] implement create/placeholder

This patch implements create/view and its corresponding test.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[TPool] Add interface to enable extensive sharing

This patch add interface to enable extensive sharing, including
reassigning or tensor pointer sharing (previously discussed as s3
sharing)

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[layer] Add modes to inplace layer execution

There are now 3 modes to inplace layer execution (compared to 2
previously - yes/no):
- none: not inplace
- restricting: if the current layer is inplace, then not every layer
right next to it can be inplace. The next layer can check its
conditions.
- non-restricting: this layer does not add any restrictions, and the
nexy layer can treat this layer just as an out-of-place layer when
deciding if the next layer should work in-place.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] Inplace support for multiout layer

This patch adds support for in-place execution of the multiout layer.
Corresponding unittests are also added.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[unittest] Implement rnncell unittest

 - Generate rnn, rnncell layer unittest
 - Generate model unittest which is composed of rnncell
 - Verified with multi timestep/stacked/loop_unrolling situation

Self evaluation:

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[rnncell] Implement rnncell

 - Implement rnncell which is only run on 1 timestep

Self evaluation:

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[Bugfix] Regenerate lstmcell unittest

 - Set lstm bias_ih to unused in init to fix the bug
 - Regenerate the lstmcell unittest
 - Fix typo

Self evaluation:

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[graph] Allocate memory based on which node runs

Currently, the memory was allocated assuming all the nodes were running
backwarding, and all the layers are trainable. But that is not true.
This patch applies the corresponding optimization and allocates memory
corresponding to the functions which will run on these nodes.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] LSTMCell bug fix

This patch applies bug fix for lstmcell.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[graph] Skip backwarding bug fix

Current implementation assumes that the graph is always linear and skips
the backwarding for contiguous set of layers at the start of the model
which should not do the backwarding.

This patch makes a fix to this issue but allowing layers in the middle
of the sorted graph to skip backwarding.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[ README ] fix typo & add link for cc APIs

fix typo & add link for cc APIs

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[meson] Add android tflite disable option

This patch add android tflite disable option

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[ README ] update readme

This patch includes readme update.

**Self evaluation:**
1. Build test:  [X]Passed [ ]Failed [ ]Skipped
2. Run test:  [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>

[layer] Support prefix for sharing weight names

This patch allows support for prefix for sharing weight names across
layers, so that layers which should share their weights can do so by
using the same prefix.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] Simplify naming scheme for requested tensors

All layers were required to make unique name for their requests.
This is now rather done by layer context by appending layer name as a
prefix to the requested name.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Trivial/CAPI] Tizen typo fix

This patch fixes tizen typo

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[layer] lstmcell minor optimize

Minor optimization for lstmcell to skip a redundant multiply.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[tensor] Optimize element wise operator functions

This patch optimizes element-wise operator functions for multiply,
divide, add and subtract by using std::transform than regular loop when
the stride along the last dimension is 1.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[tensor] Support adding result to output with multiply

Support adding result to output with multiply to fuse multiply and add
operations for LSTMcell.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Profiler] Add listener total time and percent

This patch add listener total time and percent mechanism

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[tensor] Optimize tensor apply and multiply_strided

Optimize multiply_strided and apply to use std::transform to get the best
performance, and possibility to use par_unseq for later parallelization.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] Performance optimizations to LSTMCell

Add minor performance optimizations to LSTMCell.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[tensor] Add beta for tensor sum

Add beta for tensor to enable addition to existing tensor while summing.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[API/trivial] Add LSTMCell creator

This patch add lstm cell creator

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[layer] override setBatch in bn layer

 - Override setBatch function

Self evaluation:

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>

[layer] LSTM cell batchresize bug fix

As LSTM cell uses the batch values not in the batch dimension but in the
inner dimension, it creates bug with the existing layer context interface
as it only allows updating the batch along the batch dimension.
This patch provides a quick fix to combine the time and batch values in
the batch dimension.

Once recurrent realizer allows multiple inputs, the time factor will be
removed and the reshapes can be removed.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] Optimization for LSTM cell

Optimization for LSTM cell to combine the activation function running on
the values.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Android] Add missing fopenmp

This patch add missing fopenmp.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Android] Enable test for the android build

This patch enable android build, this abling this option has an issue

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Android] Unlink ml-api-inference module

This patch unlink ml-api-inference module as it requires gstreamer

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix/pkgconfig] Fix nntrainer include dir

This patch fixes pkg config nntrainer include directory for the pkg.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[LSTMCELL] Fix compile error

This patch fix compile error in the main branch

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[layer] Bug fix for LSTMCell

Enable recurrent unrolling tests for LSTMCell with a single and stacked
LSTM cell. Add corresponding bug fixes.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] Add first draft of lstm cell

This patch adds the first draft of lstm cell. This has certain
advantages over the existing LSTM when executing for single timestep:
- does not need to divide the data over smaller chunks for computation.
This saves the overhead of creating tensors, an extra loop and combines
computation to a single call for efficiency.
- some of the operations have been avoided based on the current timestep
(which were zeros).
More optimizatios will be coming soon for LSTM cell.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[tensor] Add support for strided operators

This patch adds support for strided multiply, apply and getSharedDataTensor
operations for tesnor.
Later these will be optimized direct data access with an iterator.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Android] disable nnstreamer layer

This patch disables nnstreamer backbone because of the package issue for
android build

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Clean] remove enable-android option

This patch remove enable-android for brevity

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Android] Add prebuilt android for the built library

This patch provides prebuilt android when making android zip file

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Android] Add simple package script

This patch adds android simple package script to make tar.gz

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix] fix bug for install_headers

Fix bug that install_headers install_dir overrides subdir props.
This is undocumented behavior so changed to make it to be deterministic

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Android] Set up install directory for android build

This patch sets up install directory for android build.

It is defaulted to installed in the meson.build_root() /
android_build_result

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Android] Enable capi/ccapi build

This patch enable capi/ccapi build

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Meson] Add android nntrainer build

This patch add android meson build.
which can be triggered by ninja

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Meson] Save and apply extra defines

This patch saves extra defines as a list to be applied for the android
build as well

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Android] disable building src through ninja

This patch disables building src through ninja for android

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[android] Add platform android and install deps

This patch add platform android and install deps according to it

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[TRIVIAL] Fix error log

error log string was concated by '+' which was invalid

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Android] Remove bfd linker

bfd linker is deprecated so it's also deleted from android build

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Trivial] Add override to functions

This patch add override to virtual functions mainly to suppress compile
warnings from android build

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Optimizer] Prevent weight allocation for dependent nodes

This patch prevents optimizer weight allocation for dependent nodes

Resolves #1671

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[build] Add openmp dependency for improved performance

This patch adds openmp dependency while building for improved
performance.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[tensor] Bug fix for dropout mask generation

This patch adds fix for dropout mask generation. It was being
out-of-place which was leading to not using the tensor memory already
allocated. This patch adds the interface to use the already allocated
tensor and use it.
This patch further saves memory while generating dropout mask itself as
well.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[SVACE] Fix init problem

This patch fixes initiation problem

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[mem-opt] Add script to memory peak RSS memory

This patch adds a script to measure the peak RSS memory.
To check the memory consumption, pass the command to run the
applicaiton/test to this script.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Fix] not used variable

Not used variable is deleted from static analyzer

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix] Svace issue

Fix uinit ctor

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Concat] Add axis selection

Add axis selction mechanism if axis is not specified that if height
exist, concat to the width dimension

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Reshape] Add exportTo

This patch implements exportTo to reshape

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix] fix validate Input

This patch fixes validate Input

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[test/bn1d] Add bn1d support

This patch adds bn1d support for the transLayer_v2

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix] fix recorder registration

This patch fix registration bug when there is more than two overloaded
delgation

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix] Layer node clone configuration

layer node clone configuration was not copying realization, this is
fixed with this patch

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[layer] Create property for axis in batch normalization

This patch convert the axis member in batch normalization to a property
to allow setting it externally.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[profile] Support profiling by layer type

This patch adds support for profiling of the model based on the type of
the layers.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[network/neuralnet] Move backwarding to network

The iteration of the graph to perform the backwarding has been moved to
the network graph in this patch.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] Support concat along all the dimensions

This patch provides support for the concat layer among all the
dimensions. Other changes include:
- split_dimension property keyword has been changed to axis and reused
for concat_dimension
- TensorDim::getNumDim() is now a static function

Corresponding unittests for the concat layer are also added.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[Test] Add recurrent value compare tests

This patch adds recurrent value compares golden tests

There are 4 cases presented.

1. single fc recurrent
2. stacked fc recurrent
3. single lstm recurrent
4. stacked lstm recurrent

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[LSTM] Fix single time step backwarding

This patch updates single time step backwarding to properly set and see
the last timestep values

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Test] Add model v2 test generator

This patch add model v2 test generator using pytorch.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Test] Connect v2 test to the parameterized test

This patch connect v2 test to the model param test for convenience

+ add validateFor_v2. This is separated out because label dimension is
not given, although it should be merged to validateFor

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Test] Add models v2 test

This patch proposes models v2 test which is much lighter.

**Purpose**
1. Lighter model test as layer golden test compensates for it.
2. Easier debugging especially by checking offset, and moving metatdat
3. Give greater flexibility to the golden test generator on the model
architecture
4. Not saving golden file but making it on the fly on the builder side

**Major Difference**
1. Gradient comparison is skipped as it is automatically checked at the
next iteration.
2. Add metadata about num_iterations to file
3. Add size check while reading the golden file
4. Delete redundant information that can be infered.
5. Every iteration has it's own input information.

**Golden Format**
```
    ## file format is as below
    # [<number of iteration(int)> <Iteration> <Iteration>...<Iteration>]
    # Each iteration contains
    # [<input(Tensors)><Label(Tensors)><Parameters(Tensors)><Output(Tensors)>]
    # Each tensor contains
    # [<num_elements(int32)><data_point(float32)>...<data_point(float32)>]
```

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Trivial] Open sizeCheckedReadTensor to util

This patch opens sizeCheckedReadTensor to util for the later use

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix/Sharing] Fix gradient updation log

Gradient should be initialized at the very first (backward) access, but the
shared weight was updated at the very last (backward) access, this patch
resolves the issue by adding isFirstAccess to the var_grad.h

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[layer] Support reshape layer

This patch provides support for reshape layer and basic unittests.
The flatten layer is also updated to use reshape layer internally.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[test] GRU with reset_after=False

GRU matches with tf1 but not with tf2.
Tf2 changes the reset_after to True as default, which leads to an extra
bias. This patch forces the reset_after to be always false to enable
matching with tf1 and tf2.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[tensor] Add checks for external tensors

This patch adds checks while setting the external tensors:
- ensure that the external tensor size is more than the required memory
- if the tensor size is 0, its data must be null

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer/test] Add unittests for conv1d

This patch adds layers golden tests for conv1d.
conv1d is also added to the list of transpose layers while saving data
from tensorflow.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[layer] Add conv1d implementation

This patch adds conv1d implementation support.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[properties] Bug fix for padding compute

This patch provides bug fix for padding compute in the usage of stride.
The stride along width and height were being used wrongly.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

[LSTM] Add lstm fix and test run

This patch add lstm fix and test run.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Neuralnet] Add debug mode to check optimizer

This patch add debug mode to check optimizer

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Fix] tensor pool prerequest bug

This patch fix tensor pool prereuqestedrequest bug by that shared_name
was wrong

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Trivial] Move makeGraph function out

this patch moves makeGraph function from compiler_util -> test_util

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Test] Make model test to take arbitrary nn

This patch updates model test to accept arbitrary neuralnetwork instead
of ini only

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Trivial] Extract golden test from nnt_models

This patch extracts golden test from nntrainer models, nothing done.

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

[Trivial] Separate Watchers into a new file

This patch separates watchers into a new file

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>