Jihoon Lee [Fri, 5 Nov 2021 09:03:46 +0000 (18:03 +0900)]
[Trivial] Use if instead of template
This patch changes template to if constexpr. There seems to be false
positive with cppcheck with previous implementation, but do not need to
bother by a code used only in very small places.
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 5 Nov 2021 08:38:02 +0000 (17:38 +0900)]
Policy based tensor request for tensor pool.
This commit specifically implement createOrExtend().
From this commit, basic tensor pool request will be possible except
`reidentifySource()`.
`reidentifySource()` will come as a separate PR to get better review.
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 5 Nov 2021 08:08:13 +0000 (17:08 +0900)]
[TP] Implement extend()
This patch implements extend() and corresponding tests
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 5 Nov 2021 07:36:06 +0000 (16:36 +0900)]
[Tpool] Remove updateExternalTensor
Instead of requesting externally to update external tensor, now syncing
dependency done upon request and renamed to `fillPlaceholder` to align
with incoming changes
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 5 Nov 2021 07:22:20 +0000 (16:22 +0900)]
[Clean] Separate src spec from dep spec
As source specification and dep specification is being deviated, this
patch prepares for the further code changes with minor clearance.
**Changes proposed in this PR:**
- RequestSpec now has tensor, details
- Detail is `std::variant<SourceDetails, DependencyDetails>`
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 5 Nov 2021 07:17:58 +0000 (16:17 +0900)]
[Trivial] ZERO_LIFESPAN -> UNMANAGED
This patch renames zero_lifespan to unmanaged.
zero_lifespan suggests it should be extendable but it's precisely not.
So changed name for clarity.
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Thu, 4 Nov 2021 11:41:22 +0000 (20:41 +0900)]
[tpool] Implement view()
**Changes proposed in this PR:**
- merge updateExternalTensor()
- enable offset for view/placeholder
- corresponding tests
- s/requestSpec/RequestSpec
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Thu, 4 Nov 2021 09:07:18 +0000 (18:07 +0900)]
[TPool] implement create/placeholder
This patch implements create/view and its corresponding test.
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Thu, 4 Nov 2021 08:47:56 +0000 (17:47 +0900)]
[TPool] Add interface to enable extensive sharing
This patch add interface to enable extensive sharing, including
reassigning or tensor pointer sharing (previously discussed as s3
sharing)
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Parichay Kapoor [Wed, 3 Nov 2021 05:37:08 +0000 (14:37 +0900)]
[layer] Add modes to inplace layer execution
There are now 3 modes to inplace layer execution (compared to 2
previously - yes/no):
- none: not inplace
- restricting: if the current layer is inplace, then not every layer
right next to it can be inplace. The next layer can check its
conditions.
- non-restricting: this layer does not add any restrictions, and the
nexy layer can treat this layer just as an out-of-place layer when
deciding if the next layer should work in-place.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Mon, 1 Nov 2021 12:20:51 +0000 (21:20 +0900)]
[layer] Inplace support for multiout layer
This patch adds support for in-place execution of the multiout layer.
Corresponding unittests are also added.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
hyeonseok lee [Fri, 5 Nov 2021 07:14:32 +0000 (16:14 +0900)]
[unittest] Implement rnncell unittest
- Generate rnn, rnncell layer unittest
- Generate model unittest which is composed of rnncell
- Verified with multi timestep/stacked/loop_unrolling situation
Self evaluation:
Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>
hyeonseok lee [Fri, 29 Oct 2021 09:06:09 +0000 (18:06 +0900)]
[rnncell] Implement rnncell
- Implement rnncell which is only run on 1 timestep
Self evaluation:
Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>
hyeonseok lee [Thu, 4 Nov 2021 10:56:44 +0000 (19:56 +0900)]
[Bugfix] Regenerate lstmcell unittest
- Set lstm bias_ih to unused in init to fix the bug
- Regenerate the lstmcell unittest
- Fix typo
Self evaluation:
Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>
Parichay Kapoor [Tue, 2 Nov 2021 01:13:19 +0000 (10:13 +0900)]
[graph] Allocate memory based on which node runs
Currently, the memory was allocated assuming all the nodes were running
backwarding, and all the layers are trainable. But that is not true.
This patch applies the corresponding optimization and allocates memory
corresponding to the functions which will run on these nodes.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Mon, 1 Nov 2021 11:02:51 +0000 (20:02 +0900)]
[layer] LSTMCell bug fix
This patch applies bug fix for lstmcell.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Mon, 1 Nov 2021 06:51:42 +0000 (15:51 +0900)]
[graph] Skip backwarding bug fix
Current implementation assumes that the graph is always linear and skips
the backwarding for contiguous set of layers at the start of the model
which should not do the backwarding.
This patch makes a fix to this issue but allowing layers in the middle
of the sorted graph to skip backwarding.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
jijoong.moon [Tue, 2 Nov 2021 11:28:35 +0000 (20:28 +0900)]
[ README ] fix typo & add link for cc APIs
fix typo & add link for cc APIs
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>
Jihoon Lee [Thu, 28 Oct 2021 06:32:38 +0000 (15:32 +0900)]
[meson] Add android tflite disable option
This patch add android tflite disable option
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
jijoong.moon [Tue, 2 Nov 2021 02:59:18 +0000 (11:59 +0900)]
[ README ] update readme
This patch includes readme update.
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>
Parichay Kapoor [Mon, 1 Nov 2021 01:13:16 +0000 (10:13 +0900)]
[layer] Support prefix for sharing weight names
This patch allows support for prefix for sharing weight names across
layers, so that layers which should share their weights can do so by
using the same prefix.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Mon, 1 Nov 2021 01:08:48 +0000 (10:08 +0900)]
[layer] Simplify naming scheme for requested tensors
All layers were required to make unique name for their requests.
This is now rather done by layer context by appending layer name as a
prefix to the requested name.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Jihoon Lee [Fri, 29 Oct 2021 03:21:05 +0000 (12:21 +0900)]
[Trivial/CAPI] Tizen typo fix
This patch fixes tizen typo
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Parichay Kapoor [Fri, 29 Oct 2021 03:48:22 +0000 (12:48 +0900)]
[layer] lstmcell minor optimize
Minor optimization for lstmcell to skip a redundant multiply.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Thu, 28 Oct 2021 08:03:02 +0000 (17:03 +0900)]
[tensor] Optimize element wise operator functions
This patch optimizes element-wise operator functions for multiply,
divide, add and subtract by using std::transform than regular loop when
the stride along the last dimension is 1.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Thu, 28 Oct 2021 06:14:31 +0000 (15:14 +0900)]
[tensor] Support adding result to output with multiply
Support adding result to output with multiply to fuse multiply and add
operations for LSTMcell.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Jihoon Lee [Thu, 28 Oct 2021 04:42:30 +0000 (13:42 +0900)]
[Profiler] Add listener total time and percent
This patch add listener total time and percent mechanism
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Parichay Kapoor [Thu, 28 Oct 2021 05:15:40 +0000 (14:15 +0900)]
[tensor] Optimize tensor apply and multiply_strided
Optimize multiply_strided and apply to use std::transform to get the best
performance, and possibility to use par_unseq for later parallelization.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Wed, 27 Oct 2021 11:54:45 +0000 (20:54 +0900)]
[layer] Performance optimizations to LSTMCell
Add minor performance optimizations to LSTMCell.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Wed, 27 Oct 2021 11:36:01 +0000 (20:36 +0900)]
[tensor] Add beta for tensor sum
Add beta for tensor to enable addition to existing tensor while summing.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Jihoon Lee [Wed, 27 Oct 2021 02:59:03 +0000 (11:59 +0900)]
[API/trivial] Add LSTMCell creator
This patch add lstm cell creator
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
hyeonseok lee [Tue, 26 Oct 2021 08:44:17 +0000 (17:44 +0900)]
[layer] override setBatch in bn layer
- Override setBatch function
Self evaluation:
Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: hyeonseok lee <hs89.lee@samsung.com>
Parichay Kapoor [Tue, 26 Oct 2021 07:06:11 +0000 (16:06 +0900)]
[layer] LSTM cell batchresize bug fix
As LSTM cell uses the batch values not in the batch dimension but in the
inner dimension, it creates bug with the existing layer context interface
as it only allows updating the batch along the batch dimension.
This patch provides a quick fix to combine the time and batch values in
the batch dimension.
Once recurrent realizer allows multiple inputs, the time factor will be
removed and the reshapes can be removed.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Tue, 26 Oct 2021 06:31:20 +0000 (15:31 +0900)]
[layer] Optimization for LSTM cell
Optimization for LSTM cell to combine the activation function running on
the values.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Jihoon Lee [Tue, 26 Oct 2021 01:18:29 +0000 (10:18 +0900)]
[Android] Add missing fopenmp
This patch add missing fopenmp.
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 22 Oct 2021 14:41:02 +0000 (23:41 +0900)]
[Android] Enable test for the android build
This patch enable android build, this abling this option has an issue
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 22 Oct 2021 13:17:25 +0000 (22:17 +0900)]
[Android] Unlink ml-api-inference module
This patch unlink ml-api-inference module as it requires gstreamer
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Tue, 26 Oct 2021 05:19:15 +0000 (14:19 +0900)]
[Fix/pkgconfig] Fix nntrainer include dir
This patch fixes pkg config nntrainer include directory for the pkg.
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Tue, 26 Oct 2021 01:22:03 +0000 (10:22 +0900)]
[LSTMCELL] Fix compile error
This patch fix compile error in the main branch
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Parichay Kapoor [Mon, 25 Oct 2021 01:29:50 +0000 (10:29 +0900)]
[layer] Bug fix for LSTMCell
Enable recurrent unrolling tests for LSTMCell with a single and stacked
LSTM cell. Add corresponding bug fixes.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Sat, 23 Oct 2021 13:18:51 +0000 (22:18 +0900)]
[layer] Add first draft of lstm cell
This patch adds the first draft of lstm cell. This has certain
advantages over the existing LSTM when executing for single timestep:
- does not need to divide the data over smaller chunks for computation.
This saves the overhead of creating tensors, an extra loop and combines
computation to a single call for efficiency.
- some of the operations have been avoided based on the current timestep
(which were zeros).
More optimizatios will be coming soon for LSTM cell.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Sat, 23 Oct 2021 13:16:21 +0000 (22:16 +0900)]
[tensor] Add support for strided operators
This patch adds support for strided multiply, apply and getSharedDataTensor
operations for tesnor.
Later these will be optimized direct data access with an iterator.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Jihoon Lee [Fri, 22 Oct 2021 12:22:53 +0000 (21:22 +0900)]
[Android] disable nnstreamer layer
This patch disables nnstreamer backbone because of the package issue for
android build
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 22 Oct 2021 10:24:15 +0000 (19:24 +0900)]
[Clean] remove enable-android option
This patch remove enable-android for brevity
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 22 Oct 2021 10:22:51 +0000 (19:22 +0900)]
[Android] Add prebuilt android for the built library
This patch provides prebuilt android when making android zip file
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 22 Oct 2021 10:11:07 +0000 (19:11 +0900)]
[Android] Add simple package script
This patch adds android simple package script to make tar.gz
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 22 Oct 2021 10:10:17 +0000 (19:10 +0900)]
[Fix] fix bug for install_headers
Fix bug that install_headers install_dir overrides subdir props.
This is undocumented behavior so changed to make it to be deterministic
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 22 Oct 2021 09:36:06 +0000 (18:36 +0900)]
[Android] Set up install directory for android build
This patch sets up install directory for android build.
It is defaulted to installed in the meson.build_root() /
android_build_result
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 22 Oct 2021 08:54:29 +0000 (17:54 +0900)]
[Android] Enable capi/ccapi build
This patch enable capi/ccapi build
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 22 Oct 2021 08:01:55 +0000 (17:01 +0900)]
[Meson] Add android nntrainer build
This patch add android meson build.
which can be triggered by ninja
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 22 Oct 2021 06:48:58 +0000 (15:48 +0900)]
[Meson] Save and apply extra defines
This patch saves extra defines as a list to be applied for the android
build as well
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 22 Oct 2021 05:19:23 +0000 (14:19 +0900)]
[Android] disable building src through ninja
This patch disables building src through ninja for android
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 22 Oct 2021 03:37:10 +0000 (12:37 +0900)]
[android] Add platform android and install deps
This patch add platform android and install deps according to it
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Thu, 21 Oct 2021 07:34:06 +0000 (16:34 +0900)]
[TRIVIAL] Fix error log
error log string was concated by '+' which was invalid
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Thu, 21 Oct 2021 06:54:59 +0000 (15:54 +0900)]
[Android] Remove bfd linker
bfd linker is deprecated so it's also deleted from android build
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Thu, 21 Oct 2021 05:53:45 +0000 (14:53 +0900)]
[Trivial] Add override to functions
This patch add override to virtual functions mainly to suppress compile
warnings from android build
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Mon, 25 Oct 2021 09:26:37 +0000 (18:26 +0900)]
[Optimizer] Prevent weight allocation for dependent nodes
This patch prevents optimizer weight allocation for dependent nodes
Resolves #1671
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Parichay Kapoor [Fri, 22 Oct 2021 07:25:08 +0000 (16:25 +0900)]
[build] Add openmp dependency for improved performance
This patch adds openmp dependency while building for improved
performance.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Fri, 22 Oct 2021 06:59:48 +0000 (15:59 +0900)]
[tensor] Bug fix for dropout mask generation
This patch adds fix for dropout mask generation. It was being
out-of-place which was leading to not using the tensor memory already
allocated. This patch adds the interface to use the already allocated
tensor and use it.
This patch further saves memory while generating dropout mask itself as
well.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Jihoon Lee [Thu, 21 Oct 2021 07:55:15 +0000 (16:55 +0900)]
[SVACE] Fix init problem
This patch fixes initiation problem
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Parichay Kapoor [Mon, 13 Sep 2021 07:09:51 +0000 (16:09 +0900)]
[mem-opt] Add script to memory peak RSS memory
This patch adds a script to measure the peak RSS memory.
To check the memory consumption, pass the command to run the
applicaiton/test to this script.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Jihoon Lee [Thu, 21 Oct 2021 05:38:13 +0000 (14:38 +0900)]
[Fix] not used variable
Not used variable is deleted from static analyzer
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Thu, 21 Oct 2021 05:36:18 +0000 (14:36 +0900)]
[Fix] Svace issue
Fix uinit ctor
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Wed, 20 Oct 2021 13:37:13 +0000 (22:37 +0900)]
[Concat] Add axis selection
Add axis selction mechanism if axis is not specified that if height
exist, concat to the width dimension
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Wed, 20 Oct 2021 13:36:51 +0000 (22:36 +0900)]
[Reshape] Add exportTo
This patch implements exportTo to reshape
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Wed, 20 Oct 2021 12:33:55 +0000 (21:33 +0900)]
[Fix] fix validate Input
This patch fixes validate Input
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Thu, 21 Oct 2021 03:54:23 +0000 (12:54 +0900)]
[test/bn1d] Add bn1d support
This patch adds bn1d support for the transLayer_v2
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Wed, 20 Oct 2021 12:21:12 +0000 (21:21 +0900)]
[Fix] fix recorder registration
This patch fix registration bug when there is more than two overloaded
delgation
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Wed, 20 Oct 2021 12:20:26 +0000 (21:20 +0900)]
[Fix] Layer node clone configuration
layer node clone configuration was not copying realization, this is
fixed with this patch
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Parichay Kapoor [Wed, 20 Oct 2021 05:49:32 +0000 (14:49 +0900)]
[layer] Create property for axis in batch normalization
This patch convert the axis member in batch normalization to a property
to allow setting it externally.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Tue, 19 Oct 2021 10:52:24 +0000 (19:52 +0900)]
[profile] Support profiling by layer type
This patch adds support for profiling of the model based on the type of
the layers.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Tue, 19 Oct 2021 10:10:40 +0000 (19:10 +0900)]
[network/neuralnet] Move backwarding to network
The iteration of the graph to perform the backwarding has been moved to
the network graph in this patch.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Wed, 20 Oct 2021 04:54:42 +0000 (13:54 +0900)]
[layer] Support concat along all the dimensions
This patch provides support for the concat layer among all the
dimensions. Other changes include:
- split_dimension property keyword has been changed to axis and reused
for concat_dimension
- TensorDim::getNumDim() is now a static function
Corresponding unittests for the concat layer are also added.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Jihoon Lee [Tue, 19 Oct 2021 11:19:01 +0000 (20:19 +0900)]
[Test] Add recurrent value compare tests
This patch adds recurrent value compares golden tests
There are 4 cases presented.
1. single fc recurrent
2. stacked fc recurrent
3. single lstm recurrent
4. stacked lstm recurrent
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Tue, 19 Oct 2021 10:03:30 +0000 (19:03 +0900)]
[LSTM] Fix single time step backwarding
This patch updates single time step backwarding to properly set and see
the last timestep values
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Tue, 19 Oct 2021 10:00:56 +0000 (19:00 +0900)]
[Test] Add model v2 test generator
This patch add model v2 test generator using pytorch.
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Tue, 19 Oct 2021 08:39:59 +0000 (17:39 +0900)]
[Test] Connect v2 test to the parameterized test
This patch connect v2 test to the model param test for convenience
+ add validateFor_v2. This is separated out because label dimension is
not given, although it should be merged to validateFor
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Tue, 19 Oct 2021 08:32:39 +0000 (17:32 +0900)]
[Test] Add models v2 test
This patch proposes models v2 test which is much lighter.
**Purpose**
1. Lighter model test as layer golden test compensates for it.
2. Easier debugging especially by checking offset, and moving metatdat
3. Give greater flexibility to the golden test generator on the model
architecture
4. Not saving golden file but making it on the fly on the builder side
**Major Difference**
1. Gradient comparison is skipped as it is automatically checked at the
next iteration.
2. Add metadata about num_iterations to file
3. Add size check while reading the golden file
4. Delete redundant information that can be infered.
5. Every iteration has it's own input information.
**Golden Format**
```
## file format is as below
# [<number of iteration(int)> <Iteration> <Iteration>...<Iteration>]
# Each iteration contains
# [<input(Tensors)><Label(Tensors)><Parameters(Tensors)><Output(Tensors)>]
# Each tensor contains
# [<num_elements(int32)><data_point(float32)>...<data_point(float32)>]
```
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Tue, 19 Oct 2021 08:29:15 +0000 (17:29 +0900)]
[Trivial] Open sizeCheckedReadTensor to util
This patch opens sizeCheckedReadTensor to util for the later use
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Tue, 19 Oct 2021 08:20:00 +0000 (17:20 +0900)]
[Fix/Sharing] Fix gradient updation log
Gradient should be initialized at the very first (backward) access, but the
shared weight was updated at the very last (backward) access, this patch
resolves the issue by adding isFirstAccess to the var_grad.h
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Parichay Kapoor [Tue, 19 Oct 2021 07:10:44 +0000 (16:10 +0900)]
[layer] Support reshape layer
This patch provides support for reshape layer and basic unittests.
The flatten layer is also updated to use reshape layer internally.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Tue, 19 Oct 2021 05:14:54 +0000 (14:14 +0900)]
[test] GRU with reset_after=False
GRU matches with tf1 but not with tf2.
Tf2 changes the reset_after to True as default, which leads to an extra
bias. This patch forces the reset_after to be always false to enable
matching with tf1 and tf2.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Tue, 19 Oct 2021 02:40:52 +0000 (11:40 +0900)]
[tensor] Add checks for external tensors
This patch adds checks while setting the external tensors:
- ensure that the external tensor size is more than the required memory
- if the tensor size is 0, its data must be null
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Mon, 18 Oct 2021 08:32:46 +0000 (17:32 +0900)]
[layer/test] Add unittests for conv1d
This patch adds layers golden tests for conv1d.
conv1d is also added to the list of transpose layers while saving data
from tensorflow.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Mon, 18 Oct 2021 08:30:36 +0000 (17:30 +0900)]
[layer] Add conv1d implementation
This patch adds conv1d implementation support.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Mon, 18 Oct 2021 08:28:45 +0000 (17:28 +0900)]
[properties] Bug fix for padding compute
This patch provides bug fix for padding compute in the usage of stride.
The stride along width and height were being used wrongly.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Jihoon Lee [Mon, 18 Oct 2021 02:21:48 +0000 (11:21 +0900)]
[LSTM] Add lstm fix and test run
This patch add lstm fix and test run.
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 15 Oct 2021 09:24:42 +0000 (18:24 +0900)]
[Neuralnet] Add debug mode to check optimizer
This patch add debug mode to check optimizer
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 15 Oct 2021 09:23:51 +0000 (18:23 +0900)]
[Fix] tensor pool prerequest bug
This patch fix tensor pool prereuqestedrequest bug by that shared_name
was wrong
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 15 Oct 2021 06:29:42 +0000 (15:29 +0900)]
[Trivial] Move makeGraph function out
this patch moves makeGraph function from compiler_util -> test_util
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 15 Oct 2021 06:05:15 +0000 (15:05 +0900)]
[Test] Make model test to take arbitrary nn
This patch updates model test to accept arbitrary neuralnetwork instead
of ini only
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 15 Oct 2021 04:52:57 +0000 (13:52 +0900)]
[Trivial] Extract golden test from nnt_models
This patch extracts golden test from nntrainer models, nothing done.
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Fri, 15 Oct 2021 04:23:55 +0000 (13:23 +0900)]
[Trivial] Separate Watchers into a new file
This patch separates watchers into a new file
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Parichay Kapoor [Wed, 8 Sep 2021 11:56:50 +0000 (20:56 +0900)]
[mem-opt] Support in-place batch normalization
This patch provides support for in-place batch normalization.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Wed, 8 Sep 2021 11:48:16 +0000 (20:48 +0900)]
[batchnorm] Optimize batch norm memory usage
This patch optimized batch norm memory usage by updating the exec order
for its input as its not used in calcDerivative.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Parichay Kapoor [Tue, 7 Sep 2021 07:21:27 +0000 (16:21 +0900)]
[in-place] Support in-place activation
This patch adds support for in-place activation. Further, the in-place
optimization has been moved to the model compile phase.
This patch also adds support for running the model in in-place as well
as out-of-place mode.
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
Jihoon Lee [Thu, 14 Oct 2021 17:07:03 +0000 (02:07 +0900)]
[LayerGroups] Normalize identifiers
This patch normalize identifiers inside addWithRefrenceLayers() for further use
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Thu, 14 Oct 2021 17:06:37 +0000 (02:06 +0900)]
[Refactor] Recurrent realizer with in/out props
This patch adds recurrent realizer with in/out props
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Thu, 14 Oct 2021 16:14:06 +0000 (01:14 +0900)]
[Realizer] Implement input realizer
This patch implements input realizer which connect input to external
input
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>
Jihoon Lee [Thu, 14 Oct 2021 11:06:23 +0000 (20:06 +0900)]
[Realizer] Implement slice realizer
This patch implements slice realizer and it's test
**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped
Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>