platform/core/api/mediavision.git
5 months agoDelete Unreachable code 03/302603/1
Kwanghoon Son [Mon, 11 Dec 2023 06:40:47 +0000 (15:40 +0900)]
Delete Unreachable code

[Issue type] SVACE

Model object already checked and can not be null.

Change-Id: Ie4ce60c6221a8639a1f000966c23f3282b05cad3
Signed-off-by: Kwanghoon Son <k.son@samsung.com>
5 months agoMerge branch 'tizen_devel' into tizen accepted/tizen/unified/20231206.103842 accepted/tizen/unified/20231206.151824 accepted/tizen/unified/riscv/20231211.234031
Kwanghoon Son [Wed, 6 Dec 2023 01:37:12 +0000 (10:37 +0900)]
Merge branch 'tizen_devel' into tizen

[Version] 0.31.0

Inki Dae (8):
  mv_machine_learning: drop input and output type dependency from image
    classification
  mv_machine_learning: introduce config file for external plugin
  mv_machine_learning: use plugin config file for object detection
  mv_machine_learning: introduce MachineLearningNative module
  mv_machine_learning: drop input and output type dependency from
    landmark detection
  mv_machine_learning: drop code duplication from landmark detection
  mv_machine_learning: drop input and output type dependency from object
    detection
  mv_machine_learning: drop code duplication from object detection

Kwanghoon Son (1):
  Drop ml_only flag build

Vibhav Aggarwal (12):
  mv_machine_learning: add template to Adapter::create() function
  mv_machine_learning: convert LandmarkDetection class into template
    class
  mv_machine_learning: convert ImageClassification class into template
    class
  mv_machine_learning: refactor ImageSegmentation task group
  mv_test: integrate machine learning testcases into mv_test
  mv_machine_learning: check value range in FaceRecognition testcase
  mv_machine_learning: bug fix for invalid memory access
  mv_machine_learning: disable tests
  mv_machine_learning: update image path
  mv_machine_learing: enable inference-based tests
  mv_machine_learning: bug fix for memory leak
  mv_machine_learning: add testcase to mv_test

sangho park (2):
  mv_machine_learning: add verifying result data
  mv_machine_learning: improve comparison of selfie-segmentation result

Change-Id: Ibd8f978e8aa874f58131b574a02d63e1cd4e5fc1

5 months agomv_machine_learning: improve comparison of selfie-segmentation result
sangho park [Tue, 5 Dec 2023 06:12:57 +0000 (15:12 +0900)]
mv_machine_learning: improve comparison of selfie-segmentation result

[Issue type] code improvement

Change the judgment criteria from simple binary comparison to ssim

Change-Id: Idef31272464de702a3339674b858d9fe43935150
Signed-off-by: sangho park <sangho.g.park@samsung.com>
5 months agomv_machine_learning: drop code duplication from object detection
Inki Dae [Fri, 1 Dec 2023 08:05:31 +0000 (17:05 +0900)]
mv_machine_learning: drop code duplication from object detection

[Issue type] : code cleanup

Drop code duplication from the object detection task group by making the
this task group use MachineLearningNative module instead of internal code
for context management. In addition, this patch fixes user given model issue
by passing the user given model to each task group correctly.

Change-Id: I15993c841ae9ec5b9b1900089f88295d77b0629c
Signed-off-by: Inki Dae <inki.dae@samsung.com>
5 months agomv_machine_learning: add testcase to mv_test
Vibhav Aggarwal [Thu, 30 Nov 2023 10:49:28 +0000 (19:49 +0900)]
mv_machine_learning: add testcase to mv_test

[Issue type] code improvement

Add the FaceRecognitionAccuracy testcase to mv_test.
It is disabled by default since its execution time is high.

Change-Id: I99e789e7ee8456a5fcc56d2e3d7edd6f4264529b
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
5 months agomv_machine_learning: bug fix for memory leak
Vibhav Aggarwal [Thu, 30 Nov 2023 10:45:40 +0000 (19:45 +0900)]
mv_machine_learning: bug fix for memory leak

[Issue type] bug fix

In SimpleShot::configureModel() function, old model was not
being destroyed before creating the new model. This led to
memory leak detected by ASAN. This patch fixes the issue.

Change-Id: I6a8c3bf3df50c4b1eb39f2e7c94c777b39633323
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
5 months agomv_machine_learning: drop input and output type dependency from object detection
Inki Dae [Thu, 30 Nov 2023 06:34:24 +0000 (15:34 +0900)]
mv_machine_learning: drop input and output type dependency from object detection

[Issue type] : code refactoring

Drop input and output type dependency from object detection task group
by making the input and output types specific to the object detection
task group to be inherited from the common types,
 and then by making adapter class of the object detection task group
to use the common type instead of specific one.

Change-Id: Ie1a397cd8fc05bd507497f04637d606815c5cccc
Signed-off-by: Inki Dae <inki.dae@samsung.com>
5 months agomv_machine_learning: drop code duplication from landmark detection
Inki Dae [Wed, 29 Nov 2023 07:46:28 +0000 (16:46 +0900)]
mv_machine_learning: drop code duplication from landmark detection

[Issue type] : code cleanup

Drop code duplucation from the landmark detection task group by making the
this task group use MachineLearningNative module instead of internal code
for context management.

Change-Id: I990edccfe92cde801e665f9fc43f9f566f33c124
Signed-off-by: Inki Dae <inki.dae@samsung.com>
5 months agomv_machine_learning: drop input and output type dependency from landmark detection
Inki Dae [Wed, 29 Nov 2023 06:19:45 +0000 (15:19 +0900)]
mv_machine_learning: drop input and output type dependency from landmark detection

[Issue type] : code refactoring

Drop input and output type dependency from landmark detection task group
by making the input and output types specific to the landmark detection
task group to be inherited from the common types,
and then by making adapter class of the landmark detection task group
to use the common type instead of specific one.

Change-Id: Ie366cab3919bdbbbf315263188f4ca4f6516cd33
Signed-off-by: Inki Dae <inki.dae@samsung.com>
5 months agoDrop ml_only flag build
Kwanghoon Son [Wed, 29 Nov 2023 06:17:35 +0000 (15:17 +0900)]
Drop ml_only flag build

It was added due to the yocto-based requirements, but it is no longer used.

Change-Id: Ie238e0dc4e8fed18b2a3e2c077c9b2c7ed1007ff
Signed-off-by: Kwanghoon Son <k.son@samsung.com>
5 months agomv_machine_learning: add verifying result data
sangho park [Wed, 22 Nov 2023 23:27:21 +0000 (08:27 +0900)]
mv_machine_learning: add verifying result data

[Issue type] code improvement

verifying result data of selfie-segmentation external plugin
by restoring result data to opencv Mat and storing image file

Change-Id: I13a6f18a9bf8ec45b78fd2ae57f7e33c54c185fc
Signed-off-by: sangho park <sangho.g.park@samsung.com>
5 months agomv_machine_learing: enable inference-based tests
Vibhav Aggarwal [Mon, 27 Nov 2023 09:43:59 +0000 (18:43 +0900)]
mv_machine_learing: enable inference-based tests

[Issue type] code improvement

This patch re-enables some of the testcases which were
disabled in [1]. Specifically, all the machine learning
testcases except Face Recognition and Image Segmentation
task groups have been enabled. For Face Recognition,
there are still some memory leaks related to NNTrainer
and further investigation is required. Image Segmentation
testcases are still under development [2].

[1] https://review.tizen.org/gerrit/#/c/platform/core/api/mediavision/+/301888/
[2] https://review.tizen.org/gerrit/#/c/platform/core/api/mediavision/+/301785/

Change-Id: If97012b42165f210b1f67aaaf4a28e169c16303f
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
5 months agomv_machine_learning: update image path
Vibhav Aggarwal [Tue, 28 Nov 2023 05:11:26 +0000 (14:11 +0900)]
mv_machine_learning: update image path

[Issue type] bug fix

Update the path of sample images used for testing
in the Landmark Detection task group.

Change-Id: Ie0b4d6975a708fc77f2011504ed29a5e05984796
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
5 months agomv_machine_learning: introduce MachineLearningNative module
Inki Dae [Mon, 27 Nov 2023 01:52:12 +0000 (10:52 +0900)]
mv_machine_learning: introduce MachineLearningNative module

[Issue type] : new feature

Introduce MachineLearningNative module which can be used commonly
for all task groups for context management.

This patch allows us to eliminate the duplicated code within each task's native
API implementation module, where the module name begin with the "mv_" prefix.
By consolidating all task management code into the common module
- MachineLearningNative.cpp - and utilizing this module instead of creating
separate implementations in each task's native API module, we can streamline
development and reduce redundancy.

As a starting point, this patch makes the image classification task group
utilize the common module rather than its own internal code.
The goal of this change is to determine whether or not we can offer a unified
interface for all task groups. If successful, this will allow us to apply
the same native module to other task groups as well.

If our initial tests prove successful, we can move forward with implementing
a task manager that utilizes a inference and training pipeline graph approach
to manage individual tasks.

Change-Id: I4cf4f2a06294acae7ac94d3f562958d2ad3d0770
Signed-off-by: Inki Dae <inki.dae@samsung.com>
5 months agomv_machine_learning: use plugin config file for object detection
Inki Dae [Thu, 23 Nov 2023 04:04:45 +0000 (13:04 +0900)]
mv_machine_learning: use plugin config file for object detection

[Issue type] : new feature

Use config files for object detection external plugin by adding
face_detection_plugin.json and object_detection_plugin.json files, parsing
plugin information from the config files, and use the information
according to user desired TASK api - face detection and object detection.

With this patch, external plugin relevant code has no dependency of
the config file for object detection task group. So this patch
checks if external plugin should be used or not based on the plugin flag
value which is loaded from each plugin config file.

In case object detection task group, each task API - face detection and
object detection - supports both model types - FD_TRIV2 and OD_TRIV2 - for
external plugin because the supported external plugin supports both of them
internally. Therefore, the behavior may be changed later according to use
case change.

Change-Id: I2a82000e5d7049bb91a661711dc2cdc3ef37c57c
Signed-off-by: Inki Dae <inki.dae@samsung.com>
5 months agomv_machine_learning: disable tests
Vibhav Aggarwal [Mon, 27 Nov 2023 03:58:06 +0000 (12:58 +0900)]
mv_machine_learning: disable tests

[Issue type] temporary fix

Some memory related bug in mv_machine_learning task
groups is causing the testcases to fail. This came to
notice after machine learning testcases were integrated
into mv_test[1] and the Tizen MMFW bot reported an error
after running the testcases with ASan enabled.

This patch temporarily disables those testcases until
the issue is resolved.

[1] https://review.tizen.org/gerrit/#/c/platform/core/api/mediavision/+/301594/

Change-Id: I4f69c6a2b2eded914a5422696d3af85f3ed0be00
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
5 months agomv_machine_learning: bug fix for invalid memory access
Vibhav Aggarwal [Wed, 22 Nov 2023 03:47:35 +0000 (12:47 +0900)]
mv_machine_learning: bug fix for invalid memory access

[Issue type] bug fix

In the result() function of LandmarkDetection, ObjectDetection
and ObjectDetection3d task groups, the _result struct
was being cleared using memset to set all bytes to 0.
However, the structs include some vectors so this approach
of clearing the struct becomes invalid and resulted in
invalid memory access.

Change-Id: I98ed3e427e77b276145adbf461b3eaca44c752f6
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
5 months agomv_machine_learning: check value range in FaceRecognition testcase
Vibhav Aggarwal [Tue, 21 Nov 2023 06:19:26 +0000 (15:19 +0900)]
mv_machine_learning: check value range in FaceRecognition testcase

[Issue type] code refactoring

In the FaceRecognition.RawResultAfterInferenceShouldBeOk testcase,
check if the output raw values lie inside an acceptable range,
instead of comparing them with the exact value. The exact value
may change based on many factors like the underlying engine version.

Change-Id: Id917bb94716c6dda369dbe8ba4de1bf6311d6e73
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
5 months agomv_test: integrate machine learning testcases into mv_test
Vibhav Aggarwal [Mon, 20 Nov 2023 09:32:12 +0000 (18:32 +0900)]
mv_test: integrate machine learning testcases into mv_test

[Issue type] code refactoring

Change-Id: I881586381c9957d15cf97331c2a44ad52df75847
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
5 months agomv_machine_learning: introduce config file for external plugin
Inki Dae [Tue, 21 Nov 2023 00:38:19 +0000 (09:38 +0900)]
mv_machine_learning: introduce config file for external plugin

[Issue type] : new feature

Introduce a config file for image segmentation external plugin by adding
selfie_segmentation_plugin.json file, parsing plugin information from
the config file, and use the information.

With this patch, external plugin relevant code has no dependency of
the config file for image segmentation task group. So this patch
checks if external plugin should be used or not based on the plugin flag
value which is read from the plugin config file after reading the plugin file
name from the plugin config file.

Change-Id: I126ca1706ce021257c187f76ff54615c3d0be743
Signed-off-by: Inki Dae <inki.dae@samsung.com>
5 months agomv_machine_learning: drop input and output type dependency from image classification
Inki Dae [Wed, 15 Nov 2023 10:31:30 +0000 (19:31 +0900)]
mv_machine_learning: drop input and output type dependency from image classification

[Issue type] : code refactoring

Drop input and output type dependency from image classification task group
by introducing new common input and output types, InputBaseType and
OutputBaseType, which is located in mv_machine_learning/common directory,
and by making the input and output types specific to the image
classification task group to be inherited from the common types,
and then by making adapter class of the image classification task group
to use the common type instead of specific one.

In MachineLearningType.h which is a new common header file,
struct InputBaseType {
...
};

struct OutputBaseType {
...
};

And in image_classification_type.h which is specific to image classification
task group,
struct ImageClassificationInput : public InputBaseType {
...
};

struct ImageClassificationOutput : public OutputBaseType {
...
};

With this, native API implementation - mv_image_classification.c - of the image
classification task group has no any dependency on the file so that we can
make all native implementation portions - the code in the module files starting
with 'mv_' prefix - to use common function in the mv_machine_learning/common
directory. The common function will be introduced soon.

This is a first step to drop the code duplication from the native API
implementation module of the each task group. And it's a time for second phase
since the completion of dropping the meta file dependency from the concrete
class of each task group.

Change-Id: I14764253aeafc36d4e0f39b204b35985b3ecc73a
Signed-off-by: Inki Dae <inki.dae@samsung.com>
5 months agomv_machine_learning: refactor ImageSegmentation task group
Vibhav Aggarwal [Wed, 15 Nov 2023 05:25:38 +0000 (14:25 +0900)]
mv_machine_learning: refactor ImageSegmentation task group

[Issue type] code refactoring

Lift parsing dependency from ImageSegmentation class to
ImageSegmentationAdapter and convert ImageSegmentation
into a template class.

Change-Id: I349ac7ba04f7d0193765e8cb72b6ca1a3fb7830d
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
5 months agomv_machine_learning: convert ImageClassification class into template class
Vibhav Aggarwal [Tue, 14 Nov 2023 11:01:23 +0000 (20:01 +0900)]
mv_machine_learning: convert ImageClassification class into template class

[Issue type] code refactoring

Change-Id: I07a76684aece1773a3ef9dd2b3abbb430dad2394
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
5 months agomv_machine_learning: convert LandmarkDetection class into template class
Vibhav Aggarwal [Tue, 14 Nov 2023 09:39:21 +0000 (18:39 +0900)]
mv_machine_learning: convert LandmarkDetection class into template class

[Issue type] code refactoring

Change-Id: Ib4a040c5a6b97fd19ab3696247e2b9df11ce9ea0
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
5 months agomv_machine_learning: add template to Adapter::create() function
Vibhav Aggarwal [Tue, 14 Nov 2023 05:04:39 +0000 (14:04 +0900)]
mv_machine_learning: add template to Adapter::create() function

[Issue type] code refactoring

Convert the Adapter::create() function into a template function
to avoid nesting of switch cases for data type and task type.
This patch applies this change to ObjectDetection and
ObjectDetection3d task groups.

Change-Id: I1c65f844f8dd6bb646d3806fa898161db3c347e9
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
6 months agoFix wrong input setting 31/302031/2
Kwanghoon Son [Thu, 30 Nov 2023 00:53:30 +0000 (09:53 +0900)]
Fix wrong input setting

[Issue type] bug fix

MobilenetV1 has ASAN bug which is caused from wrong input.

==1368962==AddressSanitizer CHECK failed: ../../../../libsanitizer/asan/asan_allocator.cpp:191 "((old)) == ((kAllocBegMagic))" (0x3f1f9fa03ecececf, 0xcc6e96b9cc6e96b9)
    #0 0x7f823855f4  (/lib64/libasan.so+0xca5f4)
    #1 0x7f823a2a80  (/lib64/libasan.so+0xe7a80)
    #2 0x7f822ecfe8  (/lib64/libasan.so+0x31fe8)
    #3 0x7f822ed8c8  (/lib64/libasan.so+0x328c8)
    #4 0x7f822ede1c  (/lib64/libasan.so+0x32e1c)
    #5 0x7f8237cb50 in __interceptor_free (/lib64/libasan.so+0xc1b50)
    #6 0x7f3698d0c8 in xnn_delete_operator (/lib64/libinference-engine-tflite.so+0x5350c8)
    #7 0x7f36961d48 in xnn_delete_runtime (/lib64/libinference-engine-tflite.so+0x509d48)
    #8 0x7f3690644c  (/lib64/libinference-engine-tflite.so+0x4ae44c)
    #9 0x7f368084a0 in tflite::Subgraph::CleanupNode(int) (/lib64/libinference-engine-tflite.so+0x3b04a0)

Change-Id: I029d2354f4a96e9b362703e18febc8e6d2b68c45
Signed-off-by: Kwanghoon Son <k.son@samsung.com>
6 months agoFix dlog format specifier 87/301887/1
Kwanghoon Son [Mon, 27 Nov 2023 03:56:02 +0000 (12:56 +0900)]
Fix dlog format specifier

Change-Id: I4dc0da1cee660822b9bd9c3ce19214d20c64bb9f
Signed-off-by: Kwanghoon Son <k.son@samsung.com>
6 months agoMerge branch 'tizen_devel' into tizen accepted/tizen/unified/20231115.024850
Kwanghoon Son [Tue, 14 Nov 2023 07:53:58 +0000 (16:53 +0900)]
Merge branch 'tizen_devel' into tizen

[Version] 0.30.0

Inki Dae (11):
  mv_machine_learning: add image segmentation task group support
  mv_machine_learning: use DEFAULT_MODEL_NAME for face detection task
  mv_machine_learning: use DEFAULT_MODEL_NAME for landmark detection
    task group
  mv_machine_learning: drop parsing dependency from landmark detection
  test: fix answer table values for landmark detection
  mv_machine_learning: drop unused code
  mv_machine_learning: drop parsing dependency from object detection 3d
  mv_machine_learning: introduce MachineLearningConfig class
  mv_machine_learning: drop setTaskType member function
  mv_machine_learning: use MachineLearningConfig class for other task
    groups
  mv_machine_learning: move meta directory into common one

Kwanghoon Son (2):
  Fix wrong log level
  Update cmake version to 3.13

Seungbae Shin (1):
  mv_machine_learning: initialize class/structure member properly

Vibhav Aggarwal (10):
  mv_machine_learning: bug fix in decision weight threshold
  mv_machine_learning: reallocate parser on changing meta file
  mv_machine_learning: drop parsing dependency from object detection
  mv_machine_learning: drop parsing dependency from image classification
  mv_machine_learning: drop template from MachineLearningConfig class
  mv_machine_learning: parse label file name in
    MachineLearningConfig::parseConfigFile()
  mv_machine_learning: replace ObjectDetectionConfig by
    MachineLearningConfig
  mv_machine_learning: load meta and label files in configure()
  mv_machine_learning: convert ObjectDetection3d into a template class
  mv_machine_learning: convert ObjectDetection class into template class

sangho park (1):
  Fix build break

Change-Id: Ica0db898b2d79febeb287e03ce83b323b4cbb7cc

6 months agoUpdate cmake version to 3.13
Kwanghoon Son [Mon, 13 Nov 2023 04:24:52 +0000 (13:24 +0900)]
Update cmake version to 3.13

Range version supported from 3.12 so just set minimum version to 3.13

Change-Id: Ied6872be4fff447318709e816d84ca08ee4219dd
Signed-off-by: Kwanghoon Son <k.son@samsung.com>
6 months agomv_machine_learning: convert ObjectDetection class into template class
Vibhav Aggarwal [Fri, 10 Nov 2023 11:27:23 +0000 (20:27 +0900)]
mv_machine_learning: convert ObjectDetection class into template class

[Issue type] code refactoring

Change-Id: Ib84f8211a919c82e389659bdce1b652f4419e2fb
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
6 months agomv_machine_learning: move meta directory into common one
Inki Dae [Mon, 13 Nov 2023 05:29:15 +0000 (14:29 +0900)]
mv_machine_learning: move meta directory into common one

[Issue type] : code cleanup

Move meta directory into common one. The meta directory contains
common code to parse task own config and meta information from given files
including preprocessing and postprocessing.

However, current directory structure made it to build each task group
with the code in meta directory. It led building-duplication.

Therefore, this patch moves the meta directory into common one to combine
meta and common code. As a reault, common directory will provide one so
library - mv_ml_common.so - to each task group, and the each task group
will include only relevant header files in the common and meta directories.

Change-Id: Ib9172761ddaad0b9044f6ec02cfc71dfecb59890
Signed-off-by: Inki Dae <inki.dae@samsung.com>
6 months agomv_machine_learning: use MachineLearningConfig class for other task groups
Inki Dae [Fri, 10 Nov 2023 04:03:59 +0000 (13:03 +0900)]
mv_machine_learning: use MachineLearningConfig class for other task groups

[Issue type] : code refactoring

Use MachineLearningConfig class for other task groups instead of internal ones.

We have introduced a new common config class[1] and object detection and
object detection 3d task groups use already it. So this patch makes
other task groups - which not use the common config class - to use the
common class instead of internal ones, and drop the internal config class.

In addition, this patch applies confidence threshold config key for
landmark detection task group to MachineLearningConfig class.

[1] https://review.tizen.org/gerrit/#/c/platform/core/api/mediavision/+/301043/

Change-Id: I6e77ce33349bbdcb22f64102a7ec3f8808249b1c
Signed-off-by: Inki Dae <inki.dae@samsung.com>
6 months agomv_machine_learning: convert ObjectDetection3d into a template class
Vibhav Aggarwal [Fri, 10 Nov 2023 10:10:07 +0000 (19:10 +0900)]
mv_machine_learning: convert ObjectDetection3d into a template class

[Issue type] code refactoring

After the introduction of MachineLearningConfig, it is now possible
to convert each task group's object class into a template class.
This patch introduces this change to the object detection 3d task group.

Change-Id: Ib887846b4b1466f70d9131079f6f7ce62598fbb5
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
6 months agomv_machine_learning: load meta and label files in configure()
Vibhav Aggarwal [Fri, 10 Nov 2023 06:37:54 +0000 (15:37 +0900)]
mv_machine_learning: load meta and label files in configure()

[Issue type] bug fix

Load the model meta and label files in configure() function
instead during the object construction.

Change-Id: I0e77fb27074ebe4d6402f66d671eee37bce855f6
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
6 months agoFix wrong log level
Kwanghoon Son [Fri, 10 Nov 2023 07:38:20 +0000 (16:38 +0900)]
Fix wrong log level

[Issue type] Fix

Change-Id: Icdfc7f071f95be8b11b256c7ff8de3716466402a
Signed-off-by: Kwanghoon Son <k.son@samsung.com>
6 months agomv_machine_learning: replace ObjectDetectionConfig by MachineLearningConfig
Vibhav Aggarwal [Thu, 9 Nov 2023 05:03:52 +0000 (14:03 +0900)]
mv_machine_learning: replace ObjectDetectionConfig by MachineLearningConfig

[Issue type] code refactoring

This patch makes the ObjectDetection task group use
MachineLearningConfig class to reduce code duplication
across different task groups.

Change-Id: I07d8482ae761faa7918f4c66f5fc492ffe8debe5
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
6 months agomv_machine_learning: parse label file name in MachineLearningConfig::parseConfigFile()
Vibhav Aggarwal [Fri, 10 Nov 2023 05:53:54 +0000 (14:53 +0900)]
mv_machine_learning: parse label file name in MachineLearningConfig::parseConfigFile()

[Issue type] code improvement

Get the label file name from config file during parsing
to support the task groups that rely on a label file.

Change-Id: Ice7c8108c09b10643dfcf8573b0c4a302e98ce97
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
6 months agomv_machine_learning: drop template from MachineLearningConfig class
Vibhav Aggarwal [Fri, 10 Nov 2023 03:45:39 +0000 (12:45 +0900)]
mv_machine_learning: drop template from MachineLearningConfig class

[Issue type] code refactoring

MachineLearningConfig class introduced in [1] required the
ParserType template to initialize the _parser. This resulted in
the inclusion of parser header files of each task group in the
higher level MachineLearningConfig class which led to code smell.

This patch drops the use of template and as a first step,
modifies the object detection 3d task group to work with this
new version.

[1] https://review.tizen.org/gerrit/#/c/platform/core/api/mediavision/+/301043/

Change-Id: I90971294c89615d85ae61e789fa6200e53d367c9
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
6 months agomv_machine_learning: drop setTaskType member function
Inki Dae [Wed, 8 Nov 2023 06:23:38 +0000 (15:23 +0900)]
mv_machine_learning: drop setTaskType member function

[Issue type] : code refactoring

Drop setTaskType member function from MetaParser and its child classes.

The setTaskType member function was used to give model specific parser object
to MetaParser class so that the MetaParser can parse model specific meta
information which describes how to parse the postprocess node of each model
specific meta file.

However, we can give the specific parser object when task group specific
parser class is created so drop the setTaskType member function and update
the object at constructor of task group specific parser class.

As of now, only object detection task group will set the specific task type
for various models support. For other task groups, 0 is used in default.

Change-Id: I7156660481f1e42daf89093f0fb1097943bbae05
Signed-off-by: Inki Dae <inki.dae@samsung.com>
6 months agomv_machine_learning: introduce MachineLearningConfig class
Inki Dae [Tue, 7 Nov 2023 11:07:51 +0000 (20:07 +0900)]
mv_machine_learning: introduce MachineLearningConfig class

[Issue type] : new feature

Introduce MachineLearningConfig class which parses configuration
and meta files for each task group.

Until now, we have fixed the code smell, divergent change, of each task group
by extracting parsing portion from each concrete class of each task group,
and we conformed that the parsing portions can be used commonly for all
task groups.

This patch introduces a new common configuration class, MachineLearningConfig
by moving all parsing portions to this class. As a first use case,
this patch makes ObjectDetection3d task group to use the MachineLarningConfig
class instead of internal code, ObjectDetection3dConfig class so it drops
all relevant internal code - finally, we can drop the ugly code duplication.

Change-Id: I8f6a3c276e61903350fd1ae4390f7b125931b2b3
Signed-off-by: Inki Dae <inki.dae@samsung.com>
6 months agomv_machine_learning: drop parsing dependency from object detection 3d
Inki Dae [Mon, 6 Nov 2023 07:05:46 +0000 (16:05 +0900)]
mv_machine_learning: drop parsing dependency from object detection 3d

[Issue type] : code refactoring

Drop the configuration and meta file parsing dependency from ObjectDetection3d
class.

Until now, the concrete class of each task group got the task group
configuration information from its own configuration file, and also
included a MetaParser class object to get the tensor information,
which is corresponding to a given model file.

However, these dependencies led code smell, divergent change[1] even though
the concrete class has no any dependency from parsing the configuration and
meta files - needed only information after parsed.

As a first refactoring work, this patch extracts parsing portion from
ObjectDetection3d class and introduces as a new class, ObjectDetection3dConfig
class.

With this, adapter classes of the object detection 3d task group will parse
the configuration and meta files before creating ObjectDetection3d class.
And then it will create ObjectDetection3d class with needed information.
As a result, we could manage the ObjectDetection3d class without any
dependency on parsing work.

[1] https://refactoring.guru/smells/divergent-change

Change-Id: Ia3955a841179c802437f175e783762affc258b4d
Signed-off-by: Inki Dae <inki.dae@samsung.com>
6 months agomv_machine_learning: drop parsing dependency from image classification
Vibhav Aggarwal [Tue, 7 Nov 2023 10:01:57 +0000 (19:01 +0900)]
mv_machine_learning: drop parsing dependency from image classification

[Issue type] : code refactoring

Drop the configuration and meta file parsing dependency from ImageClassification
class.

Until now, the concrete class of each task group got the task group
configuration information from its own configuration file, and also
included a MetaParser class object to get the tensor information,
which is corresponding to a given model file.

However, these dependencies led code smell, divergent change[1] even though
the concrete class has no any dependency from parsing the configuration and
meta files - needed only information after parsed.

As a first refactoring work, this patch extracts parsing portion from
ImageClassification class and introduces as a new class, ImageClassificationConfig
class.

With this, adapter classes of the image classification task group will parse
the configuration and meta files before creating ImageClassification class.
And then it will create ImageClassification class with needed information.
As a result, we could manage the ImageClassification class without any
dependency on parsing work.

[1] https://refactoring.guru/smells/divergent-change

Change-Id: I421dcd6f796e6865dde3e5fed597ef83e7e5bc12
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
6 months agomv_machine_learning: drop parsing dependency from object detection
Vibhav Aggarwal [Mon, 6 Nov 2023 09:59:32 +0000 (18:59 +0900)]
mv_machine_learning: drop parsing dependency from object detection

[Issue type] : code refactoring

Drop the configuration and meta file parsing dependency from ObjectDetection
class.

Until now, the concrete class of each task group got the task group
configuration information from its own configuration file, and also
included a MetaParser class object to get the tensor information,
which is corresponding to a given model file.

However, these dependencies led code smell, divergent change[1] even though
the concrete class has no any dependency from parsing the configuration and
meta files - needed only information after parsed.

As a first refactoring work, this patch extracts parsing portion from
ObjectDetection class and introduces as a new class, ObjectDetectionConfig
class.

With this, adapter classes of the object detection task group will parse
the configuration and meta files before creating ObjectDetection class.
And then it will create ObjectDetection class with needed information.
As a result, we could manage the ObjectDetection class without any
dependency on parsing work.

[1] https://refactoring.guru/smells/divergent-change

Change-Id: I526d58e0012ba4daae8eb6eb7ef99bb3cf8545d6
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
6 months agomv_machine_learning: reallocate parser on changing meta file
Vibhav Aggarwal [Mon, 6 Nov 2023 09:32:59 +0000 (18:32 +0900)]
mv_machine_learning: reallocate parser on changing meta file

[Issue type] bug fix

The MetaParser needs to be reallocated when the user
calls mv_facial_landmark_set_model() or mv_pose_landmark_set_model().

Change-Id: I7f21c7d36b3ffb9b869a1998bcd8ee19a365fd38
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
6 months agomv_machine_learning: initialize class/structure member properly
Seungbae Shin [Fri, 27 Oct 2023 08:49:30 +0000 (17:49 +0900)]
mv_machine_learning: initialize class/structure member properly

[Issue type] : code cleanup

Change-Id: I3bdc0c98836ea4adf0f840ce6e3290685c1d2b6d

6 months agomv_machine_learning: drop unused code
Inki Dae [Mon, 6 Nov 2023 06:28:13 +0000 (15:28 +0900)]
mv_machine_learning: drop unused code

[Issue type] : code cleanup

Drop unused member functions from LandmarkDetection class.

Change-Id: Id6c6b33fbe79f1b381a18111f1fe201df543a46e
Signed-off-by: Inki Dae <inki.dae@samsung.com>
6 months agomv_machine_learning: bug fix in decision weight threshold
Vibhav Aggarwal [Thu, 2 Nov 2023 04:53:04 +0000 (13:53 +0900)]
mv_machine_learning: bug fix in decision weight threshold

[Issue type] bug fix

There was a minor bug in FaceRecognition::checkResult()
in which the decision weight threshold check fails to
identify the cases where multiple elements have highest
raw data value.

Change-Id: Ie859e95a72af39bc80af8dd2621f959bb0ad79d6
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
6 months agotest: fix answer table values for landmark detection
Inki Dae [Thu, 2 Nov 2023 02:21:50 +0000 (11:21 +0900)]
test: fix answer table values for landmark detection

[Issue type] bug fix

Fix answer table values for landmark detection test case.
With a new patch[1], 'confidence_threshold' value is used to check
valid detection result for pose detection task - its specific class is PldCpm
class. Without this patch, landmark detection test cases will fail.
This patch updates the answer table values of relevant test cases correctly.

[1] https://review.tizen.org/gerrit/#/c/platform/core/api/mediavision/+/300755/

Change-Id: Id0219912d9c912bb4f4b597a968a3ba9213bfa9d
Signed-off-by: Inki Dae <inki.dae@samsung.com>
6 months agomv_machine_learning: drop parsing dependency from landmark detection
Inki Dae [Tue, 31 Oct 2023 03:45:56 +0000 (12:45 +0900)]
mv_machine_learning: drop parsing dependency from landmark detection

[Issue type] : code refactoring

Drop the configuration and meta file parsing dependency from LandmarkDetection
class.

Until now, the concrete class of each task group got the task group
configuration information from its own configuration file, and also
included a MetaPaser class object to get the tensor information,
which is corresponding to a given model file.

However, these dependencies led code smell, divergent change[1] even though
the concrete class has no any dependency from parsing the configuration and
meta files - needed only information after parsed.

As a first refactoring work, this patch extracts parsing portion from
LandmarkDetection class and introduces as a new class, LandmarkDetectionConfig
class.

With this, adapter classes of the landmark detection task group will parse
the configuration and meta files before creating LandmarkDetection class.
And then it will create LandmarkDetection class with needed information.
As a result, we could manage the LandmarkDetection class without any
dependency on parsing work.

[1] https://refactoring.guru/smells/divergent-change

Change-Id: I29f2d684e2b6e698dcf36a8294c608e90dda67c0
Signed-off-by: Inki Dae <inki.dae@samsung.com>
6 months agomv_machine_learning: use DEFAULT_MODEL_NAME for landmark detection task group
Inki Dae [Wed, 25 Oct 2023 11:07:07 +0000 (20:07 +0900)]
mv_machine_learning: use DEFAULT_MODEL_NAME for landmark detection task group

[Issue type] : new feature

Use the meta file for landmark detection tasks to set the default model

With this patch, a proper concrete class object will be created
at the constructor of FacialLandmarkAdapter and PoseLandmarkAdapter classes
according to the default model names which are parsed from each meta file,
facial_landmark.json and pose_landmark.json files.

And also this patch calls the member function, create, to create the concrete
class object corresponding to the default model name instead of creating
a default concrete class object directly.

Finally, we can create user-desired concrete class object without rebuilding
by simply modifying the meta files.

Change-Id: I4ef045fe1ae24177ae4ebc7a3f48ae6e136c01dc
Signed-off-by: Inki Dae <inki.dae@samsung.com>
6 months agomv_machine_learning: use DEFAULT_MODEL_NAME for face detection task
Inki Dae [Wed, 25 Oct 2023 06:48:25 +0000 (15:48 +0900)]
mv_machine_learning: use DEFAULT_MODEL_NAME for face detection task

[Issue type] : new feature

Use the meta file for face detection task to set the default model

With this patch, a proper concrete class object will be created
at the constructor of FaceDetectionAdapter class according to the default
model name which is parsed from face_detection.json file.

And also this patch calls the member function, create, to create the concrete
class object corresponding to the default model name instead of creating
a default concrete class object directly.

Finally, we can create user-desired concrete class object without rebuilding
by simply modifying the meta file, face_detection.json.

Change-Id: Iaddd0a820180332707852bbea51afcafb49238fa
Signed-off-by: Inki Dae <inki.dae@samsung.com>
6 months agomv_machine_learning: add image segmentation task group support
Inki Dae [Tue, 17 Oct 2023 05:15:22 +0000 (14:15 +0900)]
mv_machine_learning: add image segmentation task group support

[Issue type] : new feature

Add image segmentation task group support. As a first specific task,
this patch adds a selfie segmentation task which uses the external plugin
module via ImageSegmentationExternal class instead of internal one.
Regarding the internal task, I will update it later.

Selfie segmentation task uses image data as input source and makes also image
data as output result.

Change-Id: I6d19e176164c0187bfe710ec41a050839fa90091
Signed-off-by: Inki Dae <inki.dae@samsung.com>
6 months agoFix build break
sangho park [Mon, 13 Nov 2023 05:59:06 +0000 (14:59 +0900)]
Fix build break

[Issue type] Fix

add 'mv_private.h' for fixing build braek regarding 'dlog.h' and clean up duplicate include.

Change-Id: If5ae377f1bd3ecb53279277ace834d5d4ed1209c
Signed-off-by: sangho park <sangho.g.park@samsung.com>
7 months agoMerge branch 'tizen_devel' into tizen accepted/tizen/unified/20231025.093310 accepted/tizen/unified/20231025.120201
Kwanghoon Son [Wed, 25 Oct 2023 01:54:49 +0000 (10:54 +0900)]
Merge branch 'tizen_devel' into tizen

[Version] 0.29.0

Inki Dae (16):
  mv_machine_learning: move inferenceThreadLoop into class
  mv_machine_learning: use template type for outgoing_queue
  mv_machine_learning: introduce common asynchronous inference manager
  mv_machine_learning: drop redundant member variables
  mv_machine_learning: code refactoring to AsyncManager
  mv_machine_learning: embed AsyncManager into each task group
  mv_machine_learning: drop mv_image_classification_open.h header file
  mv_machine_learning: move member functions to private
  mv_machine_learning: drop Mediavision dependency from image
    classification adapter
  mv_machine_learning: add async API for image classification task group
  mv_machine_learning: move member functions to private
  mv_machine_learning: drop Mediavision dependency from landmark
    detection adapters
  mv_machine_learning: drop FaceRecognitionRegisterInput structure
  mv_machine_learning: add  add async API for landmark detection task
    group
  mv_machine_learning: drop importLabel wrapper
  mv_machine_learning: clean up mv_private.h header inclusion

Vibhav Aggarwal (4):
  mv_machine_learning: bug fix for setting backend type and target
    device
  mv_machine_learning: clearing up resources in the beginning of each
    test case
  mv_machine_learning: bug fix when no label is registered
  mv_machine_learning: return NO_DATA error

Change-Id: I8b1b169e3d1e5092d95ce8ad9fb87a3f476f98d1

7 months agomv_machine_learning: clean up mv_private.h header inclusion
Inki Dae [Mon, 23 Oct 2023 03:04:51 +0000 (12:04 +0900)]
mv_machine_learning: clean up mv_private.h header inclusion

[Issue type] : code cleanup

Clean up the header inclusion by making mv_private.h to be included by
machine_learning_preprocess.cpp instead of machine_learning_preprocess.h
because the header inclusion is needed to use LOG_TAG but only needed by
machine_learning_preprocess.cpp file.

Change-Id: I656f00a1508ce68487d63ba2a3c1fc3d62922549
Signed-off-by: Inki Dae <inki.dae@samsung.com>
7 months agomv_machine_learning: drop importLabel wrapper
Inki Dae [Wed, 18 Oct 2023 11:11:50 +0000 (20:11 +0900)]
mv_machine_learning: drop importLabel wrapper

[Issue type] : code cleanup

Drop importLabel member function of FaceRecognition class. This wrapper
function isn't needed anymore so we can call the importLabel function of
LabelManager class instead.

And also change return type to void including removeLabel().
It's enough with throwing an exception in error case, and return NO_DATA
if label file doesn't exist when we called recognitionFace() - it means
that we tried to recognize a given input but didn't find a proper label.

Change-Id: Ib49b72261a1b8be029d58f8088507e8bece0c674
Signed-off-by: Inki Dae <inki.dae@samsung.com>
7 months agomv_machine_learning: return NO_DATA error
Vibhav Aggarwal [Fri, 13 Oct 2023 09:13:10 +0000 (18:13 +0900)]
mv_machine_learning: return NO_DATA error

[Issue type] : bug fix

When the user does not register any face,
or unregisters all the faces, inference should
return MEDIA_VISION_ERROR_NO_DATA.

Change-Id: Ie66ffc8e8ba0357983103125bf6278b79703df2e
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
7 months agomv_machine_learning: bug fix when no label is registered
Vibhav Aggarwal [Tue, 17 Oct 2023 04:46:48 +0000 (13:46 +0900)]
mv_machine_learning: bug fix when no label is registered

[Issue type] : bug fix

LabelManager::removeLabel had a bug where it did not remove
a label if it was the only label registered.
Additionally, LabelManager::importLabel did not clear the
_labels vector when label file doesn't exist.
This patch fixes this behaviour.

Change-Id: I628d3a6bfff41dbfd782d39c1b573175ce974b38
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
7 months agomv_machine_learning: clearing up resources in the beginning of each test case
Vibhav Aggarwal [Fri, 13 Oct 2023 05:27:01 +0000 (14:27 +0900)]
mv_machine_learning: clearing up resources in the beginning of each test case

[Issue type] : bug fix

The RemoveModelResources() function is being called at the end
of the testcases to remove the model files. However, when a
test case fails due to failed assertion, the model files are not
getting removed and this can affect the next testcase.
This patch removes the model files in the beginning as well.

Change-Id: Ib94e894f168c35fe783f6195366369e1b8d18dd5
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
7 months agomv_machine_learning: bug fix for setting backend type and target device
Vibhav Aggarwal [Tue, 10 Oct 2023 03:12:07 +0000 (12:12 +0900)]
mv_machine_learning: bug fix for setting backend type and target device

[Issue type] : bug fix

The set_model and set_engine APIs should have higher precedence
over the default config file, whenever used. However, for the
tasks landmark_detection, objection_detection and
object_detection_3d, the config file was overwriting model
and engine info. This patch fixes this behaviour.

Change-Id: Iafdfbbd86df98ea60d301fda2d2d8053074ac2ab
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
7 months agomv_machine_learning: add add async API for landmark detection task group
Inki Dae [Wed, 27 Sep 2023 06:52:36 +0000 (15:52 +0900)]
mv_machine_learning: add  add async API for landmark detection task group

[Issue type] : new feature

Add two asynchronous API, mv_facial_landmark_inference_async and
mv_pose_landmark_inference_async, for landmark detection task group
using AsyncManger class which is a common class for asynchronous API support
including test case.

In addition, this patch drops the use of mutex lock due to the dead lock issue
as known-issue mentioned already through the patch[1]

[1] commit-id : fdbd84f1ff3b30c9f18598b65da777f65611e4cb

Change-Id: Ifbf5a58d5053ca0226caff14913f4a0d3ba1c45e
Signed-off-by: Inki Dae <inki.dae@samsung.com>
7 months agomv_machine_learning: drop FaceRecognitionRegisterInput structure
Inki Dae [Tue, 26 Sep 2023 07:32:37 +0000 (16:32 +0900)]
mv_machine_learning: drop FaceRecognitionRegisterInput structure

[Issue type] : code cleanup

Drop FaceRecognitionRegisterInput structure. We can use inputs and labels
members of FaceRecognitionInput structure for register instead of
FaceRecognitionRegisterInput one.

Change-Id: I62fdfad7dee351baf5b4e64af59780ac2cce9165
Reported-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
Signed-off-by: Inki Dae <inki.dae@samsung.com>
7 months agomv_machine_learning: drop Mediavision dependency from landmark detection adapters
Inki Dae [Mon, 25 Sep 2023 07:55:03 +0000 (16:55 +0900)]
mv_machine_learning: drop Mediavision dependency from landmark detection adapters

[Issue type] : code refactoring

Drop Mediavision dependency from adapter classies of landmark detection task
group. There is a use case that pre-trained model file is used in private.
Therefore, the model relevant code cannot be opened. And even such users want
to configure the inference or training engines in their way.

In this case, we need to manage it properly by providing plugin approach of
behavior class. And this patch is a first step for supporting plugin based
behavior class which can be delivered as separate package.

Change-Id: I54a3f2f8de86290718129b8a44a724d3d9a3f246
Signed-off-by: Inki Dae <inki.dae@samsung.com>
7 months agomv_machine_learning: move member functions to private
Inki Dae [Mon, 25 Sep 2023 07:27:41 +0000 (16:27 +0900)]
mv_machine_learning: move member functions to private

[Issue type] : code cleanup

Move two member functions inference and preprocess from protected to private.
They are used only in LandmarkDetection class.

Change-Id: I5890575da6c926fcfb6c164c27e071bea68a3752
Signed-off-by: Inki Dae <inki.dae@samsung.com>
7 months agomv_machine_learning: add async API for image classification task group
Inki Dae [Fri, 22 Sep 2023 09:42:45 +0000 (18:42 +0900)]
mv_machine_learning: add async API for image classification task group

[Issue type] : new feature

Add asynchronous API, mv_image_classification_inference_async, for image
classification task group using AsyncManger class which is a common class
for asynchronous API support including test case.

In addition, this patch includes one fixup and cleanup.
- Fix a issue that it can access to the empty input queue after
  waitforInputQueue() call by checking if the input queue is empty or not.
- Clean up parseMetaFile and configure member functions by using const &
  as a parameter type.

Change-Id: I402760e9f585e113b06933b7d36df1767637e5c2
Signed-off-by: Inki Dae <inki.dae@samsung.com>
7 months agomv_machine_learning: drop Mediavision dependency from image classification adapter
Inki Dae [Thu, 21 Sep 2023 07:39:35 +0000 (16:39 +0900)]
mv_machine_learning: drop Mediavision dependency from image classification adapter

[Issue type] : code refactoring

Drop Mediavision dependency from adapter class of image classification task
group. There is a use case that pre-trained model file is used in private.
Therefore, the model relevant code cannot be opened. And even such users want
to configure the inference or training engines in their way.

In this case, we need to manage it properly by providing plugin approach of
behavior class. And this patch is a first step for supporting plugin based
behavior class which can be delivered as separate package.

Change-Id: Ibb7d4b5fa292378bbc5ab1fe7f5c8c00406ee3b2
Signed-off-by: Inki Dae <inki.dae@samsung.com>
7 months agomv_machine_learning: move member functions to private
Inki Dae [Thu, 21 Sep 2023 06:58:10 +0000 (15:58 +0900)]
mv_machine_learning: move member functions to private

[Issue type] : code cleanup

Move two member functions inference and preprocess from protected to private.
They are used only in ImageClassification class.

Change-Id: I2e283dd8e55bbf5a9cb45359b48ebeb08f9d1892
Signed-off-by: Inki Dae <inki.dae@samsung.com>
7 months agomv_machine_learning: drop mv_image_classification_open.h header file
Inki Dae [Thu, 21 Sep 2023 06:42:25 +0000 (15:42 +0900)]
mv_machine_learning: drop mv_image_classification_open.h header file

[Issue type] : code cleanup

Drop mv_image_classification_open.h header file which is a abandoned file.

Change-Id: I8be8bc47acd6880f5eaef3cab126ca0f60069498
Signed-off-by: Inki Dae <inki.dae@samsung.com>
7 months agomv_machine_learning: embed AsyncManager into each task group
Inki Dae [Wed, 13 Sep 2023 07:21:58 +0000 (16:21 +0900)]
mv_machine_learning: embed AsyncManager into each task group

[Issue type] : code cleanup

Embed AsyncManager into eash task group by moving all implementations from
cpp file to header one because keeping the implementations in cpp file
propergates template type declarations every time each task group
uses the AsyncManager for async API support.

Wit this patch, mv_ml_common.so library file isn't created anymore so drop
the relevant code.

Change-Id: I4fbabcfbe5cb29053858549fd77400ae9ae3ec2d
Signed-off-by: Inki Dae <inki.dae@samsung.com>
7 months agomv_machine_learning: code refactoring to AsyncManager
Inki Dae [Tue, 12 Sep 2023 09:18:58 +0000 (18:18 +0900)]
mv_machine_learning: code refactoring to AsyncManager

[Issue type] : code refactoring

Do code refactoring to AsyncManager by introducing new public member functions -
push() and pop() - and by moving existing all member functions as private ones
excepting isWorking(), popFromInput() and pushToOutput() which are used
in InferenceCallback function of the object detection task group.

By doing this, it can support async API with AsyncManager easily because
what each task group has to do for it is to call push() in PerformAsync()
and to call pop() in getOutput(). In addition, this patch moves the logic
of inferenceCallback() to lamda block of performAsync().

For async API support, what each task group have to do is,
    void TaskGroup::performAsync()
    {
        ...
        // create AsyncManager class with lamda function.
        ...
        _async_manager->push(inputVectors);
    }

    void TaskGroup::getOutput()
    {
        if (_async_manager) {
            ...
            _current_result = _async_manager->pop();
        }
        ...
    }

Change-Id: I022c71b921b5351d739ba685c188625806162000
Signed-off-by: Inki Dae <inki.dae@samsung.com>
7 months agomv_machine_learning: drop redundant member variables
Inki Dae [Tue, 12 Sep 2023 06:56:10 +0000 (15:56 +0900)]
mv_machine_learning: drop redundant member variables

[Issue type] : code cleanup

Drop a redundant member variables from ObjectDetectionInput and
AsyncInputQueue structures. The member variables - handle, inference_src,
and user_data - aren't used anymore.

Change-Id: Iecd39cb8ca0f2cafdfd09b792bdcbe9cca72ca8b
Signed-off-by: Inki Dae <inki.dae@samsung.com>
7 months agomv_machine_learning: introduce common asynchronous inference manager
Inki Dae [Thu, 7 Sep 2023 00:16:24 +0000 (09:16 +0900)]
mv_machine_learning: introduce common asynchronous inference manager

[Issue type] new feature

Introduce a new asynchronous inference manager - AsyncManager class - for
asynchronous API support of all task groups. This patch just moves the queue
management relevant code from object detection task group to common directory
so that other task groups can use it commonly for the asynchronous API
implementation.

As of the queue management, input queue can be used commonly for all
task groups even though data type of each input tensor is different each
other by storing the input tensor data with unsigned char type but not used
for output queue.

In case of output queue, how to decode the output tensor data is different
each other according to the used pre-trained model. It means that each result
structure of the used pre-trained model should be different. Therefore,
this patch uses template type for output queue so that model-driven data
structure can be used.

Each task group who wants to use AsyncManager has to follow below rules,
 - create a async manager handle with its own inference callback function.
   i.e,
       in template<typename T> void void ObjectDetection::performAsync(...),
       {
           if (!async_manager) {
               // Create async manager handler with its own inference callback.
                _async_manager = make_unique<AsyncManager<TaskResult> >(
                          [this]() { inferenceCallback<T, TaskResult(); });
           }

           if (!_async_manager->isInputQueueEmpty<T>())
               return;
           ...

           // Push a input queue for inference request.
           _async_manager->pushToInput<T>(in_queue);

           // Invoke async manager. This triggers a internal thread for
           // performing the inference with the given input queue.
           _async_manager->invoke<T>();
       }

      and in ObjectDetection::getOutput()
  {
          if (_async_manager) {
              if (!_async_manager->isWorking())
                  throw an_exception;

              _async_manager->waitforOutputQueue();
              _current_result = _async_manager->popFromQueue();
          }
          ...
     }

     and in inference callback function of each task group,
 template<typename T, typename R> void ObjectDetection::inferenceCallback()
 {
         // Get a input queue for inference.
         AsyncInputQueue<T> inputQ = _async_manager->popFromInput<T>();
         inference<T>(inputQ.inputs);
         R &resultQ = result();
         resultQ.frame_number = inputQ.frame_number;

         // push the inference result to outgoing queue.
         _async_manager->pushToOutput(resultQ);
     }

Change-Id: Ie2b68b910a73377fa4d4ad8646b134e3c2bf709a
Signed-off-by: Inki Dae <inki.dae@samsung.com>
7 months agomv_machine_learning: use template type for outgoing_queue
Inki Dae [Wed, 6 Sep 2023 05:28:54 +0000 (14:28 +0900)]
mv_machine_learning: use template type for outgoing_queue

[Issue type] : code cleanup

Use template type for _outgoing_queue. As for asynchronous inference,
we need to allow various structures to be pushed to outgoing queue
because each task group can be used for different purpose and this means
that various result structures are needed.

This is just a step for introducing asynchronous inference manager.

Change-Id: I5d7fbe59da3c95e59366f10f8232eed42a6a5e60
Signed-off-by: Inki Dae <inki.dae@samsung.com>
7 months agomv_machine_learning: move inferenceThreadLoop into class
Inki Dae [Wed, 6 Sep 2023 00:49:36 +0000 (09:49 +0900)]
mv_machine_learning: move inferenceThreadLoop into class

[Issue type] : code cleanup

Move the thread callback, inferenceThreadLoop(), for performing asynchronous
inference into ObjectDetection class. This is a first step for introducing
asynchronous inference manager which can be used commonly for other
task groups.

Change-Id: Ie018712406e7e922f92dfea8a23aead72e8e61c9
Signed-off-by: Inki Dae <inki.dae@samsung.com>
7 months agoFix vision-source API 65/300265/2
Kwanghoon Son [Thu, 19 Oct 2023 10:18:16 +0000 (19:18 +0900)]
Fix vision-source API

[Version] 0.28.19

Change-Id: I56a7582af5009149f796f48c00b15e2b075518d1
Link: https://review.tizen.org/gerrit/c/platform/core/multimedia/vision-source/+/299618
Signed-off-by: Kwanghoon Son <k.son@samsung.com>
8 months agomv_machine_learning: update test case for face recognition 32/299132/1 accepted/tizen/8.0/unified/20231005.092749 accepted/tizen/unified/20230921.171329 tizen_8.0_m2_release
Inki Dae [Wed, 20 Sep 2023 09:13:09 +0000 (18:13 +0900)]
mv_machine_learning: update test case for face recognition

[Version] : 0.28.18
[Issue type] : update test case

Update the test case 'RemoveAllLabelShouldBeOk' for face recognition.
The test case checks if unregistering all labels works well after training.
However, we didn't consider confirming the actual results
after the training-after-unregistering test.

This patch ensures that the actual results are confirmed.

Change-Id: I368a0440140d39bb3769a0a447c6cde96fb20b7b
Signed-off-by: Inki Dae <inki.dae@samsung.com>
8 months agomv_machine_learning: make sure to clean up dataset 12/299112/2
Inki Dae [Wed, 20 Sep 2023 03:16:30 +0000 (12:16 +0900)]
mv_machine_learning: make sure to clean up dataset

[Version] : 0.28.17
[Issue type] : bug fix

Make sure to clean up the training dataset after training the dataset.
If the dataset isn't cleaned up after training then new dataset will be
stacked on the previous ones. So clean up it after training.

Change-Id: I4f985e87560d5ddc237b31a16be1998ff40b1563
Signed-off-by: Inki Dae <inki.dae@samsung.com>
8 months agomv_machine_learning: fix label removal issue of face recognition 19/298919/1
Inki Dae [Fri, 15 Sep 2023 07:17:51 +0000 (16:17 +0900)]
mv_machine_learning: fix label removal issue of face recognition

[Issue type] : bug fix
[Version] : 0.28.16

Fix a label removal issue of face recognition framework.

Due to this issue, seg. fault happened when unregistering a given label
because the label table wan't updated after unregistering the label.

Therefore, this patch fixes the problem by making sure to update the label
table after unregistering. In addition, this patch makes the result CAPI
to be allowed after mv_face_recognition_inference function call so relevalt
test case is also updated properly.

Change-Id: Ib913d56c8b3625c2c6ebf9468aa19734d4ada0d2
Signed-off-by: Inki Dae <inki.dae@samsung.com>
8 months agoMerge branch 'tizen_devel' into tizen accepted/tizen/unified/20230906.014616
Kwanghoon Son [Mon, 4 Sep 2023 05:03:08 +0000 (14:03 +0900)]
Merge branch 'tizen_devel' into tizen

[Version] 0.28.15

Inki Dae (6):
  mv_machine_learning: code clean to setModelInfo function
  mv_machine_learning: do not check model info
  mv_machine_learning: introduce DEFAULT_MODEL_NAME for object detection
  mv_machine_learning: code refactoring to getOutput function
  mv_machine_learning: drop dead code
  mv_machine_learning: change async API behavior

Kwanghoon Son (4):
  Remove mv_private include
  mv_common: Drop common_c functions
  mv_3d: Drop open functions
  Remove FORCED_STATIC_BUILD option

Change-Id: If70c54ebc3f13ff35e514d208e5d8217a72aed56

8 months agoRemove FORCED_STATIC_BUILD option
Kwanghoon Son [Fri, 1 Sep 2023 08:17:22 +0000 (17:17 +0900)]
Remove FORCED_STATIC_BUILD option

Static link option is never used.

[Issue type] Refactoring

Change-Id: Ia2e916f53f84eb8ca9d00203dc7e19815d9edaba
Signed-off-by: Kwanghoon Son <k.son@samsung.com>
8 months agomv_3d: Drop open functions
Kwanghoon Son [Wed, 30 Aug 2023 06:46:08 +0000 (15:46 +0900)]
mv_3d: Drop open functions

[Issue type] refactoring

3d open functions was created for individual adapter layer
requirements, but was never used. Delete it because it slows down
execution and only increases complexity.

Change-Id: I1ad39ece35e808a4fde2f5f9bc32bf342bbb1153
Signed-off-by: Kwanghoon Son <k.son@samsung.com>
8 months agomv_common: Drop common_c functions
Kwanghoon Son [Wed, 30 Aug 2023 00:26:23 +0000 (09:26 +0900)]
mv_common: Drop common_c functions

[Issue type] refactoring

common_c functions was created for individual adapter layer
requirements, but was never used. Delete it because it slows down
execution and only increases complexity.

Change-Id: I550e5a354f413635a45277eaa0abef028bb452ab
Signed-off-by: Kwanghoon Son <k.son@samsung.com>
8 months agomv_machine_learning: change async API behavior
Inki Dae [Wed, 23 Aug 2023 01:53:51 +0000 (10:53 +0900)]
mv_machine_learning: change async API behavior

[Issue type] : code refactoring

Change async API behavior of object detection task group.

In original version, async API created an internal thread which is
different from main thread because the callback function registered by
user was called by the new thread context created by the internal framework.

With this patch, user has to create a new thread for the use of async API.
Below is an simple example,
void thread_cb(...)
{
...
while (condition) {
...
mv_object_detection_get_result(handle, &number_of_objects, ...);
...

for (unsigned int idx = 0; idx < number_of_objects; ++idx) {
...
mv_object_detection_get_label(handle, idx, &label);
...
}
}
}

In addition, this patch drops the use of mutex for thread safety due to
dead lock issue. I will introduce a new thread safety solution with
more fine-grained lock approach later.

Change-Id: I7621097a7b08456d61af79ca2659546083e0ece0
Signed-off-by: Inki Dae <inki.dae@samsung.com>
8 months agomv_machine_learning: drop dead code
Inki Dae [Wed, 23 Aug 2023 01:46:19 +0000 (10:46 +0900)]
mv_machine_learning: drop dead code

[Issue type] : code cleanup

Drop the dead code. mv_face_detection_open.h and mv_object_detection_open.h
files aren't used anymore.

Change-Id: I5cbc98ee87b59c599d2790706dc32d8d8cec2944
Signed-off-by: Inki Dae <inki.dae@samsung.com>
8 months agomv_machine_learning: code refactoring to getOutput function
Inki Dae [Fri, 11 Aug 2023 06:00:46 +0000 (15:00 +0900)]
mv_machine_learning: code refactoring to getOutput function

[Issue type] : code refactoring

Do code refactoring to getOutput function of the concrete class,
ObjectDetection, by extracting some portion of existing getOutput function code
and moving it to a new interface function, getOutputCache.

With original version of the getOutput function, it handled the specific case
that one more get-result API are called, which in turn, it made the getOutput
function to be complicated.

Therefore, this patch drops the specific case from the getOutput function
by introducing getOutputCache interface which always returns internal
result object, _current_result instead of calling the result member function
of each concrete class which decodes raw output tensor data after the completion
of the inference.

Change-Id: I4aacb8fc55be3f63a983182b919291dcb927b0c9
Signed-off-by: Inki Dae <inki.dae@samsung.com>
8 months agoRemove mv_private include
Kwanghoon Son [Fri, 11 Aug 2023 02:42:27 +0000 (11:42 +0900)]
Remove mv_private include

mv_private is not used and should not include in header file

Change-Id: Ia3149b9b048ca06ca19535e80ec5b6b23e006dbe
Signed-off-by: Kwanghoon Son <k.son@samsung.com>
8 months agomv_machine_learning: introduce DEFAULT_MODEL_NAME for object detection
Inki Dae [Wed, 9 Aug 2023 04:04:47 +0000 (13:04 +0900)]
mv_machine_learning: introduce DEFAULT_MODEL_NAME for object detection

[Issue type] : new feature

Introduce DEFAULT_MODEL_NAME for object detection task group, which is a new
key for object detection meta file.

With this patch, a proper concreate class object will be created
at the constructor of ObjectDetectionAdapter class according to the default
model name which is parsed from object_detection.json file.

And also this patch calls the member function, create, to create the concrete
class object corresponding to the default model name instead of creating
a default concrete class object directly.

Finally, we can create user-desired concrete class object without rebuilding
by simply modifying the meta file, object_detection.json.

Change-Id: Ie159991985c4ed9a28abc5a377037110fe319be6
Signed-off-by: Inki Dae <inki.dae@samsung.com>
8 months agomv_machine_learning: do not check model info
Inki Dae [Mon, 7 Aug 2023 23:45:12 +0000 (08:45 +0900)]
mv_machine_learning: do not check model info

[Issue type] : bug fix

Do not check model info given by user because the user-given model info
is optional but not mandatory. If the model info isn't set then default
one will be used instead.

Change-Id: I4b7dea2e2e3c53a6e418e5ac7f7cc22c328cdbd6
Signed-off-by: Inki Dae <inki.dae@samsung.com>
8 months agomv_machine_learning: code clean to setModelInfo function
Inki Dae [Mon, 7 Aug 2023 22:45:58 +0000 (07:45 +0900)]
mv_machine_learning: code clean to setModelInfo function

[Issue type] : code cleanup

Clean up setModelInfo function by doing function extraction.

setModelInfo function converted a given model name to its corresponding
task type internally so it made the function to be complicated.
This patch extracts the conversion portion from setModelInfo function as
another function and makes the function to be called instead.

Change-Id: I93243ffba543f9c6e9a1f0d7690d8f48a1bd14e1
Signed-off-by: Inki Dae <inki.dae@samsung.com>
9 months agoFix build warnings 54/298154/1
Kwanghoon Son [Fri, 1 Sep 2023 08:27:49 +0000 (17:27 +0900)]
Fix build warnings

Change-Id: I180eeca73b7b95c66bbc65d93afbef9ed98e08e5
Signed-off-by: Kwanghoon Son <k.son@samsung.com>
9 months agopackaging: drop BSD license 46/297946/1 accepted/tizen/unified/20230830.170531
Inki Dae [Tue, 29 Aug 2023 05:25:39 +0000 (14:25 +0900)]
packaging: drop BSD license

[Version] : 0.28.14
[Issue type] : drop license

Drop BSD license because now Mediavision uses OpenCV 4.7.0, and
OpenCV 4.5.0 and higher are licensed under Apache 2 license according to
OpenCV official site[1].

[1] https://opencv.org/license/

Change-Id: Ie44593bb68bfcc1b487e941e5d58b7213558ed14
Signed-off-by: Inki Dae <inki.dae@samsung.com>
9 months agoExclude lcov on pointcloud 69/297869/1 accepted/tizen/unified/20230828.102415
Kwanghoon Son [Mon, 28 Aug 2023 02:17:30 +0000 (11:17 +0900)]
Exclude lcov on pointcloud

[Version] 0.28.13-1

pointcloud only works on aarch64.

Change-Id: I322227f9ece9ad331dc45bfc45e61821f84f8c3d
Signed-off-by: Kwanghoon Son <k.son@samsung.com>
9 months agoFix gcov path 24/297624/1 accepted/tizen/unified/20230822.162247
Kwanghoon Son [Tue, 22 Aug 2023 02:54:09 +0000 (11:54 +0900)]
Fix gcov path

[Version] 0.28.13

Append path build end of capi-media-vision-{version}
Finally correct full path is capi-media-vision-{version}/build

error message :
```
[DEBUG]:[2023-08-22,03:42:16]:[host_cmd.py:cmd:77]:geninfo: Warning: ('gcov') skipping .gcda file
/home/ttf/jenkins_workspace/workspace/CEC_TIZEN_RPI4_GCOV_COVERAGE/CEC/res/coverage_gcov/gcov_data/capi-media-vision-0.28.12/build/mv_roi_tracker/roi_tracker/CMakeFiles/mv_roi_tracker.dir/src/ROITracker.cpp.gcda
because corresponding .gcno file is missing
```

Fixed: 00a391760b680b37647a0ebce188a9f55db38243
Change-Id: I7ab8b11f83cd41b42a6beb8c7e5b0e06e9e18512
Signed-off-by: Kwanghoon Son <k.son@samsung.com>
9 months agoAppend gcov path with package version 72/297072/1 accepted/tizen/unified/20230814.121004
Kwanghoon Son [Thu, 10 Aug 2023 02:06:27 +0000 (11:06 +0900)]
Append gcov path with package version

[Version] 0.28.12

Append path capi-media-vision-{version} end of usr\share\gcov\obj\capi-media-vision

Fixed: 7530347598dfc1629ecf260aa01bac4d5393b472
Change-Id: I181aba162c54f589c31febb41459e66d39e50433
Signed-off-by: Kwanghoon Son <k.son@samsung.com>
9 months agoMerge branch 'tizen_devel' into tizen accepted/tizen/unified/20230809.071949
Kwanghoon Son [Mon, 7 Aug 2023 04:25:59 +0000 (13:25 +0900)]
Merge branch 'tizen_devel' into tizen

[Version] 0.28.11

Inki Dae (6):
  mv_machine_learning: drop meta approach dependency from preprocess
  mv_machine_learning: pack header files for external plugin
  mv_machine_learning: rename Preprocess header file
  common: clean up to mv_private.h file
  mv_machine_learning: pass a task type to external module
  mv_machine_learning: drop create member from itask

Kwanghoon Son (1):
  Change gcov install path

Change-Id: I94359713f6b4a3b5bea24702ae61a1ddd50b0498
Signed-off-by: Kwanghoon Son <k.son@samsung.com>
9 months agoChange gcov install path
Kwanghoon Son [Thu, 27 Jul 2023 09:18:54 +0000 (18:18 +0900)]
Change gcov install path

Directory path 'build'(basename) should removed.

Change-Id: I9d1be63cc0c8b7a8e36700f5e015cfc4484e8868
Signed-off-by: Kwanghoon Son <k.son@samsung.com>
9 months agomv_machine_learning: drop create member from itask
Inki Dae [Thu, 27 Jul 2023 06:58:47 +0000 (15:58 +0900)]
mv_machine_learning: drop create member from itask

[Issue type] : code cleanup

Drop create interface from itask interface class.

The create interface isn't common but specific to certain task group.
Therefore, drop the interface from itask interface class, and change
the create member function of each adapter class to private member if needed.

Change-Id: I65264ede3d5b627c04b7fd6d8cb7555677b0d665
Signed-off-by: Inki Dae <inki.dae@samsung.com>
9 months agomv_machine_learning: pass a task type to external module
Inki Dae [Thu, 27 Jul 2023 05:44:07 +0000 (14:44 +0900)]
mv_machine_learning: pass a task type to external module

[Issue type] : code cleanup

Pass a given task type from Mediavision framework to external module
so that the external module can perform various models according to
user-given behavior.

Change-Id: I86f3254081c4bf47d3659c97ee20a4a0c5408843
Signed-off-by: Inki Dae <inki.dae@samsung.com>
9 months agocommon: clean up to mv_private.h file
Inki Dae [Tue, 25 Jul 2023 02:00:58 +0000 (11:00 +0900)]
common: clean up to mv_private.h file

[Issue type] : code cleanup

Clean up mv_private.h by dropping unnecessary inclusions and by moving
mv_private.h from include/ to mv_common/include/.
There is no reason for the header file should exist in include directory
because it is used by internal framework code only.

Change-Id: I4160942cd4966eb4104393e4193d5dfb30ed3691
Signed-off-by: Inki Dae <inki.dae@samsung.com>
9 months agomv_machine_learning: rename Preprocess header file
Inki Dae [Mon, 24 Jul 2023 00:38:53 +0000 (09:38 +0900)]
mv_machine_learning: rename Preprocess header file

[Issue type] : code cleanup

Rename Preprocess.h to machine_learning_preprocess.h.
The header file name, Preprocess, is a generic one so let's change
the file name with specific one.

Change-Id: Ic497702eb27c00172159801757c16d1faf6896a4
Signed-off-by: Inki Dae <inki.dae@samsung.com>