platform/core/api/singleo.git
11 months agoadd RunQueueManager task_manager_wip
Inki Dae [Fri, 14 Jun 2024 03:11:28 +0000 (12:11 +0900)]
add RunQueueManager

Change-Id: I7c448aa5ae579b2f61ec2e5b765789f2e82966ad
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agoauto_zoom: add face recognition support 72/312672/5
Inki Dae [Wed, 12 Jun 2024 07:24:15 +0000 (16:24 +0900)]
auto_zoom: add face recognition support

Add face recognition task node support to Autozoom service.

With this, this patch adds face recognition task for Mediavision
and then adds the face recognition task node in graph pipeline
of Autozoom service.

And also it supports runtime behavior change of the graph pipeline.
I.e., graph pipeline of Autozoom service is as follows,
    input --- face detection --- bridge --- face recognition --- endpoint
                    |                                         |
                    -------------------------------------------

In this case, we have to consider for two cases,
   - if face detection node failed then bridge node should stop to invoke
     face recognition node.
   - if face detection node worked then bridge node should invoke the
     face recognition node.

This means that bridge node has to decide whether next sub graph pipeline
should be invoked or not in runtime. This patch makes this behavior to be
possible.

Ps. face recognition needs pre-trained model so install the model file
    after training the model using face recognition training API.
    Otherwise, face recognition result will always be 'none'.

Change-Id: I0adfd553285b2e22f551e38fc95a406a0fd9acd5
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agoinference: add support for image classification task 44/312544/3
Vibhav Aggarwal [Tue, 11 Jun 2024 09:56:06 +0000 (18:56 +0900)]
inference: add support for image classification task

[Issue type] new feature

Change-Id: I30209de4a5fd2b8f45ef15a79caa6b30ea6ddc1a
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
12 months agotask_manager: spawn threads in BFS manner 47/312347/7
Vibhav Aggarwal [Fri, 7 Jun 2024 03:43:12 +0000 (12:43 +0900)]
task_manager: spawn threads in BFS manner

[Issue type] code optimization

After this change, threads for each node will be spawned
only when any of its dependency nodes has finished processing.
This change greatly reduces the number of sleeping threads in
a large graph.

Change-Id: I4ce9c8bfcf3d6395f3a5cfabc5c160242faea26a
Signed-off-by: Vibhav Aggarwal <v.aggarwal@samsung.com>
12 months agotask_manager: wait for dependency in threadCb 16/312216/6
Inki Dae [Wed, 5 Jun 2024 06:44:17 +0000 (15:44 +0900)]
task_manager: wait for dependency in threadCb

Wait for dependency in threadCb of each node.
By doing this, it will show more performance than before
because each node is performed in each thread context at the same time.

Change-Id: Ibbd1aafafa110957a0b75a39db78baf6f703b761
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agotask_manager: update graph verification 74/312074/8
Inki Dae [Tue, 4 Jun 2024 04:13:07 +0000 (13:13 +0900)]
task_manager: update graph verification

Update graph verification by adding _nexts member to check the type of
next node.

Change-Id: Id7fbac1a89d7123c9c008a781c0c3a49a29997e5
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agotask_manager: check duplicated node 72/312072/2
Inki Dae [Tue, 4 Jun 2024 01:51:24 +0000 (10:51 +0900)]
task_manager: check duplicated node

Check duplicated node by introduing isNodeDuplicated function.

Change-Id: Id7eadd1daaa8d8c87ba91df5003c2cd004f09c60
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agointroduce SharedBuffer class 30/312030/4
Inki Dae [Fri, 31 May 2024 08:56:42 +0000 (17:56 +0900)]
introduce SharedBuffer class

Introduce SharedBuffer class which can be commonly used by singleo
sub modules. The purpose of SharedBuffer is to share buffer objects
- such as ImageDataType and RawDataType - between sub modules to avoid
from memory copy.

In this patch, SharedBuffer is used to share the input data objects
between nodes. The shared buffer objects will be released if no nodes
who use the objects.

I.e., with the test case, GraphD
                   -----> face_landmark_detection --------
... ---> bridge --|                                       |----> ...
                   -----> face_detection -----------------

Bridge node creates a SharedBuffer object with a new ImageDataType buffer,
and shares the SharedBuffer object with two nodes - one for
face_landmark_detection and other for face_detection without memory copy.
And then the ImageDataType buffer will be released by release() function
of ImageDataType structure after the face_landmark_detection and
face_detection nodes are completed.

In addition, this patch fixes a critial behavior issue that it does deep-copy
even type-casting, which leads unexpected memory leak.
Therefore, this patch moves deep-copy code into clone() of ImageDataType
structure because keeping the deep-copy code in ctor of ImageDataType
structure has two behaviors - one is deep-copy and other is just type-casting.
In general, type-casting never intend deep-copy so it's a wrong behavior.

Change-Id: I29e9ffb2ab8c2bdc9f4fbb67ec5b1591596bb291
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agodrop SingleoInputManager 07/311907/2
Inki Dae [Thu, 30 May 2024 06:16:52 +0000 (15:16 +0900)]
drop SingleoInputManager

Drop SingleoInputManager and use internal queue of AutoZoom instead.

As for SingleoInputManager, I will introduce a reference counting based
generic buffer manager later, which can be used commonly by other sub modules.
As for now, the use of SingleoInputManager in AutoZoom is over-engineering.

Change-Id: I1c281e5dcbd97d5714b19f74906a0222b9c0b002
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agotask_manager: drop unnecessary type casting 93/311893/2
Inki Dae [Thu, 30 May 2024 04:50:30 +0000 (13:50 +0900)]
task_manager: drop unnecessary type casting

Change-Id: I4024fa59419fbd1f7bd234b03fe70e9a1bc4007d
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agoauto_zoom: code cleanup with clone() interface 88/311888/3
Inki Dae [Thu, 30 May 2024 02:36:56 +0000 (11:36 +0900)]
auto_zoom: code cleanup with clone() interface

Just code cleanup by adding clone() interface to BaseDataType.
With this interface, we can drop unnecessary casting.

Change-Id: Ia0056e744ff4330e33b6a2e53e3ee4e000d5109e
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agoauto_zoom: drop unnecessary code 84/311884/1
Inki Dae [Thu, 30 May 2024 00:51:40 +0000 (09:51 +0900)]
auto_zoom: drop unnecessary code

Just cleanup by dropping unnecessary code.

Change-Id: Ia28d32d57119ce796235d15ae63b84bcec96791f
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agotask_manager: delegate collecting the results to each node 01/311801/6
Inki Dae [Tue, 28 May 2024 06:32:24 +0000 (15:32 +0900)]
task_manager: delegate collecting the results to each node

Delegate collecting the results to each node. With this, we can use
only generic interface.

Change-Id: I5afd506bbc763fdcd86c18ac3980d295fd894c67
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agoclean up with copy-constructor 36/311736/5
Inki Dae [Mon, 27 May 2024 02:39:43 +0000 (11:39 +0900)]
clean up with copy-constructor

Clean up with copy-constructor. With this patch,
Manual copy to ImageDataType can be dropped.

Change-Id: I1669ac0fe1706561b903bd59933a9f217f560fce
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agotask_manager: drop getters from nodes 42/311642/2
Inki Dae [Thu, 23 May 2024 01:49:17 +0000 (10:49 +0900)]
task_manager: drop getters from nodes

Drop getters from nodes by delegating what the getters do to
specific nodes.

Change-Id: Id191f62a2153820a136c500fd29271eecd4f7603
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agotask_manager: verify if _nodes is valid graph or not 19/311519/5
Inki Dae [Wed, 22 May 2024 06:05:48 +0000 (15:05 +0900)]
task_manager: verify if _nodes is valid graph or not

Verify if _nodes is valid graph or not by checking
- DAG(Direct Acyclic Graph)
- Start node should be a task node such as InferenceNode and TrainingNode.
- Last node should be a EndpointNode.
- BridgeNode should be located bewteen two task nodes such as InferenceNode
  and TrainingNode.
- BridgeNode should have a valid callback.

As for DAG verification, this patch uses BFS(Breadth-first search) algorithm.

Change-Id: I33edf100fa81a335ab094b6cfff512c1d2cd749e
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agotest: clean up test_task_manager.cpp 50/311450/4
Inki Dae [Tue, 21 May 2024 08:56:20 +0000 (17:56 +0900)]
test: clean up test_task_manager.cpp

Clean up test_task_manager.cpp by dropping printing out messages
and verifying actual results instead.

Change-Id: I931ae19d85af584c276b3848f60616110a19ce1d
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agotask_manager: introduce new node types and dropping code smell 87/311387/8
Inki Dae [Mon, 20 May 2024 06:05:00 +0000 (15:05 +0900)]
task_manager: introduce new node types and dropping code smell

Introduce new node types and dropping implicit use of CallbackNode.
Until now, CallbackNode had been used for two purposes implicitly
- one is for bridge and other is endpoint.

So this patch separates CallbackNode into two nodes - BridgeNode and
EndpointNode - so that CallbackNode can be used explicitly, and also
separates InferenceNode into two nodes - InferenceNode and TrainingNode
- by making these two classes to be derived from TaskNode.

As summary,
   - InferenceNode and TrainingNode are derived from TaskNode which is able
     to perform inference or training through machine learning framework.
   - BridgeNode and EndpointNode are derived from CallbackNode which is able
     to call an callback registered by user.

With this patch, graph rule of task manager is as follows,
    1. Start node should be task node such as InferenceNode and TrainingNode.
    2. Last node should be EndpointNode.
    3. BridgeNode should be placed bewteen two task nodes such as InferenceNode
       and TrainingNode.
    4. BridgeNode should have a valid callback but it's optional for other types
       of callback nodes.
    5. Multiple endpoint nodes can exist.

Change-Id: I4d2917d15cededc3fd11a93afd2db4b6fe5cbe5c
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agotask_manager: cleanup to NodeCb 86/311286/1
Inki Dae [Fri, 17 May 2024 07:54:51 +0000 (16:54 +0900)]
task_manager: cleanup to NodeCb

CLeanup to NodeCb by modifying its arguments - dropping all parameters and
add new one instead, shared_ptr<INode> &node.

user_data isn't required anymore because task manager already knows whether
current node is CallbackNode or Not.

Change-Id: I1df5d89149379ed16ed1029536a08036178c0b0b
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agotask_manager: drop arguments from callback for CallbackNode 67/311267/2
Inki Dae [Fri, 17 May 2024 04:00:12 +0000 (13:00 +0900)]
task_manager: drop arguments from callback for CallbackNode

Drop unnecessary arguments from callback for CallbackNode.
Now CallbackNode can act like normal node such as InferenceNode
so drop the unnecessary arguments and get them(input and result)
from CallbackNode itself.

This patch is a step for separating CallbackNode into two nodes
- BridgeNode and EndpointNode - as I shared already, which will drop
the code smell that CallbackNode has two kinds of behaviors.

Change-Id: I25bbc87a1f1d9a82e5c7340b39b8d789599db933
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agotask_manager: get results when needed 66/311266/2
Inki Dae [Fri, 17 May 2024 00:02:56 +0000 (09:02 +0900)]
task_manager: get results when needed

Change-Id: Ifad7a833c8451435552a5ffb5f240ac6879ee02d
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agotask_manager: use nodes consistently 65/311265/1
Inki Dae [Thu, 16 May 2024 23:50:36 +0000 (08:50 +0900)]
task_manager: use nodes consistently

Use nodes - InferenceNode and CallbackNode - consistently by
adding output from CallbackNode to InferenceNode as input.
With this, we can make both kinds of nodes to have same working path.

Change-Id: Ifd1e57e5ab9c96293858e9b2320ab7714e3eece5
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agotask_manager: check if cb and user_data are valid or not 64/311264/1
Inki Dae [Thu, 16 May 2024 10:45:00 +0000 (19:45 +0900)]
task_manager: check if cb and user_data are valid or not

Check if cb and user_data to callback node are valid or not. They are
mandatorily required to call a callback of the callback node.

Change-Id: I1050c6ff7a5133ea61e61cc23cd6534ed240a357
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agorename InferenceServiceInterface to InferenceTaskInterface 95/311195/4
Inki Dae [Thu, 16 May 2024 03:09:13 +0000 (12:09 +0900)]
rename InferenceServiceInterface to InferenceTaskInterface

Rename InferenceServiceInterface to InferenceTaskInterface including
relevant code as we discussed before. The use of *service* terms made us
confusing specific service such as Autozoom.

Change-Id: Ibfcb51f333a7bbdb87c577edd5bcdd8cc6cceb0e
Signed-off-by: Inki Dae <inki.dae@samsung.com>
12 months agoservices: do not use _threads as class member 94/311194/2
Inki Dae [Thu, 16 May 2024 02:29:47 +0000 (11:29 +0900)]
services: do not use _threads as class member

Do not use _threads as class member. It's enogh with local threads handles
so drop _threads from class member and use local one instead.

Change-Id: Id99b0b5d51e5f02ff09aae1647be7e2645491e37
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agotask_manager: check if _nodes is empty or not 38/311038/1
Inki Dae [Mon, 13 May 2024 07:43:13 +0000 (16:43 +0900)]
task_manager: check if _nodes is empty or not

Check if _nodes is empty or not in output(). To get the output result,
_nodes must not be empty. In addition, use _nodes.back() instead of
nodes[nodes.size() - 1] for cleanup.

Change-Id: I22a085578a2fcba36d97685434fd6188ece60b66
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agouse observer pattern instead of callback approach 59/310959/7
Inki Dae [Fri, 10 May 2024 06:11:53 +0000 (15:11 +0900)]
use observer pattern instead of callback approach

Use observer pattern instead of callback approach to get input feeded
data in runtime.

Until now, a specific service such as AutoZoom should have registered its own
callback function to Input service to receive captured input data
from a given input service such as camera. However, this approach made
it some limit because input relevant class called a callback of specific
service's class so some wrapper functions was needed to access its own members.

For cleanup, this patch introduces observer pattern by making each specific
service class to inherit IInputObserver class. with this patch, each specific
service class can register itself as input observer to input service like below,

// override inputFeedCb interface and implement it.
class AutoZoom::public IService, public IInputObserver
{
...
void inputFeedCb(BaseDataType &data) override;
...
}

// register itself as input observer to input service.
void AutoZoom::configure(...)
{
....
    _input_service->registerObserver(this);
....
}

In addition, this patch separates input configuration from ctor
and drops redundant code.

Change-Id: I83b3cadb031b68212fcfdd783054397d09a6b36a
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agoservices: add callback node support as last node 28/310928/2
Inki Dae [Thu, 9 May 2024 10:51:11 +0000 (19:51 +0900)]
services: add callback node support as last node

Add callback node support as last node.

With this patch, we can place a callback node to last node
so that we can get the final results from the node.

For the test, this patch adds below test case - GraphC,
GraphC:
======
                             ---> callback ---> face_landmark_detection ---
 input --> face_detection --|                                              |--> callback --> output
                             ----------------------------------------------
Change-Id: Ia444b0d47480485cd81204bc0568badc8215ebfe
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agoservices: drop non-generic interfaces from INode 62/310862/1
Inki Dae [Thu, 9 May 2024 05:18:04 +0000 (14:18 +0900)]
services: drop non-generic interfaces from INode

Drop non-generic interface from INode class by moving
specific interfaces to each concrete classes, InferenceNode
and CallbackNode.

Change-Id: I58848c38584fb893e7914c01b7196fdbb56e2acf
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agotest: add task manager test case 00/310800/4
Inki Dae [Fri, 3 May 2024 02:26:07 +0000 (11:26 +0900)]
test: add task manager test case

Change-Id: I5a56b83ff46da3e4484d8148ac979127d77e41ed
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agoadd task manger support for Autozoom 87/310587/12
Inki Dae [Thu, 2 May 2024 07:28:13 +0000 (16:28 +0900)]
add task manger support for Autozoom

Change-Id: I1b9d70d5fae439122c95c84b78728282ca10a307
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agoadd task manager support 86/310586/12
Inki Dae [Tue, 30 Apr 2024 00:55:59 +0000 (09:55 +0900)]
add task manager support

Add task manager support which provides multi thread based service
pipeline configuration feature in runtime.

Each service of singleo service framework can be implemented using
one or ones of various inference services such as face detection,
face landmark detection, object detection, and so on.

As for this, task manager consists of below features,

TaskManager : Specific service such as Autozoom or service groups - which can
              be introduced later - has one TaskManager handle.
ITaskNode : Interface class which can contain inference service,
            callback service, and training service(TODO)

Basically, TaskManager provides the interface that we can configure
specific service using graph concept.

I.e., Below graph shows a service which uses two kinds of task interfaces,
  one is for inference and other is for callback.
         ---inference_node_a ----
input ---|                      |----- callback_node ----- inference_node_c
         ---inference_node_b ----

With above node graph, inference node a and b can be performed in parallel,
and both outputs of them goes to callback node. After that, the callback node
creates a new data converted by original input data and results came from
inference_node_a and inference_node_b so that new data can be allowed by
inference node c as input source, and then passes it to the inference node c
as input source.

Finally, inference node c performs the the service request with the given input
and then returns the service result to user.

Ps. task service node will be performed with its own thread context.

Below is a example which configures above pipeline graph,
task_manager = make_unique<TaskManager>();
task_manager->input(src);

auto face_detection = factory->createFaceDetection();
auto inference_node_a  = make_unique<InferenceTaskNode>();
inference_node_a->addInferenceService(face_detection);
task_manager->addNode(inference_node_a);

auto image_classification = factory->createImageClassification();
auto inference_node_b  = make_unique<InferenceTaskNode>();
inference_node_b->addInferenceService(image_classification);
task_manager->addNode(inference_node_b);

auto callback_node = make_unique<CallbackTaskNode>();
callback_node->setCb(specific_callback, callback_node.get());
callback_node->addDependency(inference_node_a);
callback_node->addDependency(inference_node_b);
task_manager->addNode(callback_node);

auto face_landmark_detection = factory->createFaceLandmarkDetection();
auto inference_node_c = make_unique<InferenceTaskNode>();
infernce_node_c->addInferenceService(face_landmark_detection);
inference_node_c->addDependency(callback_node);
task_manger->addNode(inference_node_c);

task_manager->run();

result = task_manager->output();

Change-Id: I02cc508ee26aad8312629d0c69300f21f8221ba3
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agoinference: add face landmark detection support for Mediavision 10/310210/4 inferenceservice_redesign_#01
Inki Dae [Wed, 24 Apr 2024 06:03:10 +0000 (15:03 +0900)]
inference: add face landmark detection support for Mediavision

Add face landmark detection support for Mediavision backend.

With this patch, move '_rects' member to each specific result structure,
and add _points to FldResultType for face landmark detection.

Change-Id: If0e801c6c01da247870e9ac3632d5953d90633c8
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agoinference: drop InferenceTaskInterface 58/310158/1
Inki Dae [Tue, 23 Apr 2024 08:01:41 +0000 (17:01 +0900)]
inference: drop InferenceTaskInterface

Drop IInferenceTaskInterface and its concreate classes which aren't needed
anymore. We can create targeted service through dedicated factory as
a IInferenceServiceInterface type.

Change-Id: I04e32648ff9e17cf85bc5d8d3032357e9f46379c
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agoinference: apply abstract factory pattern 53/310153/1
Inki Dae [Tue, 23 Apr 2024 06:50:16 +0000 (15:50 +0900)]
inference: apply abstract factory pattern

Apply abstract factory pattern. This patch uses abstract factory pattern
to create targeted inference service such as face detection, object detection
and so on from backend specific factory.

Regarding this, self-registeraion factory pattern is used to register
each inference service backend factory such as mediavision or mediapipe in
build time, and abstract factory pattern is used to create targeted inference
service from the inference service backend factory like below.

// create a given inference backend factory.
_factory = InferenceServiceFactory::instance().create("MvInferenceServiceFactory");

Ps. MvInferenceServiceFactory is inference service backend factory for Mediavision.
    So other backend factory such as Mediapipe will be added later.

// create a target inference task service from the given inference backend factory
_infernce_service = _factory->createFaceDetection();
or
_infernce_service = _factory->createObjectDetection();
...

Change-Id: I809bc733bd7e33a6779b6410422e3961129d7e0a
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agoinference: apply self-registeration factory pattern 26/310126/3
Inki Dae [Tue, 23 Apr 2024 01:03:33 +0000 (10:03 +0900)]
inference: apply self-registeration factory pattern

Apply self-registeration factory pattern to inference service.

With this patch, InferenceServiceDefault and InferenceServiceExternal classes
will be registered to inference factory table in build time, and
it will create InferenceServiceDefault or InferenceServiceExternal class
with a given class name in runtime. Regarding this, this patch
modifies existing build script so that inference service directory
can be built-in.

This patch is just a previous step for applying abstract factory pattern
as we discussed.

Change-Id: Icb21dc066bdf940926a9e764e40a48a82ffefed5
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agoinput: code refactoring to CameraBackendFactory class 70/310070/4
Inki Dae [Mon, 22 Apr 2024 05:08:22 +0000 (14:08 +0900)]
input: code refactoring to CameraBackendFactory class

Do code refactoring to CameraBackendFactory class by changing
the use of macro - REGISTER_CAMERA_BACKDEND() - to the use of
template.

Change-Id: I3dc3db50dfd471abf5a3ca42489cde0bff0480cb
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agoinput/camera_api: consider camera malfunctioning case 26/309926/1
Inki Dae [Thu, 18 Apr 2024 07:19:24 +0000 (16:19 +0900)]
input/camera_api: consider camera malfunctioning case

Consider camera device malfunctioning case by using timeout and
re-trying checking current camera state.

Change-Id: Icb757700b6674672447a0a9240b3a753887adf9e
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agoinput/camera_api: fix typo 10/309910/1
Inki Dae [Thu, 18 Apr 2024 06:37:16 +0000 (15:37 +0900)]
input/camera_api: fix typo

Change-Id: I763a6675dae318be01df80ab296eca2bc17ac15c
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agoinput/camera_api: drop unnecessary thread 09/309909/1
Inki Dae [Thu, 18 Apr 2024 06:32:42 +0000 (15:32 +0900)]
input/camera_api: drop unnecessary thread

Drop unnecessary thread from CameraApiBackend. previewCb callback
will be called by different thread created by Camera API internally
so we don't need to create a new thread for preview.

Change-Id: Iba8cbba1ae59253e123787797ea21ca0e05ce9d8
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agoinput/camera_api: add sync API support 92/309892/3
Inki Dae [Thu, 18 Apr 2024 01:21:38 +0000 (10:21 +0900)]
input/camera_api: add sync API support

Add sync API support by implementing capture method of CameraApiBackend class.

According to Camera-api document below,
https://docs.tizen.org/application/native/api/mobile/4.0/group__CAPI__MEDIA__CAMERA__MODULE.html
For capture, this patch follows the Camera API state diagram. However,
one tricky way is needed to make sure to wait for the "Captured" state
because completed callback since camera_start_capture call will be called
by main thread so sync API of singleo framework cannot use the callback.
Therefore, to make sure to wait for the state, we use a tricky way by
checking if current state is "Captured".

Change-Id: Ibe2ba23e0c1c9bb215976949a7d1a05cd1f98db6
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agopackaging: cleanup flags 32/309832/1
Inki Dae [Wed, 17 Apr 2024 05:52:47 +0000 (14:52 +0900)]
packaging: cleanup flags

Change-Id: Ic9a548d66097d3503962ce88e770ddc1f085004b
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agoinput/camera_api: activate user-desired camera device 95/309795/2
Inki Dae [Wed, 17 Apr 2024 02:41:24 +0000 (11:41 +0900)]
input/camera_api: activate user-desired camera device

Activate user-desired camera device by adding updateAvailableCameraDevices
and setActivateCameraDevice functions. With this patch, Camera API backend
collects valid camera device list and activates user-desired camera device
with config.device_id which is passed by user.

Change-Id: I8d44a2da1513957ee350497d328fd3c342284817
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agoinput: use InputServiceCallbackType type 94/309794/1
Inki Dae [Wed, 17 Apr 2024 02:08:21 +0000 (11:08 +0900)]
input: use InputServiceCallbackType type

Use predefined callback type - InputServiceCallbackType - for code cleanup.

Change-Id: I72d9ac29ea3081e53c95f5c4c4f38af7fa6d3721
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agoinput: add camera-api backend support 60/309760/4
Inki Dae [Tue, 16 Apr 2024 05:31:56 +0000 (14:31 +0900)]
input: add camera-api backend support

Add camera-api backend and change default input camera backend
from OpenCV to CAMERA-API.

Change-Id: Icc8aab35cbf491923045a2f7f4205ff2124af3a7
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agocode refactoring to Input service 71/309671/4
Inki Dae [Mon, 15 Apr 2024 07:26:51 +0000 (16:26 +0900)]
code refactoring to Input service

Code refactoring to Input service framework by doing,
 - introduce CameraBackendFactory class which creates Camara backend
   class corresponding to a given class name in runtime.
 - Integrate Input framework  with Service one. There is no reason to build
   separate library.
 - Drop redundant classes, CameraServiceDefault and CameraServiceExternal
   classes. Withoug Camera service classes, we can support vairous
   Input camera backends with CameraBackendFactory class.

Change-Id: I1f7090371efd2458834b14d1cbc05ce6326371af
Signed-off-by: Inki Dae <inki.dae@samsung.com>
13 months agoinput: use InputServiceCallbackType type for cleanup 42/309642/1
Inki Dae [Mon, 15 Apr 2024 02:18:39 +0000 (11:18 +0900)]
input: use InputServiceCallbackType type for cleanup

Use InputServiceCallbackType type instead of specific function call
definition.

Change-Id: Ida1786011baa3d6fdb918b1128d41190f19e5a4f
Signed-off-by: Inki Dae <inki.dae@samsung.com>
14 months agoMerge pull request #6 from inki-dae/main
대인기/Tizen Platform Lab(SR)/삼성전자 [Fri, 12 Apr 2024 05:22:34 +0000 (14:22 +0900)]
Merge pull request #6 from inki-dae/main

Update native capi

14 months agoupdate singleo_native_capi.h header file
Inki Dae [Fri, 12 Apr 2024 02:20:25 +0000 (11:20 +0900)]
update singleo_native_capi.h header file

Update singleo_native_capi.h header file by
- correct wrong descriptions
- update singleo_service_add_input_image_data and
  singleo_service_add_input_raw_data API by adding a option string which can
  describe speicific format type to user-given input data.

Signed-off-by: Inki Dae <inki.dae@samsung.com>
14 months agotest: clean up by dropping redundant code
Inki Dae [Fri, 12 Apr 2024 00:13:01 +0000 (09:13 +0900)]
test: clean up by dropping redundant code

Signed-off-by: Inki Dae <inki.dae@samsung.com>
14 months agoclean up README file (#5)
대인기/Tizen Platform Lab(SR)/삼성전자 [Thu, 11 Apr 2024 02:25:18 +0000 (11:25 +0900)]
clean up README file (#5)

* clean up README file

Signed-off-by: Inki Dae <inki.dae@samsung.com>
* Update README.md

Co-authored-by: 비브합 아가르왈/Tizen Platform Lab(SR)/삼성전자 <v.aggarwal@samsung.com>
---------

Signed-off-by: Inki Dae <inki.dae@samsung.com>
Co-authored-by: 비브합 아가르왈/Tizen Platform Lab(SR)/삼성전자 <v.aggarwal@samsung.com>
14 months agoMerge pull request #4 from inki-dae/main
대인기/Tizen Platform Lab(SR)/삼성전자 [Mon, 8 Apr 2024 23:43:41 +0000 (08:43 +0900)]
Merge pull request #4 from inki-dae/main

Implement autozoom service algorithm

14 months agoupdate README.md file
Inki Dae [Mon, 8 Apr 2024 06:08:40 +0000 (15:08 +0900)]
update README.md file

Signed-off-by: Inki Dae <inki.dae@samsung.com>
14 months agocleanup BaseResultType
Inki Dae [Mon, 8 Apr 2024 00:51:23 +0000 (09:51 +0900)]
cleanup BaseResultType

Clean up BaseResultType by moving common members from specific structure
to BaseResultType.

Signed-off-by: Inki Dae <inki.dae@samsung.com>
14 months agodrop SingleoOutputData class
Inki Dae [Fri, 5 Apr 2024 09:06:23 +0000 (18:06 +0900)]
drop SingleoOutputData class

Drop SingleoOuputData and its child classes, and use BaseResultType
and its child structures instead.

Signed-off-by: Inki Dae <inki.dae@samsung.com>
14 months agoimplement autozoom algorithm
Inki Dae [Fri, 5 Apr 2024 03:32:57 +0000 (12:32 +0900)]
implement autozoom algorithm

Implement autozoom algorithm. As for this, this patch introduces
a new interface class, IPostprocessor, and its concrete class,
Postprocessor for Autozoom service, which postprocesses a given
camera preview image by emulating pysical camera zoom-in/out
functions with zero latency.

Signed-off-by: Inki Dae <inki.dae@samsung.com>
14 months agorename API
Inki Dae [Tue, 26 Mar 2024 05:51:09 +0000 (14:51 +0900)]
rename API

Rename singleo_service_add_image_data to singleo_service_add_rgb_data.

Signed-off-by: Inki Dae <inki.dae@samsung.com>
14 months agoMerge pull request #3 from inki-dae/main
대인기/Tizen Platform Lab(SR)/삼성전자 [Tue, 26 Mar 2024 04:43:30 +0000 (13:43 +0900)]
Merge pull request #3 from inki-dae/main

visualizer support, chance service configuration schema and API refactoring

14 months agouse uppercase letters in CMakeLists.txt
Inki Dae [Tue, 26 Mar 2024 04:41:01 +0000 (13:41 +0900)]
use uppercase letters in CMakeLists.txt

Just cleanup.

Signed-off-by: Inki Dae <inki.dae@samsung.com>
14 months agodisplay camera preview image in application
Inki Dae [Tue, 26 Mar 2024 02:53:44 +0000 (11:53 +0900)]
display camera preview image in application

Display camera preview image mixed with bounding box for debug in application
instead of singleo service framework-internally.

Regarding this, this patch introduces Context structure so that
the test application can manage camera preview image and bounding box.

Signed-off-by: Inki Dae <inki.dae@samsung.com>
14 months agoadd singleo_service_add_input API series support
Inki Dae [Mon, 25 Mar 2024 07:13:16 +0000 (16:13 +0900)]
add singleo_service_add_input API series support

Add singleo_service_add_input_{image_file,rgb_data,raw_data} API support.

Singleo service API has to allow one more input data so add this API series
to add various types of data as input data. In addition, this patch
drops ISingleoCommon, SingleoImageData and SingleoRawData classes and,
introduces a new input manager, SingleoInputManager class which manages
input data queue.

Signed-off-by: Inki Dae <inki.dae@samsung.com>
14 months agochange service configuration schema
Inki Dae [Thu, 21 Mar 2024 07:01:03 +0000 (16:01 +0900)]
change service configuration schema

Change service configuration schema like below.

Before
"..., input=camera, camera_backend=opencv, ..."

After
"..., input_feed=camera, camera_id=n, ..."

Signed-off-by: Inki Dae <inki.dae@samsung.com>
14 months agoadd visualizer support for debug and test
Inki Dae [Fri, 15 Mar 2024 08:25:33 +0000 (17:25 +0900)]
add visualizer support for debug and test

Add visualizer support to display service result on the screen
through Native window system.

The visualizer will be used only for debug and test.

With this patch, test_autozoom_on_screen binary will be created,
And this binary will display the service result on the screen.

Signed-off-by: Inki Dae <inki.dae@samsung.com>
14 months agoMerge pull request #2 from inki-dae/main
대인기/Tizen Platform Lab(SR)/삼성전자 [Wed, 20 Mar 2024 02:16:23 +0000 (11:16 +0900)]
Merge pull request #2 from inki-dae/main

Bug fix,  specific exception class and face detection task support

14 months agofix seg. fault with no detection
Inki Dae [Fri, 15 Mar 2024 08:09:00 +0000 (17:09 +0900)]
fix seg. fault with no detection

Fix a seg. fault issue when detection failed by updating the result
only in case that object is detected.

Signed-off-by: Inki Dae <inki.dae@samsung.com>
14 months agochange ISingleoOutputData to SingleoOutputData
Inki Dae [Fri, 15 Mar 2024 01:06:31 +0000 (10:06 +0900)]
change ISingleoOutputData to SingleoOutputData

ISingleoOutputData class isn't interface class anymore.
So drop a prefix, 'I' from the class name.

Regarding this class, not clear to me and I'm pretty sure that
there is more better design for managing output data type and
to be able to generalize the various output data types.

Signed-off-by: Inki Dae <inki.dae@samsung.com>
14 months agoadd face detection support for Mediavision backend
Inki Dae [Thu, 14 Mar 2024 07:57:22 +0000 (16:57 +0900)]
add face detection support for Mediavision backend

Add face detection support for Mediavision backend.

With this patch, default task type for Autozoom service is changed
to face detection from object detection.

Signed-off-by: Inki Dae <inki.dae@samsung.com>
14 months agoadd specific exception support and error code
Inki Dae [Thu, 14 Mar 2024 02:33:22 +0000 (11:33 +0900)]
add specific exception support and error code

Add specific exception class support and error code for SingleO framework.

Signed-off-by: Inki Dae <inki.dae@samsung.com>
15 months agofix a memory leak to input feeding data
Inki Dae [Thu, 14 Mar 2024 01:30:01 +0000 (10:30 +0900)]
fix a memory leak to input feeding data

Fix a memory leak issue when pusing input feeding data to
incoming queue is skipped.

If the input feeding data was skipped then we have to release
the input feeding data which is a new buffer allocated and copied from
original data.

Signed-off-by: Inki Dae <inki.dae@samsung.com>
15 months agoMerge pull request #1 from inki-dae/main
대인기/Tizen Platform Lab(SR)/삼성전자 [Tue, 12 Mar 2024 04:25:39 +0000 (13:25 +0900)]
Merge pull request #1 from inki-dae/main

Add backbone version of SingleO AI service framework

15 months agoadd async manager for async API support
Inki Dae [Mon, 11 Mar 2024 08:05:35 +0000 (17:05 +0900)]
add async manager for async API support

Add async manager class for async API supprort.

Async manager is used to support async API support for each service API.

Behavior summary to async manager:
There are two queues which are performed in each thread context.
One is incoming queue which manages input data feeded from input device
such as camera, screen capture module, and other external modules
through input service framework. Simply saying, input service framework
will push input data feeded from input device.

And other is outgoing queue which manages output data after the completion
of invoke such as inference. Regarding outgoing queue, a thread context
is created by async manager, and this thead context pops input data
from incoming queue, request an invoke with the input data and then
push the result to outgoing queue.

As such, one thread puts input data obtained from an input device into
an incoming queue, and the other thread retrieves input data from
the incoming queue, performs invoke, and puts the result of the execution
into an outgoing queue in parallel to utilize HW resource in maximum.

Signed-off-by: Inki Dae <inki.dae@samsung.com>
15 months agoAdd backbone version of SingleO AI service framework
Inki Dae [Mon, 5 Feb 2024 08:04:56 +0000 (17:04 +0900)]
Add backbone version of SingleO AI service framework

Introduce backbone version of SingleO AI service framework as initial code
including test case for Auto Zoom API as example code.

SingleO AI service framework consists of like below,
- common directory
    . abstracted common data classes which is used to share data object
      between service framework and various backend services.
- services/common/singleo_native_capi.cpp
    . native CAPI implemention, which calls a user-desired service API.
- services/auto_zoom directory
    . AI service API implementation targeted to Camera auto zoom service,
  which uses service class for the use of various backend independent
  inference and training engines.
- services/common directory
    . context class
    . A unique object corresponding to user-desired service.
    . abstracted AI service
    . actual implementation to user-desired AI service using various
  generic services such as inference, training and remote computing
  services.
    . abstracted preprocess classes
    . preprocessing class for convert user-given input or internally-feeded
  input data to common data object which can be used for service
  perform framework internally.
    . abstracted input data types
    . Common data type for contarining various types of data given by user
  or internally-feeded such as camera, screen capture, image file
  and so on.
- log directory
    . message logging class which provides standard output in default and dlog
  for Tizen as a Tizen backend.
- input directory
    . abstracted input service classes which provide input feeded data through
  Camera and screen capture module in platform independent manner.
  Platform dependent camera or screen capture modules can be used as plugin
  module. In default, OpenCV based camera backend is used, which is a built
  in backend.
- inference directory
    . abstracted inference service classes which provides platform independent
  inference service.
    . inference/backends directory
    . Inference engine backend for the use of Mediavision framework.
  In default, mediavision backend is used but possible to use external
  framework through a plugin module.

Signed-off-by: Inki Dae <inki.dae@samsung.com>
16 months agoInitial commit
대인기/Tizen Platform Lab(SR)/삼성전자 [Mon, 5 Feb 2024 05:28:28 +0000 (14:28 +0900)]
Initial commit