2 .. _dev_with_OCV_on_Android:
4 Android Development with OpenCV
5 *******************************
7 This tutorial has been created to help you use OpenCV library within your Android project.
9 This guide was written with Windows 7 in mind, though it should work with any other OS supported by
12 This tutorial assumes you have the following installed and configured:
20 * ADT and CDT plugins for Eclipse
24 If you need help with anything of the above, you may refer to our :ref:`android_dev_intro` guide.
26 This tutorial also assumes you have OpenCV4Android SDK already installed on your development
27 machine and OpenCV Manager on your testing device correspondingly. If you need help with any of
28 these, you may consult our :ref:`O4A_SDK` tutorial.
30 If you encounter any error after thoroughly following these steps, feel free to contact us via
31 `OpenCV4Android <https://groups.google.com/group/android-opencv/>`_ discussion group or OpenCV
32 `Q&A forum <http://answers.opencv.org>`_ . We'll do our best to help you out.
35 Using OpenCV Library Within Your Android Project
36 ================================================
38 In this section we will explain how to make some existing project to use OpenCV.
39 Starting with 2.4.2 release for Android, *OpenCV Manager* is used to provide apps with the best
40 available version of OpenCV.
41 You can get more information here: :ref:`Android_OpenCV_Manager` and in these
42 `slides <https://docs.google.com/a/itseez.com/presentation/d/1EO_1kijgBg_BsjNp2ymk-aarg-0K279_1VZRcPplSuk/present#slide=id.p>`_.
48 Application Development with Async Initialization
49 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
51 Using async initialization is a **recommended** way for application development. It uses the OpenCV
52 Manager to access OpenCV libraries externally installed in the target system.
54 #. Add OpenCV library project to your workspace. Use menu
55 :guilabel:`File -> Import -> Existing project in your workspace`.
57 Press :guilabel:`Browse` button and locate OpenCV4Android SDK
58 (:file:`OpenCV-2.4.8-android-sdk/sdk`).
60 .. image:: images/eclipse_opencv_dependency0.png
61 :alt: Add dependency from OpenCV library
64 #. In application project add a reference to the OpenCV Java SDK in
65 :guilabel:`Project -> Properties -> Android -> Library -> Add` select ``OpenCV Library - 2.4.8``.
67 .. image:: images/eclipse_opencv_dependency1.png
68 :alt: Add dependency from OpenCV library
71 In most cases OpenCV Manager may be installed automatically from Google Play. For the case, when
72 Google Play is not available, i.e. emulator, developer board, etc, you can install it manually
73 using adb tool. See :ref:`manager_selection` for details.
75 There is a very base code snippet implementing the async initialization. It shows basic principles.
76 See the "15-puzzle" OpenCV sample for details.
81 public class Sample1Java extends Activity implements CvCameraViewListener {
83 private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
85 public void onManagerConnected(int status) {
87 case LoaderCallbackInterface.SUCCESS:
89 Log.i(TAG, "OpenCV loaded successfully");
90 mOpenCvCameraView.enableView();
94 super.onManagerConnected(status);
101 public void onResume()
104 OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_6, this, mLoaderCallback);
110 It this case application works with OpenCV Manager in asynchronous fashion. ``OnManagerConnected``
111 callback will be called in UI thread, when initialization finishes. Please note, that it is not
112 allowed to use OpenCV calls or load OpenCV-dependent native libs before invoking this callback.
113 Load your own native libraries that depend on OpenCV after the successful OpenCV initialization.
114 Default ``BaseLoaderCallback`` implementation treat application context as Activity and calls
115 ``Activity.finish()`` method to exit in case of initialization failure. To override this behavior
116 you need to override ``finish()`` method of ``BaseLoaderCallback`` class and implement your own
120 Application Development with Static Initialization
121 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
123 According to this approach all OpenCV binaries are included into your application package. It is
124 designed mostly for development purposes. This approach is deprecated for the production code,
125 release package is recommended to communicate with OpenCV Manager via the async initialization
128 #. Add the OpenCV library project to your workspace the same way as for the async initialization
129 above. Use menu :guilabel:`File -> Import -> Existing project in your workspace`,
130 press :guilabel:`Browse` button and select OpenCV SDK path
131 (:file:`OpenCV-2.4.8-android-sdk/sdk`).
133 .. image:: images/eclipse_opencv_dependency0.png
134 :alt: Add dependency from OpenCV library
137 #. In the application project add a reference to the OpenCV4Android SDK in
138 :guilabel:`Project -> Properties -> Android -> Library -> Add` select ``OpenCV Library - 2.4.8``;
140 .. image:: images/eclipse_opencv_dependency1.png
141 :alt: Add dependency from OpenCV library
144 #. If your application project **doesn't have a JNI part**, just copy the corresponding OpenCV
145 native libs from :file:`<OpenCV-2.4.8-android-sdk>/sdk/native/libs/<target_arch>` to your
146 project directory to folder :file:`libs/<target_arch>`.
148 In case of the application project **with a JNI part**, instead of manual libraries copying you
149 need to modify your ``Android.mk`` file:
150 add the following two code lines after the ``"include $(CLEAR_VARS)"`` and before
151 ``"include path_to_OpenCV-2.4.8-android-sdk/sdk/native/jni/OpenCV.mk"``
156 OPENCV_CAMERA_MODULES:=on
157 OPENCV_INSTALL_MODULES:=on
159 The result should look like the following:
164 include $(CLEAR_VARS)
167 OPENCV_CAMERA_MODULES:=on
168 OPENCV_INSTALL_MODULES:=on
169 include ../../sdk/native/jni/OpenCV.mk
171 After that the OpenCV libraries will be copied to your application :file:`libs` folder during
174 Eclipse will automatically include all the libraries from the :file:`libs` folder to the
175 application package (APK).
177 #. The last step of enabling OpenCV in your application is Java initialization code before calling
178 OpenCV API. It can be done, for example, in the static section of the ``Activity`` class:
184 if (!OpenCVLoader.initDebug()) {
185 // Handle initialization error
189 If you application includes other OpenCV-dependent native libraries you should load them
190 **after** OpenCV initialization:
196 if (!OpenCVLoader.initDebug()) {
197 // Handle initialization error
199 System.loadLibrary("my_jni_lib1");
200 System.loadLibrary("my_jni_lib2");
208 To build your own Android application, using OpenCV as native part, the following steps should be
211 #. You can use an environment variable to specify the location of OpenCV package or just hardcode
212 absolute or relative path in the :file:`jni/Android.mk` of your projects.
214 #. The file :file:`jni/Android.mk` should be written for the current application using the common
217 For detailed information see the Android NDK documentation from the Android NDK archive, in the
218 file :file:`<path_where_NDK_is_placed>/docs/ANDROID-MK.html`.
220 #. The following line:
224 include C:\Work\OpenCV4Android\OpenCV-2.4.8-android-sdk\sdk\native\jni\OpenCV.mk
226 Should be inserted into the :file:`jni/Android.mk` file **after** this line:
230 include $(CLEAR_VARS)
232 #. Several variables can be used to customize OpenCV stuff, but you **don't need** to use them when
233 your application uses the `async initialization` via the `OpenCV Manager` API.
235 .. note:: These variables should be set **before** the ``"include .../OpenCV.mk"`` line:
239 OPENCV_INSTALL_MODULES:=on
241 Copies necessary OpenCV dynamic libs to the project ``libs`` folder in order to include them
246 OPENCV_CAMERA_MODULES:=off
248 Skip native OpenCV camera related libs copying to the project ``libs`` folder.
252 OPENCV_LIB_TYPE:=STATIC
254 Perform static linking with OpenCV. By default dynamic link is used and the project JNI lib
255 depends on ``libopencv_java.so``.
257 #. The file :file:`Application.mk` should exist and should contain lines:
261 APP_STL := gnustl_static
262 APP_CPPFLAGS := -frtti -fexceptions
264 Also, the line like this one:
268 APP_ABI := armeabi-v7a
270 Should specify the application target platforms.
272 In some cases a linkage error (like ``"In function 'cv::toUtf16(std::basic_string<...>...
273 undefined reference to 'mbstowcs'"``) happens when building an application JNI library,
274 depending on OpenCV. The following line in the :file:`Application.mk` usually fixes it:
278 APP_PLATFORM := android-9
281 #. Either use :ref:`manual <NDK_build_cli>` ``ndk-build`` invocation or
282 :ref:`setup Eclipse CDT Builder <CDT_Builder>` to build native JNI lib before (re)building the Java
283 part and creating an APK.
289 Here are basic steps to guide you trough the process of creating a simple OpenCV-centric
290 application. It will be capable of accessing camera output, processing it and displaying the
293 #. Open Eclipse IDE, create a new clean workspace, create a new Android project
294 :menuselection:`File --> New --> Android Project`
296 #. Set name, target, package and ``minSDKVersion`` accordingly. The minimal SDK version for build
297 with OpenCV4Android SDK is 11. Minimal device API Level (for application manifest) is 8.
299 #. Allow Eclipse to create default activity. Lets name the activity ``HelloOpenCvActivity``.
301 #. Choose Blank Activity with full screen layout. Lets name the layout ``HelloOpenCvLayout``.
303 #. Import OpenCV library project to your workspace.
305 #. Reference OpenCV library within your project properties.
307 .. image:: images/dev_OCV_reference.png
308 :alt: Reference OpenCV library.
311 #. Edit your layout file as xml file and pass the following layout there:
316 <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
317 xmlns:tools="http://schemas.android.com/tools"
318 xmlns:opencv="http://schemas.android.com/apk/res-auto"
319 android:layout_width="match_parent"
320 android:layout_height="match_parent" >
322 <org.opencv.android.JavaCameraView
323 android:layout_width="fill_parent"
324 android:layout_height="fill_parent"
325 android:visibility="gone"
326 android:id="@+id/HelloOpenCvView"
327 opencv:show_fps="true"
328 opencv:camera_id="any" />
332 #. Add the following permissions to the :file:`AndroidManifest.xml` file:
339 <uses-permission android:name="android.permission.CAMERA"/>
341 <uses-feature android:name="android.hardware.camera" android:required="false"/>
342 <uses-feature android:name="android.hardware.camera.autofocus" android:required="false"/>
343 <uses-feature android:name="android.hardware.camera.front" android:required="false"/>
344 <uses-feature android:name="android.hardware.camera.front.autofocus" android:required="false"/>
346 #. Set application theme in AndroidManifest.xml to hide title and system buttons.
352 android:icon="@drawable/icon"
353 android:label="@string/app_name"
354 android:theme="@android:style/Theme.NoTitleBar.Fullscreen" >
356 #. Add OpenCV library initialization to your activity. Fix errors by adding requited imports.
361 private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
363 public void onManagerConnected(int status) {
365 case LoaderCallbackInterface.SUCCESS:
367 Log.i(TAG, "OpenCV loaded successfully");
368 mOpenCvCameraView.enableView();
372 super.onManagerConnected(status);
379 public void onResume()
382 OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_6, this, mLoaderCallback);
385 #. Defines that your activity implements ``CvViewFrameListener2`` interface and fix activity related
386 errors by defining missed methods. For this activity define ``onCreate``, ``onDestroy`` and
387 ``onPause`` and implement them according code snippet bellow. Fix errors by adding requited
393 private CameraBridgeViewBase mOpenCvCameraView;
396 public void onCreate(Bundle savedInstanceState) {
397 Log.i(TAG, "called onCreate");
398 super.onCreate(savedInstanceState);
399 getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
400 setContentView(R.layout.HelloOpenCvLayout);
401 mOpenCvCameraView = (CameraBridgeViewBase) findViewById(R.id.HelloOpenCvView);
402 mOpenCvCameraView.setVisibility(SurfaceView.VISIBLE);
403 mOpenCvCameraView.setCvCameraViewListener(this);
407 public void onPause()
410 if (mOpenCvCameraView != null)
411 mOpenCvCameraView.disableView();
414 public void onDestroy() {
416 if (mOpenCvCameraView != null)
417 mOpenCvCameraView.disableView();
420 public void onCameraViewStarted(int width, int height) {
423 public void onCameraViewStopped() {
426 public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
427 return inputFrame.rgba();
430 #. Run your application on device or emulator.
432 Lets discuss some most important steps. Every Android application with UI must implement Activity
433 and View. By the first steps we create blank activity and default view layout. The simplest
434 OpenCV-centric application must implement OpenCV initialization, create its own view to show
435 preview from camera and implements ``CvViewFrameListener2`` interface to get frames from camera and
438 First of all we create our application view using xml layout. Our layout consists of the only
439 one full screen component of class ``org.opencv.android.JavaCameraView``. This class is
440 implemented inside OpenCV library. It is inherited from ``CameraBridgeViewBase``, that extends
441 ``SurfaceView`` and uses standard Android camera API. Alternatively you can use
442 ``org.opencv.android.NativeCameraView`` class, that implements the same interface, but uses
443 ``VideoCapture`` class as camera access back-end. ``opencv:show_fps="true"`` and
444 ``opencv:camera_id="any"`` options enable FPS message and allow to use any camera on device.
445 Application tries to use back camera first.
447 After creating layout we need to implement ``Activity`` class. OpenCV initialization process has
448 been already discussed above. In this sample we use asynchronous initialization. Implementation of
449 ``CvCameraViewListener`` interface allows you to add processing steps after frame grabbing from
450 camera and before its rendering on screen. The most important function is ``onCameraFrame``. It is
451 callback function and it is called on retrieving frame from camera. The callback input is object
452 of ``CvCameraViewFrame`` class that represents frame from camera.
455 Do not save or use ``CvCameraViewFrame`` object out of ``onCameraFrame`` callback. This object
456 does not have its own state and its behavior out of callback is unpredictable!
458 It has ``rgba()`` and ``gray()`` methods that allows to get frame as RGBA and one channel gray scale
459 ``Mat`` respectively. It expects that ``onCameraFrame`` function returns RGBA frame that will be