3 /** @mainpage Documentation
7 @section S0_introduction Introduction
9 The ARM Computer Vision and Machine Learning library is a set of functions optimised for both ARM CPUs and GPUs using SIMD technologies.
11 Several builds of the library are available using various configurations:
12 - OS: Linux, Android or bare metal.
13 - Architecture: armv7a (32bit) or arm64-v8a (64bit)
14 - Technology: NEON / OpenCL / NEON and OpenCL
15 - Debug / Asserts / Release: Use a build with asserts enabled to debug your application and enable extra validation. Once you are sure your application works as expected you can switch to a release build of the library for maximum performance.
17 @subsection S0_1_contact Contact / Support
19 Please email developer@arm.com
21 In order to facilitate the work of the support team please provide the build information of the library you are using. To get the version of the library you are using simply run:
23 $ strings android-armv7a-cl-asserts/libarm_compute.so | grep arm_compute_version
24 arm_compute_version=v16.12 Build options: {'embed_kernels': '1', 'opencl': '1', 'arch': 'armv7a', 'neon': '0', 'asserts': '1', 'debug': '0', 'os': 'android', 'Werror': '1'} Git hash=f51a545d4ea12a9059fe4e598a092f1fd06dc858
26 @section S1_file_organisation File organisation
28 This archive contains:
29 - The arm_compute header and source files
30 - The latest Khronos OpenCL 1.2 C headers from the <a href="https://www.khronos.org/registry/cl/">Khronos OpenCL registry</a>
31 - The latest Khronos cl2.hpp from the <a href="https://www.khronos.org/registry/cl/">Khronos OpenCL registry</a> (API version 2.1 when this document was written)
32 - The sources for a stub version of libOpenCL.so to help you build your application.
33 - An examples folder containing a few examples to compile and link against the library.
34 - A @ref test_helpers folder containing headers with some boiler plate code used by the examples.
37 You should have the following file organisation:
40 ├── arm_compute --> All the arm_compute headers
43 │ │ │ ├── CLKernels.h --> Includes all the OpenCL kernels at once
44 │ │ │ ├── CL specialisation of all the generic objects interfaces (ICLTensor, ICLImage, etc.)
45 │ │ │ ├── kernels --> Folder containing all the OpenCL kernels
46 │ │ │ │ └── CL*Kernel.h
47 │ │ │ └── OpenCL.h --> Wrapper to configure the Khronos OpenCL C++ header
49 │ │ │ └── kernels --> Folder containing all the CPP kernels
50 │ │ │ │ └── CPP*Kernel.h
52 │ │ │ ├── kernels --> Folder containing all the NEON kernels
53 │ │ │ │ └── NE*Kernel.h
54 │ │ │ └── NEKernels.h --> Includes all the NEON kernels at once
55 │ │ ├── All common basic types (Types.h, Window, Coordinates, Iterator, etc.)
56 │ │ ├── All generic objects interfaces (ITensor, IImage, etc.)
57 │ │ └── Objects metadata classes (ImageInfo, TensorInfo, MultiImageInfo)
60 │ │ ├── CL objects & allocators (CLArray, CLImage, CLTensor, etc.)
61 │ │ ├── functions --> Folder containing all the OpenCL functions
63 │ │ └── CLFunctions.h --> Includes all the OpenCL functions at once
65 │ │ └── CPPScheduler.h --> Basic pool of threads to execute CPP/NEON code on several cores in parallel
67 │ │ ├── functions --> Folder containing all the NEON functions
69 │ │ └── NEFunctions.h --> Includes all the NEON functions at once
70 │ └── Basic implementations of the generic object interfaces (Array, Image, Tensor, etc.)
74 ├── documentation.xhtml -> documentation/index.xhtml
76 │ ├── cl_convolution.cpp
77 │ ├── neoncl_scale_median_gaussian.cpp
78 │ ├── neon_convolution.cpp
82 │ └── Khronos OpenCL C headers and C++ wrapper
87 │ │ └── ... (Same structure as headers)
89 │ │ └── cl_kernels --> All the OpenCL kernels
91 │ └── ... (Same structure as headers)
92 └── test_helpers --> Boiler plate code used by examples
95 @section S2_versions_changelog Versions changelog
97 @note There will be one major public release with new features per quarter. All releases in between will only contain bug fixes.
99 v16.12 (Binary release)
103 - New OpenCL kernels / functions:
104 - @ref CLActivationLayerKernel / @ref CLActivationLayer
105 - @ref CLChannelCombineKernel / @ref CLChannelCombine
106 - @ref CLDerivativeKernel / @ref CLChannelExtract
107 - @ref CLFastCornersKernel / @ref CLFastCorners
108 - @ref CLMeanStdDevKernel / @ref CLMeanStdDev
109 - New NEON kernels / functions:
110 - HOG / SVM: @ref NEHOGOrientationBinningKernel, @ref NEHOGBlockNormalizationKernel, @ref NEHOGDetectorKernel, @ref NEHOGNonMaximaSuppressionKernel / @ref NEHOGDescriptor, @ref NEHOGDetector, @ref NEHOGGradient, @ref NEHOGMultiDetection
111 - @ref NENonLinearFilterKernel / @ref NENonLinearFilter
112 - Introduced a CLScheduler to manage the default context and command queue used by the runtime library and create synchronisation events.
113 - Switched all the kernels / functions to use tensors instead of images.
114 - Updated documentation to include instructions to build the library from sources.
117 - New OpenCL kernels / functions:
118 - @ref CLLogits1DMaxKernel, @ref CLLogits1DShiftExpSumKernel, @ref CLLogits1DNormKernel / @ref CLSoftmaxLayer
119 - @ref CLPoolingLayerKernel / @ref CLPoolingLayer
120 - @ref CLIm2ColKernel @ref CLCol2ImKernel @ref CLConvolutionLayerWeightsReshapeKernel / @ref CLConvolutionLayer
121 - @ref CLRemapKernel / @ref CLRemap
122 - @ref CLGaussianPyramidHorKernel, @ref CLGaussianPyramidVertKernel / @ref CLGaussianPyramid, @ref CLGaussianPyramidHalf, @ref CLGaussianPyramidOrb
123 - @ref CLMinMaxKernel, @ref CLMinMaxLocationKernel / @ref CLMinMaxLocation
124 - @ref CLNonLinearFilterKernel / @ref CLNonLinearFilter
125 - New NEON FP16 kernels (Requires armv8.2 CPU)
126 - @ref NEAccumulateWeightedFP16Kernel
127 - @ref NEBox3x3FP16Kernel
128 - @ref NENonMaximaSuppression3x3FP16Kernel
131 - New OpenCL kernels / functions:
132 - @ref CLGradientKernel, @ref CLEdgeNonMaxSuppressionKernel, @ref CLEdgeTraceKernel / @ref CLCannyEdge
133 - GEMM refactoring + FP16 support: @ref CLGEMMInterleave4x4Kernel, @ref CLGEMMTranspose1xWKernel, @ref CLGEMMMatrixMultiplyKernel, @ref CLGEMMMatrixAdditionKernel / @ref CLGEMM
134 - @ref CLGEMMMatrixAccumulateBiasesKernel / @ref CLFullyConnectedLayer
135 - @ref CLTransposeKernel / @ref CLTranspose
136 - @ref CLLKTrackerInitKernel, @ref CLLKTrackerStage0Kernel, @ref CLLKTrackerStage1Kernel, @ref CLLKTrackerFinalizeKernel / @ref CLOpticalFlow
137 - @ref CLNormalizationLayerKernel / @ref CLNormalizationLayer
138 - @ref CLLaplacianPyramid, @ref CLLaplacianReconstruct
139 - New NEON kernels / functions:
140 - @ref NEActivationLayerKernel / @ref NEActivationLayer
141 - GEMM refactoring + FP16 support (Requires armv8.2 CPU): @ref NEGEMMInterleave4x4Kernel, @ref NEGEMMTranspose1xWKernel, @ref NEGEMMMatrixMultiplyKernel, @ref NEGEMMMatrixAdditionKernel / @ref NEGEMM
142 - @ref NEPoolingLayerKernel / @ref NEPoolingLayer
145 - Renamed the library to arm_compute
146 - New CPP target introduced for C++ kernels shared between NEON and CL functions.
147 - New padding calculation interface introduced and ported most kernels / functions to use it.
148 - New OpenCL kernels / functions:
149 - @ref CLGEMMLowpMatrixMultiplyKernel / @ref CLGEMMLowp
150 - New NEON kernels / functions:
151 - @ref NENormalizationLayerKernel / @ref NENormalizationLayer
152 - @ref NETransposeKernel / @ref NETranspose
153 - @ref NELogits1DMaxKernel, @ref NELogits1DShiftExpSumKernel, @ref NELogits1DNormKernel / @ref NESoftmaxLayer
154 - @ref NEIm2ColKernel @ref NECol2ImKernel @ref NEConvolutionLayerWeightsReshapeKernel / @ref NEConvolutionLayer
155 - @ref NEGEMMMatrixAccumulateBiasesKernel / @ref NEFullyConnectedLayer
156 - @ref NEGEMMLowpMatrixMultiplyKernel / @ref NEGEMMLowp
159 @section S3_how_to_build How to build the library and the examples
161 @subsection S3_1_build_options Build options
163 scons 2.3 or above is required to build the library.
164 To see the build options available simply run ```scons -h```:
166 debug: Debug (default=0) (0|1)
170 asserts: Enable asserts (This flag is forced to 1 for debug=1) (default=0) (0|1)
174 arch: Target Architecture (default=armv7a) (armv7a|arm64-v8a|arm64-v8.2-a|x86)
178 os: Target OS (default=linux) (linux|android|bare_metal)
182 build: Build type: (default=cross_compile) (native|cross_compile)
183 default: cross_compile
184 actual: cross_compile
186 Werror: Enable/disable the -Werror compilation flag (Default=1) (0|1)
190 opencl: Enable OpenCL support(Default=1) (0|1)
194 neon: Enable Neon support(Default=0) (0|1)
198 embed_kernels: Embed OpenCL kernels in library binary(Default=0) (0|1)
203 - With debug=1 asserts are enabled, and the library is built with symbols and no optimisations enabled.
204 - With debug=0 and asserts=1: Optimisations are enabled and symbols are removed, however all the asserts are still present (This is about 20% slower than the release build)
205 - With debug=0 and asserts=0: All optimisations are enable and no validation is performed, if the application misuses the library it is likely to result in a crash. (Only use this mode once you are sure your application is working as expected).
207 Architecture: The x86 target can only be used with neon=0 and opencl=1.
209 OS: Choose the operating system you are targeting: Linux, Android or bare metal.
210 @note bare metal can only be used for NEON (not OpenCL), only static libraries get built and NEON's multi-threading support is disabled.
212 Build type: you can either build directly on your device (native) or cross compile from your desktop machine (cross-compile). In both cases make sure the compiler is available in your path.
214 Werror: If you are compiling using the same toolchains as the ones used in this guide then there shouldn't be any warning and therefore you should be able to keep Werror=1. If with a different compiler version the library fails to build because of warnings interpreted as errors then, if you are sure the warnings are not important, you might want to try to build with Werror=0 (But please do report the issue either on Github or by an email to developer@arm.com so that the issue can be addressed).
216 OpenCL / NEON: Choose which SIMD technology you are interested targeting. (NEON for ARM Cortex-A CPUs or OpenCL for ARM Mali GPUs)
218 embed_kernels: For OpenCL only: set embed_kernels=1 if you want the OpenCL kernels to be built in the library's binaries instead of being read from separate ".cl" files. If embed_kernels is set to 0 then the application can set the path to the folder containing the OpenCL kernel files by calling CLKernelLibrary::init(). By default the path is set to "./cl_kernels".
220 @subsection S3_2_linux Linux
222 @subsubsection S3_2_1_library How to build the library ?
224 For Linux, the library was successfully built and tested using the following Linaro GCC toolchain: gcc-linaro-arm-linux-gnueabihf-4.8-2014.02_linux and gcc-linaro-6.1.1-2016.08-x86_64_arm-linux-gnueabihf
226 @note If you are building with opencl=1 then scons will expect to find libOpenCL.so either in the current directory or in "build" (See the section below if you need a stub OpenCL library to link against)
228 To cross-compile the library in debug mode, with NEON only support, for Linux 32bit:
230 scons Werror=1 -j8 debug=1 neon=1 opencl=0 os=linux arch=armv7a
232 To cross-compile the library in asserts mode, with OpenCL only support, for Linux 64bit:
234 scons Werror=1 -j8 debug=0 asserts=1 neon=0 opencl=1 embed_kernels=1 os=linux arch=arm64-v8a
236 @subsubsection S3_2_2_examples How to manually build the examples ?
238 The examples get automatically built by scons as part of the build process of the library described above. This section just describes how you can build and link your own application against our library.
240 @note The following command lines assume the arm_compute binaries are present in the current directory or in the system library path.
242 To cross compile a NEON example:
244 arm-linux-gnueabihf-g++ examples/neon_convolution.cpp test_helpers/Utils.cpp -I. -Iinclude -std=c++11 -mfpu=neon -L. -larm_compute -o neon_convolution
246 To cross compile an OpenCL example:
248 arm-linux-gnueabihf-g++ examples/cl_convolution.cpp test_helpers/Utils.cpp -I. -Iinclude -std=c++11 -mfpu=neon -L. -larm_compute -lOpenCL -o cl_convolution
250 To compile natively (i.e directly on an ARM device) for NEON:
252 g++ examples/neon_convolution.cpp test_helpers/Utils.cpp -I. -Iinclude -std=c++11 -mfpu=neon -larm_compute -o neon_convolution
254 To compile natively (i.e directly on an ARM device) for OpenCL:
256 g++ examples/cl_convolution.cpp test_helpers/Utils.cpp -I. -Iinclude -std=c++11 -larm_compute -lOpenCL -o cl_convolution
258 @note These two commands assume libarm_compute.so is available in your library path, if not add the path to it using -L
260 To run the built executable simply run:
262 LD_LIBRARY_PATH=build ./neon_convolution
266 LD_LIBRARY_PATH=build ./cl_convolution
268 @note If you built the library with support for both OpenCL and NEON you will need to link against OpenCL even if your application only uses NEON.
270 @subsection S3_3_android Android
272 For Android, the library was successfully built and tested using Google's standalone toolchains:
273 - arm-linux-androideabi-4.9 for armv7a (clang++)
274 - aarch64-linux-android-4.9 for arm64-v8a (g++)
276 Here is a guide to <a href="https://developer.android.com/ndk/guides/standalone_toolchain.html">create your Android standalone toolchains from the NDK</a>
278 - Download the NDK r14 beta 2 from here: https://developer.android.com/ndk/downloads/index.html
279 - Make sure you have Python 2 installed on your machine.
280 - Generate the 32 and/or 64 toolchains by running the following commands:
283 $NDK/build/tools/make_standalone_toolchain.py --arch arm64 --install-dir $MY_TOOLCHAINS/aarch64-linux-android-4.9 --stl gnustl
284 $NDK/build/tools/make_standalone_toolchain.py --arch arm --install-dir $MY_TOOLCHAINS/arm-linux-androideabi-4.9 --stl gnustl
286 @attention Due to some NDK issues make sure you use g++ & gnustl for aarch64 and clang++ & gnustl for armv7
288 @note Make sure to add the toolchains to your PATH: export PATH=$PATH:$MY_TOOLCHAINS/aarch64-linux-android-4.9/bin:$MY_TOOLCHAINS/arm-linux-androideabi-4.9/bin
290 @subsubsection S3_3_1_library How to build the library ?
292 @note If you are building with opencl=1 then scons will expect to find libOpenCL.so either in the current directory or in "build" (See the section below if you need a stub OpenCL library to link against)
294 To cross-compile the library in debug mode, with NEON only support, for Android 32bit:
296 CXX=clang++ CC=clang scons Werror=1 -j8 debug=1 neon=1 opencl=0 os=android arch=armv7a
298 @attention Due to some NDK issues make sure you use g++ & gnustl for aarch64 and clang++ & libc++ for armv7
300 To cross-compile the library in asserts mode, with OpenCL only support, for Android 64bit:
302 scons Werror=1 -j8 debug=0 asserts=1 neon=0 opencl=1 embed_kernels=1 os=android arch=arm64-v8a
304 @subsubsection S3_3_2_examples How to manually build the examples ?
306 The examples get automatically built by scons as part of the build process of the library described above. This section just describes how you can build and link your own application against our library.
308 @note The following command lines assume the arm_compute binaries are present in the current directory or in the system library path.
310 Once you've got your Android standalone toolchain built and added to your path you can do the following:
312 To cross compile a NEON example:
315 arm-linux-androideabi-clang++ examples/neon_convolution.cpp -I. -Iinclude -std=c++11 -larm_compute-static -L. -o neon_convolution_arm -static-libstdc++ -pie
317 aarch64-linux-android-g++ examples/neon_convolution.cpp -I. -Iinclude -std=c++11 -larm_compute-static -L. -o neon_convolution_aarch64 -static-libstdc++ -pie
319 To cross compile an OpenCL example:
322 arm-linux-androideabi-clang++ examples/cl_convolution.cpp -I. -Iinclude -std=c++11 -larm_compute-static -L. -o cl_convolution_arm -static-libstdc++ -pie -lOpenCL
324 aarch64-linux-android-g++ examples/cl_convolution.cpp -I. -Iinclude -std=c++11 -larm_compute-static -L. -o cl_convolution_aarch64 -static-libstdc++ -pie -lOpenCL
326 @note Due to some issues in older versions of the Mali OpenCL DDK (<= r13p0), we recommend to link arm_compute statically on Android.
328 Then you need to do is upload the executable and the shared library to the device using ADB:
330 adb push neon_convolution_arm /data/local/tmp/
331 adb push cl_convolution_arm /data/local/tmp/
332 adb shell chmod 777 -R /data/local/tmp/
334 And finally to run the example:
336 adb shell /data/local/tmp/neon_convolution_arm
337 adb shell /data/local/tmp/cl_convolution_arm
341 adb push neon_convolution_aarch64 /data/local/tmp/
342 adb push cl_convolution_aarch64 /data/local/tmp/
343 adb shell chmod 777 -R /data/local/tmp/
345 And finally to run the example:
347 adb shell /data/local/tmp/neon_convolution_aarch64
348 adb shell /data/local/tmp/cl_convolution_aarch64
350 @subsection S3_4_cl_stub_library The OpenCL stub library
352 In the opencl-1.2-stubs folder you will find the sources to build a stub OpenCL library which then can be used to link your application or arm_compute against.
354 If you preferred you could retrieve the OpenCL library from your device and link against this one but often this library will have dependencies on a range of system libraries forcing you to link your application against those too even though it is not using them.
356 @warning This OpenCL library provided is a stub and *not* a real implementation. You can use it to resolve OpenCL's symbols in arm_compute while building the example but you must make sure the real libOpenCL.so is in your PATH when running the example or it will not work.
358 To cross-compile the stub OpenCL library simply run:
360 <target-prefix>-gcc -o libOpenCL.so -Iinclude opencl-1.2-stubs/opencl_stubs.c -fPIC -shared
364 <target-prefix>-gcc -o libOpenCL.so -Iinclude opencl-1.2-stubs/opencl_stubs.c -fPIC -shared
366 arm-linux-gnueabihf-gcc -o libOpenCL.so -Iinclude opencl-1.2-stubs/opencl_stubs.c -fPIC -shared
368 aarch64-linux-gnu-gcc -o libOpenCL.so -Iinclude -shared opencl-1.2-stubs/opencl_stubs.c -fPIC
370 arm-linux-androideabi-clang -o libOpenCL.so -Iinclude -shared opencl-1.2-stubs/opencl_stubs.c -fPIC -shared
372 aarch64-linux-android-gcc -o libOpenCL.so -Iinclude -shared opencl-1.2-stubs/opencl_stubs.c -fPIC -shared
374 @section S4_architecture Library Architecture
376 @subsection S4_1 Core vs Runtime libraries
378 The Core library is a low level collection of algorithms implementations, it is designed to be embedded in existing projects and applications:
380 - It doesn't allocate any memory (All the memory allocations/mappings have to be handled by the caller).
381 - It doesn't perform any kind of multi-threading (but provide information to the caller about how the workload can be split).
383 The Runtime library is a very basic wrapper around the Core library which can be used for quick prototyping, it is basic in the sense that:
385 - It allocates images and tensors are allocatd using standard malloc().
386 - It multi-threads NEON code in a very basic way using a very simple pool of threads.
387 - For OpenCL it will use the default CLScheduler command queue for all mapping operations and kernels.
389 For maximum performance, it is expected that the users would re-implement an equivalent to the runtime library which suits better their needs (With a more clever multi-threading strategy, load-balancing between NEON and OpenCL, etc.)
391 @subsection S4_2_windows_kernels_mt_functions Windows, kernels, multi-threading and functions
393 @subsubsection S4_2_1_windows Windows
395 A @ref Window represents a workload to execute, it's made of up to @ref Coordinates::num_max_dimensions dimensions.
396 Each dimension is defined by a start, end and step.
398 It can split into subwindows as long as *all* the following rules remain true for all the dimensions:
400 - max[n].start() <= sub[n].start() < max[n].end()
401 - sub[n].start() < sub[n].end() <= max[n].end()
402 - max[n].step() == sub[n].step()
403 - (sub[n].start() - max[n].start()) % max[n].step() == 0
404 - (sub[n].end() - sub[n].start()) % max[n].step() == 0
406 @subsubsection S4_2_2 Kernels
408 Each implementation of the @ref IKernel interface (base class of all the kernels in the core library) works in the same way:
413 // Initialise the CLScheduler with the default context and default command queue
414 // Also initialises the CLKernelLibrary to use ./cl_kernels as location for OpenCL kernels files and sets a default device for which OpenCL programs are built.
415 CLScheduler::get().default_init();
417 cl::CommandQueue q = CLScheduler::get().queue();
418 //Create a kernel object:
420 // Initialize the kernel with the input/output and options you want to use:
421 kernel.configure( input, output, option0, option1);
422 // Retrieve the execution window of the kernel:
423 const Window& max_window = kernel.window();
424 // Run the whole kernel in the current thread:
425 kernel.run( q, max_window ); // Enqueue the kernel to process the full window on the default queue
427 // Wait for the processing to complete:
434 //Create a kernel object:
436 // Initialize the kernel with the input/output and options you want to use:
437 kernel.configure( input, output, option0, option1);
438 // Retrieve the execution window of the kernel:
439 const Window& max_window = kernel.window();
440 // Run the whole kernel in the current thread:
441 kernel.run( max_window ); // Run the kernel on the full window
444 @subsubsection S4_2_3 Multi-threading
446 The previous section shows how to run a NEON / CPP kernel in the current thread, however if your system has several CPU cores, you will probably want the kernel to use several cores. Here is how this can be done:
448 @snippet src/runtime/CPP/CPPScheduler.cpp Scheduler example
450 This is the very basic implementation used in the NEON runtime library by all the NEON functions, @sa CPPScheduler.
452 @note Some kernels like for example @ref NEHistogramKernel need some local temporary buffer to perform their calculations. In order to avoid memory corruption between threads, the local buffer must be of size: ```memory_needed_per_thread * num_threads``` and each subwindow must be initialised by calling @ref Window::set_thread_id() with a unique thread_id between 0 and num_threads.
454 @subsubsection S4_2_4 Functions
456 Functions will automatically allocate the temporary buffers mentioned above, and will automatically multi-thread kernels' executions using the very basic scheduler described in the previous section.
458 Simple functions are made of a single kernel (e.g @ref NEConvolution3x3), while more complex ones will be made of a several kernels pipelined together (e.g @ref NEGaussianPyramid, @ref NEHarrisCorners), check their documentation to find out which kernels are used by each function.
461 //Create a function object:
463 // Initialize the function with the input/output and options you want to use:
464 function.configure( input, output, option0, option1);
465 // Execute the function:
469 @warning ARM Compute libraries require Mali OpenCL DDK r8p0 or above(OpenCL kernels are compiled using the -cl-arm-non-uniform-work-group-size flag)
471 @note All OpenCL functions and objects in the runtime library use the command queue associated with CLScheduler for all operations, a real implementation would be expected to use different queues for mapping operations and kernels in order to reach a better GPU utilisation.
473 @subsubsection S4_4_1_cl_scheduler OpenCL Scheduler and kernel library
475 The ARM Compute runtime uses a single command queue and context for all the operations.
477 The user can get / set this context and command queue through the CLScheduler's interface.
479 @attention Make sure the application is using the same context as the library as in OpenCL it is forbidden to share objects across contexts. This is done by calling @ref CLScheduler::init() or @ref CLScheduler::default_init() at the beginning of your application.
481 All the OpenCL kernels used by the library are built and stored in the @ref CLKernelLibrary.
482 If the library is compiled with embed_kernels=0 the application can set the path to the OpenCL kernels by calling @ref CLKernelLibrary::init(), by default the path is set to "./cl_kernels"
484 @subsubsection S4_4_2_events_sync OpenCL events and synchronisation
486 In order to block until all the jobs in the CLScheduler's command queue are done executing the user can call @ref CLScheduler::sync() or create a sync event using @ref CLScheduler::enqueue_sync_event()
489 @snippet cl_events.cpp OpenCL events
491 @subsubsection S4_4_2_cl_neon OpenCL / NEON interoperability
493 You can mix OpenCL and NEON kernels and or functions, however it is the user's responsibility to handle the mapping unmapping of the OpenCL objects, for example:
495 @snippet neoncl_scale_median_gaussian.cpp NEON / OpenCL Interop
497 @sa main_neoncl_scale_median_gaussian
499 @subsection S4_5_algorithms Algorithms
501 All algorithms in this library have been implemented following the [OpenVX 1.1 specifications](https://www.khronos.org/registry/vx/specs/1.1/html/)
502 Please refer to the Khronos documentation for more information.
504 @subsection S4_6_images_tensors Images, padding, border modes and tensors
506 Most kernels and functions in the library process images, however, in order to be future proof most of the kernels actually accept tensors, see below for more information about they are related.
508 @attention Each memory object can be written by only one kernel, however it can be read by several kernels. Writing to the same object from several kernels will result in undefined behaviour. The kernel writing to an object must be configured before the kernel(s) reading from it.
510 @subsubsection S4_6_1_padding_and_border Padding and border modes
512 Several algorithms rely on neighbour pixels to calculate the value of a given pixel: this means the algorithm will not be able to process the borders of the image unless you give it more information about what you want to happen for border pixels, this is the @ref BorderMode.
514 You have 3 types of @ref BorderMode :
516 - @ref BorderMode::UNDEFINED : if you are missing pixel values then don't calculate the value. As a result all the pixels which are on the border will have a value which is undefined.
517 - @ref BorderMode::REPLICATE : if you are missing pixel values then assume the missing pixels have the same value as the closest valid pixel.
518 - @ref BorderMode::CONSTANT : if you are missing pixel values then assume the missing pixels all have the same constant value (The user can choose what this value should be).
520 Moreover both OpenCL and NEON use vector loads and stores instructions to access the data in buffers, so in order to avoid having special cases to handle for the borders all the images and tensors used in this library must be padded.
522 @paragraph padding Padding
524 There are different ways padding can be calculated:
528 @snippet neon_convolution.cpp Accurate padding
530 @note It's important to call allocate @b after the function is configured: if the image / tensor is already allocated then the function will shrink its execution window instead of increasing the padding. (See below for more details).
532 - Manual padding / no padding / auto padding: You can allocate your images / tensors up front (before configuring your functions), in that case the function will use whatever padding is available and will shrink its execution window if there isn't enough padding available (Which will translates into a smaller valid region for the output @sa valid_region).
533 If you don't want to manually set the padding but still want to allocate your objects upfront then you can use auto_padding.
538 // Use auto padding for the input:
539 src.info()->init_auto_padding(TensorShape(640u,480u), Format::U8);
541 // Use manual padding for the destination image
542 dst.info()->init(src.info()->tensor_shape(), Format::U8, strides_in_bytes, offset_first_element_in_bytes, total_size_in_bytes);
544 // Allocate all the images
545 src.allocator()->allocate();
546 dst.allocator()->allocate();
547 // Fill the input image with the content of the PPM image if a filename was provided:
552 // Apply a Gaussian 3x3 filter to the source image (Note: if the padding provided is not enough then the execution window and valid region of the output will be shrunk)
553 gauss.configure(&src, &dst, BorderMode::UNDEFINED);
555 //Execute the functions:
559 @warning Some kernels need up to 3 neighbour values to calculate the value of a given pixel, therefore to be safe we use a 4 pixels padding all around the image and some kernels read and write up to 32 pixels at the time, therefore we add an extra 32 pixels of padding at the end of each row to be safe. As a result auto padded buffers waste a lot of memory and are less cache friendly. It is therefore recommended to use accurate padding or manual padding wherever possible.
561 @paragraph valid_region Valid regions
563 Some kernels (like edge detectors for example) need to read values of neighbouring pixels to calculate the value of a given pixel, it is therefore not possible to calculate the values of the pixels on the edges.
565 Another case is: if a kernel processes 8 pixels per iteration then if the image's dimensions is not a multiple of 8 and not enough padding is available then the kernel will not be able to process the pixels near the right edge as a result these pixels will be left undefined.
567 In order to know which pixels have been calculated, each kernel sets a valid region for each output image or tensor @sa TensorInfo::valid_region(), @ref ValidRegion
569 @attention Valid regions and accurate padding have only been introduced in the library recently therefore not all the kernels and functions have been ported to use them yet. All the non ported kernels will set the @ref ValidRegion equal to the @ref TensorShape.
571 List of kernels which haven't been ported yet:
573 - @ref CLColorConvertKernel
574 - @ref CLEdgeNonMaxSuppressionKernel
575 - @ref CLEdgeTraceKernel
576 - @ref CLGaussianPyramidHorKernel
577 - @ref CLGaussianPyramidVertKernel
578 - @ref CLGradientKernel
579 - @ref NEChannelCombineKernel
580 - @ref NEColorConvertKernel
581 - @ref NEFillArrayKernel
582 - @ref NEGaussianPyramidHorKernel
583 - @ref NEGaussianPyramidVertKernel
584 - @ref NEHarrisScoreFP16Kernel
585 - @ref NEHarrisScoreKernel
586 - @ref NEHistogramKernel
587 - @ref NEHistogramBorderKernel
588 - @ref NEHOGBlockNormalizationKernel
589 - @ref NEHOGDetectorKernel
590 - @ref NEHOGOrientationBinningKernel
591 - @ref NELogits1DMaxKernel
592 - @ref NELogits1DShiftExpSumKernel
593 - @ref NELogits1DNormKernel
594 - @ref NELKTrackerKernel
595 - @ref NENonMaximaSuppression3x3FP16Kernel
596 - @ref NENonMaximaSuppression3x3Kernel
598 @subsubsection S4_6_2_tensors Tensors
600 Tensors are multi-dimensional arrays made of up to @ref Coordinates::num_max_dimensions dimensions.
602 A simple vector of numbers can be represented as a 1D tensor, an image is actually just a 2D tensor, a 3D tensor can be seen as an array of images, a 4D tensor as a 2D array of images, etc.
604 @note Most algorithms process images (i.e a 2D slice of the tensor), therefore only padding along the X and Y axes is required (2D slices can be stored contiguously in memory).
606 @subsubsection S4_6_3_description_conventions Images and Tensors description conventions
608 Image objects are defined by a @ref Format and dimensions expressed as [width, height, batch]
610 Tensors are defined by a @ref DataType plus a number of channels (Always expected to be 1 for now) and their dimensions are expressed as [width, height, feature_maps, batch].
612 In other words, the lower three dimensions of a tensor specify a single input in [width, height, feature_maps], while any other specified dimension represents a batch in the appropriate dimension space.
613 For example, a tensor with dimensions [128, 128, 64, 16] represents a 1D batch space with 16 batches of 128 elements in width and height and 64 feature maps each.
614 Each kernel specifies the expected layout of each of its tensors in its documentation.
616 @note Unless specified otherwise in the kernel's or function's documentation all tensors and images parameters passed must have identical dimensions.
618 @note Unless specified otherwise in the kernel's or function's documentation the number of channels for tensors is expected to be 1 (For images, the number of channels is inferred from the @ref Format).
620 @subsubsection S4_6_4_working_with_objects Working with Images and Tensors
622 In the case that no padding exists in the Image/Tensor object you can linearize the object memory and directly copy to/from it.
624 // Create a tensor object
629 unsigned char *dst = ... // Your unpadded destination buffer
630 // Copy tensor as a linear bulk of memory if no padding exists
631 if(!tensor.info()->has_padding())
633 std::copy_n(tensor.buffer(), tensor.info()->total_size(), dst);
637 On the other hand, in case of padding, each row should be carefully copied separately.
639 // Create an image object
642 const unsigned char *src = ... // Your unpadded input buffer
643 // Initialize the Image object using an RGB source image
644 for(unsigned int y = 0; y < height; ++y)
646 // Copy one RGB row at a time
647 std::copy_n(img.buffer() + img.info()->offset_element_in_bytes(Coordinates(0, y)), width * 3, src + (y * width) * 3);