ARM Compute Library
17.04
|
The ARM Computer Vision and Machine Learning library is a set of functions optimised for both ARM CPUs and GPUs using SIMD technologies.
Several builds of the library are available using various configurations:
Please email devel oper @arm. com
In order to facilitate the work of the support team please provide the build information of the library you are using. To get the version of the library you are using simply run:
$ strings android-armv7a-cl-asserts/libarm_compute.so | grep arm_compute_version arm_compute_version=v16.12 Build options: {'embed_kernels': '1', 'opencl': '1', 'arch': 'armv7a', 'neon': '0', 'asserts': '1', 'debug': '0', 'os': 'android', 'Werror': '1'} Git hash=f51a545d4ea12a9059fe4e598a092f1fd06dc858
This archive contains:
You should have the following file organisation:
. ├── arm_compute --> All the arm_compute headers │ ├── core │ │ ├── CL │ │ │ ├── CLKernels.h --> Includes all the OpenCL kernels at once │ │ │ ├── CL specialisation of all the generic objects interfaces (ICLTensor, ICLImage, etc.) │ │ │ ├── kernels --> Folder containing all the OpenCL kernels │ │ │ │ └── CL*Kernel.h │ │ │ └── OpenCL.h --> Wrapper to configure the Khronos OpenCL C++ header │ │ ├── CPP │ │ │ └── kernels --> Folder containing all the CPP kernels │ │ │ │ └── CPP*Kernel.h │ │ ├── NEON │ │ │ ├── kernels --> Folder containing all the NEON kernels │ │ │ │ └── NE*Kernel.h │ │ │ └── NEKernels.h --> Includes all the NEON kernels at once │ │ ├── All common basic types (Types.h, Window, Coordinates, Iterator, etc.) │ │ ├── All generic objects interfaces (ITensor, IImage, etc.) │ │ └── Objects metadata classes (ImageInfo, TensorInfo, MultiImageInfo) │ └── runtime │ ├── CL │ │ ├── CL objects & allocators (CLArray, CLImage, CLTensor, etc.) │ │ ├── functions --> Folder containing all the OpenCL functions │ │ │ └── CL*.h │ │ └── CLFunctions.h --> Includes all the OpenCL functions at once │ ├── CPP │ │ └── CPPScheduler.h --> Basic pool of threads to execute CPP/NEON code on several cores in parallel │ ├── NEON │ │ ├── functions --> Folder containing all the NEON functions │ │ │ └── NE*.h │ │ └── NEFunctions.h --> Includes all the NEON functions at once │ └── Basic implementations of the generic object interfaces (Array, Image, Tensor, etc.) ├── documentation │ ├── index.xhtml │ └── ... ├── documentation.xhtml -> documentation/index.xhtml ├── examples │ ├── cl_convolution.cpp │ ├── neoncl_scale_median_gaussian.cpp │ ├── neon_convolution.cpp │ └── neon_scale.cpp ├── include │ └── CL │ └── Khronos OpenCL C headers and C++ wrapper ├── opencl-1.2-stubs │ └── opencl_stubs.c ├── src │ ├── core │ │ └── ... (Same structure as headers) │ │ └── CL │ │ └── cl_kernels --> All the OpenCL kernels │ └── runtime │ └── ... (Same structure as headers) └── test_helpers --> Boiler plate code used by examples └── Utils.h
All releases are numbered vYY.MM Where YY are the last two digits of the year, and MM the month number. If there is more than one release in a month then an extra sequential number is appended at the end:
v17.03 (First release of March 2017) v17.03.1 (Second release of March 2017) v17.04 (First release of April 2017)
v17.04 Public bug fixes release The following functions have been ported to use the new accurate padding:
v17.03.1 First Major public release of the sources
v17.03 Sources preview
v17.02.1 Sources preview
v17.02 Sources preview
v16.12 Binary preview release
scons 2.3 or above is required to build the library. To see the build options available simply run scons -h
:
debug: Debug (default=0) (0|1) default: 0 actual: 0 asserts: Enable asserts (This flag is forced to 1 for debug=1) (default=0) (0|1) default: 0 actual: 0 arch: Target Architecture (default=armv7a) (armv7a|arm64-v8a|arm64-v8.2-a|x86_32|x86_64) default: armv7a actual: armv7a os: Target OS (default=linux) (linux|android|bare_metal) default: linux actual: linux build: Build type: (default=cross_compile) (native|cross_compile) default: cross_compile actual: cross_compile Werror: Enable/disable the -Werror compilation flag (Default=1) (0|1) default: 1 actual: 1 opencl: Enable OpenCL support(Default=1) (0|1) default: 1 actual: 1 neon: Enable Neon support(Default=0) (0|1) default: 0 actual: 0 embed_kernels: Embed OpenCL kernels in library binary(Default=0) (0|1) default: 0 actual: 0 scheduler: Scheduler backend(Default=cpp) (cpp|pthread|openmp) default: cpp actual: cpp set_soname: Set the library's soname and shlibversion (Requires SCons 2.4 or above) (yes|no) default: 0 actual: False extra_cxx_flags: Extra CXX flags to be appended to the build command default: actual:
Debug / asserts:
Architecture: The x86_32 and x86_64 targets can only be used with neon=0 and opencl=1.
OS: Choose the operating system you are targeting: Linux, Android or bare metal.
Build type: you can either build directly on your device (native) or cross compile from your desktop machine (cross-compile). In both cases make sure the compiler is available in your path.
Werror: If you are compiling using the same toolchains as the ones used in this guide then there shouldn't be any warning and therefore you should be able to keep Werror=1. If with a different compiler version the library fails to build because of warnings interpreted as errors then, if you are sure the warnings are not important, you might want to try to build with Werror=0 (But please do report the issue either on Github or by an email to devel so that the issue can be addressed). oper @arm. com
OpenCL / NEON: Choose which SIMD technology you want to target. (NEON for ARM Cortex-A CPUs or OpenCL for ARM Mali GPUs)
embed_kernels: For OpenCL only: set embed_kernels=1 if you want the OpenCL kernels to be built in the library's binaries instead of being read from separate ".cl" files. If embed_kernels is set to 0 then the application can set the path to the folder containing the OpenCL kernel files by calling CLKernelLibrary::init(). By default the path is set to "./cl_kernels".
set_soname: Do you want to build the versioned version of the library ? If enabled the library will contain a SONAME and SHLIBVERSION and some symlinks will automatically be created between the objects. Example: libarm_compute_core.so -> libarm_compute_core.so.1.0.0 libarm_compute_core.so.1 -> libarm_compute_core.so.1.0.0 libarm_compute_core.so.1.0.0
extra_cxx_flags: Custom CXX flags which will be appended to the end of the build command.
For Linux, the library was successfully built and tested using the following Linaro GCC toolchain: gcc-linaro-arm-linux-gnueabihf-4.8-2014.02_linux and gcc-linaro-6.1.1-2016.08-x86_64_arm-linux-gnueabihf
To cross-compile the library in debug mode, with NEON only support, for Linux 32bit:
scons Werror=1 -j8 debug=1 neon=1 opencl=0 os=linux arch=armv7a
To cross-compile the library in asserts mode, with OpenCL only support, for Linux 64bit:
scons Werror=1 -j8 debug=0 asserts=1 neon=0 opencl=1 embed_kernels=1 os=linux arch=arm64-v8a
You can also compile the library natively on an ARM device by using build=native:
scons Werror=1 -j8 debug=0 neon=1 opencl=0 os=linux arch=arm64-v8a build=native scons Werror=1 -j8 debug=0 neon=1 opencl=0 os=linux arch=armv7a build=native
For example on a 64bit Debian based system you would have to install g++-arm-linux-gnueabihf
apt-get install g++-arm-linux-gnueabihf
Then run
scons Werror=1 -j8 debug=0 neon=1 opencl=0 os=linux arch=armv7a build=cross_compile
or simply remove the build parameter as build=cross_compile is the default value:
scons Werror=1 -j8 debug=0 neon=1 opencl=0 os=linux arch=armv7a
The examples get automatically built by scons as part of the build process of the library described above. This section just describes how you can build and link your own application against our library.
To cross compile a NEON example:
arm-linux-gnueabihf-g++ examples/neon_convolution.cpp test_helpers/Utils.cpp -I. -Iinclude -std=c++11 -mfpu=neon -L. -larm_compute -o neon_convolution
To cross compile an OpenCL example:
arm-linux-gnueabihf-g++ examples/cl_convolution.cpp test_helpers/Utils.cpp -I. -Iinclude -std=c++11 -mfpu=neon -L. -larm_compute -lOpenCL -o cl_convolution
To compile natively (i.e directly on an ARM device) for NEON:
g++ examples/neon_convolution.cpp test_helpers/Utils.cpp -I. -Iinclude -std=c++11 -mfpu=neon -larm_compute -o neon_convolution
To compile natively (i.e directly on an ARM device) for OpenCL:
g++ examples/cl_convolution.cpp test_helpers/Utils.cpp -I. -Iinclude -std=c++11 -larm_compute -lOpenCL -o cl_convolution
To run the built executable simply run:
LD_LIBRARY_PATH=build ./neon_convolution
or
LD_LIBRARY_PATH=build ./cl_convolution
For Android, the library was successfully built and tested using Google's standalone toolchains:
Here is a guide to create your Android standalone toolchains from the NDK
$NDK/build/tools/make_standalone_toolchain.py --arch arm64 --install-dir $MY_TOOLCHAINS/aarch64-linux-android-4.9 --stl gnustl $NDK/build/tools/make_standalone_toolchain.py --arch arm --install-dir $MY_TOOLCHAINS/arm-linux-androideabi-4.9 --stl gnustl
To cross-compile the library in debug mode, with NEON only support, for Android 32bit:
CXX=clang++ CC=clang scons Werror=1 -j8 debug=1 neon=1 opencl=0 os=android arch=armv7a
To cross-compile the library in asserts mode, with OpenCL only support, for Android 64bit:
scons Werror=1 -j8 debug=0 asserts=1 neon=0 opencl=1 embed_kernels=1 os=android arch=arm64-v8a
The examples get automatically built by scons as part of the build process of the library described above. This section just describes how you can build and link your own application against our library.
Once you've got your Android standalone toolchain built and added to your path you can do the following:
To cross compile a NEON example:
#32 bit: arm-linux-androideabi-clang++ examples/neon_convolution.cpp -I. -Iinclude -std=c++11 -larm_compute-static -L. -o neon_convolution_arm -static-libstdc++ -pie #64 bit: aarch64-linux-android-g++ examples/neon_convolution.cpp -I. -Iinclude -std=c++11 -larm_compute-static -L. -o neon_convolution_aarch64 -static-libstdc++ -pie
To cross compile an OpenCL example:
#32 bit: arm-linux-androideabi-clang++ examples/cl_convolution.cpp -I. -Iinclude -std=c++11 -larm_compute-static -L. -o cl_convolution_arm -static-libstdc++ -pie -lOpenCL #64 bit: aarch64-linux-android-g++ examples/cl_convolution.cpp -I. -Iinclude -std=c++11 -larm_compute-static -L. -o cl_convolution_aarch64 -static-libstdc++ -pie -lOpenCL
Then you need to do is upload the executable and the shared library to the device using ADB:
adb push neon_convolution_arm /data/local/tmp/ adb push cl_convolution_arm /data/local/tmp/ adb shell chmod 777 -R /data/local/tmp/
And finally to run the example:
adb shell /data/local/tmp/neon_convolution_arm adb shell /data/local/tmp/cl_convolution_arm
For 64bit:
adb push neon_convolution_aarch64 /data/local/tmp/ adb push cl_convolution_aarch64 /data/local/tmp/ adb shell chmod 777 -R /data/local/tmp/
And finally to run the example:
adb shell /data/local/tmp/neon_convolution_aarch64 adb shell /data/local/tmp/cl_convolution_aarch64
In the opencl-1.2-stubs folder you will find the sources to build a stub OpenCL library which then can be used to link your application or arm_compute against.
If you preferred you could retrieve the OpenCL library from your device and link against this one but often this library will have dependencies on a range of system libraries forcing you to link your application against those too even though it is not using them.
To cross-compile the stub OpenCL library simply run:
<target-prefix>-gcc -o libOpenCL.so -Iinclude opencl-1.2-stubs/opencl_stubs.c -fPIC -shared
For example:
<target-prefix>-gcc -o libOpenCL.so -Iinclude opencl-1.2-stubs/opencl_stubs.c -fPIC -shared #Linux 32bit arm-linux-gnueabihf-gcc -o libOpenCL.so -Iinclude opencl-1.2-stubs/opencl_stubs.c -fPIC -shared #Linux 64bit aarch64-linux-gnu-gcc -o libOpenCL.so -Iinclude -shared opencl-1.2-stubs/opencl_stubs.c -fPIC #Android 32bit arm-linux-androideabi-clang -o libOpenCL.so -Iinclude -shared opencl-1.2-stubs/opencl_stubs.c -fPIC -shared #Android 64bit aarch64-linux-android-gcc -o libOpenCL.so -Iinclude -shared opencl-1.2-stubs/opencl_stubs.c -fPIC -shared
The Core library is a low level collection of algorithms implementations, it is designed to be embedded in existing projects and applications:
The Runtime library is a very basic wrapper around the Core library which can be used for quick prototyping, it is basic in the sense that:
For maximum performance, it is expected that the users would re-implement an equivalent to the runtime library which suits better their needs (With a more clever multi-threading strategy, load-balancing between NEON and OpenCL, etc.)
A Window represents a workload to execute, it's made of up to Coordinates::num_max_dimensions dimensions. Each dimension is defined by a start, end and step.
It can split into subwindows as long as all the following rules remain true for all the dimensions:
Each implementation of the IKernel interface (base class of all the kernels in the core library) works in the same way:
OpenCL kernels:
NEON / CPP kernels:
The previous section shows how to run a NEON / CPP kernel in the current thread, however if your system has several CPU cores, you will probably want the kernel to use several cores. Here is how this can be done:
This is the very basic implementation used in the NEON runtime library by all the NEON functions.
memory_needed_per_thread * num_threads
and each subwindow must be initialised by calling Window::set_thread_id() with a unique thread_id between 0 and num_threads.Functions will automatically allocate the temporary buffers mentioned above, and will automatically multi-thread kernels' executions using the very basic scheduler described in the previous section.
Simple functions are made of a single kernel (e.g NEConvolution3x3), while more complex ones will be made of a several kernels pipelined together (e.g NEGaussianPyramid, NEHarrisCorners), check their documentation to find out which kernels are used by each function.
The ARM Compute runtime uses a single command queue and context for all the operations.
The user can get / set this context and command queue through the CLScheduler's interface.
All the OpenCL kernels used by the library are built and stored in the CLKernelLibrary. If the library is compiled with embed_kernels=0 the application can set the path to the OpenCL kernels by calling CLKernelLibrary::init(), by default the path is set to "./cl_kernels"
In order to block until all the jobs in the CLScheduler's command queue are done executing the user can call CLScheduler::sync() or create a sync event using CLScheduler::enqueue_sync_event()
For example:
You can mix OpenCL and NEON kernels and or functions, however it is the user's responsibility to handle the mapping unmapping of the OpenCL objects, for example:
All algorithms in this library have been implemented following the OpenVX 1.1 specifications Please refer to the Khronos documentation for more information.
Most kernels and functions in the library process images, however, in order to be future proof most of the kernels actually accept tensors, see below for more information about they are related.
Several algorithms rely on neighbour pixels to calculate the value of a given pixel: this means the algorithm will not be able to process the borders of the image unless you give it more information about what you want to happen for border pixels, this is the BorderMode.
You have 3 types of BorderMode :
Moreover both OpenCL and NEON use vector loads and stores instructions to access the data in buffers, so in order to avoid having special cases to handle for the borders all the images and tensors used in this library must be padded.
There are different ways padding can be calculated:
Some kernels (like edge detectors for example) need to read values of neighbouring pixels to calculate the value of a given pixel, it is therefore not possible to calculate the values of the pixels on the edges.
Another case is: if a kernel processes 8 pixels per iteration then if the image's dimensions is not a multiple of 8 and not enough padding is available then the kernel will not be able to process the pixels near the right edge as a result these pixels will be left undefined.
In order to know which pixels have been calculated, each kernel sets a valid region for each output image or tensor. See also TensorInfo::valid_region(), ValidRegion
List of kernels which haven't been ported yet:
Tensors are multi-dimensional arrays made of up to Coordinates::num_max_dimensions dimensions.
A simple vector of numbers can be represented as a 1D tensor, an image is actually just a 2D tensor, a 3D tensor can be seen as an array of images, a 4D tensor as a 2D array of images, etc.
Image objects are defined by a Format and dimensions expressed as [width, height, batch]
Tensors are defined by a DataType plus a number of channels (Always expected to be 1 for now) and their dimensions are expressed as [width, height, feature_maps, batch].
In other words, the lower three dimensions of a tensor specify a single input in [width, height, feature_maps], while any other specified dimension represents a batch in the appropriate dimension space. For example, a tensor with dimensions [128, 128, 64, 16] represents a 1D batch space with 16 batches of 128 elements in width and height and 64 feature maps each. Each kernel specifies the expected layout of each of its tensors in its documentation.
For example, to read the element located at the coordinates (x,y) of a float tensor:
The library provides some iterators to access objects' data. Iterators are created by associating a data object (An image or a tensor for example) with an iteration window.
Iteration windows are defined by an array of dimension, each of which is made of a start, end and step.
The execute_window_loop function takes an execution window, a lambda function and one or more iterators. It will iterate through every element of the execution window and for each element it will update the iterators accordingly and call the lambda function.
Here is a couple of examples of how to use the iterators to fill / read tensors: