[BYOC][Contrib] Arm Compute Library integration (#5915)
authorlhutton1 <35535092+lhutton1@users.noreply.github.com>
Tue, 21 Jul 2020 15:30:26 +0000 (16:30 +0100)
committerGitHub <noreply@github.com>
Tue, 21 Jul 2020 15:30:26 +0000 (08:30 -0700)
commitd8c9bb18907ed57916d94325f47f1c3dcfd34784
tree060f9351a09759e9a08cb8bb77b97c10a35158ff
parent679fd29a7f6cae17d8c8b1b734a0abb0592ab3a8
[BYOC][Contrib] Arm Compute Library integration (#5915)

* [BYOC][Contrib] Arm Compute Library integration

Arm Compute Library (ACL) integration using the BYOC infrastructure. This will enable offloading select operators from a relay
graph to ACL so we can achieve faster inference times on Arm CPU's due to hand crafted optimized routines. The PR adds initial
support for offloading FP32 conv2d, maxpool2d and reshape to ACL. ACL codegen is used to generate a JSON representation of an
operator or 'ACL layer', the ACL runtime then uses this representation to construct a layer, cache it and create a packed
function to for the graph runtime to call into.

RFC here: https://discuss.tvm.ai/t/rfc-byoc-arm-compute-library-integration/7082

Change-Id: If756dcea787ea346b1508e9a191b7eed7bd02b7f

* Refactor ACL integration to support JSON runtime

* Now uses JSON runtime
* Addresses tutorial comments
* Rename acl to arm_compute_lib in user facing api

Change-Id: I3b5ef80607f713e898363e82ab4398fbc2cf267a

* Address comments

Change-Id: I041fda14f3bf9975f3518ba8a4e3ab43ba98403d

* Address comments

* correct mistakes in tutorial
* reshuffle runtime to use fewer macro blocks
* preprocess module using "optimize" functionality
* use new module api

Change-Id: I219488e617e5767edd7489b43b8bfce876cd24b8

* Enable ACL codegen tests in CI

* Skips runtime tests as these are not supported on x86.

Change-Id: I6843c003a2604afe95cfdccf2323d2a336b56fe5

* Fix check for runtime

Change-Id: I3f9eec15c599f01b1105d624fb053b73bfb6ed41

* Address comments

* Add warning to ACL engine creation
* Correct runtime check so it doesn't fail when codegen not present
* Improve testing to check acl partitions is what is expected
* Check results of multiple runs test

Change-Id: I9522950930805b9b601dad03269adcf8ed3138cc

* Address comments

* Multiple style improvements
* Use base class for creating json node for single op
* Move GetSource to base class
* Improve annotation checks

Change-Id: I8219659c4b99e86df887cd914720157cb94c61a0

* Improve tutorial

Change-Id: I8f610bd37af1e3740fd48c2d502bcc4727d9d712

* Initialize conv with nullptr

Change-Id: I6c37f0d75a064001c74e171ff83b9f7a7c3f1918
27 files changed:
CMakeLists.txt
cmake/config.cmake
cmake/modules/contrib/ArmComputeLib.cmake [new file with mode: 0644]
docs/deploy/arm_compute_lib.rst [new file with mode: 0644]
docs/deploy/index.rst
python/tvm/relay/backend/graph_runtime_factory.py
python/tvm/relay/op/contrib/__init__.py
python/tvm/relay/op/contrib/arm_compute_lib.py [new file with mode: 0644]
src/relay/backend/contrib/arm_compute_lib/codegen.cc [new file with mode: 0644]
src/relay/backend/contrib/codegen_c/codegen_c.h
src/relay/backend/contrib/dnnl/codegen.cc
src/relay/backend/utils.h
src/runtime/contrib/arm_compute_lib/acl_allocator.cc [new file with mode: 0644]
src/runtime/contrib/arm_compute_lib/acl_allocator.h [new file with mode: 0644]
src/runtime/contrib/arm_compute_lib/acl_runtime.cc [new file with mode: 0644]
src/runtime/contrib/arm_compute_lib/acl_utils.cc [new file with mode: 0644]
src/runtime/contrib/arm_compute_lib/acl_utils.h [new file with mode: 0644]
src/runtime/contrib/json/json_node.h
src/runtime/contrib/json/json_runtime.h
tests/python/contrib/test_arm_compute_lib/__init__.py [new file with mode: 0644]
tests/python/contrib/test_arm_compute_lib/infrastructure.py [new file with mode: 0644]
tests/python/contrib/test_arm_compute_lib/test_conv2d.py [new file with mode: 0644]
tests/python/contrib/test_arm_compute_lib/test_network.py [new file with mode: 0644]
tests/python/contrib/test_arm_compute_lib/test_pooling.py [new file with mode: 0644]
tests/python/contrib/test_arm_compute_lib/test_reshape.py [new file with mode: 0644]
tests/python/contrib/test_arm_compute_lib/test_runtime.py [new file with mode: 0644]
tests/scripts/task_config_build_cpu.sh