[RFC] [Contrib] Minimal runtime (~12kb .text on ARMv7/x86) for subset of TVM models...
authorAndrew Tulloch <andrew@tullo.ch>
Thu, 12 Sep 2019 19:32:01 +0000 (12:32 -0700)
committerTianqi Chen <tqchen@users.noreply.github.com>
Thu, 12 Sep 2019 19:32:01 +0000 (03:32 +0800)
commit1de52bb0109af3e7051a573c2b8fa2f039c3d05d
tree9d4257288bae1bf595ff1a2c82865da681a346dd
parent4e2d707f2d2160a30375267269c4f5da9a810cfa
[RFC] [Contrib] Minimal runtime (~12kb .text on ARMv7/x86) for subset of TVM models (#3567)

This is an alternative implementation of a subset of the TVM runtime API (and
graph runtime) that focuses entirely on reducing code size, at the expense of
functionality (no tvm.extern(..) calls via PackedFunc, CPU only, etc). It might
be worth incrementally expanding the surface area if there's interest.

The motivation for this work was seeing what the minimal useful subset of the
TVM runtime is. This is relevant for e.g. super code-size constrained
applications in e.g. embedded/mobile. The current runtime is more like O(100KiB)
or so, so this might be compelling for some users.

The smaller surface area for auditing might make this relevant for
https://github.com/dmlc/tvm/issues/3159, or the usecases I was thinking about in
https://github.com/dmlc/tvm/issues/2523#issuecomment-459165815 re: the Rust
runtime.

The symbols in the tvm::minimalruntime space (i.e. excluding std:: and
picojson::) are about 5KiB, so I think there's a bunch of room here (i.e. we
could replace picojson:: with [`jsmn`](https://zserge.com/jsmn.html) or
something, and we could replace more of the `std::unordered_map` usage, etc with
custom primitives as well (similar to the `DynArray`).
14 files changed:
3rdparty/picojson/README.md [new file with mode: 0644]
3rdparty/picojson/picojson.h [new file with mode: 0644]
CMakeLists.txt
cmake/config.cmake
cmake/modules/contrib/MicroStandaloneRuntime.cmake [new file with mode: 0644]
include/tvm/runtime/micro/standalone/utvm_runtime.h [new file with mode: 0644]
src/runtime/micro/standalone/README.md [new file with mode: 0644]
src/runtime/micro/standalone/minimal_vector.h [new file with mode: 0644]
src/runtime/micro/standalone/utvm_graph_runtime.cc [new file with mode: 0644]
src/runtime/micro/standalone/utvm_graph_runtime.h [new file with mode: 0644]
src/runtime/micro/standalone/utvm_runtime.cc [new file with mode: 0644]
src/runtime/micro/standalone/utvm_runtime_api.cc [new file with mode: 0644]
src/runtime/micro/standalone/utvm_runtime_api.h [new file with mode: 0644]
tests/cpp/utvm_runtime_standalone_test.cc [new file with mode: 0644]