[mlir] Move symbol loading from mlir-cpu-runner to ExecutionEngine.
authorIngo Müller <ingomueller@google.com>
Thu, 15 Jun 2023 14:33:22 +0000 (14:33 +0000)
committerIngo Müller <ingomueller@google.com>
Fri, 16 Jun 2023 14:50:14 +0000 (14:50 +0000)
commit0b3841eb97f5c7951c2375e5b1a38ea89d9d6a37
treeb7ece8db787dacac95ea7865e5b5eb1fb46c5e69
parent55ed7ba7ab84271a30eb73639c90d83f5bd72af5
[mlir] Move symbol loading from mlir-cpu-runner to ExecutionEngine.

Both the mlir-cpu-runner and the execution engine allow to provide a
list of shared libraries that should be loaded into the process such
that the jitted code can use the symbols from those libraries. The
runner had implemented a protocol that allowed libraries to control
which symbols it wants to provide in that context (with a function
called __mlir_runner_init). In absence of that, the runner would rely on
the loading mechanism of the execution engine, which didn't do anything
particular with the symbols, i.e., only symbols with public visibility
were visible to jitted code.

Libraries used a mix of the two mechanisms: while the runner utils and C
runner utils libs (and potentially others) used public visibility, the
async runtime lib (as the only one in the monorepo) used the loading
protocol. As a consequence, the async runtime library could not be used
through the Python bindings of the execution engine.

This patch moves the loading protocol from the runner to the execution
engine. For the runner, this should not change anything: it lets the
execution engine handle the loading which now implements the same
protocol that the runner had implemented before. However, the Python
binding now get to benefit from the loading protocol as well, so the
async runtime library (and potentially other out-of-tree libraries) can
now be used in that context.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D153029
mlir/include/mlir/ExecutionEngine/ExecutionEngine.h
mlir/lib/ExecutionEngine/AsyncRuntime.cpp
mlir/lib/ExecutionEngine/ExecutionEngine.cpp
mlir/lib/ExecutionEngine/JitRunner.cpp