From: MyungJoo Ham Date: Tue, 10 Dec 2019 09:35:09 +0000 (+0900) Subject: Import edgetpu runtime library. X-Git-Tag: submit/tizen/20191218.020717^0 X-Git-Url: http://review.tizen.org/git/?a=commitdiff_plain;h=36cec71a0adbba0644676f716bd823583649d8ed;p=platform%2Fadaptation%2Fnpu%2Fgoogle-edgetpu-runtime.git Import edgetpu runtime library. Get required files from https://github.com/google-coral/edgetpu and their binary packages. Signed-off-by: MyungJoo Ham --- 36cec71a0adbba0644676f716bd823583649d8ed diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..f4f87bd --- /dev/null +++ b/LICENSE @@ -0,0 +1,203 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + \ No newline at end of file diff --git a/libedgetpu/BUILD b/libedgetpu/BUILD new file mode 100644 index 0000000..f17ee8c --- /dev/null +++ b/libedgetpu/BUILD @@ -0,0 +1,14 @@ +# Description: +# Edgetpu custom op api for tflite. + +package(default_visibility = ["//visibility:public"]) + +licenses(["notice"]) # Apache 2.0 + +cc_library( + name = "header", + hdrs = ["edgetpu.h", "edgetpu_c.h"], + deps = [ + "@org_tensorflow//tensorflow/lite:context", + ], +) diff --git a/libedgetpu/LICENSE.txt b/libedgetpu/LICENSE.txt new file mode 100644 index 0000000..07bfbb5 --- /dev/null +++ b/libedgetpu/LICENSE.txt @@ -0,0 +1,7 @@ +Copyright 2019 Google LLC. This software is provided as-is, without warranty +or representation for any use or purpose. Your use of it is subject to your +agreements with Google covering this software, or if no such agreement +applies, your use is subject to a limited, non-transferrable, non-exclusive +license solely to run the software for your testing use, unless and until +revoked by Google. + diff --git a/libedgetpu/direct/aarch64/libedgetpu.so.1 b/libedgetpu/direct/aarch64/libedgetpu.so.1 new file mode 120000 index 0000000..90ac68c --- /dev/null +++ b/libedgetpu/direct/aarch64/libedgetpu.so.1 @@ -0,0 +1 @@ +libedgetpu.so.1.0 \ No newline at end of file diff --git a/libedgetpu/direct/aarch64/libedgetpu.so.1.0 b/libedgetpu/direct/aarch64/libedgetpu.so.1.0 new file mode 100755 index 0000000..b01a19e Binary files /dev/null and b/libedgetpu/direct/aarch64/libedgetpu.so.1.0 differ diff --git a/libedgetpu/direct/armv6/libedgetpu.so.1 b/libedgetpu/direct/armv6/libedgetpu.so.1 new file mode 120000 index 0000000..90ac68c --- /dev/null +++ b/libedgetpu/direct/armv6/libedgetpu.so.1 @@ -0,0 +1 @@ +libedgetpu.so.1.0 \ No newline at end of file diff --git a/libedgetpu/direct/armv6/libedgetpu.so.1.0 b/libedgetpu/direct/armv6/libedgetpu.so.1.0 new file mode 100755 index 0000000..c59055b Binary files /dev/null and b/libedgetpu/direct/armv6/libedgetpu.so.1.0 differ diff --git a/libedgetpu/direct/armv7a/libedgetpu.so.1 b/libedgetpu/direct/armv7a/libedgetpu.so.1 new file mode 120000 index 0000000..90ac68c --- /dev/null +++ b/libedgetpu/direct/armv7a/libedgetpu.so.1 @@ -0,0 +1 @@ +libedgetpu.so.1.0 \ No newline at end of file diff --git a/libedgetpu/direct/armv7a/libedgetpu.so.1.0 b/libedgetpu/direct/armv7a/libedgetpu.so.1.0 new file mode 100755 index 0000000..0213c35 Binary files /dev/null and b/libedgetpu/direct/armv7a/libedgetpu.so.1.0 differ diff --git a/libedgetpu/direct/k8/libedgetpu.so.1 b/libedgetpu/direct/k8/libedgetpu.so.1 new file mode 120000 index 0000000..90ac68c --- /dev/null +++ b/libedgetpu/direct/k8/libedgetpu.so.1 @@ -0,0 +1 @@ +libedgetpu.so.1.0 \ No newline at end of file diff --git a/libedgetpu/direct/k8/libedgetpu.so.1.0 b/libedgetpu/direct/k8/libedgetpu.so.1.0 new file mode 100755 index 0000000..be5d9bc Binary files /dev/null and b/libedgetpu/direct/k8/libedgetpu.so.1.0 differ diff --git a/libedgetpu/edgetpu-accelerator.rules b/libedgetpu/edgetpu-accelerator.rules new file mode 100644 index 0000000..60e034c --- /dev/null +++ b/libedgetpu/edgetpu-accelerator.rules @@ -0,0 +1,2 @@ +SUBSYSTEM=="usb",ATTRS{idVendor}=="1a6e",GROUP="plugdev" +SUBSYSTEM=="usb",ATTRS{idVendor}=="18d1",GROUP="plugdev" diff --git a/libedgetpu/edgetpu.h b/libedgetpu/edgetpu.h new file mode 100644 index 0000000..6e80c39 --- /dev/null +++ b/libedgetpu/edgetpu.h @@ -0,0 +1,292 @@ +/* +Copyright 2018 Google LLC + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +// +// This header file defines EdgeTpuManager, and EdgeTpuContext. +// EdgeTpuContext is an object associated with one or more tflite::Interpreter. +// Instances of this class should be allocated through +// EdgeTpuManager::NewEdgeTpuContext. +// More than one Interpreter instances can point to the same context. This means +// the tasks from both would be executed under the same TPU context. +// The lifetime of this context must be longer than all associated +// tflite::Interpreter instances. +// +// Typical usage with NNAPI: +// +// std::unique_ptr interpreter; +// tflite::ops::builtin::BuiltinOpResolver resolver; +// auto model = +// tflite::FlatBufferModel::BuildFromFile(model_file_name.c_str()); +// +// // Registers edge TPU custom op handler with Tflite resolver. +// resolver.AddCustom(edgetpu::kCustomOp, edgetpu::RegisterCustomOp()); +// +// tflite::InterpreterBuilder(*model, resolver)(&interpreter); +// +// interpreter->AllocateTensors(); +// .... (Prepare input tensors) +// interpreter->Invoke(); +// .... (retrieving the result from output tensors) +// +// // Releases interpreter instance to free up resources associated with +// // this custom op. +// interpreter.reset(); +// +// Typical usage with Non-NNAPI: +// +// // Sets up the tpu_context. +// auto tpu_context = +// edgetpu::EdgeTpuManager::GetSingleton()->OpenDevice(); +// +// std::unique_ptr interpreter; +// tflite::ops::builtin::BuiltinOpResolver resolver; +// auto model = +// tflite::FlatBufferModel::BuildFromFile(model_file_name.c_str()); +// +// // Registers edge TPU custom op handler with Tflite resolver. +// resolver.AddCustom(edgetpu::kCustomOp, edgetpu::RegisterCustomOp()); +// +// tflite::InterpreterBuilder(*model, resolver)(&interpreter); +// +// // Binds a context with a specific interpreter. +// interpreter->SetExternalContext(kTfLiteEdgeTpuContext, +// tpu_context.get()); +// +// // Note that all edge TPU context set ups should be done before this +// // function is called. +// interpreter->AllocateTensors(); +// .... (Prepare input tensors) +// interpreter->Invoke(); +// .... (retrieving the result from output tensors) +// +// // Releases interpreter instance to free up resources associated with +// // this custom op. +// interpreter.reset(); +// +// // Closes the edge TPU. +// tpu_context.reset(); + +#ifndef TFLITE_PUBLIC_EDGETPU_H_ +#define TFLITE_PUBLIC_EDGETPU_H_ + +// If the ABI changes in a backward-incompatible way, please increment the +// version number in the BUILD file. +// LINT.IfChange() + +#include +#include +#include +#include +#include + +#include "tensorflow/lite/context.h" + +#if defined(_WIN32) +#ifdef EDGETPU_COMPILE_LIBRARY +#define EDGETPU_EXPORT __declspec(dllexport) +#else +#define EDGETPU_EXPORT __declspec(dllimport) +#endif // EDGETPU_COMPILE_LIBRARY +#else +#define EDGETPU_EXPORT +#endif // _WIN32 + +namespace edgetpu { + +// EdgeTPU custom op. +static const char kCustomOp[] = "edgetpu-custom-op"; + +enum class DeviceType { + kApexPci = 0, + kApexUsb = 1, +}; + +class EdgeTpuContext; + +// Singleton edge TPU manager for allocating new TPU contexts. +// Functions in this interface are thread-safe. +class EDGETPU_EXPORT EdgeTpuManager { + public: + using DeviceOptions = std::unordered_map; + struct DeviceEnumerationRecord { + DeviceType type; + std::string path; + + // Returns true if two enumeration records point to the same device. + friend bool operator==(const DeviceEnumerationRecord& lhs, + const DeviceEnumerationRecord& rhs) { + return (lhs.type == rhs.type) && (lhs.path == rhs.path); + } + + // Returns true if two enumeration records point to defferent devices. + friend bool operator!=(const DeviceEnumerationRecord& lhs, + const DeviceEnumerationRecord& rhs) { + return !(lhs == rhs); + } + }; + + // Returns pointer to the singleton object, or nullptr if not supported on + // this platform. + static EdgeTpuManager* GetSingleton(); + + // NewEdgeTpuContext family functions has been deprecated and will be removed + // in the future. Please use OpenDevice for new code. + // + // These functions return an unique_ptr to EdgeTpuContext, with + // the intention that the device will be closed, and associate resources + // released, when the unique_ptr leaves scope. + // + // These functions seek exclusive ownership of the opened devices. As they + // cannot open devices already opened by OpenDevice, and vice versa. + // Devices opened through these functions would have attribute + // "ExclusiveOwnership", which can be queried through + // #EdgeTpuContext::GetDeviceOptions(). + + // Creates a new Edge TPU context to be assigned to Tflite::Interpreter. The + // Edge TPU context is associated with the default TPU device. May be null + // if underlying device cannot be found or open. Caller owns the returned new + // context and should destroy the context either implicity or explicitly after + // all interpreters sharing this context are destroyed. + virtual std::unique_ptr NewEdgeTpuContext() = 0; + + // Same as above, but the created context is associated with the specified + // type. + virtual std::unique_ptr NewEdgeTpuContext( + DeviceType device_type) = 0; + + // Same as above, but the created context is associated with the specified + // type and device path. + virtual std::unique_ptr NewEdgeTpuContext( + DeviceType device_type, const std::string& device_path) = 0; + + // Same as above, but the created context is associated with the given device + // type, path and options. + // + // Available options are: + // - "Performance": ["Low", "Medium", "High", "Max"] (Default is "Max") + // - "Usb.AlwaysDfu": ["True", "False"] (Default is "False") + // - "Usb.MaxBulkInQueueLength": ["0",.., "255"] (Default is "32") + virtual std::unique_ptr NewEdgeTpuContext( + DeviceType device_type, const std::string& device_path, + const DeviceOptions& options) = 0; + + // Enumerates all connected Edge TPU devices. + virtual std::vector EnumerateEdgeTpu() const = 0; + + // OpenDevice family of functions return a shared_ptr to EdgeTpuContext, with + // the intention that the device can be shared among multiple software + // components. + // + // These functions seek shared ownership of the opened devices. As they + // cannot open devices already opened by NewEdgeTpuContext, and vice versa. + // The device would be closed after the last reference leaves scope. + + // Opens the default Edge TPU device. + // + // Multiple invocations of this function could return handle to the same + // device, but there is no guarantee. + // + // Returns a shared pointer to Edge TPU device. The shared_ptr could point to + // nullptr in case of error. + virtual std::shared_ptr OpenDevice() = 0; + + // Same as above, but the returned context is associated with the specified + // type. + virtual std::shared_ptr OpenDevice( + DeviceType device_type) = 0; + + // Same as above, but the returned context is associated with the specified + // type and device path. If path is empty, any device of the specified type + // could be returned. + virtual std::shared_ptr OpenDevice( + DeviceType device_type, const std::string& device_path) = 0; + + // Same as above, but the specified options would used to create a new context + // if no existing device is compatible with the specified type and path. + // + // If a device of compatible type and path can be found, the options could be + // ignored. It is the caller's responsibility to verify if the returned + // context is desirable, through #EdgeTpuContext::GetDeviceOptions(). + // + // Available options are: + // - "Performance": ["Low", "Medium", "High", "Max"] (Default is "Max") + // - "Usb.AlwaysDfu": ["True", "False"] (Default is "False") + // - "Usb.MaxBulkInQueueLength": ["0",.., "255"] (Default is "32") + virtual std::shared_ptr OpenDevice( + DeviceType device_type, const std::string& device_path, + const DeviceOptions& options) = 0; + + // Returns a snapshot of currently opened shareable devices. + // Exclusively owned Edge TPU devices cannot be returned here, as they're + // owned by unique pointers. + virtual std::vector> GetOpenedDevices() + const = 0; + + // Sets verbosity of operating logs related to edge TPU. + // Verbosity level can be set to [0-10], in which 10 is the most verbose. + virtual TfLiteStatus SetVerbosity(int verbosity) = 0; + + // Returns the version of EdgeTPU runtime stack. + virtual std::string Version() const = 0; + + protected: + // No deletion for this singleton instance. + virtual ~EdgeTpuManager() = default; +}; + +// External context to be assigned through +// tflite::Interpreter::SetExternalContext. +// One should get hold of either shared_ptr from EdgeTpuManager::OpenDevice, or +// unique_ptr from EdgeTpuManager::NewEdgeTpuContext, to ensure ownership, and +// avoid using this pointer directly. +// Functions in this interface are thread-safe. +class EdgeTpuContext : public TfLiteExternalContext { + public: + virtual ~EdgeTpuContext() = 0; + + // Returns a pointer to the device enumeration record for this device, + // if available. + virtual const EdgeTpuManager::DeviceEnumerationRecord& GetDeviceEnumRecord() + const = 0; + + // Returns a snapshot of the options used to open this + // device, and current state, if available. + // + // Supported attributes are: + // - "ExclusiveOwnership": present when it is under exclusive ownership + // (unique_ptr returned by NewEdgeTpuContext). + // - "IsReady": present when it is ready for further requests. + virtual EdgeTpuManager::DeviceOptions GetDeviceOptions() const = 0; + + // Returns true if the device is most likely ready to accept requests. + // When there are fatal errors, including unplugging of an USB device, the + // state of this device would be changed. + virtual bool IsReady() const = 0; +}; + +// Returns pointer to an instance of TfLiteRegistration to handle +// EdgeTPU custom ops, to be used with +// tflite::ops::builtin::BuiltinOpResolver::AddCustom +EDGETPU_EXPORT TfLiteRegistration* RegisterCustomOp(); + +// Inserts name of device type into ostream. Returns the modified ostream. +EDGETPU_EXPORT std::ostream& operator<<(std::ostream& out, + DeviceType device_type); + +} // namespace edgetpu + +// LINT.ThenChange(BUILD:version) + +#endif // TFLITE_PUBLIC_EDGETPU_H_ diff --git a/libedgetpu/edgetpu_c.h b/libedgetpu/edgetpu_c.h new file mode 100644 index 0000000..ead4b68 --- /dev/null +++ b/libedgetpu/edgetpu_c.h @@ -0,0 +1,106 @@ +/* +Copyright 2019 Google LLC + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +// This header defines C API to provide edge TPU support for TensorFlow Lite +// framework. It is only available for non-NNAPI use cases. +// +// Typical API usage from C++ code involves serveral steps: +// +// 1. Create tflite::FlatBufferModel which may contain edge TPU custom op. +// +// auto model = +// tflite::FlatBufferModel::BuildFromFile(model_file_name.c_str()); +// +// 2. Create tflite::Interpreter. +// +// tflite::ops::builtin::BuiltinOpResolver resolver; +// std::unique_ptr interpreter; +// tflite::InterpreterBuilder(model, resolver)(&interpreter); +// +// 3. Enumerate edge TPU devices. +// +// size_t num_devices; +// std::unique_ptr devices( +// edgetpu_list_devices(&num_devices), &edgetpu_free_devices); +// +// assert(num_devices > 0); +// const auto& device = devices.get()[0]; +// +// 4. Modify interpreter with the delegate. +// +// auto* delegate = +// edgetpu_create_delegate(device.type, device.path, nullptr, 0); +// interpreter->ModifyGraphWithDelegate({delegate, edgetpu_free_delegate}); +// +// 5. Prepare input tensors and run inference. +// +// interpreter->AllocateTensors(); +// .... (Prepare input tensors) +// interpreter->Invoke(); +// .... (Retrieve the result from output tensors) + +#ifndef TFLITE_PUBLIC_EDGETPU_C_H_ +#define TFLITE_PUBLIC_EDGETPU_C_H_ + +#include "tensorflow/lite/context.h" + +#ifdef __cplusplus +extern "C" { +#endif + +enum edgetpu_device_type { + EDGETPU_APEX_PCI = 0, + EDGETPU_APEX_USB = 1, +}; + +struct edgetpu_device { + enum edgetpu_device_type type; + const char* path; +}; + +struct edgetpu_option { + const char* name; + const char* value; +}; + +// Returns array of connected edge TPU devices. +struct edgetpu_device* edgetpu_list_devices(size_t* num_devices); + +// Frees array returned by `edgetpu_list_devices`. +void edgetpu_free_devices(struct edgetpu_device* dev); + +// Creates a delegate which handles all edge TPU custom ops inside +// `tflite::Interpreter`. Options must be available only during the call of this +// function. +TfLiteDelegate* edgetpu_create_delegate(enum edgetpu_device_type type, + const char* name, + const struct edgetpu_option* options, + size_t num_options); + +// Frees delegate returned by `edgetpu_create_delegate`. +void edgetpu_free_delegate(TfLiteDelegate* delegate); + +// Sets verbosity of operating logs related to edge TPU. +// Verbosity level can be set to [0-10], in which 10 is the most verbose. +void edgetpu_verbosity(int verbosity); + +// Returns the version of edge TPU runtime stack. +const char* edgetpu_version(); + +#ifdef __cplusplus +} // extern "C" +#endif + +#endif // TFLITE_PUBLIC_EDGETPU_C_H_ diff --git a/libedgetpu/throttled/aarch64/libedgetpu.so.1 b/libedgetpu/throttled/aarch64/libedgetpu.so.1 new file mode 120000 index 0000000..90ac68c --- /dev/null +++ b/libedgetpu/throttled/aarch64/libedgetpu.so.1 @@ -0,0 +1 @@ +libedgetpu.so.1.0 \ No newline at end of file diff --git a/libedgetpu/throttled/aarch64/libedgetpu.so.1.0 b/libedgetpu/throttled/aarch64/libedgetpu.so.1.0 new file mode 100755 index 0000000..ed06f00 Binary files /dev/null and b/libedgetpu/throttled/aarch64/libedgetpu.so.1.0 differ diff --git a/libedgetpu/throttled/armv6/libedgetpu.so.1 b/libedgetpu/throttled/armv6/libedgetpu.so.1 new file mode 120000 index 0000000..90ac68c --- /dev/null +++ b/libedgetpu/throttled/armv6/libedgetpu.so.1 @@ -0,0 +1 @@ +libedgetpu.so.1.0 \ No newline at end of file diff --git a/libedgetpu/throttled/armv6/libedgetpu.so.1.0 b/libedgetpu/throttled/armv6/libedgetpu.so.1.0 new file mode 100755 index 0000000..7b51d60 Binary files /dev/null and b/libedgetpu/throttled/armv6/libedgetpu.so.1.0 differ diff --git a/libedgetpu/throttled/armv7a/libedgetpu.so.1 b/libedgetpu/throttled/armv7a/libedgetpu.so.1 new file mode 120000 index 0000000..90ac68c --- /dev/null +++ b/libedgetpu/throttled/armv7a/libedgetpu.so.1 @@ -0,0 +1 @@ +libedgetpu.so.1.0 \ No newline at end of file diff --git a/libedgetpu/throttled/armv7a/libedgetpu.so.1.0 b/libedgetpu/throttled/armv7a/libedgetpu.so.1.0 new file mode 100755 index 0000000..8aaba50 Binary files /dev/null and b/libedgetpu/throttled/armv7a/libedgetpu.so.1.0 differ diff --git a/libedgetpu/throttled/k8/libedgetpu.so.1 b/libedgetpu/throttled/k8/libedgetpu.so.1 new file mode 120000 index 0000000..90ac68c --- /dev/null +++ b/libedgetpu/throttled/k8/libedgetpu.so.1 @@ -0,0 +1 @@ +libedgetpu.so.1.0 \ No newline at end of file diff --git a/libedgetpu/throttled/k8/libedgetpu.so.1.0 b/libedgetpu/throttled/k8/libedgetpu.so.1.0 new file mode 100755 index 0000000..491001c Binary files /dev/null and b/libedgetpu/throttled/k8/libedgetpu.so.1.0 differ diff --git a/packaging/edgetpu-runtime.manifest b/packaging/edgetpu-runtime.manifest new file mode 100644 index 0000000..017d22d --- /dev/null +++ b/packaging/edgetpu-runtime.manifest @@ -0,0 +1,5 @@ + + + + + diff --git a/packaging/edgetpu-runtime.spec b/packaging/edgetpu-runtime.spec new file mode 100644 index 0000000..13ac267 --- /dev/null +++ b/packaging/edgetpu-runtime.spec @@ -0,0 +1,123 @@ +Name: edgetpu-runtime +Summary: Support library for Edge TPU +Version: 12.1 +Release: 0 +Group: Development/Libraries +Packager: MyungJoo Ham +License: Apache-2.0 +Source0: edgetpu-runtime-%{version}.tar.gz +Source1001: edgetpu-runtime.manifest +Source1002: edgetpu.pc.in +ExclusiveArch: aarch64 x86_64 + +%description +This is repackaging of libedgetpu1-*.deb of Ubuntu at +https://packages.cloud.google.com/apt + +%prep +%setup -q +cp %{SOURCE1001} . +cp %{SOURCE1002} . + +%build +sed -i 's|@PREFIX@|%{_prefix}|' edgetpu.pc.in +sed -i 's|@EXEC_PREFIX@|%{_exec_prefix}|' edgetpu.pc.in +sed -i 's|@LIB_INSTALL_DIR@|%{_libdir}|' edgetpu.pc.in +sed -i 's|@INCLUDE_INSTALL_DIR@|%{_includedir}|' edgetpu.pc.in +sed -i 's|@VERSION@|%{version}|' edgetpu.pc.in +mv edgetpu.pc.in edgetpu.pc + +%install +mkdir -p %{buildroot}%{_libdir} +mkdir -p %{buildroot}%{_libdir}/pkgconfig +mkdir -p %{buildroot}%{_includedir}/edgetpu +%ifarch aarch64 +install -p -m 0755 libedgetpu/direct/aarch64/libedgetpu.so.1.0 %{buildroot}%{_libdir}/libedgetpu-max.so.1.0 +install -p -m 0755 libedgetpu/throttled/aarch64/libedgetpu.so.1.0 %{buildroot}%{_libdir}/libedgetpu-std.so.1.0 +%endif +%ifarch x86_64 +install -p -m 0755 libedgetpu/direct/k8/libedgetpu.so.1.0 %{buildroot}%{_libdir}/libedgetpu-max.so.1.0 +install -p -m 0755 libedgetpu/throttled/k8/libedgetpu.so.1.0 %{buildroot}%{_libdir}/libedgetpu-std.so.1.0 +%endif +install -p -m 0644 libedgetpu/*.h %{buildroot}%{_includedir}/ + +pushd %{buildroot}%{_libdir} +ln -sf libedgetpu-max.so.1.0 libedgetpu-max.so.1 +ln -sf libedgetpu-std.so.1.0 libedgetpu-std.so.1 +ln -sf libedgetpu.so.1 libedgetpu.so +popd +install -p -m 0644 edgetpu.pc %{buildroot}%{_libdir}/pkgconfig/ + +%package -n libedgetpu1 +Summary: Metapackage for edgetpu-runtime library +Requires: libedgetpu1-driver = %{version}-%{release} +Recommends: libedgetpu1-std = %{version}-%{release} +%description -n libedgetpu1 +Metapacakge for edgetpu-runtime library. +By default it uses "std" library instead of "max" library. +%files -n libedgetpu1 + +%package -n libedgetpu1-std +Summary: Standard throttling version of edgetpu-runtime library +Provides: libedgetpu1-driver = %{version}-%{release} +Conflicts: libedgetpu1-max +%description -n libedgetpu1-std +Standard throttled edgetpu-runtime library + +%files -n libedgetpu1-std +%manifest edgetpu-runtime.manifest +%license LICENSE +%{_libdir}/libedgetpu-std.so.1 +%{_libdir}/libedgetpu-std.so.1.0 + +%post -n libedgetpu1-std +pushd %{_libdir} +ln -sf libedgetpu-std.so.1 libedgetpu.so.1 +ln -sf libedgetpu-std.so.1.0 libedgetpu.so.1.0 +popd +/sbin/ldconfig +%postun -n libedgetpu1-std +# If it's uninstall +if [ "$1" == "0" ]; then +rm %{_libdir}/libedgetpu.so.1 +rm %{_libdir}/libedgetpu.so.1.0 +fi +/sbin/ldconfig + +%package -n libedgetpu1-max +Summary: Max-performance version of edgetpu-runtime library +Provides: libedgetpu1-driver = %{version}-%{release} +Conflicts: libedgetpu1-std +%description -n libedgetpu1-max +Non-throttled, max-performance edgetpu-runtime library + +%files -n libedgetpu1-max +%manifest edgetpu-runtime.manifest +%license LICENSE +%{_libdir}/libedgetpu-max.so.1 +%{_libdir}/libedgetpu-max.so.1.0 + +%post -n libedgetpu1-max +pushd %{_libdir} +ln -sf libedgetpu-max.so.1 libedgetpu.so.1 +ln -sf libedgetpu-max.so.1.0 libedgetpu.so.1.0 +popd +/sbin/ldconfig +%postun -n libedgetpu1-max +# If it's uninstall +if [ "$1" == "0" ]; then +rm %{_libdir}/libedgetpu.so.1 +rm %{_libdir}/libedgetpu.so.1.0 +fi +/sbin/ldconfig + +%package -n libedgetpu-dev +Summary: Developmental support for libedgetpu +Requires: libedgetpu1 = %{version}-%{release} +%description -n libedgetpu-dev +Developmental support for libedgetpu + +%files -n libedgetpu-dev +%{_libdir}/libedgetpu.so +%{_libdir}/pkgconfig/* +%{_includedir}/* diff --git a/packaging/edgetpu.pc.in b/packaging/edgetpu.pc.in new file mode 100644 index 0000000..d6825c6 --- /dev/null +++ b/packaging/edgetpu.pc.in @@ -0,0 +1,13 @@ +# Package Information for pkg-config, for libedgetpu users + +prefix=@PREFIX@ +exec_prefix=@EXEC_PREFIX@ +libdir=@LIB_INSTALL_DIR@ +includedir=@INCLUDE_INSTALL_DIR@ + +Name: edgetpu +Description: Dev Kit of Coral Edge TPU +Version: @VERSION@ +Requires: +Libs: -L${libdir} -ledgetpu +Cflags: -I${includedir}/edgetpu