edgetpu: tflite-runtime (2.12 and newer) no longer supports EdgeTPU

Description

While tflite-runtime can be successfully compile from source (using the full TensorFlow source) the whl produced does not have support for EdgeTPU (it’s not recognized within tflite-runtime). In the past coral.ai used to provide custom-made tflite-runtime with EdgeTPU support, yet, those are either no longer available, or they are built again obsolete versions of TF/Python. Furthermore, while coral.ai page indicate that such whl can be found in here, that page refers back to the original coral.ai page for support for EdgeTPU. This endless loop is confusing and pointless. The version on Pypi doesn’t have support for EdgeTPU either. One may suggest the use of PyCoral. Well PyCoral uses EdgeTPU, and without it it just doesn’t work.

The last version of TensorFlow Lite to support EdgeTPU is 2.11.1. Any later version leads to a segmentation fault. Which leaves me with no way to run a model on the EdgeTPU with a recent release of TF (> 2.11.x). I tested every release since then (currently up to 2.15.0) and none works.

it would be great if Google would acknowledge that development (or at least maintenance) of TFlite for EdgeTPU is still ongoing, or if its dead.

Click to expand!

Issue Type

Build/Install, Support, Documentation Feature Request, Documentation Bug

Operating System

Linux, Ubuntu, Mac OS

Coral Device

USB Accelerator, M.2 Accelerator A+E, M.2 Accelerator B+M

Other Devices

Raspberry Pi 3

Programming Language

Python 3.8, Python 3.9, Other

Relevant Log Output

No response

About this issue

Original URL
State: open
Created 7 months ago
Reactions: 2
Comments: 26

Most upvoted comments

OK, I reuploaded debs that should work. Raspbian is still based on bullseye, not on bookworm, hence the libc6 issue.

Please let me know if you run into issues.

feranick on Jan 29, 2024

I just became aware of this issue. This is because I cross-compiled from Ubuntu (with the newer library). I am setting a VM with debian at the moment…

feranick on Jan 29, 2024

Builds for aarch64 and armv7a are successful using my llibedgetpu fork. They are compiled with Docker+Bazel.

Deb packages (amd64, arm64, armv7a) are available. Refer to updated README.md there for build instructions.

feranick on Jan 29, 2024

A bit of a breakthrough. Not only gasket-dkms, but also is libedgetpu, which is what is used to compile the edgetpu drivers. The liibedgetpu drivers repo is linked directly to TF, and the version that is linked against is TF2.5.

So I forked both gasket-dkms and libedgetpu and added support for newer versions of TF, at the moment up to TF2.13.1. After compiling and installing gasket-dkms (needed only for the m2-based edgeTPU boards) and libedgetpu, edgeTPU works perfectly with tflite_runtime 2.13.1, 2.14.1, and 2.15.0. No more segmentation faults.

Currently libedgetpu does not compile with TF 2.14.0 and newer due to some python issues, but I am on working on.

All this is to say that edgetpu is back and working fine with support for modern platforms.

NOTE: the recommended way to compile libedgeetpu is via Bazel, which needs to be in the same version as TF is compiled against.

Currently tested and working with Ubuntu 22.04 x86_64 and MacOS 14.3.

feranick on Jan 29, 2024

I’d note that gasket-dkms is only needed for the edgetpu in the M2 format, which is the one I am using (not with the USB-accelerator). I have not tried the USB accelerator on linux in recent times.

feranick on Jan 27, 2024

Actually, gasket-dkms is open-source and support for kernel 6.4+ has been added. I can confirm that a very quick build and reboot restores functionality of the edgetpu (with tflite_runtime <=2.11.1).

https://github.com/google/gasket-driver

feranick on Jan 27, 2024

@feranick Huge thanks but sadly since I’m on a raspi 5 the wheels won’t work. Did you use this guide to make the wheel?

Yes, pretty much to the letter.

feranick on Jan 24, 2024

@feranick do you have a example on how you got it working? I’ve tried with many different tflite_runtime versions and tensorflow versions and it continues to segfault. I’ve posted a issue here: #821 where there’s more detail.

I don’t have something simple I can share. However I use it within a larger software that I wrote:

https://github.com/feranick/SpectralMachine/blob/master/SpectraKeras/libSpectraKeras.py

Here’s the wheel for the version that works for me (Ubuntu 22.04):

https://www.dropbox.com/scl/fi/uyif5p9g1pgn85e4u80wo/tflite_runtime-2.11.1-cp310-cp310-linux_x86_64.whl?rlkey=r7am6q6k854k0snqatde14wju&dl=0

or for MacOS:

https://www.dropbox.com/scl/fi/wikg9kdq1kij623dil66t/tflite_runtime-2.11.1-cp311-cp311-macosx_14_0_x86_64.whl?rlkey=u4e899j3tjqv15p8edr5eftfx&dl=0

feranick on Jan 23, 2024