edgetpu: RuntimeError: Encountered unresolved custom op: edgetpu-custom-op.Node number 0 (edgetpu-custom-op) failed to prepare.

My setup:

Using a Coral M.2 Accelerator A+E key in a device running Ubuntu 16.04 The model file is a MobileNet v2 SSD object detection model compiled for edgetpu, and I have successfully run this model file without issue on other similar devices.

Running:

import tflite_runtime.interpreter as tflite

model = tflite.Interpreter(
    "/src/object_detection/model/model_edgetpu.tflite", 
    experimental_delegates=[tflite.load_delegate('libedgetpu.so.1')]
)
model.allocate_tensors()

The model.allocate_tensors() call results in the following RuntimeError:

RuntimeError: Encountered unresolved custom op: edgetpu-custom-op.Node number 0 (edgetpu-custom-op) failed to prepare.

Given that I’ve seen this code + model work on other devices, I’m guessing that the issue here is either a hardware connection or perhaps a corrupt/missing installation. Under what conditions should I expect to see this error? Do you know what’s likely causing it now?

Thanks!

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 15

Most upvoted comments

@mrusme oh, that’s most likely because EDGETPU_SHARED_LIB didn’t mapped to the correct platform

Namburger on Sep 10, 2020

Had the same issue for a Python script I’ve been working on, solved it by replacing the tflite.Interpreter(... call from @BotScutters (up there in the issue description) with a call to this function:

def make_interpreter(model_file):
  model_file, *device = model_file.split('@')
  return tflite.Interpreter(
      model_path=model_file,
      experimental_delegates=[
          tflite.load_delegate(EDGETPU_SHARED_LIB,
                               {'device': device[0]} if device else {})
      ])

Also added the following to the script head:

EDGETPU_SHARED_LIB = {
  'Linux': 'libedgetpu.so.1',
  'Darwin': 'libedgetpu.1.dylib',
  'Windows': 'edgetpu.dll'
}[platform.system()]

That solved the issue, at least in my case.

mrusme on Sep 9, 2020

Doesn’t seem to fix the issue, unfortunately. The device has been taken offline, so I won’t be able to investigate this for a bit, but once it’s back up I should have improved access to it and will revisit this issue. Thanks Nam!

BotScutters on Jul 8, 2020

Good morning, @Namburger, I just got some data regarding this issue.

The response from the two commands you asked to run:

➜ dpkg -l | grep libedgetpu
ii libedgetpu1-std:amd64 14.0 amd64 Support library for Edge TPU

➜ python3 -c 'print(_import("tflite_runtime").version_)'
2.1.0.post1

And here’s an excerpt from the strace log from when it was trying to load libedgetpu.so.1, and I’ve send you the complete log separately. It looks like it’s searching in the wrong directories, and actually in some directories associated with OpenVINO (we’ve run previous experiments with an Intel Neural Compute Stick on this device), so that seems interesting. What determines the path(s) it checks while trying to load libedgetpu?

open("/opt/intel//computer_vision_sdk_2018.5.455/deployment_tools/inference_engine/external/hddl/lib/libedgetpu.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/opt/intel//computer_vision_sdk_2018.5.455/deployment_tools/inference_engine/external/gna/lib/libedgetpu.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/opt/intel//computer_vision_sdk_2018.5.455/deployment_tools/inference_engine/external/mkltiny_lnx/lib/libedgetpu.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/opt/intel//computer_vision_sdk_2018.5.455/deployment_tools/inference_engine/external/omp/lib/libedgetpu.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/opt/intel//computer_vision_sdk_2018.5.455/deployment_tools/inference_engine/lib/ubuntu_16.04/intel64/libedgetpu.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("tls/x86_64/libedgetpu.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("tls/libedgetpu.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("x86_64/libedgetpu.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("libedgetpu.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 8
fstat(8, {st_mode=S_IFREG|0644, st_size=130228, ...}) = 0
mmap(NULL, 130228, PROT_READ, MAP_PRIVATE, 8, 0) = 0x7f751b6fd000
close(8)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/usr/lib/x86_64-linux-gnu/libedgetpu.so.1", O_RDONLY|O_CLOEXEC) = 8
read(8, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832
fstat(8, {st_mode=S_IFREG|0644, st_size=942584, ...}) = 0
mmap(NULL, 953984, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 8, 0) = 0x7f7515bc8000

Thanks again for all of your help on this!

BotScutters on Jul 2, 2020