compute-engine: lce_converter build from source error: /tmp/lce_android

perfect.

@Tombana, @AdamHillier thank you all for the support. still wonder why such an error during compilation is thrown by newer versions than bazel 3.1.0 but I consider this issue resolved

Hi @simonmaurer, sorry for the delay in getting back to you. Thanks for investigating the Bazel errors further 😃

My gut instinct, without any evidence, is that this is a bug in Bazel that has been introduced, since as you say that release is meant to be backwards-compatible. Unfortunately, it’s hard to have any insight in how to get this working with newer Bazel versions without actually trying to do upgrade. Since TensorFlow master is currently on Bazel 3.7.2, we will certainly have to address this when we update our TF dependency (which we probably do when/soon after the TF 2.5 release branch gets cut).

It’s true that our approach of hard-coding this link to /tmp/lce_android is not the approach TF takes (perhaps it once was); they have dynamic configuration of paths to Android SDK/NDK environments as part of their configure.py, so likely the easiest thing for us to do would be to copy these functions into our own configure.py and delete the hard-coded lines. Of course, that’s actually a much nicer solution anyway since it’s much less brittle and allows interactive configuration.

Happy to hear that you got it working!

one thing I noticed is when I’m directly in the root folder of the larq_compute_engine repo. launching the python interpreter and importing larq_compute_engine results in:

import larq_compute_engine as lce Traceback (most recent call last): File “”, line 1, in File “…/larq_compute_engine/larq_compute_engine/init.py”, line 1, in from larq_compute_engine.mlir.python.converter import convert_keras_model File “…/larq_compute_engine/mlir/python/converter.py”, line 6, in from larq_compute_engine.mlir._graphdef_tfl_flatbuffer import ( ModuleNotFoundError: No module named ‘larq_compute_engine.mlir._graphdef_tfl_flatbuffer’

is that expected behavior?

This is expected. When in the root folder, the python import statement gives the larq_compute_engine folder higher priority than the package, and the _graphdef_tfl_flatbuffer.so (compiled library) file is not present there, it is only present in the compiled package.

so built again with exactly the aforementioned flags

Hmm, that is weird. I don’t know why it would default to a higher value than your native OS version 😕

That said, we successfully build release wheels for MacOS 10.13, you can see the command flags we pass in order to do that here:

https://github.com/larq/compute-engine/blob/579c8fb02f69d6f08e1581206c7b9b00b5394028/.github/workflows/release.yml#L134-L135

this time the compilation resulted in a correctlarq_compute_engine-0.5.0-cp37-cp37m-macosx_10_13_x86_64.whl, installing it via pip in the virtual environment was successful. importing larq_compute_engine in the python interpreter works as expected. very nice!

one thing I noticed is when I’m directly in the root folder of the larq_compute_engine repo. launching the python interpreter and importing larq_compute_engine results in:

import larq_compute_engine as lce Traceback (most recent call last): File “<stdin>”, line 1, in <module> File “…/larq_compute_engine/larq_compute_engine/init.py”, line 1, in <module> from larq_compute_engine.mlir.python.converter import convert_keras_model File “…/larq_compute_engine/mlir/python/converter.py”, line 6, in <module> from larq_compute_engine.mlir._graphdef_tfl_flatbuffer import ( ModuleNotFoundError: No module named ‘larq_compute_engine.mlir._graphdef_tfl_flatbuffer’

is that expected behavior?

will compile again with these optimization flags

Hmm, that is weird. I don’t know why it would default to a higher value than your native OS version 😕

That said, we successfully build release wheels for MacOS 10.13, you can see the command flags we pass in order to do that here: https://github.com/larq/compute-engine/blob/579c8fb02f69d6f08e1581206c7b9b00b5394028/.github/workflows/release.yml#L134-L135

within the virtual environment the which python command states :

Python 3.7.7

so that should be correct when executing python configure.py

I’ll do a reconfigure (including the latest bazelisk version from Homebrew that chooses the right bazel version 3.1.0 given the .bazelversion in larq_compute_engine repo) and report back.