tensorflow: Code completion of Keras Module no longer work since 2.6

Thanks for the information.

Based on my rough knowledge for auto completion, I think the issue probably caused by difference between package/directory/module.

Most of the IDE will try to auto complete by the package tree structure, so if there is a directory with init under tensorflow call “keras”, then it will resolve correctly when you do “from tensorflow.keras.xxxx import yyy”. This was working before 2.7 since Keras was part of the Tensorflow PIP package.

Note that the “import tensorflow as tf; tf.keras.xxx.yyy” is resolved differently. tf.keras is firstly resolved into the logic in the tensorflow/init.py, and then the keras becomes the “keras.apis._v2.keras”, which resolved correctly to the keras PIP package.

Before TF 2.7, I think the we did some manual mimic for the keras APIs to tensorflow/keras folder from tensorflow/python/keras/apis, so that it the package level auto completion works, but that is not the case anymore after 2.7 since the keras is a standalone package.

As a quick test on my local, I manually created a symlink under venv/lib/python3.10/site-packages/tensorflow by ln -s ../keras/api/_v2/keras/ keras (assuming you have tensorflow and keras pip package installed properly), and then the IDE can resolve the package level auto completion correctly.

I am not sure if this fix can be applied to TF direclty, since the symlink is not well supported on windows, and might be in different location if the package is installed differently.

@cpuimage Oh nice! maybe my work is something wrong. I will read again your posts.

By the way, I found good feature in PEP563. https://www.python.org/dev/peps/pep-0563/#runtime-annotation-resolution-and-type-checking

There are example code in the document.

import typing

if typing.TYPE_CHECKING:
    import expensive_mod

def a_func(arg: expensive_mod.SomeClass) -> None:
    a_var: expensive_mod.SomeClass = arg
    ...

This code is work good in the VSCode. (I’m not check on the PyCharm)

import typing
if typing.TYPE_CHECKING:
  from tensorflow import keras
else:
  _keras_module = "keras.api._v2.keras"
  keras = _LazyLoader("keras", globals(), _keras_module)
  _module_dir = _module_util.get_parent_dir_for_name(_keras_module)
  if _module_dir:
    _current_module.__path__ = [_module_dir] + _current_module.__path__
  setattr(_current_module, "keras", keras)

good job. it work good in PyCharm.

@cpuimage Oh nice! maybe my work is something wrong. I will read again your posts.

By the way, I found good feature in PEP563. https://www.python.org/dev/peps/pep-0563/#runtime-annotation-resolution-and-type-checking

There are example code in the document.

import typing

if typing.TYPE_CHECKING:
    import expensive_mod

def a_func(arg: expensive_mod.SomeClass) -> None:
    a_var: expensive_mod.SomeClass = arg
    ...

This code is work good in the VSCode. (I’m not check on the PyCharm)

import typing
if typing.TYPE_CHECKING:
  from tensorflow import keras
else:
  _keras_module = "keras.api._v2.keras"
  keras = _LazyLoader("keras", globals(), _keras_module)
  _module_dir = _module_util.get_parent_dir_for_name(_keras_module)
  if _module_dir:
    _current_module.__path__ = [_module_dir] + _current_module.__path__
  setattr(_current_module, "keras", keras)

I can’t believe so few people are complaining about this, what IDE are other people using to write tensorflow code, notepad++? 😄

Vim 😛

But coming back to the issue, I think this is due to the keras moving to a separate repository. Probably the solution would also come from there.

Having said that, all the import should always work, since the auto complete and python import works differently. Tensorflow has extra logic in the tensorflow/init.py to ensure all the public API are populated correctly via the _api folder, instead of directly using the directory structure.

@cpuimage Thanks. A module completion such as from tensorflow.keras import ~ is works good. But tf.keras.~ is not work code completions.

Can you work completion about tf.keras.~ on the PyCharm. My VSCode is not work code completion after changes.

Maybe PyCharm is more intelligence.