tensorflow: Inspecting the Tensorflow 2.0 python package: unable to find or import modules

I’m analyzing the Tensorflow 2.0 Python package using importlib to loop through the symbols of the tensorflow package (see: https://github.com/galeone/rtf/blob/master/rtf/generators/base.py#L72 ).

There are certain modules that importlib is not able to load or find, e.g. the tf.losses tf.keras.losses, tf.optmizers (and all the module alias) etc. So far, recursively using importlib.import_module I get this structure

.
├── autograph
├── compat
│   ├── v1
│   │   ├── autograph
│   │   ├── config
│   │   ├── data
│   │   ├── distribute
│   │   ├── estimator
│   │   ├── io
│   │   ├── keras
│   │   │   ├── applications
│   │   │   ├── datasets
│   │   │   ├── mixed_precision
│   │   │   ├── optimizers
│   │   │   ├── preprocessing
│   │   │   └── wrappers
│   │   ├── layers
│   │   ├── lite
│   │   │   └── experimental
│   │   ├── lookup
│   │   ├── nn
│   │   ├── random
│   │   ├── saved_model
│   │   ├── tpu
│   │   ├── train
│   │   └── xla
│   └── v2
│       ├── autograph
│       ├── config
│       ├── data
│       ├── distribute
│       ├── estimator
│       ├── io
│       ├── keras
│       │   ├── applications
│       │   ├── datasets
│       │   ├── mixed_precision
│       │   ├── optimizers
│       │   ├── preprocessing
│       │   └── wrappers
│       ├── lookup
│       ├── random
│       ├── tpu
│       ├── train
│       └── xla
├── config
├── data
├── distribute
├── estimator
├── io
├── keras
│   ├── applications
│   ├── datasets
│   ├── mixed_precision
│   ├── optimizers
│   ├── preprocessing
│   └── wrappers
├── lite
│   ├── experimental
│   │   └── examples
│   │       └── lstm
│   ├── python
│   │   └── optimize
│   └── toco
├── lookup
├── random
├── tools
│   └── docs
├── tpu
├── train
└── xla

that is incomplete.

Is there a way to use the importlib module to get the complete structure of the tensorflow package and to successfully load the module?

Here’s a minimum reproducible example:

Install

pip install --upgrade tf-nightly-2.0-preview

Loading losses

import importlib 
importlib.import_module("tensorflow.losses")

Produces the error: “ModuleNotFoundError: No module named ‘tensorflow.losses’”

This happens only in Tensorflow 2.0, if I install Tensorflow 1.14 I can inspect more or less every package without any failure (haven’t tested in depth).

Disclosure: this is a crosspost from https://groups.google.com/a/tensorflow.org/forum/#!topic/developers/jibh60HPGA0

Greetings

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 20 (8 by maintainers)

Most upvoted comments

Hi,

Yes, lazy loading makes the modules only available via getattr

You can also try looking into sys.modules, even if the module is lazy loaded it should be as a key there (though probably not all of its submodules)

Does something like the following work for your usecase @galeone? tf = importlib.import_module(“tensorflow”) getattr(tf, “losses”)

As far as I know, you are correct. @annarev just in case something has changed since I last touched that code.