tensorflow: Pylint incorrectly identifies tensorflow public API functions in tensorflow 2.2+

System information

  • Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Apart from the example below, no
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04/OSX 10.15.6
  • Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: ❌
  • TensorFlow installed from (source or binary): binary
  • TensorFlow version (use command below): 2.2+
  • Python version: 3.7.4
  • Bazel version (if compiling from source): ❌
  • GCC/Compiler version (if compiling from source): ❌
  • CUDA/cuDNN version: ❌
  • GPU model and memory: ❌

Describe the current behavior When using pylint, a lot of functions from the public API are misidentified. For example, the public api function tf.split, which is publicly defined as tensorflow/python/ops/array_ops.split is misidentified (I think as tensorflow/python/ops/gen_array_ops.split).

Other examples include tf.random.uniform, tf.concat and the list goes on.

Let’s take the code snipped below:

example.py:

import tensorflow as tf

tensor = tf.random.uniform((2, 4), minval=0, maxval=256)
tf.split(tensor, num_or_size_splits=2, axis=-1)

This is perfectly fine code, it runs as expected.

However, when running pylint both functions are misidentified and lots of linting errors are raised. Running pylint on the module (pylint -E example.py) gives:

example.py:3:9: E1123: Unexpected keyword argument 'minval' in function call (unexpected-keyword-arg)
example.py:3:9: E1123: Unexpected keyword argument 'maxval' in function call (unexpected-keyword-arg)
example.py:3:9: E1120: No value for argument 'dtype' in function call (no-value-for-parameter)
example.py:4:0: E1123: Unexpected keyword argument 'num_or_size_splits' in function call (unexpected-keyword-arg)
example.py:4:0: E1124: Argument 'axis' passed by position and keyword in function call (redundant-keyword-arg)
example.py:4:0: E1120: No value for argument 'value' in function call (no-value-for-parameter)
example.py:4:0: E1120: No value for argument 'num_split' in function call (no-value-for-parameter)

Describe the expected behavior Running pylint -E example.py should not give any errors.

Standalone code to reproduce the issue

import tensorflow as tf

tensor = tf.random.uniform((2, 4), minval=0, maxval=256)
tf.split(tensor, num_or_size_splits=2, axis=-1)

Other info / logs This occurs with tensorflow>=2.2.0. Previous versions of tensorflow 2.X (e.g. tensorflow 2.1.X and tensorflow 2.0.X) do not have these problems.

I have used pylint==2.6.0 for the example, but previous versions have the same behaviour.

One of the things that might have caused this (just a guess) is the upgrade to a new version of gast that occurred in tensorflow 2.2, where they went from gast==0.2.2 to gast==0.3.3.

Now, I know that this issue is not a code breaking issue, but it is a workflow breaking issue when using tensorflow in a professional setting. For example, one of the requirements for passing all steps in the CI may be running pylint, which now fails. Pylint allow disabling errors for specific third-party packages, so really the only solution is to add a pylint: disable=... comment every time you use a tensorflow function which is misidentified or to disable pylint for the project all together. Both options aren’t desirable.

This issue was also raised in the pylint repo (https://github.com/PyCQA/pylint/issues/3613) and probably also related to https://github.com/PyCQA/pylint/issues/3596. But I don’t think these issues belong in the pylint repo(orastroid` for that matter), but here in the tensorflow repo as it’s probably caused by the import structure of tensorflow.

One lead might be that a wildstar import overwrites functions. An examples might be the wildstar import in https://github.com/tensorflow/tensorflow/blob/v2.2.0/tensorflow/python/ops/array_ops.py#L40 which overwrites tensorflow/python/ops/array_ops.split with the starred import tensorflow/python/ops/gen_array_ops.split. I’m not sure, but not performing wildstar imports might solve this linter problem.

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 5
  • Comments: 16 (9 by maintainers)

Most upvoted comments

@mihaimaruseac Please reopen this.

Was able to reproduce the issue with TF v2.3 and TF-nightly. Please find the gist of it here. Thanks!

@ScaleRunner Yes let’s see what they want to do after having an overview on the LOG.

I did some more digging with older versions of pylint.

No errors on example.py:

  • last 1.x release (1.9.5): ✅
  • 2.0.1 up until 2.2.3 gave an error that probably has something to do with the astroid dependency https://github.com/PyCQA/astroid/issues/649
  • 2.3.3: ✅
  • 2.4.4: ✅
  • 2.5.3: ❌
  • 2.6.0: ❌

Now these only check the minimal example above, but might give an indication of what might still work.

I saw that 69a975e6a35279a91035f601b36f6771331d6619 upgrades to 2.6.0 and the CI failed. Maybe we could try upgrading to one of the versions above?