tensorflow: Pylint incorrectly identifies tensorflow public API functions in tensorflow 2.2+
System information
- Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Apart from the example below, no
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04/OSX 10.15.6
- Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: ❌
- TensorFlow installed from (source or binary): binary
- TensorFlow version (use command below): 2.2+
- Python version: 3.7.4
- Bazel version (if compiling from source): ❌
- GCC/Compiler version (if compiling from source): ❌
- CUDA/cuDNN version: ❌
- GPU model and memory: ❌
Describe the current behavior
When using pylint, a lot of functions from the public API are misidentified.
For example, the public api function tf.split, which is publicly defined as tensorflow/python/ops/array_ops.split is misidentified (I think as tensorflow/python/ops/gen_array_ops.split).
Other examples include tf.random.uniform, tf.concat and the list goes on.
Let’s take the code snipped below:
example.py:
import tensorflow as tf
tensor = tf.random.uniform((2, 4), minval=0, maxval=256)
tf.split(tensor, num_or_size_splits=2, axis=-1)
This is perfectly fine code, it runs as expected.
However, when running pylint both functions are misidentified and lots of linting errors are raised.
Running pylint on the module (pylint -E example.py) gives:
example.py:3:9: E1123: Unexpected keyword argument 'minval' in function call (unexpected-keyword-arg)
example.py:3:9: E1123: Unexpected keyword argument 'maxval' in function call (unexpected-keyword-arg)
example.py:3:9: E1120: No value for argument 'dtype' in function call (no-value-for-parameter)
example.py:4:0: E1123: Unexpected keyword argument 'num_or_size_splits' in function call (unexpected-keyword-arg)
example.py:4:0: E1124: Argument 'axis' passed by position and keyword in function call (redundant-keyword-arg)
example.py:4:0: E1120: No value for argument 'value' in function call (no-value-for-parameter)
example.py:4:0: E1120: No value for argument 'num_split' in function call (no-value-for-parameter)
Describe the expected behavior
Running pylint -E example.py should not give any errors.
Standalone code to reproduce the issue
import tensorflow as tf
tensor = tf.random.uniform((2, 4), minval=0, maxval=256)
tf.split(tensor, num_or_size_splits=2, axis=-1)
Other info / logs
This occurs with tensorflow>=2.2.0.
Previous versions of tensorflow 2.X (e.g. tensorflow 2.1.X and tensorflow 2.0.X) do not have these problems.
I have used pylint==2.6.0 for the example, but previous versions have the same behaviour.
One of the things that might have caused this (just a guess) is the upgrade to a new version of gast that occurred in tensorflow 2.2, where they went from gast==0.2.2 to gast==0.3.3.
Now, I know that this issue is not a code breaking issue, but it is a workflow breaking issue when using tensorflow in a professional setting. For example, one of the requirements for passing all steps in the CI may be running pylint, which now fails. Pylint allow disabling errors for specific third-party packages, so really the only solution is to add a pylint: disable=... comment every time you use a tensorflow function which is misidentified or to disable pylint for the project all together. Both options aren’t desirable.
This issue was also raised in the pylint repo (https://github.com/PyCQA/pylint/issues/3613) and probably also related to https://github.com/PyCQA/pylint/issues/3596. But I don’t think these issues belong in the pylint repo(orastroid` for that matter), but here in the tensorflow repo as it’s probably caused by the import structure of tensorflow.
One lead might be that a wildstar import overwrites functions. An examples might be the wildstar import in https://github.com/tensorflow/tensorflow/blob/v2.2.0/tensorflow/python/ops/array_ops.py#L40 which overwrites tensorflow/python/ops/array_ops.split with the starred import tensorflow/python/ops/gen_array_ops.split. I’m not sure, but not performing wildstar imports might solve this linter problem.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 5
- Comments: 16 (9 by maintainers)
@mihaimaruseac Please reopen this.
Was able to reproduce the issue with TF v2.3 and TF-nightly. Please find the gist of it here. Thanks!
@ScaleRunner Yes let’s see what they want to do after having an overview on the LOG.
I did some more digging with older versions of pylint.
No errors on
example.py:1.xrelease (1.9.5): ✅2.0.1up until2.2.3gave an error that probably has something to do with the astroid dependency https://github.com/PyCQA/astroid/issues/6492.3.3: ✅2.4.4: ✅2.5.3: ❌2.6.0: ❌Now these only check the minimal example above, but might give an indication of what might still work.
I saw that 69a975e6a35279a91035f601b36f6771331d6619 upgrades to
2.6.0and the CI failed. Maybe we could try upgrading to one of the versions above?