agents: AttributeError: 'NoneType' object has no attribute 'dumps' for tf_py_environment

OS: 5.6.15-arch1-1 (Arch Linux) Python: 3.8 Pip list:

absl-py 0.9.0
appdirs 1.4.4
astor 0.8.1
astroid 2.4.1
astunparse 1.6.3
backcall 0.1.0
beautifulsoup4 4.9.1
Brlapi 0.7.0
btrfsutil 1.2.0
CacheControl 0.12.6
cachetools 4.1.0
ceph-volume 1.0.0
cephfs 2.0.0
cephfs-shell 0.0.1
certifi 2020.4.5.2
chardet 3.0.4
cloudpickle 1.3.0
colorama 0.4.3
contextlib2 0.6.0.post1
cycler 0.10.0
decorator 4.4.2
distlib 0.3.0
distro 1.5.0
dm-tree 0.1.5
EasyProcess 0.3
future 0.18.2
gast 0.3.3
gin-config 0.3.0
google 2.0.3
google-auth 1.17.0
google-auth-oauthlib 0.4.1
google-pasta 0.2.0
grpcio 1.29.0
gym 0.17.2
h5py 2.10.0
html5lib 1.0.1
idna 2.9
imageio 2.8.0
ipython 7.15.0
ipython-genutils 0.2.0
isort 4.3.21
jedi 0.17.0
Jinja2 2.11.2
joblib 0.15.1
Keras 2.3.1
Keras-Applications 1.0.8
Keras-Preprocessing 1.1.2
kiwisolver 1.2.0
lazy-object-proxy 1.4.3
lensfun 0.3.95
louis 3.14.0
Markdown 3.2.2
MarkupSafe 1.1.1
matplotlib 3.2.1
mccabe 0.6.1
mock 4.0.2
msgpack 1.0.0
numpy 1.18.5
oauthlib 3.1.0
opt-einsum 3.2.1
ordered-set 3.1.1
packaging 20.4
pandas 1.0.4
parso 0.7.0
pep517 0.8.2
pexpect 4.8.0
pickleshare 0.7.5
Pillow 7.1.2
pip 20.0.2
progress 1.5
prompt-toolkit 3.0.5
protobuf 3.12.2
ptyprocess 0.6.0
pwquality 1.4.2
pyasn1 0.4.8
pyasn1-modules 0.2.8
pybind11 2.5.0
pycairo 1.19.1
pyglet 1.5.0
Pygments 2.6.1
PyGObject 3.36.1
pylint 2.5.2
PyOpenGL 3.1.5
pyparsing 2.4.7
python-dateutil 2.8.1
python-libtorrent 1.2.7
pytoml 0.1.21
pytz 2020.1
PyVirtualDisplay 1.3.2
pyxdg 0.26
PyYAML 5.3.1
rados 2.0.0
rbd 2.0.0
Reflector 2020.3.21.11.40.36 requests 2.23.0
requests-oauthlib 1.3.0
retrying 1.3.3
rgw 2.0.0
rsa 4.1
scikit-learn 0.23.1
scipy 1.4.1
setuptools 47.1.1
six 1.15.0
sklearn 0.0
soupsieve 2.0.1
tb-nightly 2.3.0a20200611
team 1.0
tensorboard-plugin-wit 1.6.0.post3
termcolor 1.1.0
tf-agents-nightly 0.6.0.dev20200611
tf-estimator-nightly 2.3.0.dev2020061101 tf-nightly 2.3.0.dev20200611
tfp-nightly 0.11.0.dev20200611 threadpoolctl 2.1.0
toml 0.10.1
torch 1.5.0
traitlets 4.3.3
urllib3 1.25.9
wcwidth 0.1.9
webencodings 0.5.1
Werkzeug 1.0.1
wheel 0.34.2
wrapt 1.12.1

Following this guide: https://www.tensorflow.org/agents/tutorials/1_dqn_tutorial

These lines: train_env = tf_py_environment.TFPyEnvironment(train_py_env) eval_env = tf_py_environment.TFPyEnvironment(eval_py_env)

Produce this error: Exception ignored in: <function Pool.__del__ at 0x7f4ba9686670> Traceback (most recent call last): File "/usr/lib/python3.8/multiprocessing/pool.py", line 268, in __del__ File "/usr/lib/python3.8/multiprocessing/queues.py", line 362, in put AttributeError: 'NoneType' object has no attribute 'dumps' Exception ignored in: <function Pool.__del__ at 0x7f4ba9686670> Traceback (most recent call last): File "/usr/lib/python3.8/multiprocessing/pool.py", line 268, in __del__ File "/usr/lib/python3.8/multiprocessing/queues.py", line 362, in put AttributeError: 'NoneType' object has no attribute 'dumps'

About this issue

Original URL
State: open
Created 4 years ago
Reactions: 9
Comments: 15 (2 by maintainers)

Most upvoted comments

I’m facing the same problem. When I run:

env = suite_gym.load('CartPole-v0')
env = tfa.environments.tf_py_environment.TFPyEnvironment(env)

I get:

2021-02-28 16:36:26.147133: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
Exception ignored in: <function Pool.__del__ at 0x7f66b923b280>
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 268, in __del__
  File "/usr/lib/python3.8/multiprocessing/queues.py", line 362, in put
AttributeError: 'NoneType' object has no attribute 'dumps'

Process finished with exit code 0

My venv:

Keras-Preprocessing | 1.1.2 | 1.1.2
Markdown | 3.3.4 | 3.3.4
Pillow | 7.2.0 | 8.1.0
Werkzeug | 1.0.1 | 1.0.1
absl-py | 0.11.0 | 0.11.0
astunparse | 1.6.3 | 1.6.3
cachetools | 4.2.1 | 4.2.1
certifi | 2020.12.5 | 2020.12.5
chardet | 4.0.0 | 4.0.0
cloudpickle | 1.6.0 | 1.6.0
decorator | 4.4.2 | 4.4.2
dm-tree | 0.1.5 | 0.1.5
flatbuffers | 1.12 | 1.12
future | 0.18.2 | 0.18.2
gast | 0.3.3 | 0.4.0
gin-config | 0.4.0 | 0.4.0
google-auth | 1.27.0 | 1.27.0
google-auth-oauthlib | 0.4.2 | 0.4.2
google-pasta | 0.2.0 | 0.2.0
grpcio | 1.32.0 | 1.36.0
gym | 0.18.0 | 0.18.0
h5py | 2.10.0 | 3.1.0
idna | 2.10 | 3.1
numpy | 1.19.5 | 1.20.1
oauthlib | 3.1.0 | 3.1.0
opt-einsum | 3.3.0 | 3.3.0
pip | 21.0 | 21.0.1
protobuf | 3.15.3 | 3.15.3
pyasn1 | 0.4.8 | 0.4.8
pyasn1-modules | 0.2.8 | 0.2.8
pyglet | 1.5.0 | 1.5.15
requests | 2.25.1 | 2.25.1
requests-oauthlib | 1.3.0 | 1.3.0
rsa | 4.7.2 | 4.7.2
scipy | 1.6.1 | 1.6.1
setuptools | 52.0.0 | 54.0.0
six | 1.15.0 | 1.15.0
tensorboard | 2.4.1 | 2.4.1
tensorboard-plugin-wit | 1.8.0 | 1.8.0
tensorflow | 2.4.1 | 2.4.1
tensorflow-estimator | 2.4.0 | 2.4.0
tensorflow-probability | 0.12.1 | 0.12.1
termcolor | 1.1.0 | 1.1.0
tf-agents | 0.7.1 | 0.7.1
typing-extensions | 3.7.4.3 | 3.7.4.3
urllib3 | 1.26.3 | 1.26.3
wheel | 0.36.2 | 0.36.2
wrapt | 1.12.1 | 1.12.1

JacobFV on Feb 28, 2021

Hi, @ebrevdo and everyone,

I think I find out the location where is bug happens. First you need to set isolation = True to create a ThreadPool.

Then at the very last of your script, add train_env.close().

BUT, BUT, this is not the end of story,

TFPyEnvironment.close methods call self._pool.join() first then self._pool.close() which raise another error: Pool is still running. https://github.com/tensorflow/agents/blob/74cff73d8d985bbf08d7d2dac1b675e3c4bd82c2/tf_agents/environments/tf_py_environment.py#L200-L203

To sove this problem, simply correct TFPyEnvironment.close method with

self._pool.close() first, then self._pool.join()

reference: https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool.join

You are good to go~~~

QuantHao on May 20, 2021

ISSUE FIXED put the code with distributed training code to a separate method and call it inside the main method. Git hub Code

SaralaSewwandi on Oct 5, 2021

@QuantHao thanks for the pointer. I was experiencing this issue while trying to run parallel TF-Agents drivers using mpi4py. Just adding env.close() at the end of my script solved it for me, I did not need to set env.isolation = True.

samarth-robo on Jul 2, 2021

I received the same issue, if anyone has resolved the issue please share.

In my case, if I ended the code in train_env = tf_py_environment.TFPyEnvironment(train_py_env), it will throw this exception. But if I created the agent and ran the session, it didn’t throw any exceptions. I guess it is lazy loaded. If you didn’t create a session in the code, it throws exception. If you create a session, the line is executed in the session.

git4sun on Apr 18, 2021