ray: [Bug] Can't pickle function objects error on ray.init
Search before asking
- I searched the issues and found no similar issues.
Ray Component
Ray Core
What happened + What you expected to happen
Running ray.init() throws this error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/_private/client_mode_hook.py", line 89, in wrapper
return func(*args, **kwargs)
File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/worker.py", line 950, in init
job_config=job_config)
File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/worker.py", line 1451, in connect
lambda worker_info: sys.path.insert(1, current_directory))
File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/worker.py", line 386, in run_function_on_all_workers
pickled_function = pickle.dumps(function)
File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 73, in dumps
cp.dump(obj)
File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 580, in dump
return Pickler.dump(self, obj)
File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/pickle5_files/pickle5/pickle.py", line 485, in dump
self.save(obj)
File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/pickle5_files/pickle5/pickle.py", line 601, in save
self.save_reduce(obj=obj, *rv)
File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/pickle5_files/pickle5/pickle.py", line 689, in save_reduce
save(func)
File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/pickle5_files/pickle5/pickle.py", line 601, in save
self.save_reduce(obj=obj, *rv)
File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/pickle5_files/pickle5/pickle.py", line 689, in save_reduce
save(func)
File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/pickle5_files/pickle5/pickle.py", line 576, in save
rv = reduce(self.proto)
TypeError: can't pickle function objects
>>>
Versions / Dependencies
Ubuntu 20.04, Python 3.7.1 via pyenv, ray 1.7.1
Reproduction script
import ray
ray.init()
Anything else
Original discussion in Modin project: https://github.com/modin-project/modin/issues/3601
Are you willing to submit a PR?
- Yes I am willing to submit a PR!
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 24 (7 by maintainers)
Same problem with 3.7.0. I have no
tensorflowon my system without virtual environment orpickle5Problem disappeared with python==3.7.10.It might be a issue related to python.
It looks like there are many people going through the issue. Can you try setting up the priority cc @ericl ?
This issue has been solved for Python 3.8.2rc1, Python 3.9.0 alpha 4 or late versions.