ray: [Bug] Can't pickle function objects error on ray.init

Search before asking

  • I searched the issues and found no similar issues.

Ray Component

Ray Core

What happened + What you expected to happen

Running ray.init() throws this error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/_private/client_mode_hook.py", line 89, in wrapper
    return func(*args, **kwargs)
  File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/worker.py", line 950, in init
    job_config=job_config)
  File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/worker.py", line 1451, in connect
    lambda worker_info: sys.path.insert(1, current_directory))
  File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/worker.py", line 386, in run_function_on_all_workers
    pickled_function = pickle.dumps(function)
  File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 73, in dumps
    cp.dump(obj)
  File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 580, in dump
    return Pickler.dump(self, obj)
  File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/pickle5_files/pickle5/pickle.py", line 485, in dump
    self.save(obj)
  File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/pickle5_files/pickle5/pickle.py", line 601, in save
    self.save_reduce(obj=obj, *rv)
  File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/pickle5_files/pickle5/pickle.py", line 689, in save_reduce
    save(func)
  File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/pickle5_files/pickle5/pickle.py", line 601, in save
    self.save_reduce(obj=obj, *rv)
  File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/pickle5_files/pickle5/pickle.py", line 689, in save_reduce
    save(func)
  File "/home/boris/.pyenv/versions/3.7.1/lib/python3.7/site-packages/ray/pickle5_files/pickle5/pickle.py", line 576, in save
    rv = reduce(self.proto)
TypeError: can't pickle function objects
>>> 

Versions / Dependencies

Ubuntu 20.04, Python 3.7.1 via pyenv, ray 1.7.1

Reproduction script

import ray
ray.init()

Anything else

Original discussion in Modin project: https://github.com/modin-project/modin/issues/3601

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 24 (7 by maintainers)

Most upvoted comments

Same problem with 3.7.0. I have no tensorflow on my system without virtual environment or pickle5 Problem disappeared with python==3.7.10.

It might be a issue related to python.

It looks like there are many people going through the issue. Can you try setting up the priority cc @ericl ?

This issue has been solved for Python 3.8.2rc1, Python 3.9.0 alpha 4 or late versions.