spyder: joblib muliprocessing not working on > spyder 5?

Issue Report Checklist

  • Searched the issues page for similar reports
  • Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
  • Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
  • Could not reproduce inside jupyter qtconsole (if console-related)
  • Tried basic troubleshooting (if a bug/error)
    • Restarted Spyder
    • Reset preferences with spyder --reset
    • Reinstalled the latest version of Anaconda
    • Tried the other applicable steps from the Troubleshooting Guide
  • Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

I am using joblib for multiprocessing.

I get the following error with 5.1.3 and later: BrokenProcessPool: A task has failed to un-serialize. Please ensure that the arguments of the function are all picklable.

I seem not to get the problem with 5.0.3.

What steps reproduce the problem?

  1. run joblib multiprocessing with 5.0.3
  2. run joblib multiprocessing with 5.1.4

What is the expected output? What do you see instead?

Both of the above should work. I obsrve today that only step #1 works.

Paste Traceback/Error Below (if applicable)


PASTE TRACEBACK HERE

Versions

  • Spyder version: 5.0.3 / 5.1.4
  • Python version: 3.8.10
  • Qt version:
  • PyQt version:
  • Operating System name/version: Windows 10

Dependencies

PASTE DEPENDENCIES HERE

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 15 (7 by maintainers)

Most upvoted comments

You omitted this crucial part, which shows that this problem has nothing to do with Spyder. Please recreate your environment and install everything from conda-forge. The problem is conda-forge and defaults packages are binary incompatible, so you can’t mix them (as you’re doing right now).

That worked - works find now.