spyder: Project's folder is not added to sys.path when creating/loading a project with Spyder 5.3.2

Issue Report Checklist

  • Searched the issues page for similar reports
  • Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
  • Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
  • Could not reproduce inside jupyter qtconsole (if console-related)
  • Tried basic troubleshooting (if a bug/error)
    • Restarted Spyder
    • Reset preferences with spyder --reset
    • Reinstalled the latest version of Anaconda
    • Tried the other applicable steps from the Troubleshooting Guide
  • Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

Using Spyder 5 in a miniconda environment in Windows 10, Spyder does not add the folder to sys.path when loading or creating a project. Therefore, I cannot import functions from other files as expected. Uninstalling miniconda, manually removing leftover folders (like ~.spyder-py3), and starting from a fresh miniconda installation does not help.

What steps reproduce the problem?

  1. Fresh install of miniconda latest.
  2. Create and activate Python 3.9 environment: conda create -n myenv python=3.9, conda activate myenv
  3. Install numpy, scipy, matplotlib, and spyder in myenv: conda install package (no version specified)
  4. Create a new project or load an existing one.
  5. Can’t import packages from folders in project. Check sys.path. Folder is not there.

What is the expected output? What do you see instead?

sys.path should include the project folder. It works in Ubuntu 20, but not in two different machines I’ve tried with Windows 10.

The command window returns an error: zmq.error.ZMQError: not a socket

Versions

  • Spyder version: 5.2.2
  • Python version: 3.9.12 64-bit
  • Qt version: 5.15.12
  • PyQt version: PyQt5 5.15.7
  • Operating System name/version:

Dependencies

# Mandatory:
atomicwrites >=1.2.0          :  1.4.0 (OK)
chardet >=2.0.0               :  4.0.0 (OK)
cloudpickle >=0.5.0           :  2.0.0 (OK)
cookiecutter >=1.6.0          :  1.7.3 (OK)
diff_match_patch >=20181111   :  20200713 (OK)
intervaltree >=3.0.2          :  3.1.0 (OK)
IPython >=7.6.0;<8.0.0        :  7.31.1 (OK)
jedi >=0.17.2;<0.19.0         :  0.18.1 (OK)
jellyfish >=0.7               :  0.9.0 (OK)
jsonschema >=3.2.0            :  4.4.0 (OK)
keyring >=17.0.0              :  23.4.0 (OK)
nbconvert >=4.0               :  6.4.4 (OK)
numpydoc >=0.6.0              :  1.4.0 (OK)
paramiko >=2.4.0              :  2.8.1 (OK)
parso >=0.7.0;<0.9.0          :  0.8.3 (OK)
pexpect >=4.4.0               :  4.8.0 (OK)
pickleshare >=0.4             :  0.7.5 (OK)
psutil >=5.3                  :  5.9.0 (OK)
pygments >=2.0                :  2.11.2 (OK)
pylint >=2.5.0                :  2.14.5 (OK)
pyls_spyder >=0.4.0           :  0.4.0 (OK)
pylsp >=1.3.2;<1.4.0          :  1.3.3 (OK)
pylsp_black >=1.0.0           :  1.0.0 (OK)
qdarkstyle =3.0.2             :  3.0.2 (OK)
qstylizer >=0.1.10            :  0.1.10 (OK)
qtawesome >=1.0.2             :  1.0.3 (OK)
qtconsole >=5.2.1;<5.3.0      :  5.2.2 (OK)
qtpy >=1.5.0                  :  2.0.1 (OK)
rtree >=0.9.7                 :  0.9.7 (OK)
setuptools >=49.6.0           :  63.4.1 (OK)
sphinx >=0.6.6                :  5.0.2 (OK)
spyder_kernels >=2.2.1;<2.3.0 :  2.2.1 (OK)
textdistance >=4.2.0          :  4.2.1 (OK)
three_merge >=0.1.1           :  0.1.1 (OK)
watchdog >=0.10.3             :  2.1.6 (OK)
zmq >=17                      :  23.2.0 (OK)

# Optional:
cython >=0.21                 :  None (NOK)
matplotlib >=2.0.0            :  3.5.2 (OK)
numpy >=1.7                   :  1.21.5 (OK)
pandas >=1.1.1                :  None (NOK)
scipy >=0.17.0                :  1.7.3 (OK)
sympy >=0.7.3                 :  None (NOK)

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 15 (12 by maintainers)

Most upvoted comments

Thanks for the follow-up everyone!

Ok, so why don’t we add the project’s path to spyder_pythonpath in MainWindow? That should be another way to solve this, right?

Yes, I think so; and I think that is the root of the problem.

It appears that main.save_python_path intends to update spyder_pythonpath with active paths from path, not_active_path, and project path, but this never gets called when projects is updated. The reason is that main.pythonpath_changed (only called when projects are opened/closed) updates main.project_path attribute, calls main.get_spyder_pythonpath_dict (which will now have updated project path) and sends this to main.update_python_path. However, main.update_python_path calls main.get_spyder_pythonpath_dict, expecting it to be an “old” path list (but, oops, it’s already been updated) to compare to this “new” path list, and only calls main.save_python_path if they are different. Thus, main.pythonpath_changed unwittingly ensures that main.save_python_path is not called and spyder_pythonpath will never have project paths.

I’ll make a new proposal shortly that ensures spyder_pythonpath is properly updated with the project path.

The apparent cause for this issue seems to be that upon opening a new project, the signal to update the paths is received by the existing IPython Console, but then the signal is sent to restart the console and upon restart the SpyderKernelSpec only retrieves the spyder_pythonpath in the configuration, which is up-to-date but does not carry the project path.

This issue is present in the 5.3.1 artifact, which predates both #17512 (updating PYTHONPATH) and #17408 (updating bootstrap), so I’m not sure when it was introduced. Nevertheless, I think we have a few options:

  1. In the projects plugin we can emit the sig_pythonpath_changed after restarting consoles. If sig_pythonpath_changed is emitted after starting consoles in all situations, then SPY_PYTHONPATH could be eliminated from spydercustomize.py altogether.
  2. We could combine the project paths with spyder_pythonpath when assigning SPY_PYTHONPATH in SpyderKernelSpec. SpyderKernelSpec does not have access to the main window’s get_spyder_pythonpath_dict, which performs this sythesis, so SPY_PYTHONPATH may need to be updated in get_new_kernel in the ipythonconsole main_widget.

@spyder-ide/core-developers, I favor the first option; does anybody have a different suggestion?

Ok, thanks for checking that @DanielReyesLastiri. I can reproduce this problem by doing the following:

  • Resetting Spyder preferences.
  • Start Spyder
  • Opening a project.
  • Checking sys.path in a console and noticing that the project’s path was not added to sys.path.

@mrclary, please take care of this one because it looks related to the changes you did in PR #17511.