spyder: PYTHONPATH not being used in Windows Subsystem for Linux Ubuntu

Problem Description

The PYTHONPATH in my terminal does not seem to be used by spyder (and I’ve started spyder from the command line). In the console, sys.path just contains

['/home/<username>/miniconda3/envs/<env name>/lib/python310.zip',
 '/home/<username>/miniconda3/envs/<env name>/lib/python3.10',
 '/home/<username>/miniconda3/envs/<env name>/lib/python3.10/lib-dynload',
 '',
 '/home/<username>/miniconda3/envs/<env name>/lib/python3.10/site-packages']

os.environ[‘PYTHONPATH’] also does not exist, though other custom environment variables are present in os.environ, seeming to suggest that something has acted to exclude importing the PYTHONPATH variable. If I use ipython in the terminal in the same environment, sys.path and os.environ[‘PYTHONPATH’] contain all the directories in PYTHONPATH.

This is in a fresh spyder install in a miniconda environment. I’m opening Spyder from a MobaXterm terminal in a Windows 10 Subsystem for Linux Ubuntu installation.

I’m coming back to spyder after a while, so perhaps there are some extra steps I need to perform that I don’t know - I don’t remember needing to do anything special to use PYTHONPATH in spyder before, and I can’t find documentation on it.

Versions

  • Spyder version: 5.5.0
  • Python version: 3.10.13
  • Qt version: 5.15.8
  • PyQt version: 5.15.9
  • Operating System name/version: Ubuntu 20.04 in Windows 10 Subsystem for Linux

Dependencies

Mandatory:

atomicwrites >=1.2.0 : 1.4.1 (OK) chardet >=2.0.0 : 5.2.0 (OK) cloudpickle >=0.5.0 : 3.0.0 (OK) cookiecutter >=1.6.0 : 2.5.0 (OK) diff_match_patch >=20181111 : 20230430 (OK) intervaltree >=3.0.2 : 3.1.0 (OK) IPython >=8.13.0,<9.0.0,!=8.17.1 : 8.18.1 (OK) jedi >=0.17.2,<0.20.0 : 0.19.1 (OK) jellyfish >=0.7 : 1.0.3 (OK) jsonschema >=3.2.0 : 4.20.0 (OK) keyring >=17.0.0 : 24.3.0 (OK) nbconvert >=4.0 : 7.12.0 (OK) numpydoc >=0.6.0 : 1.6.0 (OK) parso >=0.7.0,<0.9.0 : 0.8.3 (OK) pexpect >=4.4.0 : 4.8.0 (OK) pickleshare >=0.4 : 0.7.5 (OK) psutil >=5.3 : 5.9.7 (OK) pygments >=2.0 : 2.17.2 (OK) pylint >=2.5.0,❤️.1 : 3.0.3 (OK) pylint_venv >=3.0.2 : 3.0.3 (OK) pyls_spyder >=0.4.0 : 0.4.0 (OK) pylsp >=1.9.0,<1.10.0 : 1.9.0 (OK) pylsp_black >=1.2.0,❤️.0.0 : 1.3.0 (OK) qdarkstyle >=3.2.0,❤️.3.0 : 3.2 (OK) qstylizer >=0.2.2 : 0.2.2 (OK) qtawesome >=1.2.1 : 1.3.0 (OK) qtconsole >=5.5.0,<5.6.0 : 5.5.1 (OK) qtpy >=2.1.0 : 2.4.1 (OK) rtree >=0.9.7 : 1.1.0 (OK) setuptools >=49.6.0 : 68.2.2 (OK) sphinx >=0.6.6 : 7.2.6 (OK) spyder_kernels >=2.5.0,<2.6.0 : 2.5.0 (OK) textdistance >=4.2.0 : 4.5.0 (OK) three_merge >=0.1.1 : 0.1.1 (OK) watchdog >=0.10.3 : 3.0.0 (OK) xdg >=0.26 : 0.28 (OK) zmq >=22.1.0 : 25.1.2 (OK)

Optional:

cython >=0.21 : None (NOK) matplotlib >=3.0.0 : 3.8.2 (OK) numpy >=1.7 : 1.26.2 (OK) pandas >=1.1.1 : None (NOK) scipy >=0.17.0 : 1.11.4 (OK) sympy >=0.7.3 : None (NOK)

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 1
  • Comments: 30 (19 by maintainers)

Most upvoted comments

I’m surprised by that because this “recent change” is almost all* required to resolve the current issue. If that is changed we are back to this inability to actively customize the environment

Yes, it does work in your favor at the moment. I say it may change because the that particular fix bifurcates the behavior of Spyder based on its launch mechanism, which I don’t think is a good idea. Spyder should behave the same way, regardless of launch mechanism.

@Flamefire and @smax1, I am committed to resolving the issue regarding your situations, e.g. accommodating disparate package locations and compatibility with environment modules. However, I think that this is separate from (though related to) the current issue. I recommend continuing our conversation at #22681.

However, we’re talking about overriding our cleaning of PYTHONPATH at startup and when creating kernels through an env var (see #22681).

I should clarify regarding this issue as well. The proposed option to not clean PYTHONPATH will only affect Spyder runtime and Spyder-kernel runtime and would have no impact on user’s code. At least that’s how I am envisioning its implementation.

The treatment of PYTHONPATH for user’s code (i.e. IPython Console sys.path and os.environ) is a separate issue. I can see that for @Flamefire’s situation in #22681, we must have some discussion regarding lib/.../[site|dist]-package type paths. But let’s isolate that conversation to #22681 so that we don’t hijack this issue.

@mrclary What I found is that it is $SPY_PYTHONPATH that ends up in the console and that is determined from spyder_pythonpath in the config which in turns is the result of https://github.com/spyder-ide/spyder/blob/5f4c5cd326ff3b2bbf7b96b95d9ad787aa2c0bb5/spyder/plugins/pythonpath/utils.py#L33

However there is some filtering: All non-existant paths are removed (which is why my /foo didn’t show up) and all lib[...]site-packages or lib[...]dist-packages are also removed.

The latter will be a problem for our use case where we do the equivalent of

$ pip install foo --prefix=/opt/foo
$ export PYTHONPATH=/opt/foo/lib/python3.8/site-packages:$PYTHONPATH

@PAGWatson Is it something similar for you? What is the content of your PYTHONPATH?

Same problem. It’s important when you’re managing a bunch of environment data for multiple tools in a Linux system. I want to connect the user to a specific set of python modules that can be simply imported from a script that they run. I don’t want to install them into Python because the version may change or it might affect other users. I don’t want to use conda virtual environments because I need to manage the environment of many tools - not just Spyder - and this is best done with environment variables and environment modules.