notebook: Jupyter Notebook is loading incorrect Python kernel

Solved this issue by executing

python3 -m ipykernel install --user

I got a similar problem

I solved it by sudo pip install --upgrade ipykernel

For now, you can probably work around it by changing the first item in argv to ‘python3’ in the Python 3 kernel.json file.

as reported by others, sys.path and sys.executable agree from within the virtualenv (I’m using pipenv, btw), the latter correctly pointing to the virtualenv:

>>> sys.executable
'/Users/tullio/.local/share/virtualenvs/data_science-V36xIoci/bin/python'

but from within jupyter lab, sys.executable moves to pointing to

>>> sys.executable
'/usr/local/opt/python/bin/python3.7'

so far, 2 solutions have been proposed, neither of which worked for me:

Running python3 -m ipykernel install --user

did not fix the issue for me (I’m assuming it has to be run after activating the venv, and before opening jupyter lab, right?)

Editing the kernel.json file

Running

jupyter kernelspec list

returns me the same, single kernel.json file regardless of whether I do it from the virtualenv or from the jupyter lab instance I launch from within it. argv[0] really looks fine to me, no reason to change it:

$ more kernel.json
{
 "argv": [
  "/Users/tullio/.local/share/virtualenvs/data_science-V36xIoci/bin/python3",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Python 3",
 "language": "python"
}

And seriously, jupyter: threads like this one have been going on for ages. I’ve installed jupyter lab from within the virtualenv - WHY ON EARTH would I want sys.executable to point to anywhere but to the same value that will be returned when launching python from the virtualenv?

This is also happening to me. Using ipython3 notebook works.

Yup, it’s the same. Running ipython kernel install for Python 2 and 3 will resolve the ambiguity by installing with sys.executable’s abspath that we can’t do in data_files. While the new data_files work better in envs of all kinds and with just one Python, two Pythons running on the same prefix (e.g. --user on linux, but not mac) run into ambiguity.

The problem with which python inside notebook not pointing to the same as sys.executable is caused by a difference in system paths. In order to make sure they are pointing to the same executable:

• make sure you are using the right python version and it’s pip when creating the ipykernel - ensured by python -m ...
• set up paths for your ipykernel by passing --env PATH=...

For my setup with virtualenv it looks like this

/home/user/venv/bin/python -m pip install ipython ipykernel
/home/user/venv/bin/python -m ipykernel install --user --name=kernel-name --env PATH /home/user/venv/bin:$PATH

In general:

<YOUR_PYTHON_EXECUTABLE> -m pip install ipython ipykernel
<YOUR_PYTHON_EXECUTABLE> -m ipykernel install --user --name=kernel-name --env PATH <YOUR_PYTHON_BIN_DIRECTORY>:$PATH

Solved mine with python3 -m ipykernel install --user

I’m having a similar issue on Mac OS. When trying to run the kernel within a virtual environment, the system python (brew installed) is loaded instead.

When the virtualenv is active:

(venv) lubaroli@machine: ~$ which python
/Users/lubaroli/.env/venv/bin/python

Likewise:

(venv) lubaroli@machine: ~$ which IPython
/Users/lubaroli/.env/venv/bin/IPython

When running: python kernel install --user --name=venv

The kernel.json file reads:

{
 "argv": [
  "/usr/local/opt/python/bin/python3.7",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "venv",
 "language": "python"
}

Changing the kernel.json path manually to the venv path does not fix the issue. When running a notebook, sys.executable still returns: '/usr/local/opt/python/bin/python3.7'

Any ideas?

works for me with python3 -m ipykernel install --user - THANK YOU nvdv 👍

Solved this issue by executing

python3 -m ipykernel install --user

works like charm

solved my problem with python3 -m ipykernel install --user on macOS Catalina 10.15.3 with anaconda

My jupyter notebook was using a python3 kernel even though I wanted it to use a python2 kernel (and the web interface was showing python2).

1. Verify that the actual kernel is python3 by running the following from within the jupyter notebook:

import platform 
print(platform.python_version())

1. Verify kernel is missing:

jupyter kernelspec list

1. Install new kernel:

sudo -H pip2 install jupyter

1. Restart jupyter notebook (repeat step 1 to confirm output is what’s expected).

See https://stackoverflow.com/a/37857536/5658349

It seems like the crux of the issue is that the correct python is in use by the kernel, but the PATH (from the kernel) doesn’t reflect the python in use. As a result, while python statements (e.g., import foo) produce the desire results, any shell commands (e.g., !pip install foo) do not. In such cases, you could fully-qualify the shell command and, I believe, get the desire results.

I found the IPython’s documentation link in the Python Environments section of the troubleshooting documentation to be fairly helpful. In particular the portion about installing the kernel with the --prefix option to point at an env different that the current env.

Jupyter, by default, doesn’t activate environments for a given kernel. This is why the PATH-based commands (sans prefixing) can be problematic. If you find that activation may resolve things for you, then perhaps taking a look at nb_conda_kernels may be helpful. This package updates your jupyter’s configuration to use an alternate KernelSpecManager that is used for discovering kernels (which some find invasive). These kernel specifications contain information that, AFAIK, activate the environment associated with that kernel prior to its launch.

Here’s a screenshot of a kernel I’ve started that uses a python from an environment other than the notebook server. The notebook server is launched in my activated notebook-dev env, while the kernel I want to run is in my enterprise-gateway-dev env. Aside from the PATH and sys.executable difference, you’ll find that the import statement produces the desired result (my notebook-dev env doesn’t have kubernetes installed). In addition, my EG env doesn’t have nose installed until I fully qualify the pip command to install it into the kernel’s env (i.e., enterprise-gateway-dev).

I hope you find this helpful and apologize for the frustration.

same issue here as tbagnoli and lubaroli using conda environments.

I generate the environment using the code below:

conda create --name venv python pip ipykernel
conda activate venv
pip install -e /path/to/local/package

Within terminal the environments agree:

~
venv ❯ which python
/Users/nicholasmi/opt/anaconda3/envs/venv/bin/python

~
venv ❯ which pip   
/Users/nicholasmi/opt/anaconda3/envs/venv/bin/pip

I then install the kernelspec via:

~
venv ❯ python -m ipykernel install --user --name venv --display-name "Python (venv)"

Using the kernelspec in Jupyter then yields:

Does a solution exist? The methods above don’t help. For example, the kernel.json file seems to be set up correctly already.

~/Library/Jupyter/kernels/venv 19s
base ❯ cat kernel.json      
{
 "argv": [
  "/Users/nicholasmi/opt/anaconda3/envs/venv/bin/python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Python (venv)",
 "language": "python"
}```

I didn’t fully understand what @minrk said above, but running ipython kernel install --name python3 didn’t solve the issue, if that helps.

As @takluyver suggested, changing argv solved it for me, thank you. I changed the first argument of argv in python3 folder’s kernel.json from python2 path to python3 path, and everything works as expected.

@nemusa Thank you so much, this worked for me! It was driving me crazy that I couldn’t figure out why the PATH inside the shell didn’t match my $PATH

@jianboy I solved it with this

run pip install pandas, it installed to /usr/bin/python2 instead of /opt/python3.9

import sys
! {sys.executable} -m pip install pandas
import pandas

You may need to restart the kernel

ipython kernel install --user worked for me

Hi @yasirroni - try using the %pip magic. This is exactly the problem it resolves.

The inability to save edits to a file directly is more of a system thing. You’ll need to look at the kernel.json’s permissions and change users and/or permissions to complete the save operation. Since this is a server, you may need to enlist the help of your system’s administrator.

kernel.json in python2 directory:

{
 "display_name": "Python 2", 
 "language": "python", 
 "argv": [
  "python", 
  "-m", 
  "ipykernel_launcher", 
  "-f", 
  "{connection_file}"
 ]
}

kernel.json in python3 directory:

{
 "argv": [
  "python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Python 3",
 "language": "python"
}