jupyter: Broken jupyter notebook kernel integration with conda environments

conda install nb_conda_kernels failed with:

UnsatisfiableError: The following specifications were found to be in conflict:
  - nb_conda_kernels -> _nb_ext_conf -> nb_anacondacloud -> python 2.7* -> openssl 1.0.1*
  - python 3.6*

So I installed nb_conda_kernels from conda-forge using:

conda install --channel=conda-forge nb_conda_kernels

Now I can choose from my conda environments in the Jupyter kernel menu. Phew!

Note that the nb_conda_kernels README states:

To use a conda environment as a kernel, don’t forget to install ipykernel in this environment or it won’t show up in the kernel list.

I think Continuum stopped shipping their environments-as-kernels plugin by default - you can probably still install it

@takluyver, thanks for that info! It was enough for me to track down nb_conda_kernels (anaconda cloud, repo). Once I’m back at the affected machine, I’ll let you know if this fixes my issues.

nb_conda also looks interesting (allows you to manage conda environments in a Jupyter browser tab).

In worst case, I should be able to use the single kernel installation commands you link to:

source activate myenv
python -m ipykernel install --user --name myenv --display-name "Python (myenv)"
source activate other-env
python -m ipykernel install --user --name other-env --display-name "Python (other-env)"

But I’m not a huge fan of this solution as it’s a manual step. Furthermore, when we have several people collaborating on a repo, we want them to all have the same kernel names for conda environments.

I think Continuum stopped shipping their environments-as-kernels plugin by default - you can probably still install it, or you can set up kernels for environments manually. We’re thinking about better ways to expose environments to Jupyter.

Such a useful thread. IMO it is more than a little counterintuitive that notebooks do not use the active conda env by default, nor that they are even visible!

For future Googlers the simplest solution seems to be to install the nb_conda package: https://stackoverflow.com/a/38880722/209882

I changed my .condarc from

channels:
  - defaults
ssl_verify: true

to

channels:
  - defaults
  - conda-forge
create_default_packages:
  - pip
  - ipykernel
  - nb_conda_kernels
ssl_verify: true

and now whenever I am creating a new virtualenv it gets automatically added to the list of environments. I don’t even have to run the commands anymore.

And they got removed when I remove the env using conda remove env -n NAME_OF_ENV_HERE

For what it’s worth, I am experiencing the similar issues. I am on Windows 10, and I completely removed Anaconda using the recommended steps (anaconda-clean, then uninstall from add/remove programs). I installed the Python 3.6 version of Anaconda, and now my Jupyter Notebook is completely borked. I can launch the server, but I can’t access any of my conda environments (old or new). Only one kernel is listed: “Python 3.” If I install Jupyter from within the environment and launch the notebook server while the environment is activated, sys.path still refers to the base installation path. Is something broken with respect to 3.6?

I was facing an issue with this method. If i freshly create a conda environment and then create a kernel with it, I am being able to access those kernels from the base environment, however, if I update the conda environment, then jupyter from the base environment is not being able to connect with this kernel.

The problem repeates itself in 2019 with a fresh installation of Anaconda 3 on a Mac with Mojave. The kernel does not exist and the jupyter notebook cannot get access to Python.

[W 16:56:44.626 NotebookApp] KernelRestarter: restart failed
[W 16:56:44.627 NotebookApp] Kernel b656dc1a-b221-4d35-b473-dfbcf1d593a3 died, removing from map.
[W 16:57:29.624 NotebookApp] Timeout waiting for kernel_info reply from b656dc1a-b221-4d35-b473-dfbcf1d593a3
[E 16:57:29.627 NotebookApp] Error opening stream: HTTP 404: Not Found (Kernel does not exist: b656dc1a-b221-4d35-b473-dfbcf1d593a3)

The conda info message displays what follows:

Last login: Mon Jun 24 18:36:34 on ttys000
Restored session: Lun 24 Giu 2019 18:45:55 CEST
(base) MacBook-Pro:~ anthony$ conda info

     active environment : base
    active env location : /Users/anthony/anaconda3
            shell level : 1
       user config file : /Users/anthony/.condarc
 populated config files : /Users/anthony/.condarc
          conda version : 4.6.14
    conda-build version : 3.17.8
         python version : 3.7.3.final.0
       base environment : /Users/anthony/anaconda3  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/osx-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/osx-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/osx-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /Users/anthony/anaconda3/pkgs
                          /Users/anthony/.conda/pkgs
       envs directories : /Users/anthony/anaconda3/envs
                          /Users/anthony/.conda/envs
               platform : osx-64
             user-agent : conda/4.6.14 requests/2.21.0 CPython/3.7.3 Darwin/18.6.0 OSX/10.14.5
                UID:GID : 501:20
             netrc file : None
           offline mode : False

I did the following:

python -m ipykernel install --user --name StoAU2 --display-name "Python (StoAU2)"
/home/user/anaconda3/envs/StoAU2/bin/python: No module named ipykernel
[Errno 13] Permission denied: '/usr/local/share/jupyter'`

So then I tried:

source deactivate
python -m ipykernel install --user --name StoAU2 --display-name "Python (StoAU2)"
Installed kernelspec StoAU2 in /home/user/.local/share/jupyter/kernels/stoau2
python -m ipykernel install --user --name StoAU --display-name "Python 3 (StoAU)"
Changed Kernel -> Python (StoAU2).

Then in a Jupyter notebook:

import sys
print(sys.executable)
from jupyter_core.paths import jupyter_data_dir
print(jupyter_data_dir())`

/home/user/anaconda3/bin/python
/home/user/.local/share/jupyter

Still not referring to the right kernel… This is quite an annoying change that terminal kernel and Notebook terminal are not the same.

This is what I tried (based on: https://github.com/Anaconda-Platform/nb_conda_kernels/blob/fe861722902df9c97f47bbf6e16c8c634d17144f/README.md):

conda install -c conda-forge nb_conda_kernels
conda create -n test
package plan Proceed ([y]/n)? y
conda install -n nb_conda_kernels --file requirements.txt -c r

But I got an error that I don’t have the requirement.txt (sorry lost actual error). So I reinstalled everything and did the following (which finally worked):

conda create --name StoAU
source activate StoAU
conda install ipykernel
python -m ipykernel install --user --name StoAU --display-name "Python 3 (StoAU)"

I might be a beginner, but I don’t understand what the purpose is of creating an environment and installing all packages there, if in the end the notebook doesn’t use this kernel with what you’ve just installed.