nb_conda_kernels: Conda environments not detected

So I was having a similar issue, possibly the same as FBen3, I’m going to try to outline the steps I took to get my kernels showing up the way I want them to:

I wanted to make sure I was starting fresh with kernels so I decided to try this.

Removing /Users/${USER}/Library/Jupyter/kernels fixes the issue.

Well it didn’t fix my issues but it seemed to remove the kernels that looked like the ones in FBen3’s post.

Next step is to make sure you really follow the steps in this post especially for manually registering a kernel that already exists. I’ll try to outline the important ones:

1. Update your conda, idk if this is essential but I haven’t been using mine in a while and I wanted to make sure I was on the most up to date.
2. Do conda install nb_conda_kernels on the base environment, I believe this is supposed to auto detect new environments in the future but I actually haven’t tested it yet.
3. Do conda activate into the environment you want to add
4. Here are the crucial parts: Make sure ipykernel is installed with conda install notebook ipykernel. I thought nb_conda_kernels should be picking up the new kernel but it was not so I had to do the next step.
5. You need install the kernel. If ipython kernel install doesn’t work try ipython kernel install --user, I believe the --user flag is for if your kernelspec is located in /Users/… rather than /usr/… If you want a to add a specific name do ipython kernel install --user --name [kernel_name] --display-name [kernel_name]. IF YOU DON’T SPECIFY THE NAME, I THINK IT WILL OVERWRITE YOUR PYTHON3 KERNEL. I’m not a 100% sure on that but I think that’s why they don’t show up.
6. conda deactivate and jupyter kernelspec list. You should hopefully start seeing the env names there. In my case this is

Available kernels: cs682 /Users/kenn/Library/Jupyter/kernels/cs682 py37 /Users/kenn/Library/Jupyter/kernels/py37 python3 /Users/kenn/Library/Jupyter/kernels/python3

1. jupyter notebook on (base) and you should hopefully see the kernels as normal now

As explain in the README, you can optionally install the virtual conda kernel specs in a place searched for by conda:

1. Create a configuration file for jupyter named jupyter_config.json in the folder returned by jupyter --config-dir.
2. Add the following configuration to install all kernel spec for the current user:

{
  "CondaKernelSpecManager": {
    "kernelspec_path": "--user"
  }
}

1. Execute the command (or open the classical Notebook or JupyterLab UI):

python -m nb_conda_kernels list 

1. Check that the conda environment kernels are discovered by jupyter:

jupyter kernelspec list

The kernelspec path needs to be writable by the users as kernelspecs will be dynamically updated by nb_conda_kernels.

conda install nb_conda_kernels fixed the problem quite easily since it auto-detects and picks up the existing conda envs

okay, so I had a bunch of problems with this and it sucked up far too many hours of my life. Google ranked this thread up pretty high, so I’m posting it here for posterity. For anyone trying to do add conda-installed kernels in Docker, here is the process:

FROM jupyter/minimal-notebook:7a0c7325e470
ARG conda_env=py36
COPY environment.yml /home/${NB_USER}/tmp/

RUN cd /home/${NB_USER}/tmp/ && \
    conda env create -p $CONDA_DIR/envs/$conda_env -f environment.yml

# create Python 3.6 environment and link it to jupyter
RUN $CONDA_DIR/envs/${conda_env}/bin/python -m ipykernel install --user --name=${conda_env}
# any additional pip installs:
RUN $CONDA_DIR/envs/${conda_env}/bin/pip install numpy

# prepend to path
ENV PATH $CONDA_DIR/envs/${conda_env}/bin:$PATH
# optional defaulting
ENV CONDA_DEFAULT_ENV ${conda_env}

# some cleanup because we used the docker-stacks images.
RUN fix-permissions $CONDA_DIR && \
    fix-permissions /home/$NB_USER

(--display-name "Other Python" in the ipykernel step if you want a different display name in the Jupyter dropdown, and you can avoid the COPY with conda create changed to conda create --quiet --yes -n $conda_env python=3.6 pip ipython ipykernel )

@michaelaye a couple of things. First, where exactly is your jupyter_notebook_config.py file located, and what non-trivial content does it have in it? For instance, mine is in ~/.jupyter/; and I can see what’s active with this grep command:

grep -v '^\s*\(#.*\)\?$' ~/.jupyter/jupyter_notebook_config.py

where exactly is your jupyter_notebook_config.py located; and what exactly does it have inside it (ignoring the comments)? For instance, I created one in ~/.jupyter,

Second, you could try using the verbose enable/disable commands:

python -m nb_conda_kernels.install -v --enable
python -m nb_conda_kernels.install -v --disable
python -m nb_conda_kernels.install -v --status

This should help us see what’s going on.

The error you’re seeing above is telling me that, somehow, your jupyter_notebook_config.py file already has some sort of setting in it that is overriding what nb_conda_kernels is trying to achieve locally.