notebook: Installed package won't import in notebook

Complete reinstallation of Anaconda fixed the issue.

Check sys.executable and sys.path inside the notebook - I bet your kernel is not running in the environment you intended it to. Run jupyter kernelspec list to see where it finds the python3 kernel.

There’ll be a kernelspec somewhere which is pointing it to a different Python. We really need to redesign that system. Run jupyter kernelspec list in a terminal to see what kernels it knows about. It will point you to directories which contain kernel.json files - one of those will be pointing it to the wrong Python.

As it turns out, this does not fix this issue as I first claimed. It is not restricted to the seaborn package either.

Wow, that’s bizarre. Its an extremely odd list. I’ve deleted ~/Library/Jupyter/kernels and that appears to have straightened things out.

Thanks.

It sounds like you’re trying an import in two different installations of Python, or two different environments. Check sys.executable to see which Python and environment you’re running in, and sys.path to see where it looks to import modules.

Python packages must be installed separately for each copy of Python you use, and if you are using virtualenvs or conda envs, packages must be installed into each environment where you need them. Either the package is not installed in one, or a different version of the package is installed.

To install packages to a particular Python installation or environment using pip, you can run it like this:

path/to/python -m pip install ...
# Replace path/to/python with the path of the Python executable (sys.executable)

If you are using conda environments, you can install packages from conda like this:

source activate myenv  # On Windows, just 'activate myenv' (no 'source')
conda install ...

I have written this up as a blog post as well: http://takluyver.github.io/posts/i-cant-import-it.html

[ This is a saved reply because I answer similar questions often - sorry if it doesn’t exactly fit your case ]

I think you need to restart the kernel for this to work. Besides, you have to be sure to be using the right pip (e.g. pip that comes with Anaconda instead of system pip).

Steps to replicate:

1. Create conda env with:

   conda create --name tester notebook pymc3 numpy ipython

2. Activate environment:

   source activate tester

3. Open IPython, and import pymc3 from the command line. Assert that it imports as expected.

4. Open a Jupyter notebook session:

   jupyter notebook

5. Create a new Python3 notebook and import pymc3. Failure occurs:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-37bf2a3357ff> in <module>()
----> 1 import pymc3

ModuleNotFoundError: No module named 'pymc3'

Despite all the helpful comments, it took me a while to figure this one out. I’m using pip, not conda, and I got it to work reliably without having to manually change the Python executable path as follows:

1. Enter virtual environment: source venv/bin/activate
2. Make sure to reinstall jupyter: pip3 install jupyter
3. Restart venv by deactivating and reactivating.

Checking sys.executable should now point to the venv’s Python kernel. I prefer this method over manually changing kernel.json since that would get tedious if working with multiple different venvs. Pretty much the same idea as @vtslab

I faced the same issue too. What was happening was that my ipython notebook was using the global instance of notebook installation since my newly created conda environment did not have jupyter notebook installed in it. And it was also searching for all the packages referenced in the notebook globally.

The fix was : conda install jupyter-notebook in the newly created environment.

After doing this, when I launched ipynb, it started using the packages in my environment and not the global ones. I also used conda install command for installing all the packages.

To test whether it is a global/environment issue export your ipynb as .python file and try running it from bash. If it works in the environment’s bash, you know that your notebook is trying to find packages in the global env.

To install on notebook and have any effect the correct way to do it is:

import sys
!conda install --yes --prefix {sys.prefix} seaborn

It will install in all conda environments.

If you do:

import sys
!{sys.executable} -m pip install seaborn

then it is even better! It will install in all environments (not only conda environments)

Where it says "python" in that file, give it the path to the Python executable you want it to use.

conda install nb_conda_kernels I tried this. It worked. Original post: https://stackoverflow.com/a/44786736/4110004

Install jupyter in that particular environment.

ok, sys.executable in just python in terminal gives me /Users/hotte/miniconda2/envs/jupyterlab/bin/python (after I source activate the conda env) Within the notebook I do the same and it gives me /Users/hotte/miniconda2/bin/python . Which is pretty puzzling? Any ideas why the notebook itself does not point to my environment that I started it from? (notebook and everything WAS installed from within the environment)

Facing the same issue @mgooty !jupyter kernelspec list will display the path but that didn’t solve the issue for me. Maybe more of a conda question, that fixes the issue for me : http://stuartmumford.uk/blog/jupyter-notebook-and-conda.html

@fonnesbeck Even I am facing the similar issue. I have installed keras but I am not able to import it in the notebook. I am using Mac. Where can I find ~/Library/Jupyter/kernels so that I can follow your strategy?

Thanks all for the infomatic post for the issue that I also encountered. And agree with @sandsece i.e.

==========

PROBLEM:

From within myENV created from conda, run jupyter notebook, but in this notebook some module that could be loaded from myENV just through out error like:

ModuleNotFoundError Traceback (most recent call last) <ipython-input-6-ae142f98063f> in <module>() 1 # import libraries ----> 2 import folium

ModuleNotFoundError: No module named ‘folium’

===========

Check:

1. which jupyter since even sys.path show correct path of myENV, the sys.executable can be different in myENV and in Jupyter notebook. This can simply because the jupyter you run is not installed in myENV and it has to fallback to the base install. It happens that in the base environment the module you want to use is not installed. ----> error!!!

===========

Solution:

For the case above, it is straightforward: conda install jupyter

Hope this will help.

BR//Kangqiao

Thanks! I reset the path in the kernel.json file as you suggested. Everything works now!

I had the same problem. Changing kernels did not work for me. I installed pandas in my venv but It wont show up in jupyter which I ran through my venv.

I have Ubuntu, with a virtual env in python 3.7 python3.7 -m venv my_env Then I added the venv python site-packages folder to system path and it worked. Added following code to jupyter notebook and it was able to find pandas easily

import sys
sys.path.append('/home/aseem/venv/general_venv/lib/python3.7/site-packages')

Just to clarify, I am using a conda env based on this yml:

name: dev

channels:
 - conda-forge
 
dependencies:
- python=3.6
- arrow
- basemap
- blaze
- bokeh
- cython
- dask
- engarde
- ipython
- jupyter
- notebook
- ipyparallel
- ipywidgets
- joblib
- jupyter_client
- jupyter_core
- jupyter_contrib_nbextensions
- jupyter_nbextensions_configurator
- jupyterlab
- line_profiler
- mkl-service
- numexpr
- numba
- numpy
- openpyxl
- pandas
- patsy
- pillow
- pip
- pylint
- scikit-learn
- scipy
- seaborn
- snakeviz
- sympy
- tensorflow
- theano
- xlrd
- pip:
  - GPy
  - PyCap
  - pystan
  - rpy2
  - git+https://github.com/GPflow/GPflow.git#egg=GPflow
  - git+https://github.com/pymc-devs/pymc3.git#egg=pymc3

The env was created and all packages were installed without error, but some packages can be loaded into the notebook, while others were not.

On Ubuntu, I found that using sudo ~/anaconda3/bin/pip instead of pip or pip3 worked for me.

Additionally, I’ve just started using an alias for pip for convenience with alias pip="sudo ~/anaconda3/bin/pip" in the terminal.

Having the same issue, any ideas / solutions?

I can also confirm that the package is present in the site-packages directory for the active env:

drwxr-xr-x   24 fonnescj  staff      816 Apr  2 22:17 pymc3
drwxr-xr-x    7 fonnescj  staff      238 Apr  2 22:17 pymc3-3.0-py3.5.egg-info
-rw-r--r--    1 fonnescj  staff  1124023 Apr  2 14:30 pymc3-3.1rc3-py3.5.egg

And again, it imports from the IPython console; its only the notebook where it fails.

I faced a similar issue. While the imports were working fine on terminal in the virtual environment, they failed to import in the jupyter notebook. The problem was that I had not installed a local jupyter package for the virtual environment. Therefore, the jupyter notebook on my newly created virtual environment was using the global instance of notebook installation, i.e. from the base environment.

I used the command

conda install jupyter notebook

to install notebook on my virtual environment, and this fixed all issues for me. I could import all the packages local to the newly created virtual environment.

@1998anwesha Just check where is the Python Interpreter is running? Are they same or different? Both Jupyter and your interpreter must have same interpreter.

import sys print(sys.executable)

had the same problem (installed python module works in python but not in ipython/jupyter). I found a simple solution to this problem: just look up manually the location of the module and add to the sys.path.

For example, I installed the tensorflow in a conda environment named ‘tensorflow3’. The location of tensorflow module can be found as following:

wei@wei-Precision-T7600:~$ source activate tensorflow3 (tensorflow3) wei@wei-Precision-T7600:~$ python Python 3.6.4 |Anaconda, Inc.| (default, Mar 13 2018, 01:15:57) [GCC 7.2.0] on linux Type “help”, “copyright”, “credits” or “license” for more information.

import tensorflow tensorflow.__file__ ‘/home/wei/anaconda3/envs/tensorflow3/lib/python3.6/site-packages/tensorflow/init.py’

Now I just add the path: ‘/home/wei/anaconda3/envs/tensorflow3/lib/python3.6/site-packages’ to sys.path in ipython / jupyter, and then the tensorflow module can be imported in ipython or jupyter notebook !

In [1]: sys.path.append(‘/home/wei/anaconda3/envs/tensorflow3/lib/python3.6/site-packages’)

In [3]: import tensorflow as tf /home/wei/anaconda3/lib/python3.6/site-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type. from ._conv import register_converters as _register_converters

In [4]:

Wish this helps.

Even installing the old fashioned way with python setup.py install does not result in the notebook seeing the package. I’m at a loss here.

Run the following in the jupyter notebook cell:

import sys

sys.path

sys.executable

It may not be pointing to your virtual environment but to the root

The fix is to install the jupyter notebook from inside your virtual environment

$ . your_env/bin/activiate

(your_env)$ python -m pip install jupyter

Now you can import tensorflow or keras

I also had the problem of differences in the sys.executable for the console and the jupyter notebook. Both point to different pythons within the same virtual environment. My solution was to install jupyter notebook in the virtual environment source activate <your virtual env> conda install jupyter notebook After killing and restarting jupyter notebook they are the same 😃

I had a similar issue. I was trying to install few python packages in the conda environment. But I was getting ImportError in the jupyter notebook. Though it worked fine in the terminal with the conda env on.

I looked at the sys.executable and sys.path from inside the jupyter notebook. I found that the kernel wasn’t running the intended version of python I wanted. The notebook was taking the python host version since I hadn’t installed jupyter in my environment.

So to solve it I installed jupyter notebook from inside of my conda environment. That solved my problem.

Tried installing on an AWS instance. Same problem.

I’ve restarted the kernel multiple times, but this continues to occur. I have even tried deactivating and reactivating the conda env. I can also confirm that it is the pip installed into the current env, rather than the one in the root env. I can import this library from the ipython console without issue.

I can also confirm that sys.executable is what I expect it to be:

'/Users/fonnescj/anaconda3/bin/python'

Hi guys,

After all these years this problem is still annoying. I just wanted to quote what I did to solve a similar problem started by @fonnesbeck a time ago.

After checking the comment of @Vaishak I decided to open my kernel.json file that is inside the environment that I created for my project: /anaconda/envs/MYPROJECT/share/jupyter/kernels/python3

and I changed “python” by “/Users/myname/anaconda/envs/MYPROJECT/bin/python”

{
 "argv": [
  "/Users/myname/anaconda/envs/MYPROJECT/bin/python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Python 3",
 "language": "python"
}

This was possible because of the discussion between @ShayekhBinIslam and @Vaishak

Debugged the issue, at least for my case. (Note that as I was debugging this, I found that I did sometimes have to delete the corresponding folder in ~/Library/Jupyter/kernels/ else jupyter would not update, e.g., different versions of the same package even though ipython did).

This doesn’t work:

conda env create -f environment.yml -n test
python -m ipykernel install --user --name=test
jupyter lab

Same issue here: import works when in ipython command line but not in jupyter (notebook or lab).

This does work:

conda env create -f environmentyml -n test
source activate test
ipython kernel install --user --name=test
jupyter lab

Haven’t tested doing conda activate test but imagine that should work too.

I had the same problem. Changing kernels did not work for me. I installed pandas in my venv but It wont show up in jupyter which I ran through my venv.

I have Ubuntu, with a virtual env in python 3.7 python3.7 -m venv my_env Then I added the venv python site-packages folder to system path and it worked. Added following code to jupyter notebook and it was able to find pandas easily

import sys
sys.path.append('/home/aseem/venv/general_venv/lib/python3.7/site-packages')

This worked for me and seems like a very easy solution.

This is quite a weird bug. Like @iamjli mentioned, if I just reactivate the env by deactivating and activating again, it works fine, even though literally nothing else changed. So that means if you install package and jupyter at the same time, without reactivating env, it will not be able to import, until you reactivate. I just tested this again in a new virtualenv, it proved to be correct.

mkdir -p /tmp/test
cd /tmp/test
virtualenv -p python3 .venv
source .venv/bin/activate
pip install numpy && pip install jupyter && jupyter notebook

create a new notebook and try to import numpy. It failed. Try deactivate, and activate again, it works fine. I have a hard time believing such fundamental issue has existed since 2017 and remain unfixed.

On a side note: it has nothing to do with the path inside kernel.json or sys path not appended or anything.

I’ve tried summarizing options and common troubles in this Q&A: https://stackoverflow.com/questions/58068818/how-to-use-jupyter-notebooks-in-a-conda-environment

Same issue here, just installed sklearn by conda, by: conda install scikit-leanr I can import it from command line, but not from jupyter.

Well if it will only work if we install a new jupyter notebook in each new environment, what on earth is the point of having these stupid kernelspec files and separate kernels in the base jupyter notebook installation? Also the recommended way of handling multiple environments according to the docs is to use kernels and kernelspecs. But this doesn’t seem to work, instead everyone is doing this workaround of installing a new jupyter notebook on each environment.

Why is this issue closed? It is still WIDE open in my opinion!

My problem solved by install Jupyternotebook in the conda environment and use it there. The packages can be imported in the same environment.

Without this step, I think you are using Jupyternotebook from the general setting, for that import packages in a specific conda environment is not working.

For me, I installed Jupyter after creating an environment, but then was trying to run a module installed from the base env. I found by “jupyter kernelspec list” (https://github.com/jupyter/notebook/issues/2563), my kernel.json at C:\Users\username\Anaconda37\share\jupyter\kernels\python3\kernel.json was pointing to the python.exe in my working env. Changed the path and solved it.

This was an exhaustive description of python path setting. And here is my solution.

Do print(mypy) in the console to see where it’s imported from. Is it in the working directory? In that case, it will only be picked up when running in that directory. Otherwise, check sys.path for differences.

@takluyver Thanks for the blog post!

Check sys.executable anyway, it might be running the kernel somewhere different to where you expect.

edit: just noticed that this isn’t the jupyterlab thread. sorry for the spam. But maybe the problem is transferable. It works for my “normal” Jupyter notebooks

Thanks for the tips! However: I installed jupyterlab in its own conda env and installed everything in there via conda install … so it should be there, correct? Still - I cannot import it (restarting kernel, etc. does not work, I am using a python 2.7 kernel, it’s running on a Mac).

I was only using pip in the notebook to list the installed package. It was installed first with conda when I created the env and then using pip from a conventional terminal.

What if you use a proper terminal instead of trying to run !pip in the notebook?