superset: ModuleNotFoundError installing Superset 2.0.1 in a fresh virtual environment

@zhaoyongjie I ended up using this one liner shared in another issue. It installs the exact dependencies for Apache Superset 2.0.1, from its requirements/base.txt:

$ curl -sS https://raw.githubusercontent.com/apache/superset/2.0.1/requirements/base.txt | \
       tail -n +10 | \
       awk -v ORS=" " '/^[A-z]/{print}' | \
       xargs pip install apache-superset==2.0.1

I still don’t understand why pip installing apache-superset==2.0.1 doesn’t work. This is a recurring issue with Superset for me.

@alanorth try this snippet

pip uninstall cryptography
pip uninstall pyopenssl
pip install cryptography==3.4.7

Thanks @XinbeiGong, it’s worked !

I think it would be necessary to update the installation documentation, because it is a bit tedious to search all over the internet to install and operate this magnificent tool.

Indirectly this repels a lot of potential users.

Closing because we realized that pip install apache-superset is apparently not meant to work. In other threads the maintainers have commented that we’re supposed to be using the requirements file with pip. 🤷

For better or worse, now I always install Superset using a one liner based on the example given earlier in this issue:

$ curl -sS https://raw.githubusercontent.com/apache/superset/2.0.1/requirements/base.txt | \
       tail -n +10 | \
       awk -v ORS=" " '/^[A-z]/{print}' | \
       xargs pip install apache-superset==2.0.1

FWIW, there is a merged PR that removes the use of the private cryptography API _Certificate class by replacing it with the public version (Certificate) and at the same time pins the cryptography version.

https://github.com/apache/superset/pull/22744

I got the same error on Fedora 37.

Just so you know I couldn’t get it to work with Python default version on Fedora (3.11) since it’s not compatible with lots of libraries and dependencies in Superset.

In case anyone is interested I first had to build Python 3.8 from source:

sudo dnf groupinstall "Development Tools"
sudo dnf install python3-devel openssl-devel zlib-devel bzip2-devel sqlite-devel libffi-devel
wget https://www.python.org/ftp/python/3.8.16/Python-3.8.16.tgz
tar xzf Python-3.8.16.tgz Python-3.8.16/
cd Python-3.8.16/
./configure --enable-optimizations --with-ensurepip=install --with-lto=full --enable-profiling
sudo make altinstall

This last line helps you leave your system-wide Python 3.11 intact. You can run python3.8 after that and create a superset virtual enrivonment using

python3.8 -m venv superset

After activating the new virtual environment and installing apache-superset with pip I got errors when I ran

superset db upgrade

as stated in the documentation (https://superset.apache.org/docs/installation/installing-superset-from-scratch/):

(superset) [alonso@fedora pythonvenvs]$ superset db upgrade
/home/alonso/pythonvenvs/superset/lib/python3.8/site-packages/pandas/compat/__init__.py:124: UserWarning: Could not import the lzma module. Your installed Python is incomplete. Attempting to use lzma compression will result in a RuntimeError.
  warnings.warn(msg)
Traceback (most recent call last):
  File "/home/alonso/pythonvenvs/superset/bin/superset", line 5, in <module>
    from superset.cli.main import superset
  File "/home/alonso/pythonvenvs/superset/lib/python3.8/site-packages/superset/__init__.py", line 21, in <module>
    from superset.app import create_app
  File "/home/alonso/pythonvenvs/superset/lib/python3.8/site-packages/superset/app.py", line 23, in <module>
    from superset.initialization import SupersetAppInitializer
  File "/home/alonso/pythonvenvs/superset/lib/python3.8/site-packages/superset/initialization/__init__.py", line 33, in <module>
    from superset.extensions import (
  File "/home/alonso/pythonvenvs/superset/lib/python3.8/site-packages/superset/extensions/__init__.py", line 32, in <module>
    from superset.utils.cache_manager import CacheManager
  File "/home/alonso/pythonvenvs/superset/lib/python3.8/site-packages/superset/utils/cache_manager.py", line 24, in <module>
    from superset.utils.core import DatasourceType
  File "/home/alonso/pythonvenvs/superset/lib/python3.8/site-packages/superset/utils/core.py", line 76, in <module>
    from cryptography.hazmat.backends.openssl.x509 import _Certificate
ModuleNotFoundError: No module named 'cryptography.hazmat.backends.openssl.x509'

In order to fix this I had to run the lines @zhaoyongjie provided, which I reproduce below:

@alanorth try this snippet

pip uninstall cryptography
pip uninstall pyopenssl
pip install cryptography==3.4.7

After that I ran

export FLASK_APP=superset
superset db upgrade

And I got the following errors:

  File "/home/alonso/pythonvenvs/superset/lib/python3.8/site-packages/superset/initialization/__init__.py", line 476, in configure_data_sources
    ConnectorRegistry.register_sources(module_datasource_map)
  File "/home/alonso/pythonvenvs/superset/lib/python3.8/site-packages/superset/connectors/connector_registry.py", line 42, in register_sources
    module_obj = __import__(module_name, fromlist=class_names)
  File "/home/alonso/pythonvenvs/superset/lib/python3.8/site-packages/superset/connectors/sqla/__init__.py", line 17, in <module>
    from . import models, views
  File "/home/alonso/pythonvenvs/superset/lib/python3.8/site-packages/superset/connectors/sqla/views.py", line 28, in <module>
    from wtforms.ext.sqlalchemy.fields import QuerySelectField
ModuleNotFoundError: No module named 'wtforms.ext'

@jsanko9 helped with this, I’m partly quoting the reply:

pip uninstall WTForms 
pip install WTForms==2.3.2

With that I could successfully run

superset db upgrade

Thanks for that info @jsanko9. It looks like we can pin WTForms to >=2.3.2 ❤️.0 and that should help with that issue. It looks like there may be a similar issue with the cryptography package. We know that cryptography==3.4.7 works. We currently require "cryptography>=3.3.2". Does anyone know which version is failing? cc @zhaoyongjie?

this one liner worked for me too, thank you!