superset: ModuleNotFoundError: No module named 'sqlglot' error across many containers

Same goes for me and I have done exactly the same steps. Trying to push this in my company but is hard when the local thing just doesn’t works …

Thanks @akshayjain3450

Resolved ‘sqlglot’ error after adding sqlglot==20.8.0.

All containers are up, but “http://localhost:8088/superset/welcome/” is continuously loading.

When I did port forwarding, it shows me favicon icon. http://localhost:8089/static/assets/images/favicon.png

Also not able to connect 5432 database.

Are you able to run, Can you suggest something?

I found one more error with postgres container where it is looking for a test table which does not exist. What I did is added CREATE DATABASE test command in ./docker/examples-init.sh. And tried again. I am still in the process to confirm if the UI opens up or not.

My two containers are down even after the module fix:

1. superset_websocket with error: config.json file not found
2. superset_tests_worker with error: 2024-02-08 16:45:28 2024-02-08 11:15:28,136:ERROR:flask_appbuilder.security.sqla.manager:DB Creation and initialization failed: (psycopg2.OperationalError) connection to server at "localhost" (::1), port 5432 failed: Connection refused 2024-02-08 16:45:28 Is the server running on that host and accepting TCP/IP connections? 2024-02-08 16:45:28 connection to server at "localhost" (127.0.0.1), port 5432 failed: FATAL: database "test" does not exist Also, I am not able to login with username: admin and password: admin

Need help to make this working.

@SbstnErhrdt Thanks for your reply. You did not specify what machine you are using.

• Even if this is the case, did you had to specify platform: linux/x86_64/v8 in the docker file of each container?
• What are your logs for superset-websocket now?
• I also had to create a superset_config.py file and export its path to my superset dir so that the app reads it and override the secret key that is weak by default.
• If I checkout the tag_3.1.0, I get a new log error for superset-websocket that says its looking for a config.json file now?? when its clearly using config.py for configuration

superset-websocket@0.0.1 start node dist/index.js start config.json file not found {“date”:“Wed Feb 07 2024 16:00:22 GMT+0000 (Coordinated Universal Time)”,“error”:{},“exception”:true,“level”:“error”,“message”:“uncaughtException: Please provide a JWT secret at least 32 bytes long\nError: Please provide a JWT secret at least 32 bytes long\n at Object.<anonymous> (/home/superset-websocket/dist/index.js:76:11)\n at Module._compile (node:internal/modules/cjs/loader:1198:14)\n at Object.Module._extensions…js (node:internal/modules/cjs/loader:1252:10)\n at Module.load (node:internal/modules/cjs/loader:1076:32)\n at Function.Module._load (node:internal/modules/cjs/loader:911:12)\n at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)\n at node:internal/main/run_main_module:22:47”,“os”:{“loadavg”:[29.07,19.59,13.16],“uptime”:87469.63},“process”:{“argv”:[“/usr/local/bin/node”,“/home/superset-websocket/dist/index.js”,“start”],“cwd”:“/home/superset-websocket”,“execPath”:“/usr/local/bin/node”,“gid”:1000,“memoryUsage”:{“arrayBuffers”:74962,“external”:948746,“heapTotal”:18386944,“heapUsed”:15244176,“rss”:0},“pid”:22,“uid”:1000,“version”:“v16.20.2”},“stack”:“Error: Please provide a JWT secret at least 32 bytes long\n at Object.<anonymous> (/home/superset-websocket/dist/index.js:76:11)\n at Module._compile (node:internal/modules/cjs/loader:1198:14)\n at Object.Module._extensions…js (node:internal/modules/cjs/loader:1252:10)\n at Module.load (node:internal/modules/cjs/loader:1076:32)\n at Function.Module._load (node:internal/modules/cjs/loader:911:12)\n at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)\n at node:internal/main/run_main_module:22:47”,“trace”:[{“column”:11,“file”:“/home/superset-websocket/dist/index.js”,“function”:null,“line”:76,“method”:null,“native”:false},{“column”:14,“file”:“node:internal/modules/cjs/loader”,“function”:“Module._compile”,“line”:1198,“method”:“_compile”,“native”:false},{“column”:10,“file”:“node:internal/modules/cjs/loader”,“function”:“Module._extensions…js”,“line”:1252,“method”:“.js”,“native”:false},{“column”:32,“file”:“node:internal/modules/cjs/loader”,“function”:“Module.load”,“line”:1076,“method”:“load”,“native”:false},{“column”:12,“file”:“node:internal/modules/cjs/loader”,“function”:“Module._load”,“line”:911,“method”:“_load”,“native”:false},{“column”:12,“file”:“node:internal/modules/run_main”,“function”:“Function.executeUserEntryPoint [as runMain]”,“line”:81,“method”:“executeUserEntryPoint [as runMain]”,“native”:false},{“column”:47,“file”:“node:internal/main/run_main_module”,“function”:null,“line”:22,“method”:null,“native”:false}]}

I resolved the issues by checking out the latest stable release

My system:

Ubuntu 22.04.3 LTS (GNU/Linux 5.15.0-92-generic x86_64)

Steps

git clone https://github.com/apache/superset.git
git checkout tags/3.1.0
docker compose up

If all runs smoothly

docker compose up -d

Quick note that we met with the devx sub-team and I said I’d pick up doing a set of improvements for the docker-compose workflows addressing the core issues here. Main idea is referencing the branch’s Dockerfile as opposed to a baked image like we do now.

Hi, You might want to install it yourself (by editing to .superset/docker/requirements-local.txt).

This dependency was cherry picked and added to master, how ever you all still using outdated images. (For example me who use based on 3.1.0 multiplatform docker image.)

+sqlglot==20.8.0

Cheers

The config.json file issue with WebSocket is in the master also. It’s not just related to tags/3.1.0. The solution for ModuleNotFoundError, for now, is to create a requirements-local.txt in ./docker path and put this sqlglot==20.8.0. Then if you run docker-compose again, you will not find this error. @yashagv @vikramwalia @ricokali96 @andrekef

@rusackas We are missing the dependency in the docker-image being used by the docker-compose. The image seems to be old and does not have the additional changes of base.txt. This needs to be fixed.

same issue on Amazon x86-64 EC2 instance with Ubuntu

Virtualization: amazon
Operating System: Ubuntu 22.04.3 LTS
          Kernel: Linux 6.2.0-1018-aws
    Architecture: x86-64
 Hardware Vendor: Amazon EC2
  Hardware Model: t3.large

the current behavior probably helps us find bugs on master a lot faster 😉

Quite true! You can tell when master breaks as people come flooding into Slack reporting the same problem. But I don’t think using new users as test subjects is probably good for the long-term health of the project. I expect for every person who reports a GitHub issue about broken master branch, many more give up silently and say Superset is not ready for production.

I will put it on the town hall list, good idea.

Digging a bit into this, I understand some things, but I’m not 100% clear on how this is setting or exactly how things are supposed to work here. But few things I know

docker-compose.yml has this ->

x-superset-image: &superset-image apachesuperset.docker.scarf.sh/apache/superset:${TAG:-latest-dev}

Which points to latest-dev which right now is 3.1.0 which does not have sqlglot. So if you git checkout 3.1.0 you can docker-compose up and things line up.

Pointing to master-dev seems more appropriate and much more likely to work, but if you’re on arm we don’t build that particular variation on these days.

With this PR -> https://github.com/apache/superset/pull/27146 we’ll be having multi-platform build for master, so that would make things work better.

Though I’m not sure what’s a normal setup for docker-compose here and how the repo is supposed to line up with the images. My best bet is recent master should work on top of a recent image build off of master (as in the master-dev tag), but that seems fragile, there’s no guarantee that any particular SHA should match the latest image. I guess if you have a freshly rebases branch and we have a fresh image, things should line up most of the time.

Thanks @akshayjain3450 Resolved ‘sqlglot’ error after adding sqlglot==20.8.0.

All containers are up, but “http://localhost:8088/superset/welcome/” is continuously loading. When I did port forwarding, it shows me favicon icon. http://localhost:8089/static/assets/images/favicon.png Also not able to connect 5432 database.

Are you able to run, Can you suggest something?

Are we looking for any update soon on this.

not resolved yet ! what [SbstnErhrdt] said does not work for me.

Pinging @betodealmeida and @john-bodley since it sounds related to their recent consolidation efforts.