superset: superset-init-db job on 1.5.0rc4 image fails with no error messages
I’m using helm chart 0.6.0 with 1.5.0rc4, however superset-init-db fails with no obvious errors:
Upgrading DB schema...
logging was configured successfully
2022-04-25 13:32:15,777:INFO:superset.utils.logging_configurator:logging was configured successfully
2022-04-25 13:32:15,781:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
Loaded your LOCAL configuration at [/app/pythonpath/superset_config.py]
Here is my superset config:
import os
from cachelib.redis import RedisCache
def env(key, default=None):
return os.getenv(key, default)
MAPBOX_API_KEY = env('MAPBOX_API_KEY', '')
CACHE_CONFIG = {
'CACHE_TYPE': 'redis',
'CACHE_DEFAULT_TIMEOUT': 300,
'CACHE_KEY_PREFIX': 'superset_',
'CACHE_REDIS_HOST': env('REDIS_HOST'),
'CACHE_REDIS_PORT': env('REDIS_PORT'),
'CACHE_REDIS_PASSWORD': env('REDIS_PASSWORD'),
'CACHE_REDIS_DB': env('REDIS_DB', 1),
}
DATA_CACHE_CONFIG = CACHE_CONFIG
SQLALCHEMY_DATABASE_URI = f"postgresql+psycopg2://{env('DB_USER')}:{env('DB_PASS')}@{env('DB_HOST')}:{env('DB_PORT')}/{env('DB_NAME')}"
SQLALCHEMY_TRACK_MODIFICATIONS = True
SECRET_KEY = env('SECRET_KEY', 'thisISaSECRET_1234')
# Flask-WTF flag for CSRF
WTF_CSRF_ENABLED = True
# Add endpoints that need to be exempt from CSRF protection
WTF_CSRF_EXEMPT_LIST = []
# A CSRF token that expires in 1 year
WTF_CSRF_TIME_LIMIT = 60 * 60 * 24 * 365
class CeleryConfig(object):
CELERY_IMPORTS = ('superset.sql_lab', )
CELERY_ANNOTATIONS = {'tasks.add': {'rate_limit': '10/s'}}
BROKER_URL = f"redis://:{env('REDIS_PASSWORD')}@{env('REDIS_HOST')}:{env('REDIS_PORT')}/0"
CELERY_RESULT_BACKEND = f"redis://:{env('REDIS_PASSWORD')}@{env('REDIS_HOST')}:{env('REDIS_PORT')}/0"
CELERY_CONFIG = CeleryConfig
RESULTS_BACKEND = RedisCache(
host=env('REDIS_HOST'),
password=env('REDIS_PASSWORD'),
port=env('REDIS_PORT'),
key_prefix='superset_results'
)
# Overrides
# cache_config
FILTER_STATE_CACHE_CONFIG = {
'CACHE_TYPE': 'redis',
'CACHE_DEFAULT_TIMEOUT': 600,
'CACHE_KEY_PREFIX': 'superset_filter_state_',
'CACHE_REDIS_HOST': env('REDIS_HOST'),
'CACHE_REDIS_PORT': env('REDIS_PORT'),
'CACHE_REDIS_PASSWORD': env('REDIS_PASSWORD'),
'CACHE_REDIS_DB': env('REDIS_DB', 1),
}
EXPLORE_FORM_DATA_CACHE_CONFIG = {
'CACHE_TYPE': 'redis',
'CACHE_DEFAULT_TIMEOUT': 600,
'CACHE_KEY_PREFIX': 'superset_explore_form_data_',
'CACHE_REDIS_HOST': env('REDIS_HOST'),
'CACHE_REDIS_PORT': env('REDIS_PORT'),
'CACHE_REDIS_PASSWORD': env('REDIS_PASSWORD'),
'CACHE_REDIS_DB': env('REDIS_DB', 1),
}
I realized superset fab create-admin is the part that is failing [returning exit code 1]
My superset installation is already in place [older version 1.4.2] and I can login to superset. I can see all dashboards, create databases, datasets, …
Essentially everything works, except this init job.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 1
- Comments: 23 (6 by maintainers)
Same thing happens on 1.5.2 image :
Chart version: 0.7.1
I’m struggling here with the same error, tried many versions of the chart, no success so far. Is there any workaround?
After manually applying changes from https://github.com/apache/superset/pull/22991 I can see the following:
superset 1.5.3 postgres 12.12 chart 0.7.6 (also tested with 0.5.10 and 0.8.2)
The
alembic_versiontable contains one row:Inspecting
/app/superset/migrations/versionsinside the container I found that revision9d8a8d575284is not present there. I created it and pasted there contents of this file but it didn’t help:Apparently, there are several revisions missing. I took the list from here and put it through some bash’n’grep until I got this list of missing revisions:
I downgraded from helm chart 0.7.7 to 0.7.6 and it worked.
I’ve resolved this issue with removing all deployments and re-deploy after that. Looks like when you migrate from one version to another sometimes there are some problems with DB initialisation
For me it works with
1.5.1but fails withlatest, just FYIHaving the same issue with
1.5.0or1.5.1rc1tags. Thesuperset-init-jobfails but nothing seems wrong in the logs. Deleting the pod, deployment, restarting, nothing helps.Here’s a full log:
Upon further inspection, this command
superset db upgradereturnsexit code 1.And upon even further inspection, the hidden errors appear to be related to this: https://github.com/miguelgrinberg/Flask-Migrate/issues/434 . There’s a hackish way to see the actual error message, though, it’s in the linked issue.
I have the same error, superset-init-db never starts
`postgresql 14:51:46.43 postgresql 14:51:46.43 Welcome to the Bitnami postgresql container postgresql 14:51:46.43 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-postgresql postgresql 14:51:46.43 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-postgresql/issues postgresql 14:51:46.43 postgresql 14:51:46.44 INFO ==> ** Starting PostgreSQL setup ** postgresql 14:51:46.47 INFO ==> Validating settings in POSTGRESQL_* env vars… postgresql 14:51:46.48 INFO ==> Loading custom pre-init scripts… postgresql 14:51:46.48 INFO ==> Initializing PostgreSQL database… postgresql 14:51:46.49 INFO ==> pg_hba.conf file not detected. Generating it… postgresql 14:51:46.50 INFO ==> Generating local authentication configuration postgresql 14:51:46.51 INFO ==> Deploying PostgreSQL with persisted data… postgresql 14:51:46.52 INFO ==> Configuring replication parameters postgresql 14:51:46.56 INFO ==> Configuring fsync postgresql 14:51:46.59 INFO ==> Configuring synchronous_replication postgresql 14:51:46.63 INFO ==> Loading custom scripts… postgresql 14:51:46.63 INFO ==> Enabling remote connections postgresql 14:51:46.64 INFO ==> ** PostgreSQL setup finished! **
postgresql 14:51:46.66 INFO ==> ** Starting PostgreSQL ** 2022-05-04 14:51:46.677 GMT [1] FATAL: database files are incompatible with server 2022-05-04 14:51:46.677 GMT [1] DETAIL: The data directory was initialized by PostgreSQL version 11, which is not compatible with this version 14.2.`
This is for 0.6.0 Chart.
When I change to 0.5.10
superset-init-db does not start
client.go:596: [debug] superset-init-db: Jobs active: 1, jobs failed: 0, jobs succeeded: 0 Error: INSTALLATION FAILED: failed post-install: timed out waiting for the condition helm.go:84: [debug] failed post-install: timed out waiting for the condition