superset: Migrations for 0.11 hangs with posgresql

OS Debian GNU/Linux 8 Postgresql 9.5 Caravel version 0.11.0

To reproduce :

pip install caravel --upgrade caravel db upgrade

Additional information

Webserver not running
No application using the database
Hangs even if db upgrade is called after postgresql restart

Postgres activity will migration hangs :

blocked_pid | blocked_user | blocking_pid | blocking_user |                         blocked_statement                          |                                  current_statement_in_blocking_process                                   
-------------+--------------+--------------+---------------+--------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------
        8050 | logs         |         8049 | logs          |                                                                   +| UPDATE alembic_version SET version_num='ab3d66c4246e' WHERE alembic_version.version_num = 'eca4694defa7'
             |              |              |               |             SELECT a.attname,                                     +| 
             |              |              |               |               pg_catalog.format_type(a.atttypid, a.atttypmod),    +| 
             |              |              |               |               (SELECT pg_catalog.pg_get_expr(d.adbin, d.adrelid)  +| 
             |              |              |               |                 FROM pg_catalog.pg_attrdef d                      +| 
             |              |              |               |                WHERE d.adrelid = a.attrelid AND d.adnum = a.attnum+| 
             |              |              |               |                AND a.atthasdef)                                   +| 
             |              |              |               |               AS DEFAULT,                                         +| 
             |              |              |               |               a.attnotnull, a.attnum, a.attrelid as table_oid     +| 
             |              |              |               |             FROM pg_catalog.pg_attribute a                        +| 
             |              |              |               |             WHERE a.attrelid = 16962                              +| 
             |              |              |               |             AND a.attnum > 0 AND NOT a.attisdropped               +| 
             |              |              |               |             ORDER BY a.attnum                                     +|

Migrations logs :

INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade 960c69cb1f5b -> f162a1dea4c4, d3format_by_metric
INFO  [alembic.runtime.migration] Running upgrade f162a1dea4c4 -> ad82a75afd82, Update models to support storing the queries.
INFO  [alembic.runtime.migration] Running upgrade ad82a75afd82 -> 3c3ffe173e4f, add_sql_string_to_table
INFO  [alembic.runtime.migration] Running upgrade 3c3ffe173e4f -> 41f6a59a61f2, database options for sql lab
INFO  [alembic.runtime.migration] Running upgrade 41f6a59a61f2 -> 4500485bde7d, allow_run_sync_async
INFO  [alembic.runtime.migration] Running upgrade 4500485bde7d -> 65903709c321, allow_dml
INFO  [alembic.runtime.migration] Running upgrade 41f6a59a61f2 -> 33d996bcc382
INFO  [alembic.runtime.migration] Running upgrade 33d996bcc382, 65903709c321 -> b347b202819b, empty message
INFO  [alembic.runtime.migration] Running upgrade b347b202819b -> 5e4a03ef0bf0, Add access_request table to manage requests to access datastores.
INFO  [alembic.runtime.migration] Running upgrade 5e4a03ef0bf0 -> eca4694defa7, sqllab_setting_defaults
INFO  [alembic.runtime.migration] Running upgrade eca4694defa7 -> ab3d66c4246e, add_cache_timeout_to_druid_cluster
INFO  [alembic.runtime.migration] Running upgrade eca4694defa7 -> 3b626e2a6783, Sync DB with the models.py.

Edit :

First query to hangs is :

SELECT ab_permission_view.id AS ab_permission_view_id, ab_permission_view.permission_id AS ab_permission_view_permission_id, ab_permission_view.view_menu_id AS ab_permission_view_view_menu_id

But even after killing it, migration still hangs at the same point

About this issue

Original URL
State: closed
Created 8 years ago
Reactions: 2
Comments: 20 (8 by maintainers)

Commits related to this issue

fix(plugin-chart-echarts): missing dnd control in tree chart (#1260) — committed to zhaoyongjie/incubator-superset by kgabryje 3 years ago
fix(plugin-chart-echarts): missing dnd control in tree chart (#1260) — committed to zhaoyongjie/incubator-superset by kgabryje 3 years ago
fix(plugin-chart-echarts): missing dnd control in tree chart (#1260) — committed to zhaoyongjie/incubator-superset by kgabryje 3 years ago
fix(plugin-chart-echarts): missing dnd control in tree chart (#1260) — committed to zhaoyongjie/incubator-superset by kgabryje 3 years ago

Most upvoted comments

Encountered hang with postgres 9.4, with postgres 9.6 it works perfectly.

sohelsarder on Jul 4, 2018

@rhunwicks thanks for the analysis! Can we add the session cleanup in a overriden add_permissions_menu in security/sqla/manager.py? Like the following untested:

diff --git a/flask_appbuilder/security/sqla/manager.py b/flask_appbuilder/security/sqla/manager.py
index c75ba548..138914e1 100644
--- a/flask_appbuilder/security/sqla/manager.py
+++ b/flask_appbuilder/security/sqla/manager.py
@@ -251,6 +251,10 @@ class SecurityManager(BaseSecurityManager):
                 log.error(c.LOGMSG_ERR_SEC_DEL_PERMISSION.format(str(e)))
                 self.get_session.rollback()
 
+    def add_permissions_menu(self, view_menu_name):
+        super(SecurityManager, self).add_permissions_menu(self, view_menu_name)
+        self.get_session.close()
+
     # ----------------------------------------------
     #       PRIMITIVES VIEW MENU
     #----------------------------------------------

xrmx on Oct 8, 2017

nope, had to kill the queries

GlombSt on Jan 8, 2017

I’ve done a bit more fiddling with this issue this AM and found one interesting tidbit: If the migration is run one-step at a time in sequence, the upgrade succeeds without hanging. Perhaps there’s something wrong with the branch/merge?

My starting point was ‘eca4694defa7’ I tried to just simply run upgrade, which should have installed ‘ab3d66c4246e’, ‘3b626e2a6783’, ‘ef8843b41dac’ and ‘b46fa1b0b39e’, but it hangs when attempting to install ‘3b626e2a6783’.

However, running each upgrade individually, one-after-the-other worked just fine: ./caravel/bin/caravel db upgrade ab3d66c4246e ./caravel/bin/caravel db upgrade 3b626e2a6783 ./caravel/bin/caravel db upgrade ef8843b41dac ./caravel/bin/caravel db upgrade b46fa1b0b39e

This isn’t so much a fix as it is a workaround as I can’t really explain what is going wrong with the straight upgrade… but it solved my immediate issue.

woel0007 on Oct 18, 2016