sqlalchemy: MetaData.reflect() fails on FTS5 db

I can not debug this further. The error below occurs when trying to reflect on sqlite:////home/anatoli/.cache/tracker/meta.db which seems to use https://sqlite.org/fts5.html extension.

Traceback (most recent call last):
  File "/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
    cursor, statement, parameters, context
  File "/venv/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: no such tokenizer: TrackerTokenizer

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "sqlalchemy_schemadisplay.py", line 278, in <module>
    sys.exit(main(arg_dburl, arg_outpng))
  File "sqlalchemy_schemadisplay.py", line 250, in main
    concentrate=False # Don't try to join the relation lines together
  File "sqlalchemy_schemadisplay.py", line 170, in create_schema_graph
    metadata.reflect()
  File "/venv/lib/python3.7/site-packages/sqlalchemy/sql/schema.py", line 4249, in reflect
    Table(name, self, **reflect_opts)
  File "<string>", line 2, in __new__
  File "/venv/lib/python3.7/site-packages/sqlalchemy/util/deprecations.py", line 128, in warned
    return fn(*args, **kwargs)
  File "/venv/lib/python3.7/site-packages/sqlalchemy/sql/schema.py", line 496, in __new__
    metadata._remove_table(name, schema)
  File "/venv/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/venv/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 153, in reraise
    raise value
  File "/venv/lib/python3.7/site-packages/sqlalchemy/sql/schema.py", line 491, in __new__
    table._init(name, metadata, *args, **kw)
  File "/venv/lib/python3.7/site-packages/sqlalchemy/sql/schema.py", line 585, in _init
    resolve_fks=resolve_fks,
  File "/venv/lib/python3.7/site-packages/sqlalchemy/sql/schema.py", line 609, in _autoload
    _extend_on=_extend_on,
  File "/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1612, in run_callable
    return callable_(self, *args, **kwargs)
  File "/venv/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 431, in reflecttable
    table, include_columns, exclude_columns, resolve_fks, **opts
  File "/venv/lib/python3.7/site-packages/sqlalchemy/engine/reflection.py", line 646, in reflecttable
    table_name, schema, **table.dialect_kwargs
  File "/venv/lib/python3.7/site-packages/sqlalchemy/engine/reflection.py", line 373, in get_columns
    self.bind, table_name, schema, info_cache=self.info_cache, **kw
  File "<string>", line 2, in get_columns
  File "/venv/lib/python3.7/site-packages/sqlalchemy/engine/reflection.py", line 56, in cache
    ret = fn(self, con, *args, **kw)
  File "/venv/lib/python3.7/site-packages/sqlalchemy/dialects/sqlite/base.py", line 1638, in get_columns
    connection, "table_info", table_name, schema=schema
  File "/venv/lib/python3.7/site-packages/sqlalchemy/dialects/sqlite/base.py", line 2050, in _get_table_pragma
    cursor = connection.execute(statement)
  File "/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 982, in execute
    return self._execute_text(object_, multiparams, params)
  File "/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1155, in _execute_text
    parameters,
  File "/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1253, in _execute_context
    e, statement, parameters, cursor, context
  File "/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1473, in _handle_dbapi_exception
    util.raise_from_cause(sqlalchemy_exception, exc_info)
  File "/venv/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/venv/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 152, in reraise
    raise value.with_traceback(tb)
  File "/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
    cursor, statement, parameters, context
  File "/venv/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such tokenizer: TrackerTokenizer
[SQL: PRAGMA main.table_info("fts5")]
(Background on this error at: http://sqlalche.me/e/e3q8)

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 15 (15 by maintainers)

Most upvoted comments

sorry this is difficult. sqlite is really a much more rough database than people realize. handy for sure but its issue and development process is a bit opaque.