mlflow: "ValueError: invalid interpolation syntax" with username containing "%"

System information

• Have I written custom code (as opposed to using a stock example script provided in MLflow):
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
• MLflow installed from (source or binary): pip install mlflow==1.0.0
• MLflow version (run mlflow --version): mlflow, version 1.0.0
• Python version: Python 3.6.8
• **npm version (if running the dev UI): N/A
• Exact command to reproduce:

mlflow server \
    --backend-store-uri "postgresql://mlflow%40mlflow:__password__@mydb.postgres.database.azure.com/mlflow" \
    --default-artifact-root "N/A" \
    --host 0.0.0.0 \
    --port 3000

Describe the problem

The hosted version of microsoft azure (stupidly) requires an @ symbol in the username. Regaredless of whether or not in the backend-store-uri you uri_encode it to become a %40, it will be escaped before being saved in an alembic config.

Source code / logs

2019/06/21 10:15:59 INFO mlflow.store.sqlalchemy_store: Creating initial MLflow database tables...
2019/06/21 10:16:04 INFO mlflow.store.db.utils: Updating database tables at postgresql://mlflow%40mlflow:__password__@mydb.postgres.database.azure.com/mlflow
2019/06/21 10:16:04 ERROR mlflow.cli: Error initializing backend store
2019/06/21 10:16:04 ERROR mlflow.cli: invalid interpolation syntax in 'postgresql://mlflow%40mlflow:__password__@mydb.postgres.database.azure.com/mlflow' at position 19
Traceback (most recent call last):
  File "/home/sam/.virtualenvs/mlflow-tracking-server/lib/python3.6/site-packages/mlflow/cli.py", line 247, in server
    _get_store(backend_store_uri, default_artifact_root)
  File "/home/sam/.virtualenvs/mlflow-tracking-server/lib/python3.6/site-packages/mlflow/server/handlers.py", line 35, in _get_store
    _store = SqlAlchemyStore(store_dir, artifact_root)
  File "/home/sam/.virtualenvs/mlflow-tracking-server/lib/python3.6/site-packages/mlflow/store/sqlalchemy_store.py", line 85, in __init__
    SqlAlchemyStore._initialize_tables(self.engine)
  File "/home/sam/.virtualenvs/mlflow-tracking-server/lib/python3.6/site-packages/mlflow/store/sqlalchemy_store.py", line 103, in _initialize_tables
    _upgrade_db(engine_url)
  File "/home/sam/.virtualenvs/mlflow-tracking-server/lib/python3.6/site-packages/mlflow/store/db/utils.py", line 53, in _upgrade_db
    config = _get_alembic_config(url)
  File "/home/sam/.virtualenvs/mlflow-tracking-server/lib/python3.6/site-packages/mlflow/store/db/utils.py", line 36, in _get_alembic_config
    config.set_main_option('sqlalchemy.url', db_url)
  File "/home/sam/.virtualenvs/mlflow-tracking-server/lib/python3.6/site-packages/alembic/config.py", line 237, in set_main_option
    self.set_section_option(self.config_ini_section, name, value)
  File "/home/sam/.virtualenvs/mlflow-tracking-server/lib/python3.6/site-packages/alembic/config.py", line 264, in set_section_option
    self.file_config.set(section, name, value)
  File "/usr/lib/python3.6/configparser.py", line 1193, in set
    super().set(section, option, value)
  File "/usr/lib/python3.6/configparser.py", line 894, in set
    value)
  File "/usr/lib/python3.6/configparser.py", line 402, in before_set
    "position %d" % (value, tmp_value.find('%')))

Quick hack to fix

I modified the _get_alembic_config function from this:

def _get_alembic_config(db_url, alembic_dir=None):
    from alembic.config import Config
    final_alembic_dir = os.path.join(_get_package_dir(), 'alembic')\
        if alembic_dir is None else alembic_dir
    config = Config(os.path.join(final_alembic_dir, 'alembic.ini'))
    config.set_main_option('script_location', final_alembic_dir)
    config.set_main_option('sqlalchemy.url', db_url)
    return config

To this:

def _get_alembic_config(db_url, alembic_dir=None):
    from alembic.config import Config
    final_alembic_dir = os.path.join(_get_package_dir(), 'alembic')\
        if alembic_dir is None else alembic_dir
    config = Config(os.path.join(final_alembic_dir, 'alembic.ini'))
    config.set_main_option('script_location', final_alembic_dir)
    # NEW CODE HERE TO ESCAPE THE '%'
    db_url = db_url.replace('%', '%%')
    config.set_main_option('sqlalchemy.url', db_url)
    return config

And I am now up and running. I’m also a bit confused about why this has not come up yet in an issue (couldn’t find one anyway) as special characters in a password are quite common. Normally to work around I would just change the name or the password but in this case I cannot because azure decided to require an @ in all of its database usernames.