Zappa: werkzeug dependency version conflict for zappa and flask

Context

I use pyenv and pipenv to manage the python version and virtualenv for my project. I have zappa installed successfully, but an attempt to update to the latest packages via pipenv update fails due to a dependency error. The relevant context from the dependency resolver is:

There are incompatible versions in the resolved dependencies: werkzeug<1.0 (from zappa==0.52.0->-r /tmp/pipenv61nng6jnrequirements/pipenv-oxt4aw92-constraints.txt (line 4)) werkzeug>=2.0 (from flask==2.0.1->flask-cors==3.0.10->-r /tmp/pipenv61nng6jnrequirements/pipenv-oxt4aw92-constraints.txt (line 2)) I think this is because flask 2.x was recently released, and it naturally is requiring werkzeug 2.0 or greater. I saw there is an old issue #818 from back in 2020 where zappa pinned the wekzeug dependency to less than 1.0, but it’s not clear (to me) where the import error from the original issue was happening.

If it was in Flask, would it make sense to test this again now that new major versions of Flask and werkzeug have been released? If it was in zappa, I can’t seem to find the code that references “secure_filename” in the original issue. However, it looks like the function has just been relocated so the following would work:

from werkzeug.utils import secure_filename

Expected Behavior

I expected that the dependencies should be resolvable. I suppose that if the dependency resolver was clever enough, it would determine that no update was possible and just let me know everything was as up-to-date as possible, but I’m wondering if the old werkzeug constraint is actually valid.

Actual Behavior

Dependency resolution failed because the main project is pinned to an old version of wekzeug but the newer Flask dependency requires a newer werkzeug.

Possible Fix

One idea would be to simply require a version of Flask < 2.0. I don’t really like this, as I’d think Zappa should probably be tracking Flask improvements. Another idea would be to test whether allowing newer Flask and wekzeug dependencies will now actually work together.

Steps to Reproduce

waldbiec@waldbiec-Latitude-7490:~/temp/zappa-dev$ cat Pipfile 
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
zappa = "*"
flask-cors = "*"
flask-talisman = "*"
flask-seasurf = "*"

[requires]
python_version = "3.8"
waldbiec@waldbiec-Latitude-7490:~/temp/zappa-dev$ pipenv install
Creating a virtualenv for this project…
Pipfile: /home/waldbiec/temp/zappa-dev/Pipfile
Using /home/waldbiec/.pyenv/versions/3.8.6/bin/python3.8 (3.8.6) to create virtualenv…
⠹ Creating virtual environment...created virtual environment CPython3.8.6.final.0-64 in 108ms
  creator CPython3Posix(dest=/home/waldbiec/.virtualenvs/zappa-dev-830jh23n, clear=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/waldbiec/.local/share/virtualenv)
    added seed packages: pip==21.0.1, setuptools==56.1.0, wheel==0.36.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

✔ Successfully created virtual environment! 
Virtualenv location: /home/waldbiec/.virtualenvs/zappa-dev-830jh23n
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Building requirements...
Resolving dependencies...
✘ Locking Failed! 
[ResolutionFailure]:   File "/home/waldbiec/.local/lib/python3.6/site-packages/pipenv/resolver.py", line 785, in _main
[ResolutionFailure]:       resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages)
[ResolutionFailure]:   File "/home/waldbiec/.local/lib/python3.6/site-packages/pipenv/resolver.py", line 746, in resolve_packages
[ResolutionFailure]:       results, resolver = resolve(
[ResolutionFailure]:   File "/home/waldbiec/.local/lib/python3.6/site-packages/pipenv/resolver.py", line 728, in resolve
[ResolutionFailure]:       return resolve_deps(
[ResolutionFailure]:   File "/home/waldbiec/.local/lib/python3.6/site-packages/pipenv/utils.py", line 1378, in resolve_deps
[ResolutionFailure]:       results, hashes, markers_lookup, resolver, skipped = actually_resolve_deps(
[ResolutionFailure]:   File "/home/waldbiec/.local/lib/python3.6/site-packages/pipenv/utils.py", line 1093, in actually_resolve_deps
[ResolutionFailure]:       resolver.resolve()
[ResolutionFailure]:   File "/home/waldbiec/.local/lib/python3.6/site-packages/pipenv/utils.py", line 818, in resolve
[ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: Could not find a version that matches werkzeug<1.0,>=2.0 (from zappa==0.52.0->-r /tmp/pipenv4_gls8o7requirements/pipenv-ioee6zwm-constraints.txt (line 4))
Tried: 0.1, 0.2, 0.3, 0.3.1, 0.4, 0.4.1, 0.5, 0.5.1, 0.6, 0.6.1, 0.6.2, 0.7, 0.7.1, 0.7.2, 0.8, 0.8.1, 0.8.2, 0.8.3, 0.9, 0.9.1, 0.9.2, 0.9.3, 0.9.4, 0.9.5, 0.9.6, 0.10, 0.10.1, 0.10.2, 0.10.2, 0.10.4, 0.10.4, 0.11, 0.11, 0.11.1, 0.11.1, 0.11.2, 0.11.2, 0.11.3, 0.11.3, 0.11.4, 0.11.4, 0.11.5, 0.11.5, 0.11.6, 0.11.6, 0.11.7, 0.11.7, 0.11.8, 0.11.8, 0.11.9, 0.11.9, 0.11.10, 0.11.10, 0.11.11, 0.11.11, 0.11.12, 0.11.12, 0.11.13, 0.11.13, 0.11.14, 0.11.14, 0.11.15, 0.11.15, 0.12, 0.12, 0.12.1, 0.12.1, 0.12.2, 0.12.2, 0.13, 0.13, 0.14, 0.14, 0.14.1, 0.14.1, 0.15.0, 0.15.0, 0.15.1, 0.15.1, 0.15.2, 0.15.2, 0.15.3, 0.15.3, 0.15.4, 0.15.4, 0.15.5, 0.15.5, 0.15.6, 0.15.6, 0.16.0, 0.16.0, 0.16.1, 0.16.1, 1.0.0, 1.0.0, 1.0.1, 1.0.1, 2.0.0, 2.0.0, 2.0.1, 2.0.1
Skipped pre-versions: 1.0.0rc1, 1.0.0rc1, 2.0.0rc1, 2.0.0rc1, 2.0.0rc2, 2.0.0rc2, 2.0.0rc3, 2.0.0rc3, 2.0.0rc4, 2.0.0rc4, 2.0.0rc5, 2.0.0rc5
There are incompatible versions in the resolved dependencies:
  werkzeug<1.0 (from zappa==0.52.0->-r /tmp/pipenv4_gls8o7requirements/pipenv-ioee6zwm-constraints.txt (line 4))
  werkzeug>=2.0 (from flask==2.0.1->flask-cors==3.0.10->-r /tmp/pipenv4_gls8o7requirements/pipenv-ioee6zwm-constraints.txt (line 3))

Your Environment

Zappa version used: zappa==0.52.0
Operating System and Python version: Ubuntu Linux 18.04 / Python 3.8.6
The output of pip freeze: No packages installed
Link to your project (optional): N/A
Your zappa_settings.json: N/A

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 7
Comments: 25 (1 by maintainers)

Most upvoted comments

I agree. Basically using Werkzeug<=1 limits us to only 1.x of flask. After reading through the reasoning for the pinning outlined here https://github.com/Miserlou/Zappa/issues/2036 I don’t believe this should be a blocking reason as to why we don’t bump Werkzeug to 2.x

Verified that it works with Flask 2.0.1 with Werkzeug>=2.0. I have a fork with the change as well and have been using this on a local monkey patch for a while.

There are no tests for the issue in this project and that makes sense because the secure_filename feature is not used here. That function has simply moved into a utils package. from werkzeug.utils import secure_filename should fix the problems of those that are using secure_filename in its since deprecated path.

fergyfresh on Jun 5, 2021

Any ETA as to when we might see this updated in PyPi? If the maintainers are waiting for a PR, I could put one together.

cwaldbieser on Jul 20, 2021

Just wondering if someone from the project has any insight as to the disposition of this issue? Are there any plans to handle it in the near term or even eventually? Seems like it’s low-hanging fruit that could help out a lot of deployers.

cwaldbieser on Aug 13, 2021

I confirm this is a serious problem. Flask is now running with Werkzeug>=2.0. So a Pipfile as simple as the following will crash.

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
flask = "*"
zappa = "*"

[dev-packages]

[requires]
python_version = "3.8"

Output

There are incompatible versions in the resolved dependencies:
  werkzeug<1.0 (from zappa==0.52.0->-r /tmp/pipenvuz4e6phprequirements/pipenv-np25ru7c-constraints.txt (line 2))
  werkzeug>=2.0 (from flask==2.0.1->-r /tmp/pipenvuz4e6phprequirements/pipenv-np25ru7c-constraints.txt (line 3))

loic001 on Jun 7, 2021

I think the solution is clear, but waiting for maintainers. Here I summarized the workaround here.

dissolved on Aug 27, 2021

The problem is that get_deps_list function returns deps with zappa==0.52.0, and not your forked repo.

As a quick and dirty fix, just add your forked repo in the package list here.

export SLIM_HANDLER_ZAPPA_PACKAGE=git+https://github.com/<forked_zappa>.git#egg=zappa

pkg_list = ["{0!s}=={1!s}".format(dep, version) for dep, version in zappa_deps if not dep.startswith('zappa')]
pkg_list.append(os.environ.get('SLIM_HANDLER_ZAPPA_PACKAGE'))

I can’t wait for the official repo to fix this problem to avoid ugly things like this 😃

loic001 on Jun 18, 2021