docker-py: Breaks with requests 2.32.0: Not supported URL scheme http+docker
With requests 2.32.0 (released about an hour ago as I write this), the docker library as called by tox-docker fails with the following exception:
Traceback (most recent call last):
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/requests/adapters.py", line 532, in send
conn = self._get_connection(request, verify, proxies=proxies, cert=cert)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/requests/adapters.py", line 400, in _get_connection
conn = self.poolmanager.connection_from_host(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/urllib3/poolmanager.py", line 304, in connection_from_host
return self.connection_from_context(request_context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/urllib3/poolmanager.py", line 326, in connection_from_context
raise URLSchemeUnknown(scheme)
urllib3.exceptions.URLSchemeUnknown: Not supported URL scheme http+docker
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/docker/api/client.py", line 214, in _retrieve_server_version
return self.version(api_version=False)["ApiVersion"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/docker/api/daemon.py", line 181, in version
return self._result(self._get(url), json=True)
^^^^^^^^^^^^^^
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/docker/utils/decorators.py", line 46, in inner
return f(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/docker/api/client.py", line 237, in _get
return self.get(url, **self._set_request_timeout(kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/requests/sessions.py", line 602, in get
return self.request("GET", url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/requests/adapters.py", line 534, in send
raise InvalidURL(e, request=request)
requests.exceptions.InvalidURL: Not supported URL scheme http+docker
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/tox/session/cmd/run/single.py", line 48, in _evaluate
code, outcomes = run_commands(tox_env, no_test)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/tox/session/cmd/run/single.py", line 79, in run_commands
MANAGER.tox_before_run_commands(tox_env)
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/tox/plugin/manager.py", line 88, in tox_before_run_commands
self.manager.hook.tox_before_run_commands(tox_env=tox_env)
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/pluggy/_hooks.py", line 513, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/pluggy/_manager.py", line 120, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/pluggy/_callers.py", line 139, in _multicall
raise exception.with_traceback(exception.__traceback__)
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/pluggy/_callers.py", line 103, in _multicall
res = hook_impl.function(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/tox_docker/tox4/plugin.py", line 73, in tox_before_run_commands
docker_build_or_pull(container_config, log)
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/tox_docker/plugin.py", line 57, in docker_build_or_pull
docker_pull(container_config, log)
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/tox_docker/plugin.py", line 65, in docker_pull
docker = docker_module.from_env(version="auto")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/docker/client.py", line 96, in from_env
return cls(
^^^^
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/docker/client.py", line 45, in __init__
self.api = APIClient(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/docker/api/client.py", line 197, in __init__
self._version = self._retrieve_server_version()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/eagle/dvl/venvs/gafaelfawr/lib/python3.12/site-packages/docker/api/client.py", line 221, in _retrieve_server_version
raise DockerException(
docker.errors.DockerException: Error while fetching server API version: Not supported URL scheme http+docker
Reverting to requests 2.31.0 without any other changes fixes the problem.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 169
- Comments: 63 (15 by maintainers)
Commits related to this issue
- Avoid requests 2.32.0 during testings Related: https://github.com/docker/docker-py/issues/3256 Related: https://github.com/ansible-collections/community.docker/issues/860 Related: https://github.com/... — committed to ansible/molecule by ssbarnea 3 years ago
- Avoid requests 2.32.0 during testing (#4185) Apparently requests==2.32.0 is incompatible with vendored versions of docker-py from inside community.docker collection. We do not have a direct depende... — committed to ansible/molecule by ssbarnea 3 years ago
- docker: lock docker controller requests pip module to 2.31.0 https://github.com/docker/docker-py/issues/3256 — committed to saltyorg/Saltbox by saltydk 3 years ago
- Pin requests as a temporary workaround for https://github.com/docker/docker-py/issues/3256 (#12077) Signed-off-by: harupy <17039389+harupy@users.noreply.github.com> — committed to mlflow/mlflow by harupy 3 years ago
- [build]: pin to requests 2.31 to resolve docker library error A temporary workaround for https://github.com/docker/docker-py/issues/3256 — committed to SeleniumHQ/docker-selenium by VietND96 3 years ago
- [CHORE] Pin requests to fix docker-py CI issue (#2289) The latest version of docker-py fails on the latest version of requests See: https://github.com/docker/docker-py/issues/3256 --------- ... — committed to Eventual-Inc/Daft by jaychia 3 years ago
- fix requests dependency using the latest version of requests causes a failure with docker-py, see https://github.com/docker/docker-py/issues/3256 — committed to josibake/warnet by josibake 3 years ago
- fix requests dependency using the latest version of requests causes a failure with docker-py, see https://github.com/docker/docker-py/issues/3256 — committed to josibake/warnet by josibake 3 years ago
- fix requests dependency using the latest version of requests causes a failure with docker-py, see https://github.com/docker/docker-py/issues/3256 — committed to josibake/warnet by josibake 3 years ago
- fix requests dependency using the latest version of requests causes a failure with docker-py, see https://github.com/docker/docker-py/issues/3256 — committed to josibake/warnet by josibake 3 years ago
- Pin requests in CI until docker package updates See https://github.com/docker/docker-py/issues/3256 — committed to DigitalSlideArchive/digital_slide_archive by manthey 3 years ago
- add pin for requests - issue with latest release: https://github.com/docker/docker-py/issues/3256 — committed to unity-sds/unity-initiator by pymonger 3 years ago
- pin requests <2.32 until docker/docker-py#3256 is resolved — committed to GoogleCloudPlatform/functions-framework-python by jrmfg 3 years ago
- fix: pin requests <2.32 until docker/docker-py#3256 is resolved — committed to GoogleCloudPlatform/functions-framework-python by jrmfg 3 years ago
- fix: pin requests <2.32 until docker/docker-py#3256 is resolved — committed to GoogleCloudPlatform/functions-framework-python by jrmfg 3 years ago
- fix: exclude broken python requests versions which cause docker.py failures (docker/docker-py#3256 and #3257) — committed to GoogleCloudPlatform/functions-framework-python by jrmfg 3 years ago
- fix: exclude broken python requests versions which cause docker.py failures (docker/docker-py#3256 and #3257) — committed to GoogleCloudPlatform/functions-framework-python by jrmfg 3 years ago
- Fix Docker requests incompatibility in CI. (#817) Bug that broke with latest requests release: https://github.com/ansible-collections/community.docker/pull/861 https://github.com/docker/docker-py/is... — committed to run-house/runhouse by rohinb2 3 years ago
- Change pinning for requests library See https://github.com/docker/docker-py/issues/3256 Signed-off-by: Hannes Baum <hannes.baum@cloudandheat.com> — committed to rak-n-rok/Krake by cah-hbaum 2 years ago
- Change pinning for requests library See https://github.com/docker/docker-py/issues/3256 Signed-off-by: Hannes Baum <hannes.baum@cloudandheat.com> — committed to rak-n-rok/Krake by cah-hbaum 2 years ago
Half of the internet is broken now… ansible, docker etc.
im:
Sorry for the delay, we had some unexpected complications to manage for this release.
Version 7.1.0 should now be live on pypi, thx for the patience ❤️
Pinning dependencies, in particular very general dependencies such as
requests, generally causes a lot more problems than it solves. The best place where to pin dependencies is at the end of the chain, i.e. where the venv is created that contains all the Python libraries and programs that you need.same!
pip install requests==2.31got me past this problemThe fix is included in docker-py 7.1.0.
@milas I saw you approved the latest PR:s to this repo.
Do you have any information on if we could expect this to be fixed and released anytime soon?
Our team wasted all day today trying to triage and put a fix to this problem. After this incident, is there any plan to revisit this unpinned dependency specification: https://github.com/docker/docker-py/blob/b6464dbed92b14b2c61d5ee49805fce041a3e083/pyproject.toml#L33
In case anyone is using the
ansiblepackage, you need to manually upgrade community.docker to 3.10.2. The version included in Ansible 9.6.0, community.docker 3.10.1, only has a fix for requests 2.32.0 and 2.32.1 since it got released before the changes for requests 2.32.2 were made (the Ansible 9.6.0 release also happened shortly before the requests changes were made).If you need to use community.docker as part of the Ansible community package (https://pypi.org/project/ansible/), you have to stick to requests < 2.32.2 until the next Ansible release (Ansible 9.7.0 roughly in four weeks, and Ansible 10.0.0 one or two weeks before that).
If you only pin direct dependencies, you’re usually missing A LOT of indirect dependencies that can cause all kind of problems - such as in this case.
You might want to use
pip freeze(https://pip.pypa.io/en/stable/cli/pip_freeze/) or pip-tools (https://pypi.org/project/pip-tools/) to pin all dependencies. Especially with pip-tools you can create a workflow that pins everything, and you have a CI workflow that regularly tries to upgrades dependencies and checks whether the result passes CI. Also check out Renovate (https://github.com/renovatebot/renovate) and Dependabot, which help a lot with that.We had the same problem with ansible docker while using the below
My coworker saw this fix https://github.com/ansible-collections/community.docker/pull/864. She updated to version
3.10.2and it seems it solved the problem.Dear Felix. We appreciate you following the case and submitting a corresponding patch. We are looking forward that it will be merged soon, and released on behalf of a relevant bugfix release.
requests 2.32.2 made the new interface public, but Docker SDK for Python still needs to support it first. Right now it does not, there’s a PR for it, and that (or something equivalent) needs to be merged and a new release of Docker SDK for Python has to be made. Otherwise you need to restrict to requests < 2.32.0.
A simple fix is adding
at the bottom of https://github.com/docker/docker-py/blob/main/docker/transport/basehttpadapter.py.
Not sure how sustainable that is, but it works for now 😃
我把docker版本升级到最新解决了这个问题
translated:
What version of docker-py are you using? I think this should be fixed by https://github.com/docker/docker-py/pull/3257#issuecomment-2173911243 ?
I am still facing this error Successfully installed requests-2.32.3
stack from console
Traceback (most recent call last): File “/home/ubuntu/.local/lib/python3.10/site-packages/requests/adapters.py”, line 633, in send conn = self.get_connection_with_tls_context( File “/home/ubuntu/.local/lib/python3.10/site-packages/requests/adapters.py”, line 489, in get_connection_with_tls_context conn = self.poolmanager.connection_from_host( File “/home/ubuntu/.local/lib/python3.10/site-packages/urllib3/poolmanager.py”, line 303, in connection_from_host return self.connection_from_context(request_context) File “/home/ubuntu/.local/lib/python3.10/site-packages/urllib3/poolmanager.py”, line 325, in connection_from_context raise URLSchemeUnknown(scheme) urllib3.exceptions.URLSchemeUnknown: Not supported URL scheme http+docker
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File “/usr/lib/python3/dist-packages/docker/api/client.py”, line 214, in _retrieve_server_version return self.version(api_version=False)[“ApiVersion”] File “/usr/lib/python3/dist-packages/docker/api/daemon.py”, line 181, in version return self._result(self._get(url), json=True) File “/usr/lib/python3/dist-packages/docker/utils/decorators.py”, line 46, in inner return f(self, *args, **kwargs) File “/usr/lib/python3/dist-packages/docker/api/client.py”, line 237, in _get return self.get(url, **self._set_request_timeout(kwargs)) File “/home/ubuntu/.local/lib/python3.10/site-packages/requests/sessions.py”, line 602, in get return self.request(“GET”, url, **kwargs) File “/home/ubuntu/.local/lib/python3.10/site-packages/requests/sessions.py”, line 589, in request resp = self.send(prep, **send_kwargs) File “/home/ubuntu/.local/lib/python3.10/site-packages/requests/sessions.py”, line 703, in send r = adapter.send(request, **kwargs) File “/home/ubuntu/.local/lib/python3.10/site-packages/requests/adapters.py”, line 637, in send raise InvalidURL(e, request=request) requests.exceptions.InvalidURL: Not supported URL scheme http+docker
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File “/usr/bin/docker-compose”, line 33, in <module> sys.exit(load_entry_point(‘docker-compose==1.29.2’, ‘console_scripts’, ‘docker-compose’)()) File “/usr/lib/python3/dist-packages/compose/cli/main.py”, line 81, in main command_func() File “/usr/lib/python3/dist-packages/compose/cli/main.py”, line 200, in perform_command project = project_from_options(‘.’, options) File “/usr/lib/python3/dist-packages/compose/cli/command.py”, line 60, in project_from_options return get_project( File “/usr/lib/python3/dist-packages/compose/cli/command.py”, line 152, in get_project client = get_client( File “/usr/lib/python3/dist-packages/compose/cli/docker_client.py”, line 41, in get_client client = docker_client( File “/usr/lib/python3/dist-packages/compose/cli/docker_client.py”, line 170, in docker_client client = APIClient(use_ssh_client=not use_paramiko_ssh, **kwargs) File “/usr/lib/python3/dist-packages/docker/api/client.py”, line 197, in init self._version = self._retrieve_server_version() File “/usr/lib/python3/dist-packages/docker/api/client.py”, line 221, in _retrieve_server_version raise DockerException( docker.errors.DockerException: Error while fetching server API version: Not supported URL scheme http+docker
I got the same error too. Seems like this has broken the internet. downgraded to 2.31 and it worked.
requests 2.32.2+docker 6.1.1is still not working for me:Now, updating to
requests 2.32.2Its still error Traceback (most recent call last): File “/usr/bin/docker-compose”, line 33, in <module> sys.exit(load_entry_point(‘docker-compose==1.29.2’, ‘console_scripts’, ‘docker-compose’)()) File “/usr/lib/python3/dist-packages/compose/cli/main.py”, line 81, in main command_func() File “/usr/lib/python3/dist-packages/compose/cli/main.py”, line 200, in perform_command project = project_from_options(‘.’, options) File “/usr/lib/python3/dist-packages/compose/cli/command.py”, line 60, in project_from_options return get_project( File “/usr/lib/python3/dist-packages/compose/cli/command.py”, line 152, in get_project client = get_client( File “/usr/lib/python3/dist-packages/compose/cli/docker_client.py”, line 41, in get_client client = docker_client( File “/usr/lib/python3/dist-packages/compose/cli/docker_client.py”, line 170, in docker_client client = APIClient(use_ssh_client=not use_paramiko_ssh, **kwargs) File “/usr/lib/python3/dist-packages/docker/api/client.py”, line 197, in init self._version = self._retrieve_server_version() File “/usr/lib/python3/dist-packages/docker/api/client.py”, line 221, in _retrieve_server_version raise DockerException( docker.errors.DockerException: Error while fetching server API version: Not supported URL scheme http+docker
npm list urllib
└── urllib@2.2.2
For me, setting docker==7.1.0 fixed it.
The developers released a fix a within a few days, in version 7.1.0. Setting requests==2.31.0 didn’t work for me.
Here is an example where it is still failing, despite using the new versions of
requestsanddocker-py.Could it be a caching issue? If not, could you provide steps to reproduce the issue?
Tests on my end show the new version working as expected
so what is the final verdict i still cant run the code
So now the question is, do i
I can’t make a decision :p help! What would you do?
same problem
I faced the same problem. requests version is 2.28.2. Ubuntu 22.04. Docker 27.1.2 Any solutions? Thank you
I faced the same problem. This problem was avoided by lowering the version of requests and the version of docker.
In case someone are interested, here is the version that worked, in my environment.
Docker Version: 24.0.6 Docker Compose Version: v2.22.0
requests version: 2.28.2
I suspect that many of the reports here saying the problem wasn’t fixed may be due to some other package pinning the docker library to an older version. For example, tox-docker currently pins to <7, which prevents the new version with the fix from being used. (This is already tracked in https://github.com/tox-dev/tox-docker/issues/181.)
For those still having problems, making sure, using
pip listor the like, that you really have version 7.1.0 of the docker library.@krissetto community.docker 3.10.2 has that fix for its vendored Docker SDK for Python code since a few hours after requests 2.32.2 was released. The modules in it that use the ‘real’ Docker SDK for Python (mostly the Docker Swarm modules) should only still fail if Docker SDK for Python hasn’t been upgraded, but everything else should just work.
I’ve seen some reports that folks are still seeing that error with community.docker 3.10.2 with modules using the vendored Docker SDK for Python parts, but I cannot reproduce it (and CI cannot either).
(In any case, now with the latest Docker SDK for Python released, CI is green except in cases which fail for some ansible-test pip proxy reasons…)
Happened to me as well. Downgrading
requeststo a version less then 2.32.2 fixed it I did it using poetrypoetry add "requests<2.32.2"which installed version2.31.0Sorry @jsf9k, I gave a very quick look at your project and it seems it does not depend on docker-py. Apparently you (like the other above mentioning issues) are using ansible to do some things with docker, so that’s an issue for the ansible project or the ansible community’s modules and plugins for docker.
This might be the correct place to ask for help: https://github.com/ansible-collections/community.docker
I understand you might’ve seen the error mentioned in this thread pop up in your logs so you ended up here. Many projects depend on this library so they might also have to be updated to use the 7.1.0 version of the docker library to avoid this issue.
In some cases, it might be enough to reinstall the affected packages, in other cases changes to the projects using docker might be needed.
To be able to diagnose any further issues with docker-py, it’d help a ton to have a simple example project that we can run on our machines to reproduce the problem.
Hope this helps ✌️
Issue persists with
Error connecting: Error while fetching server API version: Not supported URL scheme http+docker [0.0s]Same as @saltydk, this error was returned from Ansible:
this is one of those cases where every solution is a bad one
same issue here, the fix from @felixfontein works for me! for now I just replaced the dependency docker for git+https://github.com/felixfontein/docker-py@requests-hotfix