urllib3: DNS lookup error exception

I am using requests to check domain status for k8s.io domains (https://github.com/kubernetes/k8s.io/pull/30) and I can’t find a good way to check for DNS lookup error like ERR_NAME_NOT_RESOLVED in Chrome. Exceptions output seems platform specific:

    Traceback (most recent call last):
      File "check.py", line 11, in <module>
        status = requests.head('http://' + site)
      File "C:\Python27\lib\site-packages\requests\api.py", line 93, in head
        return request('head', url, **kwargs)
      File "C:\Python27\lib\site-packages\requests\api.py", line 53, in request
        return session.request(method=method, url=url, **kwargs)
      File "C:\Python27\lib\site-packages\requests\sessions.py", line 468, in request
        resp = self.send(prep, **send_kwargs)
      File "C:\Python27\lib\site-packages\requests\sessions.py", line 576, in send
        r = adapter.send(request, **kwargs)
      File "C:\Python27\lib\site-packages\requests\adapters.py", line 437, in send
        raise ConnectionError(e, request=request)
    requests.exceptions.ConnectionError: HTTPConnectionPool(host='jenkins.k8s.io', port=80):
        Max retries exceeded with url: / (Caused by
        NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at
        0x029E43B0>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed',))

So I was redirected here from https://github.com/kennethreitz/requests/issues/3630#issuecomment-255031285 to see if urllib3 could expose more fine-grained DNSLookupError exception and maybe remove MaxRetryError wrapper.

About this issue

Original URL
State: closed
Created 8 years ago
Reactions: 2
Comments: 15 (10 by maintainers)

Commits related to this issue

Add NameResolutionError for failed DNS lookups (#1003) Test case: from urllib3.util.connection import create_connection create_connection(('wowsucherror', 80)) On Windows, it gives: Tr... — committed to techtonik/urllib3 by techtonik 8 years ago
Add NameResolutionError for failed DNS lookups (#1003) Test case: from urllib3.util.connection import create_connection create_connection(('wowsucherror', 80)) On Windows, it gives: Tr... — committed to techtonik/urllib3 by techtonik 8 years ago
Add NameResolutionError for failed DNS lookups (#1003) Test case: from urllib3.util.connection import create_connection create_connection(('wowsucherror', 80)) On Windows, it gives: Tr... — committed to techtonik/urllib3 by techtonik 8 years ago

Most upvoted comments

Ok, so.

We’re certainly not going to remove the MaxRetryError: that’s not a winner. But we can probably be more clear about the specific error by hoisting the getaddrinfo call on line 75 of urllib3/util/connection.py out of the for statement and shoving it in a try...except that raises an appropriate urllib3 exception.

That would at least give you a specific exception to try to haul out. MaxRetryError has a reason attribute which you can use to fetch the underlying exception, so you can from that point take the exception.

Note, however, that this change will take a while to propagate into a released version of Requests. In the interim, you may find it more helpful to check for DNS errors by simply calling socket.getaddrinfo yourself before making the request.

Lukasa on Oct 20, 2016