urllib3: ValueError: file descriptor cannot be a negative integer (-1)

A user attempted to make a connection from within a ProcessPoolExecutor and somehow got an error from urllib3 about a -1 file descriptor. Any ideas on how this could happen? I didn’t see any place that would be assigned.

File /opt/conda/lib/python3.7/site-packages/blobfile/ops.py, line 764, in exists
--
  | isfile, _ = _google_isfile(path)
  | File /opt/conda/lib/python3.7/site-packages/blobfile/ops.py, line 738, in _google_isfile
  | resp = _execute_google_api_request(req)
  | File /opt/conda/lib/python3.7/site-packages/blobfile/ops.py, line 389, in _execute_google_api_request
  | return _execute_request(build_req)
  | File /opt/conda/lib/python3.7/site-packages/blobfile/ops.py, line 412, in _execute_request
  | redirect=False,
  | File /opt/conda/lib/python3.7/site-packages/urllib3/request.py, line 76, in request
  | method, url, fields=fields, headers=headers, **urlopen_kw
  | File /opt/conda/lib/python3.7/site-packages/urllib3/request.py, line 97, in request_encode_url
  | return self.urlopen(method, url, **extra_kw)
  | File /opt/conda/lib/python3.7/site-packages/urllib3/poolmanager.py, line 330, in urlopen
  | response = conn.urlopen(method, u.request_uri, **kw)
  | File /opt/conda/lib/python3.7/site-packages/urllib3/connectionpool.py, line 654, in urlopen
  | conn = self._get_conn(timeout=pool_timeout)
  | File /opt/conda/lib/python3.7/site-packages/urllib3/connectionpool.py, line 265, in _get_conn
  | if conn and is_connection_dropped(conn):
  | File /opt/conda/lib/python3.7/site-packages/urllib3/util/connection.py, line 24, in is_connection_dropped
  | return wait_for_read(sock, timeout=0.0)
  | File /opt/conda/lib/python3.7/site-packages/urllib3/util/wait.py, line 146, in wait_for_read
  | return wait_for_socket(sock, read=True, timeout=timeout)
  | File /opt/conda/lib/python3.7/site-packages/urllib3/util/wait.py, line 99, in poll_wait_for_socket
  | poll_obj.register(sock, mask)
  |  
  | ValueError: file descriptor cannot be a negative integer (-1)
File /opt/conda/lib/python3.7/site-packages/urllib3/util/connection.py, line 24, in is_connection_dropped
--
  | 7    def is_connection_dropped(conn):  # Platform-specific
  | (...)
  | 20       if sock is None:  # Connection already closed (such as by httplib).
  | 21           return True
  | 22       try:
  | 23           # Returns True if readable, which here means it's been dropped
  | --> 24           return wait_for_read(sock, timeout=0.0)
  | 25       except NoWayToWaitForSocketError:  # Platform-specific: AppEngine
  | ..................................................
  | conn = <urllib3.connection.VerifiedHTTPSConnection object at 0x7f11
  | 2ff79470>
  | sock = <ssl.SSLSocket fd=-1, family=AddressFamily.AF_INET, type=Soc
  | ketKind.SOCK_STREAM, proto=6>
  | NoWayToWaitForSocketError = <class 'urllib3.util.wait.NoWayToWaitForSocketError'>

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Comments: 31 (11 by maintainers)

Most upvoted comments

Ah, you mean you still got the error without using processes (and threads?) at all? Strange.

As you say, I don’t think we have anything actionable left here. I would not be comfortable adding an explicit check for a -1 fd without understanding how we can get there.

I’m going to close this, but I’d be happy to continue hearing about the investigation.