urllib3: ValueError: file descriptor cannot be a negative integer (-1)
A user attempted to make a connection from within a ProcessPoolExecutor and somehow got an error from urllib3 about a -1 file descriptor. Any ideas on how this could happen? I didn’t see any place that would be assigned.
File /opt/conda/lib/python3.7/site-packages/blobfile/ops.py, line 764, in exists
--
| isfile, _ = _google_isfile(path)
| File /opt/conda/lib/python3.7/site-packages/blobfile/ops.py, line 738, in _google_isfile
| resp = _execute_google_api_request(req)
| File /opt/conda/lib/python3.7/site-packages/blobfile/ops.py, line 389, in _execute_google_api_request
| return _execute_request(build_req)
| File /opt/conda/lib/python3.7/site-packages/blobfile/ops.py, line 412, in _execute_request
| redirect=False,
| File /opt/conda/lib/python3.7/site-packages/urllib3/request.py, line 76, in request
| method, url, fields=fields, headers=headers, **urlopen_kw
| File /opt/conda/lib/python3.7/site-packages/urllib3/request.py, line 97, in request_encode_url
| return self.urlopen(method, url, **extra_kw)
| File /opt/conda/lib/python3.7/site-packages/urllib3/poolmanager.py, line 330, in urlopen
| response = conn.urlopen(method, u.request_uri, **kw)
| File /opt/conda/lib/python3.7/site-packages/urllib3/connectionpool.py, line 654, in urlopen
| conn = self._get_conn(timeout=pool_timeout)
| File /opt/conda/lib/python3.7/site-packages/urllib3/connectionpool.py, line 265, in _get_conn
| if conn and is_connection_dropped(conn):
| File /opt/conda/lib/python3.7/site-packages/urllib3/util/connection.py, line 24, in is_connection_dropped
| return wait_for_read(sock, timeout=0.0)
| File /opt/conda/lib/python3.7/site-packages/urllib3/util/wait.py, line 146, in wait_for_read
| return wait_for_socket(sock, read=True, timeout=timeout)
| File /opt/conda/lib/python3.7/site-packages/urllib3/util/wait.py, line 99, in poll_wait_for_socket
| poll_obj.register(sock, mask)
|
| ValueError: file descriptor cannot be a negative integer (-1)
File /opt/conda/lib/python3.7/site-packages/urllib3/util/connection.py, line 24, in is_connection_dropped
--
| 7 def is_connection_dropped(conn): # Platform-specific
| (...)
| 20 if sock is None: # Connection already closed (such as by httplib).
| 21 return True
| 22 try:
| 23 # Returns True if readable, which here means it's been dropped
| --> 24 return wait_for_read(sock, timeout=0.0)
| 25 except NoWayToWaitForSocketError: # Platform-specific: AppEngine
| ..................................................
| conn = <urllib3.connection.VerifiedHTTPSConnection object at 0x7f11
| 2ff79470>
| sock = <ssl.SSLSocket fd=-1, family=AddressFamily.AF_INET, type=Soc
| ketKind.SOCK_STREAM, proto=6>
| NoWayToWaitForSocketError = <class 'urllib3.util.wait.NoWayToWaitForSocketError'>
About this issue
- Original URL
- State: open
- Created 4 years ago
- Comments: 31 (11 by maintainers)
Ah, you mean you still got the error without using processes (and threads?) at all? Strange.
As you say, I don’t think we have anything actionable left here. I would not be comfortable adding an explicit check for a -1 fd without understanding how we can get there.
I’m going to close this, but I’d be happy to continue hearing about the investigation.