filesystem_spec: AttributeError: 'HadoopFileSystem' object has no attribute 'close'
When running the following code:
import fsspec
f = fsspec.open('hdfs:///path/to/file.csv').open()
...
I get the following error when the script exits (but complete successfully):
AttributeError: 'HadoopFileSystem' object has no attribute 'close'
Exception ignored in: 'pyarrow.lib.HadoopFileSystem.__dealloc__'
AttributeError: 'HadoopFileSystem' object has no attribute 'close'
I’m also getting the error when running:
import fsspec
f = fsspec.open('hdfs:///path/to/file.csv').open()
...
f.close()
The versions in my environment are:
python: 3.7.8
fsspec: 0.8.0
pyarrow: 1.0.1
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 24 (24 by maintainers)
How about with https://github.com/intake/filesystem_spec/pull/435/commits/289b77ca92443273ca83b1e368c34ae1067578cc ?
Well I suppose that
closecan be added to the list of known attributes here; but interesting that this was not apparently needed before. Note that this is a method for the class instance, not the file, to close and free the connection, which is happening during garbage collection.