terracotta: Getting a 500 when using the newest version of Rasterio (1.0.22)
When deploying a Lambda function through Zappa, using the newest version of Rasterio (1.0.22), we get a 500 for some tiles, while others load fine.
When reverting back to Rasterio 1.0.13 everything works fine.
zappa tail gives the following:
[1554888957364] [ERROR] 2019-04-10T09:35:57.326Z b24c3c65-85ed-4390-a18b-c0cf347feec0 Exception on /rgb/italy/2018/7/70/47.png [GET]
Traceback (most recent call last):
File "rasterio/_base.pyx", line 213, in rasterio._base.DatasetBase.__init__
File "rasterio/_shim.pyx", line 64, in rasterio._shim.open_dataset
File "rasterio/_err.pyx", line 205, in rasterio._err.exc_wrap_pointer
rasterio._err.CPLE_OpenFailedError: '/vsis3/italy-composite/rasters/italy_2018_red.tif' not recognized as a supported file format.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/task/terracotta/drivers/raster_base.py", line 464, in _get_raster_tile
src = es.enter_context(rasterio.open(path))
File "/var/task/rasterio/env.py", line 423, in wrapper
return f(*args, **kwds)
File "/var/task/rasterio/__init__.py", line 216, in open
s = DatasetReader(path, driver=driver, **kwargs)
File "rasterio/_base.pyx", line 215, in rasterio._base.DatasetBase.__init__
rasterio.errors.RasterioIOError: '/vsis3/italy-composite/rasters/italy_2018_red.tif' not recognized as a supported file format.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/task/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/var/task/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/var/task/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/var/task/flask/_compat.py", line 35, in reraise
raise value
File "/var/task/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/var/task/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/var/task/terracotta/server/flask_api.py", line 50, in inner
return fun(*args, **kwargs)
File "/var/task/terracotta/server/rgb.py", line 85, in get_rgb
return _get_rgb_image(keys, tile_xyz=tile_xyz)
File "/var/task/terracotta/server/rgb.py", line 135, in _get_rgb_image
some_keys, rgb_values, stretch_ranges=stretch_ranges, tile_xyz=tile_xyz, **options
File "/var/lang/lib/python3.6/contextlib.py", line 52, in inner
return func(*args, **kwds)
File "/var/task/terracotta/handlers/rgb.py", line 85, in rgb
band_data = band_data_future.result()
File "/var/lang/lib/python3.6/concurrent/futures/_base.py", line 432, in result
return self.__get_result()
File "/var/lang/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/var/lang/lib/python3.6/concurrent/futures/thread.py", line 56, in run
result = self.fn(*self.args, **self.kwargs)
File "/var/task/cachetools/__init__.py", line 87, in wrapper
v = method(self, *args, **kwargs)
File "/var/lang/lib/python3.6/contextlib.py", line 52, in inner
return func(*args, **kwds)
File "/var/task/terracotta/drivers/raster_base.py", line 466, in _get_raster_tile
raise IOError('error while reading file {}'.format(path))
OSError: error while reading file s3://italy-composite/rasters/italy_2018_red.tif
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 1
- Comments: 24 (4 by maintainers)
Commits related to this issue
- disable GDAL_DISABLE_READDIR_ON_OPEN (fixes #139) — committed to DHI/terracotta by dionhaefner 5 years ago
New insights. We can get rid of the issue with
rasterio.errors.RasterioIOError: '/vsis3/italy-composite/rasters/italy_2018_red.tif' not recognized as a supported file format.by settingGDAL_DISABLE_READDIR_ON_OPEN=FALSE. We were usingEMPTY_DIRbefore.We now tried
FALSEto see whether we would get a different error message than thenot recognized as supported file format, following https://github.com/mapbox/rasterio/issues/1706.The
EMPTY_DIRwas recommended among others here: https://github.com/mapbox/rasterio/issues/987.Whether the change from
EMPTY_DIRtoFALSEactually prohibits the actual cause of the S3 read failure or just prevents some race condition because it changes the timing of things, we do not know. But it seems to work consistently.Small update on this. I’m seeing quite a few changes to
/vsiscurl/and/vsis3/in the GDAL 2.4.0 changelog. I’m currently trying to build the Rasterio wheels with an older GDAL version to see if that fixes it.