shapely: Segmentation fault when calling geometry.MultiPolygon with shapes from mask array
Expected behavior and actual behavior.
Segmentation fault when trying to create a MultiPolygon, from shapes generated from rasterio.features.shapes.
Steps to reproduce the problem.
The problem was initially encountered in terracotta, an XYZ tile server that uses Shapely as a dependency.
The specific line of code that triggers the segfault is this one https://github.com/DHI-GRAS/terracotta/blob/b7c67c3c2736401295644c1e8882b3f0f013bb5c/terracotta/raster.py#L157
Below is a minimal example that triggers the segfault.
Python 3.9.11 (main, Apr 7 2022, 11:47:24)
Type 'copyright', 'credits' or 'license' for more information
IPython 8.4.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import numpy as np
In [2]: from rasterio import features
In [3]: from shapely import geometry
In [4]: mask = np.random.randint(0, 2, (2000, 2000), dtype=int)
In [5]: mask = mask.astype("uint8")
In [6]: hull_shapes = (geometry.shape(s) for s, _ in features.shapes(np.ones(mask.shape, 'uint8'), mask=ma
...: sk))
In [7]: geometry.MultiPolygon(hull_shapes)
[1] 44843 segmentation fault ipython
Operating system
arch: macOS/arm64
macOS Monterey 12.4, Apple Silicon M1 chip.
Shapely version and provenance
The issue is present in Shapely 1.8.2 (newest as of the time of writing) and Shapely 1.8.1. Shapely 1.8.0 is the latest version that is not affected by the issue as of the time of writing.
About this issue
- Original URL
- State: open
- Created 2 years ago
- Comments: 28 (15 by maintainers)
Commits related to this issue
- Replace c_void_p with c_geom_p In an attempt to fix the issue reported in #1449 and perhaps solved in https://github.com/django/django/pull/15214. — committed to shapely/shapely by sgillies 2 years ago
- Replace c_void_p with c_geom_p (#1479) * Replace c_void_p with c_geom_p In an attempt to fix the issue reported in #1449 and perhaps solved in https://github.com/django/django/pull/15214. * En... — committed to shapely/shapely by sgillies 2 years ago
Yes, rasterio is built from source (as is the default via
pip install rasterio).The problem is fixed if I also build shapely from source, as you probably suspected:
I ran the same as dionhaefner, upgrading too to 1.8.4 (I was in 1.8.2) and the problem dissapeared!
This feels like a GEOS issue to me. Wheels for versions 1.8.0 and 1.8.1 include GEOS 3.9.1. Wheels for 1.8.2 include GEOS 3.10.2. For 2.0a1: GEOS 3.10.3.
@mrpgraae the best we can do for 1.8 is upgrade to GEOS 3.10.3 for the 1.8.3 wheels (coming soon https://github.com/shapely/shapely/issues/1472).
It’s fine to keep open until a final release. Or if it somehow gets fixed for the 1.8 series.
@misuzu thanks for verifying!
No, there are no tests for that platform. The usual warning from GitHub Actions for “Build and publish” (e.g.) is:
So unfortunately, this investigation can only be done by folks with that platform. For now, at least.
I can reproduce this on my M1 Mac when using the
macosx_11_0_arm64wheels.