shapely: Segmentation fault when calling geometry.MultiPolygon with shapes from mask array

Expected behavior and actual behavior.

Segmentation fault when trying to create a MultiPolygon, from shapes generated from rasterio.features.shapes.

Steps to reproduce the problem.

The problem was initially encountered in terracotta, an XYZ tile server that uses Shapely as a dependency.

The specific line of code that triggers the segfault is this one https://github.com/DHI-GRAS/terracotta/blob/b7c67c3c2736401295644c1e8882b3f0f013bb5c/terracotta/raster.py#L157

Below is a minimal example that triggers the segfault.

Python 3.9.11 (main, Apr  7 2022, 11:47:24)
Type 'copyright', 'credits' or 'license' for more information
IPython 8.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import numpy as np

In [2]: from rasterio import features

In [3]: from shapely import geometry

In [4]: mask = np.random.randint(0, 2, (2000, 2000), dtype=int)

In [5]: mask = mask.astype("uint8")

In [6]: hull_shapes = (geometry.shape(s) for s, _ in features.shapes(np.ones(mask.shape, 'uint8'), mask=ma
   ...: sk))

In [7]: geometry.MultiPolygon(hull_shapes)
[1]    44843 segmentation fault  ipython

Operating system

arch: macOS/arm64

macOS Monterey 12.4, Apple Silicon M1 chip.

Shapely version and provenance

The issue is present in Shapely 1.8.2 (newest as of the time of writing) and Shapely 1.8.1. Shapely 1.8.0 is the latest version that is not affected by the issue as of the time of writing.

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 28 (15 by maintainers)

Commits related to this issue

Most upvoted comments

Yes, rasterio is built from source (as is the default via pip install rasterio).

The problem is fixed if I also build shapely from source, as you probably suspected:

$ pip install shapely --no-binary shapely

I ran the same as dionhaefner, upgrading too to 1.8.4 (I was in 1.8.2) and the problem dissapeared!

This feels like a GEOS issue to me. Wheels for versions 1.8.0 and 1.8.1 include GEOS 3.9.1. Wheels for 1.8.2 include GEOS 3.10.2. For 2.0a1: GEOS 3.10.3.

@mrpgraae the best we can do for 1.8 is upgrade to GEOS 3.10.3 for the 1.8.3 wheels (coming soon https://github.com/shapely/shapely/issues/1472).

It’s fine to keep open until a final release. Or if it somehow gets fixed for the 1.8 series.

@misuzu thanks for verifying!

No, there are no tests for that platform. The usual warning from GitHub Actions for “Build and publish” (e.g.) is:

While arm64 wheels can be built on x86_64, they cannot be tested. The ability to test the arm64 wheels will be added in a future release of cibuildwheel, once Apple Silicon CI runners are widely available.

So unfortunately, this investigation can only be done by folks with that platform. For now, at least.

I can reproduce this on my M1 Mac when using the macosx_11_0_arm64 wheels.