tippecanoe: Segmentation fault

I have been processing a file which is 81GB of newline-delimited JSON. I have processed it a couple of times successfully on this machine. Just now I changed the parameters slightly:

tippecanoe --output=mbtiles/parcels.mbtiles --force --read-parallel --minimum-zoom=13 --maximum-zoom=15 --generate-ids --simplification=10 --detect-shared-borders --simplify-only-low-zooms --attribute-type='ATTOM ID:int' --include='ATTOM ID' --include=APN --include=CountyFIPS --coalesce-smallest-as-needed --base-zoom=14 --accumulate-attribute=APN:comma --accumulate-attribute='ATTOM ID:comma' --layer=parcels data/parcels.json

and got a seg fault:

...
Going to try keeping the biggest 19.34% of the features to make it fit
tile 13/2352/3121 size is 502684 with detail 12, >500000
Going to try keeping the biggest 17.31% of the features to make it fit
Segmentation fault (core dumped)

Now, following this I’ll try running it again to actually get the core dump, which seems to not be written by default.

The changes between “worked” and “crashed” are:

// worked:
                    minimumZoom: 13,
                    maximumZoom: 15, 
                    generateIds: true,
                    simplification:10,
                    filename: `${options.datapath}/parcels${options.suffix}.json`,
                    detectSharedBorders: true,
                    dropSmallestAsNeeded: true,
                    simplifyOnlyLowZooms: true,
                    attributeType: 'ATTOM ID:int',
                    include: ['ATTOM ID', 'APN','CountyFIPS'],

// crashed (all the above, plus these:)
                    dropSmallestAsNeeded: false,
                    coalesceSmallestAsNeeded: true,
                    baseZoom: 14,
                    accumulateAttribute: ['APN:comma', 'ATTOM ID:comma'],

Somewhere in the log I also noticed there was one tile that tippecanoe couldn’t make small enough. What does it actually do in that situation? Previously I thought I had seen it abort altogether.

I have also managed to process a subset of this dataset successfully, with these exact settings.

I’m not sure how to share the source file with you, given its size. Any ideas?

tippecanoe v1.32.12

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Comments: 19 (18 by maintainers)

Most upvoted comments

@stevage I am facing the exact problem. Did you get a solution over the past year and half now?

Could you please try https://github.com/mapbox/tippecanoe/pull/845 and see if it fixes the problem for you?

Sorry to be unclear. The “separate metadata” file is basically a holdover from an older version of Tippecanoe in which most of the characteristics of a feature were stored in a separate file so they wouldn’t have to be duplicated across temporary files when a feature was clipped into many sub-features in different tiles. I gradually moved more and more things into the “geometry” file, so all it has in it now is a table of indices of key/value attribute pairs into the attribute pool, and is only populated when a feature is expected to span a large number of tiles.

I should probably just get rid of it, since it doesn’t seem to work reliably and is another thing that has to be handled as a special case when it exists.