fastparquet: RuntimeError: Decompression 'SNAPPY' not available. Options: ['GZIP', 'UNCOMPRESSED']

Output:

RuntimeError: Decompression 'SNAPPY' not available.  Options: ['GZIP', 'UNCOMPRESSED']

Code:

from fastparquet import ParquetFile

filename = 'somefile.parquet'
pf = ParquetFile(filename)

Environment:

$ python -V; pip list | grep -e fastparquet -e snapp
Python 3.6.5
fastparquet      0.1.6
python-snappy    0.5.3

I’ve tried install snappy instead of python-snappy. Still no joy because with these installed…

python-snappy    0.5.3
snappy           2.6.1
snappy-manifolds 1.0

I get the error:

AttributeError: module 'snappy' has no attribute 'compress'

About this issue

Original URL
State: closed
Created 6 years ago
Comments: 17 (8 by maintainers)

Most upvoted comments

RuntimeError: Decompression ‘SNAPPY’ not available. Options: [‘GZIP’, ‘UNCOMPRESSED’]

Try this please :

pip install fastparquet pyarrow dask

+13

RabiaMelhem on Jan 26, 2020

On Ubuntu linux, I simply installed python-snappy from conda-forge, and it simply avoids the error.

matanox on Jul 23, 2019

Ooh - I think I might have got somewhere with the following…

$ brew install snappy # snappy library from Google
$ CPPFLAGS="-I/usr/local/include -L/usr/local/lib" pip install python-snappy

Taken from the bottom of your link https://github.com/andrix/python-snappy/

@martindurant thanks for the help - fingers crossed I can finally start to have a play with this module.

lqueryvg on Aug 27, 2018

I only had to install python-snappy, which had the dependency: sudo apt-get install libsnappy-dev

ek-nath on Jul 9, 2020

The block that inserts Snappy into the list of compressions looks like this:

    import snappy
    def snappy_decompress(data, uncompressed_size):
        return snappy.decompress(data)
    compressions['SNAPPY'] = snappy.compress
    decompressions['SNAPPY'] = snappy_decompress

You can try to run that yourself to find out where it is failing.

martindurant on Aug 27, 2018