duckdb: IO Error when installing httpfs extension from Python

What happens?

Executing INSTALL 'httpfs'; returns the following error:

RuntimeError: IO Error: Failed to download extension http://extensions.duckdb.org/b52293246/linux_amd64/httpfs.duckdb_extension.gz

To Reproduce

First, install latest Python version of duckdb, then execute the following code:

import duckdb
con = duckdb.connect(database=":memory:", read_only=False)
con.execute("INSTALL 'httpfs';")

Environment (please complete the following information):

I’m using the following dockerfile:

FROM mcr.microsoft.com/vscode/devcontainers/python:3.9
RUN pip3 install duckdb --pre --upgrade
ENTRYPOINT "/bin/bash"

An alternative way of reproducing the issue:

docker run -it mcr.microsoft.com/vscode/devcontainers/python:3.9 /bin/bash
pip install duckdb
python -c 'import duckdb; con = duckdb.connect(database=":memory:", read_only=False); con.execute("INSTALL 'httpfs';");'

Before Submitting

Have you tried this on the latest master branch?

Python: pip install duckdb --upgrade --pre
R: install.packages("https://github.com/duckdb/duckdb/releases/download/master-builds/duckdb_r_src.tar.gz", repos = NULL)
Other Platforms: You can find binaries here or compile from source.

Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 22 (10 by maintainers)

Most upvoted comments

we’re working on a solution that should be up for PR soon, we’re adding binary builds of extensions with the GLIBCXX_USE_CXX11_ABI=0 flag for python

samansmink on Mar 28, 2022

As a temporary work-around for this, you can set the dlopen flags yourself before loading the duckdb module:

import sys, os
sys.setdlopenflags(os.RTLD_GLOBAL | os.RTLD_NOW)

import duckdb
con = duckdb.connect()
con.execute('load httpfs')

Mytherin on Mar 28, 2022

It worked - I was able to pull the build from pypi and load the json extension. Thanks @samansmink!

gotdan on Mar 22, 2022

It works! Thanks a lot @samansmink!

davidgasquez on Mar 22, 2022

The python package has not been uploaded yet due to an issue with pypi. I believe this should be fixed now, which means the next master push will trigger an upload of the latest master build for python. You can either await this, or manually build the python package.

I think currently the cli master binaries are not uploaded indeed, i’m not sure why they aren’t uploaded, I can take a look at this tomorrow.

samansmink on Mar 21, 2022

yw! ci canceling issue pr is currently under review (https://github.com/duckdb/duckdb/pull/3245) after that is merged and the binaries uploaded, you should be able to test it out, though I will also add a test similar to the Node one in tools/nodejs/test/extension.test.js when i have time, so you could also wait that out to be sure its actually working 😃

samansmink on Mar 16, 2022