manylinux: Cannot build C++14 in docker images

For anyone landing on this issue, I had the same issue but with C++17 features required to build the pysimdjson wheels. I’ve uploaded a derived manylinux image with gcc8.3 pre-built and is available here -> https://hub.docker.com/r/tktechdocker/manylinux.

If you want to build your own, here’s a usable dockerfile:

FROM quay.io/pypa/manylinux1_x86_64

LABEL description="A docker image for building portable Python linux binary wheels using modern GCC"
LABEL maintainer="tk@tkte.ch"

ARG GCC_VERSION=8.3.0
ARG GCC_PATH=/usr/local/gcc-$GCC_VERSION

RUN yum -y update && yum -y install \
    curl \
    bison \
    flex \
    && yum clean all

RUN cd /tmp \
    && curl -L -o gcc.tar.gz "https://ftp.gnu.org/gnu/gcc/gcc-${GCC_VERSION}/gcc-${GCC_VERSION}.tar.gz" \
    && tar xf gcc.tar.gz \
    && cd /tmp/gcc-$GCC_VERSION \
    && contrib/download_prerequisites \
    && mkdir build \
    && cd build \
    && ../configure -v \
        --build=x86_64-linux-gnu \
        --host=x86_64-linux-gnu \
        --target=x86_64-linux-gnu \
        --prefix=/usr/local/gcc-$GCC_VERSION \
        --enable-checking=release \
        --enable-languages=c,c++,fortran \
        --disable-multilib \
        --program-suffix=-$GCC_VERSION \
    && make -j4 \
    && make install-strip \
    && cd /tmp \
    && rm -rf /tmp/gcc-$GCC_VERSION /tmp/gcc.tar.gz

If you want to change the version of GCC built into the image just set $GCC_VERSION when building. Fun fact, a macbook pro building gcc will cause first-degree burns.

@YannickJadoul it is, but you can just change: FROM quay.io/pypa/manylinux1_x86_64 to FROM quay.io/pypa/manylinux1_x86 … to use the 32bit base.

I’m using this image to produce binaries for a tool that uses processor features only available on x86_64 so 32bit wasn’t of interest to me.

If you use the Steam hardware survey as a (biased) baseline, about 1.3% of Windows users are 32bit, and ~0% of Linux users. https://store.steampowered.com/hwsurvey/.

32-bit Linux is getting pretty fringe these days, I bet you can get away without. That’s not so for 32-bit Windows, there’s still a reasonable percentage of Windows Python installs that are 32-bit. I estimated it at 35% or a year or so ago: https://github.com/statsmodels/statsmodels/issues/4657#issuecomment-390726164

I’m attempting to automate it with Github Actions so in the future new images should become available any time manylinux or gcc have a release.

Updating the query now, for the last 30 days of Numpy downloads from PyPI, 32-bit Windows wheels were at 45% for Python 3.7, 50% for Python 2.7, and 27% (oddly) for Python 3.6. So, yes, still a big appetite for 32-bit on Windows.

I’ve rebuilt @TkTech’s excellent docker image for GCC 9.1.0, you can find it at dockerhub or just pull tstenner/manylinux:gcc9.1.0.

Anaconda’s download page puts 64-bit first and the (unlabeled) “Download” button downloads 64-bit: https://www.anaconda.com/distribution/#download-section This would suggest doing so can’t be causing that many problems for a reasonably diverse category of Python users with modern computers.

@YannickJadoul I symlinked cc to gcc in my 7.2 build.

I also just set the LD_LIBRARY_PATH in my builds. You’re welcome to make a pull request to the build script if you want to make that unnecessary, though I probably won’t rebuild the binaries until there’s a new version of at least one of the components.

@Noctem Amazing, it’s working, thanks! 😃 There’s just two things I need to fix in order to get it working: setting the variables LD_LIBRARY_PATH (because it couldn’t find libisl.so.15) and CC (because otherwise CMake looks for cc in the path and considers that as the system-wide compiler). So just as some feedback/suggestion, if you’d be interested: I know the latter is fixed by @squeaky-pl by just adding a symlink cc to gcc when packaging. And the former would maybe be solved by setting the rpath? But I don’t know enough about those details. (But things perfectly build without all that, and I need to set CXXFLAGS="-static-libstdc++" anyway, so these are just minor details 😉 )

Side note. I pushed GCC 7.1 release: https://github.com/squeaky-pl/centos-devtools/releases/tag/7.1

I think it would be great if we could add some of the information in this thread as instructions for getting modern C++ working in the manylinux1 container to the wiki or readme or something. I think many people would benefit from them.

Sent from my iPhone

On Jun 15, 2017, at 2:23 PM, Yannick Jadoul notifications@github.com wrote:

@njsmith Thanks for the insight! And I dó like the fact that I’ve now got binary manylinux wheels 😃

I’d actually still be interested in helping with the manylinux2/3, on the longer term, though I don’t think I have all the necessary practical knowledge and experience to do all this.

Also, the main ‘magic’ of Parselmouth is still in the pybind11 library, as far as I’m concerned; that thing is really awesome 😉

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

@YannickJadoul what about feeding -static-libstdc++ to the compiler/linker?