black: Installing black 19.10b0 without gcc installed causes an error

There should be an alternative to a hard dependency of regex for a year old low priority bug that causes widespread disruption to users. Also relying on third-party library in pull requests and packaging availability must be questioned.

The author of regex is not going to provide wheels:

https://bitbucket.org/mrabarnett/mrab-regex/issues/343/wheel-for-linux

@Seth Morton: I, personally, am not interested in building wheels for anything other than Windows (and I’m not sure how long I’ll be building ones for Python 2, what with it about to reach end-of-life), but I would accept a contribution that would enable others to do build them.

Why not revert the fix and dependency to reconsider a better solution?

This makes using black pretty hard on modern CI/CD which usually uses docker. Can we please find a better tool other than regex or MR something into that library, so that it builds binary linux wheels

Instead of looking for an alternative to regex itself, I would offer to make it better by contributing a CI pipeline to it that builds wheels on all platforms.

There’s a pull request merged that will soon produce linux wheels for regex.

Everywhere in code regex can be safely replaced by re, so regex can be imported with a fallback to re and installed as an extra.

I ran into this issue in an image that uses continuumio/miniconda3:latest which in turn uses debian:latest. I addressed it by adding the following line:

RUN apt-get update && apt-get -y install gcc

I didn’t need to install all of build-essential. In any case, this resolution is now going to stress the debian repositories.

FWIW, I’ve just asked @ambv to revert the change again given the low priority nature of the bug and the impact it has on many people.

@impredicative I have a library that generates Python source code from jinja templates and then formats it with black, so it is a direct dependency.

For people looking for a compact Alpine Docker image with Black 19.10b0, installing Black with GCC can be done in a separate build stage to reduce the final image size. Example (see original Dockerfile):

FROM alpine:3.10.3 AS black
RUN apk add --no-cache \
    gcc~=8.3.0 \
    musl-dev~=1.1.22 \
    python3-dev~=3.7.5 \
  && pip3 install --target /opt/black black==19.10b0

FROM alpine:3.10.3
RUN apk add --no-cache python3~=3.7.5
COPY --from=black /opt/black /opt/black
ENV PATH="${PATH}:/opt/black/bin" \
  PYTHONPATH="${PYTHONPATH}:/opt/black"

This one results in an image of about 64.8MB in size. I hope this helps.

If an example of the problematic Unicode pattern and input string is known, perhaps it could be shared here so someone could report it to https://bugs.python.org/ in the hopes that is could be improved in the re module.

The regex author doesn’t seem to be willing to add pre-built linux wheels. We also find that requiring a source build is inconvenient, both in terms of added complexity and build times.