setup-python: Installing psycopg2 on custom runners is giving error for python3.11

Description:

Installing psycopg2 on python3.11 on custom runners gives error

Action version:

Runner version 2.299.1

Platform:

  • Ubuntu
  • macOS
  • Windows

Runner type:

  • Hosted
  • Self-hosted

Tools version:

Python 3.11.0

Repro steps:

Once this is running on a custom runner, here is what I had to do reproduce the issue

mkdir ~/.x
cd ~/.x
/home/ubuntu/actions-runner/_work/_tool/Python/3.11.0/x64/bin/python3.11 -m venv venv
source venv/bin/activate
pip install -U pip wheel
pip install psycopg2

Expected behavior:

psycopg2 should install correctly

Actual behavior:

Collecting psycopg2
  Using cached psycopg2-2.9.5.tar.gz (384 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: psycopg2
  Building wheel for psycopg2 (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [40 lines of output]
      /home/ranjith/.x/venv/lib/python3.11/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
        warnings.warn(msg, warning_class)
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-311
      creating build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/pool.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/tz.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/_range.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/_ipaddress.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/errors.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/errorcodes.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/__init__.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/extras.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/sql.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/extensions.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/_json.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      running build_ext
      building 'psycopg2._psycopg' extension
      creating build/temp.linux-x86_64-cpython-311
      creating build/temp.linux-x86_64-cpython-311/psycopg
      gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC "-DPSYCOPG_VERSION=2.9.5 (dt dec pq3 ext lo64)" -DPSYCOPG_DEBUG=1 -DPG_VERSION_NUM=140005 -DHAVE_LO64=1 -DPSYCOPG_DEBUG=1 -I/home/ranjith/.x/venv/include -I/opt/hostedtoolcache/Python/3.11.0/x64/include/python3.11 -I. -I/usr/include/postgresql -I/usr/include/postgresql/14/server -I/usr/include/libxml2 -c psycopg/adapter_asis.c -o build/temp.linux-x86_64-cpython-311/psycopg/adapter_asis.o -Wdeclaration-after-statement
      In file included from psycopg/adapter_asis.c:28:
      ./psycopg/psycopg.h:35:10: fatal error: Python.h: No such file or directory
         35 | #include <Python.h>
            |          ^~~~~~~~~~
      compilation terminated.

      It appears you are missing some prerequisite to build the package from source.

      You may install a binary package by installing 'psycopg2-binary' from PyPI.
      If you want to install psycopg2 from source, please install the packages
      required for the build and try again.

      For further information please check the 'doc/src/install.rst' file (also at
      <https://www.psycopg.org/docs/install.html>).

      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for psycopg2
  Running setup.py clean for psycopg2
Failed to build psycopg2
Installing collected packages: psycopg2
  Running setup.py install for psycopg2 ... error
  error: subprocess-exited-with-error

  × Running setup.py install for psycopg2 did not run successfully.
  │ exit code: 1
  ╰─> [42 lines of output]
      /home/ranjith/.x/venv/lib/python3.11/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
        warnings.warn(msg, warning_class)
      running install
      /home/ranjith/.x/venv/lib/python3.11/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-311
      creating build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/pool.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/tz.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/_range.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/_ipaddress.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/errors.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/errorcodes.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/__init__.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/extras.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/sql.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/extensions.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      copying lib/_json.py -> build/lib.linux-x86_64-cpython-311/psycopg2
      running build_ext
      building 'psycopg2._psycopg' extension
      creating build/temp.linux-x86_64-cpython-311
      creating build/temp.linux-x86_64-cpython-311/psycopg
      gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC "-DPSYCOPG_VERSION=2.9.5 (dt dec pq3 ext lo64)" -DPSYCOPG_DEBUG=1 -DPG_VERSION_NUM=140005 -DHAVE_LO64=1 -DPSYCOPG_DEBUG=1 -I/home/ranjith/.x/venv/include -I/opt/hostedtoolcache/Python/3.11.0/x64/include/python3.11 -I. -I/usr/include/postgresql -I/usr/include/postgresql/14/server -I/usr/include/libxml2 -c psycopg/adapter_asis.c -o build/temp.linux-x86_64-cpython-311/psycopg/adapter_asis.o -Wdeclaration-after-statement
      In file included from psycopg/adapter_asis.c:28:
      ./psycopg/psycopg.h:35:10: fatal error: Python.h: No such file or directory
         35 | #include <Python.h>
            |          ^~~~~~~~~~
      compilation terminated.

      It appears you are missing some prerequisite to build the package from source.

      You may install a binary package by installing 'psycopg2-binary' from PyPI.
      If you want to install psycopg2 from source, please install the packages
      required for the build and try again.

      For further information please check the 'doc/src/install.rst' file (also at
      <https://www.psycopg.org/docs/install.html>).

      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> psycopg2

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 12
  • Comments: 22 (6 by maintainers)

Most upvoted comments

Hello @ranjith19 , it looks like you do not have python headers and libraries installed on the host you are using to run the builds. Please try to execute the following commands on the machine

sudo apt-get install python-dev
sudo apt-get install python3-dev

and run the build once more time. The command above might be run once either in the terminal or as a step in the pipeline

+16
dsame on Dec 13, 2022

@ranjith19 It sees the correct name of the ubuntu package is libpython3.11-dev now

Please try to execute

sudo apt-get install libpython3.11-dev

According to this it should install missing headers

+14
dsame on Dec 26, 2022

@dsame @ranjith19 I got it working by installing python3.11-dev

+6
alesisjoan on Dec 20, 2022

This still seems to be a valid issue. Installing system dev headers does not seem to solve the issue for us.

We are also currently experiencing this in python 3.10.8

As a quite hacky fix to temporarily get our pipeline to run we were able to fix it by copying the folder …/actions-runner/_work/_tools/Python to /opt/hostedtoolcache/Python. This will probably break again pretty quickly.

It is expecting the headers to be in the path that Github hosted runners use (/opt/hostedtoolcache), but a self hosted runner uses ./_work/_tool instead. Making a symlink to point the Github hosted runner location to the actual one solves the problem for us.

You can see the mismatch above:

 /home/jsk/actions-runner/_work/_tool/Python/3.10.8/x64/lib/python3.10/site-packages/setuptools/config/setupcfg.py:508
-I/opt/hostedtoolcache/Python/3.10.8/x64/include/python3.10

This seems to be a rather new-ish error as it started happening in December-ish for us.

+2
AngellusMortis on Mar 23, 2023

Hello @ranjith19 ! Thanks for reporting this! We will investigate the issue and come back to you as soon as we have any updates

+2
MaksimZhukov on Nov 30, 2022

hello there , use this command : pip install psycopg2-binary this command did work for me on windows 10.

I tried this but still got an error. The error I am getting is "Running setup.py install for psycopg2 … error error: subprocess-exited-with-error "

" note: This error originates from a subprocess, and is likely not a problem with pip. error: legacy-install-failure

× Encountered error while trying to install package. ╰─> psycopg2

note: This is an issue with the package mentioned above, not pip. hint: See above for output from the failure. "

+1
dayo777 on Jan 27, 2024

hello there , use this command : pip install psycopg2-binary this command did work for me on windows 10.

+1
parsarsalan on Dec 23, 2023

This solved it for me after a lot of searching for a solution!!

sudo apt-get install libpython3.x-dev

thanks @dsame

+1
LeonrdoFgundes on Jan 9, 2023
Read more comments on GitHub
← setup-python: Error: Version X with arch x64 not found
setup-python: May I ask why? pip cache is not found →