Multicorn: Build error against PostgreSQL 13

Hi,

PostgreSQL is in beta3, and we are in the process of testing software. Multicorn fails to build against 13. Can you please take a look?

Thanks! Devrim

gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -g -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fPIC -I/usr/include/python3.8 -I/usr/include/python3.8 -I. -I./ -I/usr/pgsql-13/include/server -I/usr/pgsql-13/include/internal -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include -c -o src/python.o src/python.c src/python.c: In function 'compareColumns': src/python.c:527:17: warning: passing argument 1 of 'lnext' from incompatible pointer type [-Wincompatible-pointer-types] 527 | cell1 = lnext(cell1); | ^~~~~ | | | ListCell * In file included from /usr/pgsql-13/include/server/access/tupdesc.h:19, from /usr/pgsql-13/include/server/access/htup_details.h:19, from /usr/pgsql-13/include/server/access/relscan.h:17, from src/multicorn.h:3, from src/python.c:4: /usr/pgsql-13/include/server/nodes/pg_list.h:321:19: note: expected 'const List *' but argument is of type 'ListCell *' 321 | lnext(const List *l, const ListCell *c) | ~~~~~~~~~~~~^ src/python.c:527:11: error: too few arguments to function 'lnext' 527 | cell1 = lnext(cell1); | ^~~~~ In file included from /usr/pgsql-13/include/server/access/tupdesc.h:19, from /usr/pgsql-13/include/server/access/htup_details.h:19, from /usr/pgsql-13/include/server/access/relscan.h:17, from src/multicorn.h:3, from src/python.c:4: /usr/pgsql-13/include/server/nodes/pg_list.h:321:1: note: declared here 321 | lnext(const List *l, const ListCell *c) | ^~~~~ src/python.c:528:17: warning: passing argument 1 of 'lnext' from incompatible pointer type [-Wincompatible-pointer-types] 528 | cell2 = lnext(cell2); | ^~~~~ | | | ListCell * In file included from /usr/pgsql-13/include/server/access/tupdesc.h:19, from /usr/pgsql-13/include/server/access/htup_details.h:19, from /usr/pgsql-13/include/server/access/relscan.h:17, from src/multicorn.h:3, from src/python.c:4: /usr/pgsql-13/include/server/nodes/pg_list.h:321:19: note: expected 'const List *' but argument is of type 'ListCell *' 321 | lnext(const List *l, const ListCell *c) | ~~~~~~~~~~~~^ src/python.c:528:11: error: too few arguments to function 'lnext' 528 | cell2 = lnext(cell2); | ^~~~~ In file included from /usr/pgsql-13/include/server/access/tupdesc.h:19, from /usr/pgsql-13/include/server/access/htup_details.h:19, from /usr/pgsql-13/include/server/access/relscan.h:17, from src/multicorn.h:3, from src/python.c:4: /usr/pgsql-13/include/server/nodes/pg_list.h:321:1: note: declared here 321 | lnext(const List *l, const ListCell *c) | ^~~~~ src/python.c:534:17: warning: passing argument 1 of 'lnext' from incompatible pointer type [-Wincompatible-pointer-types] 534 | cell1 = lnext(cell1); | ^~~~~ | | | ListCell * In file included from /usr/pgsql-13/include/server/access/tupdesc.h:19, from /usr/pgsql-13/include/server/access/htup_details.h:19, from /usr/pgsql-13/include/server/access/relscan.h:17, from src/multicorn.h:3, from src/python.c:4: /usr/pgsql-13/include/server/nodes/pg_list.h:321:19: note: expected 'const List *' but argument is of type 'ListCell *' 321 | lnext(const List *l, const ListCell *c) | ~~~~~~~~~~~~^ src/python.c:534:11: error: too few arguments to function 'lnext' 534 | cell1 = lnext(cell1); | ^~~~~ In file included from /usr/pgsql-13/include/server/access/tupdesc.h:19, from /usr/pgsql-13/include/server/access/htup_details.h:19, from /usr/pgsql-13/include/server/access/relscan.h:17, from src/multicorn.h:3, from src/python.c:4: /usr/pgsql-13/include/server/nodes/pg_list.h:321:1: note: declared here 321 | lnext(const List *l, const ListCell *c) | ^~~~~ src/python.c:535:17: warning: passing argument 1 of 'lnext' from incompatible pointer type [-Wincompatible-pointer-types] 535 | cell2 = lnext(cell2); | ^~~~~ | | | ListCell * In file included from /usr/pgsql-13/include/server/access/tupdesc.h:19, from /usr/pgsql-13/include/server/access/htup_details.h:19, from /usr/pgsql-13/include/server/access/relscan.h:17, from src/multicorn.h:3, from src/python.c:4: /usr/pgsql-13/include/server/nodes/pg_list.h:321:19: note: expected 'const List *' but argument is of type 'ListCell *' 321 | lnext(const List *l, const ListCell *c) | ~~~~~~~~~~~~^ src/python.c:535:11: error: too few arguments to function 'lnext' 535 | cell2 = lnext(cell2); | ^~~~~ In file included from /usr/pgsql-13/include/server/access/tupdesc.h:19, from /usr/pgsql-13/include/server/access/htup_details.h:19, from /usr/pgsql-13/include/server/access/relscan.h:17, from src/multicorn.h:3, from src/python.c:4: /usr/pgsql-13/include/server/nodes/pg_list.h:321:1: note: declared here 321 | lnext(const List *l, const ListCell *c) | ^~~~~ src/python.c:541:17: warning: passing argument 1 of 'lnext' from incompatible pointer type [-Wincompatible-pointer-types] 541 | cell1 = lnext(cell1); | ^~~~~ | | | ListCell * In file included from /usr/pgsql-13/include/server/access/tupdesc.h:19, from /usr/pgsql-13/include/server/access/htup_details.h:19, from /usr/pgsql-13/include/server/access/relscan.h:17, from src/multicorn.h:3, from src/python.c:4: /usr/pgsql-13/include/server/nodes/pg_list.h:321:19: note: expected 'const List *' but argument is of type 'ListCell *' 321 | lnext(const List *l, const ListCell *c) | ~~~~~~~~~~~~^ src/python.c:541:11: error: too few arguments to function 'lnext' 541 | cell1 = lnext(cell1); | ^~~~~ In file included from /usr/pgsql-13/include/server/access/tupdesc.h:19, from /usr/pgsql-13/include/server/access/htup_details.h:19, from /usr/pgsql-13/include/server/access/relscan.h:17, from src/multicorn.h:3, from src/python.c:4: /usr/pgsql-13/include/server/nodes/pg_list.h:321:1: note: declared here 321 | lnext(const List *l, const ListCell *c) | ^~~~~ src/python.c:542:17: warning: passing argument 1 of 'lnext' from incompatible pointer type [-Wincompatible-pointer-types] 542 | cell2 = lnext(cell2); | ^~~~~ | | | ListCell * In file included from /usr/pgsql-13/include/server/access/tupdesc.h:19, from /usr/pgsql-13/include/server/access/htup_details.h:19, from /usr/pgsql-13/include/server/access/relscan.h:17, from src/multicorn.h:3, from src/python.c:4: /usr/pgsql-13/include/server/nodes/pg_list.h:321:19: note: expected 'const List *' but argument is of type 'ListCell *' 321 | lnext(const List *l, const ListCell *c) | ~~~~~~~~~~~~^ src/python.c:542:11: error: too few arguments to function 'lnext' 542 | cell2 = lnext(cell2); | ^~~~~ In file included from /usr/pgsql-13/include/server/access/tupdesc.h:19, from /usr/pgsql-13/include/server/access/htup_details.h:19, from /usr/pgsql-13/include/server/access/relscan.h:17, from src/multicorn.h:3, from src/python.c:4: /usr/pgsql-13/include/server/nodes/pg_list.h:321:1: note: declared here 321 | lnext(const List *l, const ListCell *c) | ^~~~~ make[1]: *** [<builtin>: src/python.o] Error 1 make[1]: *** Waiting for unfinished jobs.... src/errors.c: In function 'reportException': src/errors.c:71:6: error: too many arguments to function 'errstart' 71 | if (errstart(severity, __FILE__, __LINE__, PG_FUNCNAME_MACRO, TEXTDOMAIN)) | ^~~~~~~~ In file included from /usr/pgsql-13/include/server/postgres.h:47, from src/multicorn.h:2, from src/errors.c:15: /usr/pgsql-13/include/server/utils/elog.h:149:13: note: declared here 149 | extern bool errstart(int elevel, const char *domain); | ^~~~~~~~ src/errors.c:73:7: error: too many arguments to function 'errstart' 73 | if (errstart(severity, __FILE__, __LINE__, PG_FUNCNAME_MACRO, TEXTDOMAIN)) | ^~~~~~~~ In file included from /usr/pgsql-13/include/server/postgres.h:47, from src/multicorn.h:2, from src/errors.c:15: /usr/pgsql-13/include/server/utils/elog.h:149:13: note: declared here 149 | extern bool errstart(int elevel, const char *domain); | ^~~~~~~~ src/errors.c:84:2: error: too few arguments to function 'errfinish' 84 | errfinish(0); | ^~~~~~~~~ In file included from /usr/pgsql-13/include/server/postgres.h:47, from src/multicorn.h:2, from src/errors.c:15: /usr/pgsql-13/include/server/utils/elog.h:150:13: note: declared here 150 | extern void errfinish(const char *filename, int lineno, const char *funcname); | ^~~~~~~~~ make[1]: *** [<builtin>: src/errors.o] Error 1 src/multicorn.c:146:13: error: use of undeclared identifier 'oid_hash'; did you mean 'uuid_hash'? ctl.hash = oid_hash; ^~~~~~~~ uuid_hash /usr/pgsql-13/include/server/utils/fmgrprotos.h:1702:14: note: 'uuid_hash' declared here extern Datum uuid_hash(PG_FUNCTION_ARGS); ^ src/multicorn.c:146:11: warning: incompatible pointer types assigning to 'HashValueFunc' (aka 'unsigned int (*)(const void *, unsigned long)') from 'Datum (FunctionCallInfo)' (aka 'unsigned long (struct FunctionCallInfoBaseData *)') [-Wincompatible-pointer-types] ctl.hash = oid_hash; ^ ~~~~~~~~ 1 warning and 1 error generated. make[1]: *** [/usr/pgsql-13/lib/pgxs/src/makefiles/../../src/Makefile.global:1051: src/multicorn.bc] Error 1 src/errors.c:71:35: error: too many arguments to function call, expected 2, have 5 if (errstart(severity, __FILE__, __LINE__, PG_FUNCNAME_MACRO, TEXTDOMAIN)) ~~~~~~~~ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <scratch space>:199:1: note: expanded from here 71 ^ /usr/pgsql-13/include/server/utils/elog.h:149:1: note: 'errstart' declared here extern bool errstart(int elevel, const char *domain); ^ src/errors.c:73:36: error: too many arguments to function call, expected 2, have 5 if (errstart(severity, __FILE__, __LINE__, PG_FUNCNAME_MACRO, TEXTDOMAIN)) ~~~~~~~~ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <scratch space>:201:1: note: expanded from here 73 ^ /usr/pgsql-13/include/server/utils/elog.h:149:1: note: 'errstart' declared here extern bool errstart(int elevel, const char *domain); ^ src/errors.c:84:13: error: too few arguments to function call, expected 3, have 1 errfinish(0); ~~~~~~~~~ ^ /usr/pgsql-13/include/server/utils/elog.h:150:1: note: 'errfinish' declared here extern void errfinish(const char *filename, int lineno, const char *funcname); ^ 3 errors generated. make[1]: *** [/usr/pgsql-13/lib/pgxs/src/makefiles/../../src/Makefile.global:1051: src/errors.bc] Error 1 src/python.c:527:22: error: too few arguments to function call, expected 2, have 1 cell1 = lnext(cell1); ~~~~~ ^ /usr/pgsql-13/include/server/nodes/pg_list.h:320:1: note: 'lnext' declared here static inline ListCell * ^ src/python.c:528:22: error: too few arguments to function call, expected 2, have 1 cell2 = lnext(cell2); ~~~~~ ^ /usr/pgsql-13/include/server/nodes/pg_list.h:320:1: note: 'lnext' declared here static inline ListCell * ^ src/python.c:534:22: error: too few arguments to function call, expected 2, have 1 cell1 = lnext(cell1); ~~~~~ ^ /usr/pgsql-13/include/server/nodes/pg_list.h:320:1: note: 'lnext' declared here static inline ListCell * ^ src/python.c:535:22: error: too few arguments to function call, expected 2, have 1 cell2 = lnext(cell2); ~~~~~ ^ /usr/pgsql-13/include/server/nodes/pg_list.h:320:1: note: 'lnext' declared here static inline ListCell * ^ src/python.c:541:22: error: too few arguments to function call, expected 2, have 1 cell1 = lnext(cell1); ~~~~~ ^ /usr/pgsql-13/include/server/nodes/pg_list.h:320:1: note: 'lnext' declared here static inline ListCell * ^ src/python.c:542:22: error: too few arguments to function call, expected 2, have 1 cell2 = lnext(cell2); ~~~~~ ^ /usr/pgsql-13/include/server/nodes/pg_list.h:320:1: note: 'lnext' declared here static inline ListCell * ^ 6 errors generated. make[1]: *** [/usr/pgsql-13/lib/pgxs/src/makefiles/../../src/Makefile.global:1051: src/python.bc] Error 1 src/query.c:844:16: error: too few arguments to function call, expected 2, have 1 lc = lnext(lc); ~~~~~ ^ /usr/pgsql-13/include/server/nodes/pg_list.h:320:1: note: 'lnext' declared here static inline ListCell * ^ src/query.c:847:16: error: too few arguments to function call, expected 2, have 1 lc = lnext(lc); ~~~~~ ^ /usr/pgsql-13/include/server/nodes/pg_list.h:320:1: note: 'lnext' declared here static inline ListCell * ^ src/query.c:850:16: error: too few arguments to function call, expected 2, have 1 lc = lnext(lc); ~~~~~ ^ /usr/pgsql-13/include/server/nodes/pg_list.h:320:1: note: 'lnext' declared here static inline ListCell * ^ src/query.c:853:16: error: too few arguments to function call, expected 2, have 1 lc = lnext(lc); ~~~~~ ^ /usr/pgsql-13/include/server/nodes/pg_list.h:320:1: note: 'lnext' declared here static inline ListCell * ^ src/query.c:859:16: error: too few arguments to function call, expected 2, have 1 lc = lnext(lc); ~~~~~ ^ /usr/pgsql-13/include/server/nodes/pg_list.h:320:1: note: 'lnext' declared here static inline ListCell * ^ 5 errors generated. make[1]: *** [/usr/pgsql-13/lib/pgxs/src/makefiles/../../src/Makefile.global:1051: src/query.bc] Error 1

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Reactions: 1
  • Comments: 16 (2 by maintainers)

Most upvoted comments

@rusty-jules: The patch in the Debian package is simply what I pushed as #260, but as said there, the patch is incomplete. Someone with an understanding of PG’s and Multicorn’s list handling should sit down a bit with that to fix my naive attempt.

@wieck it seems that the debian folks have patched this issue. I got this to build in the postgres:13-buster container. Note that their patch is a version of postgresql-13-python3-multicorn.

FROM postgres:13-buster

RUN apt-get update && apt-get install -y \
   python3-multicorn \
   libpython3-dev \
   python3-dev \
   postgresql-server-dev-13 libpq-dev \
   python3-setuptools
ENV PYTHON_OVERRIDE python3.7

# postgresql-13-python3-multicorn is only available on bullseye and bookworm,
# not buster so we need to build it ourselves
RUN git clone --depth 1 --branch debian/1.4.0-3 https://salsa.debian.org/postgresql/postgresql-multicorn && \
  cd postgresql-multicorn && \
  # apply the debian patches
  git apply debian/patches/drop-install-target.diff && \
  git apply debian/patches/dual-python-build.diff && \
  git apply debian/patches/pg13 && \
  make && make install && ldconfig && \
  echo "create extension multicorn;" > /docker-entrypoint-initdb.d/multicorn.sql

Attached is my own patch file for s3csv_fdw which I got working in this container. Their README does not seem correct, creating a server should be done like so:

CREATE SERVER multicorn_csv FOREIGN DATA WRAPPER multicorn OPTIONS (wrapper 's3fdw.s3fdw.S3Fdw');

This patch also completely ignores the user mapping settings, and relies on boto3’s credential finder instead.

s3csv_fdw_python3.patch.zip

Any word on supporting PostgreSQL 13?