docker-gitlab: Gitlab 16 incompatible with PostgreSQL 12, requires >=13

docker-compose.yml contained in the 16 release uses sameersbn/postgresql:12-20200524 as a PostgreSQL container. When starting the containers, we get following error:

You are using PostgreSQL 12.3 for the ci database, but this version of GitLab requires PostgreSQL >= 13.

Please upgrade your environment to a supported PostgreSQL version.
See https://docs.gitlab.com/ee/install/requirements.html#database for details.

Despite the message, Gitlab itself appear to work at a cursory glance, but it will probably cause problems further down the line.

About this issue

  • Original URL
  • State: open
  • Created a year ago
  • Reactions: 16
  • Comments: 30 (8 by maintainers)

Most upvoted comments

We’ll provide a workaround. But this will take some time.

+15
sachilles on May 26, 2023

I just updated the PostgreSQL version in my GitLab Docker environment. I went from using the image ‘sameersbn/postgresql:12-20200524’ to ‘sameersbn/postgresql:14-20230628’, and then upgraded again to ‘sameersbn/postgresql:15-20230628’. During each transition, I performed a Docker Compose up.

PostgreSQL seamlessly detected the need for an upgrade when I switched to the newer versions, and I’m pleased to report that the entire process went smoothly without any issues.

So, to avoid any problem seems to be a good idea to upgrade from 12 to 14 and then 15.

+9
drjoju on Aug 8, 2023
How I did it - or very inofficial instructions for Docker Compose and Postgres image from Bitnami.

Credits to @f-io.

  1. Shutdown approx. 4 containers: docker compose down. Make sure you are using the newest software with the new syntax; without the - between docker and compose.
  2. Rename postgresql: to postgresql-old: in docker-compose.yml file.
  3. Insert new section into docker-compose.yml file:
    postgresql:
        environment:
            - POSTGRESQL_USERNAME=gitlab
            - POSTGRESQL_PASSWORD=***
            - POSTGRESQL_DATABASE=gitlabhq_production
        image: bitnami/postgresql:13.11.0
        networks:
            ***:
        restart: unless-stopped
        volumes:
            - /etc/apt/apt.conf:/etc/apt/apt.conf
            ###- ./volume_postgresql/:/bitnami/postgresql/
        # docker compose exec postgresql psql -U gitlab gitlabhq_production
        # CREATE EXTENSION IF NOT EXISTS pg_trgm;
        # CREATE EXTENSION IF NOT EXISTS btree_gist;
  1. Power up postgresql-old: docker compose up -d postgresql-old.
  2. Do the backup: sudo bash -c 'docker compose exec postgresql-old /bin/bash -c "PGPASSWORD=*** pg_dump --username gitlab gitlabhq_production" > ./gitlab_dump_.sql'. Notice the space at the beginning of the command. On some distros this will prevent the command saved to the bash history. Useful because of the password.
  3. Shutdown again with docker compose down.
  4. Comment the old section and rename the volume:
    #postgresql-old:
    #    environment:
    #        - DB_USER=gitlab
    #        - DB_PASS=***
    #        - DB_NAME=gitlabhq_production
    #        - DB_EXTENSION=pg_trgm,btree_gist
    #    image: sameersbn/postgresql:12-20200524
    #    networks:
    #        ***:
    #    restart: unless-stopped
    #    volumes:
    #        - /etc/apt/apt.conf:/etc/apt/apt.conf
    #        - ./volume_postgresql_old:/var/lib/postgresql:Z
  1. Rename the old volume: sudo mv volume_postgresql volume_postgresql_old.
  2. Power up the new database: docker compose up -d postgresql.
  3. Copy out the conf and data folder:
POSTGRES_CONTAINER_ID=$(docker compose ps --format json | grep postgres | grep -oP '"ID":"[0-9a-fA-F]+"' | grep -oP '[0-9a-fA-F]+' | tail -n 1)
[ $(echo -n $POSTGRES_CONTAINER_ID | wc -c) -eq 64 ] && echo 'Fine, got Postgres container ID.' || echo 'Something went wrong!?'
# Do the copy:
sudo docker cp ${POSTGRES_CONTAINER_ID}:/bitnami/postgresql/ .
# Rename the new host folder:
sudo mv postgresql volume_postgresql
cd volume_postgresql
# Fix permissions:
sudo chown 1001:root data -R
  1. Remove the comment ### from new volume: section in docker-compose.yml file.
  2. Shutdown again: docker compose down.
  3. Power up the new database again: docker compose up -d postgresql.
  4. Activate 2 needed extensions:
docker compose exec postgresql psql -U gitlab gitlabhq_production
# Enter password
CREATE EXTENSION IF NOT EXISTS pg_trgm;
CREATE EXTENSION IF NOT EXISTS btree_gist;
  1. Do the restore: docker compose exec -T postgresql /bin/bash -c "PGPASSWORD=*** psql --username gitlab gitlabhq_production" < ./gitlab_dump_.sql
  2. Shutdown again: docker compose down.
  3. Start in foreground: docker compose up and see if it works…
  4. Break foreground with [Ctrl]+[C] and start in background: docker compose up -d.
+7
bor8 on Jun 5, 2023

@sachilles a good first step would be - if you have the permissions needed at all, if not, the discussion on that topic is probably moot - to provide a PQ13 version of sameersbn/postgresql that people could test drive with a manual upgrade process. If that isn’t possible (I’m not sure the original author is still around at all), we should probably discuss the best part forward. A fork, maybe ?

It’s probably not very productive if people report back on their issues with random hand rolled deployments of official PG13 images and/or custom patched/upgraded versions of sameersbn/postgresql and will just end up muddling the discussion.

+5
Gaibhne on May 30, 2023

We moved to official postgres docker image and also used pg_dump. Database dump ~6G, almost 8y of collected garbage.

Just make sure that the required extensions are available in your new database.

I know there are some convenient features within custom postgres images, but how often do you plan to migrate to a new major version? In my experience using a custom image will most likely lead to more work than it saves.

#dump

docker exec -i postgresql_current /bin/bash -c "PGPASSWORD=……. pg_dump --username gitlab gitlabhq_production" > ./postgres_backup/gitlab_dump_$(date +%y-%m-%d-%H:%M).sql

#import

docker exec -i postgresql_new /bin/bash -c "PGPASSWORD=…….  psql --username gitlab gitlabhq_production" < ./postgres_backup/gitlab_dump__.sql
+5
f-io on May 30, 2023

The latest image: sameersbn/postgresql:latest is now pg14.8 and stops the warnings on gitlab startup

+4
davidglassborow on Jun 27, 2023

Note: Here are some special features of sameersbn/postgresql:

+3
kkimurak on Oct 20, 2023

Sorry for being slow. Actually I wanted to have a look at the special features of the sameersbn/postgresql image this week. Unfortunately, I was very busy this week, so I did not get around to it.

However, I guess it makes more sense to use an official postgres image and thus, the documentation needs to be updated.

+3
sachilles on Jun 2, 2023

Not saying it’s right but was already using sameersbn/postgresql:latest in docker-compose, so I just did docker-compose pull and then rebuild the containers.

The logs show PG detecting the change, and auto updating:

postgresql_1  | Initializing datadir...
postgresql_1  | Initializing certdir...
postgresql_1  | Initializing logdir...
postgresql_1  | Initializing rundir...
postgresql_1  | Setting resolv.conf ACLs...
postgresql_1  | Initializing database...
postgresql_1  | ‣ Migrating PostgreSQL 12 data to 14...
postgresql_1  | ‣ Installing PostgreSQL 12...
postgresql_1  | debconf: delaying package configuration, since apt-utils is not installed
postgresql_1  | ‣ Migration in progress. Please be patient...Performing Consistency Checks
postgresql_1  | -----------------------------
postgresql_1  | Checking cluster versions                                   ok
postgresql_1  | Checking database user is the install user                  ok
postgresql_1  | Checking database connection settings                       ok
postgresql_1  | Checking for prepared transactions                          ok
postgresql_1  | Checking for system-defined composite types in user tables  ok
postgresql_1  | Checking for reg* data types in user tables                 ok
postgresql_1  | Checking for contrib/isn with bigint-passing mismatch       ok
<snip>
postgresql_1  |   /var/lib/postgresql/12/main/base/16400/2683
postgresql_1  |                                                             ok
postgresql_1  | Setting next OID for new cluster                            ok
postgresql_1  | Sync data directory to disk                                 ok
postgresql_1  | Creating script to delete old cluster                       ok
postgresql_1  | Checking for extension updates                              notice
postgresql_1  | 
postgresql_1  | Your installation contains extensions that should be updated
postgresql_1  | with the ALTER EXTENSION command.  The file
postgresql_1  |     update_extensions.sql
postgresql_1  | when executed by psql by the database superuser will update
postgresql_1  | these extensions.
postgresql_1  | 
postgresql_1  | 
postgresql_1  | Upgrade Complete
postgresql_1  | ----------------
postgresql_1  | Optimizer statistics are not transferred by pg_upgrade.
postgresql_1  | Once you start the new server, consider running:
postgresql_1  |     /usr/lib/postgresql/14/bin/vacuumdb --all --analyze-in-stages
postgresql_1  | 
postgresql_1  | Running this script will delete the old cluster's data files:
postgresql_1  |     ./delete_old_cluster.sh
postgresql_1  | 
postgresql_1  | Configuring hot standby...
postgresql_1  | ‣ Setting postgresql.conf parameter: wal_level = 'hot_standby'
postgresql_1  | ‣ Setting postgresql.conf parameter: max_wal_senders = '16'
postgresql_1  | ‣ Setting postgresql.conf parameter: checkpoint_segments = '8'
postgresql_1  | ‣ Setting postgresql.conf parameter: wal_keep_segments = '32'
postgresql_1  | ‣ Setting postgresql.conf parameter: hot_standby = 'on'
postgresql_1  | ‣ Setting postgresql.conf parameter: data_directory = '/var/lib/postgresql/14/main'
postgresql_1  | ‣ Setting postgresql.conf parameter: log_directory = '/var/log/postgresql'
postgresql_1  | ‣ Setting postgresql.conf parameter: log_filename = 'postgresql-14-main.log'
postgresql_1  | ‣ Setting postgresql.conf parameter: ssl = 'off'
postgresql_1  | Creating database user: gitlab
postgresql_1  | Creating database: gitlabhq_production...
postgresql_1  | ‣ Loading pg_trgm extension...
postgresql_1  | ‣ Loading btree_gist extension...
postgresql_1  | ‣ Granting access to gitlab user...
postgresql_1  | Starting PostgreSQL 14...
postgresql_1  | 2023-06-27 07:07:52.883 UTC [1] LOG:  starting PostgreSQL 14.8 (Ubuntu 14.8-1.pgdg18.04+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0, 64-bit
postgresql_1  | 2023-06-27 07:07:52.883 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
postgresql_1  | 2023-06-27 07:07:52.883 UTC [1] LOG:  listening on IPv6 address "::", port 5432
postgresql_1  | 2023-06-27 07:07:52.884 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgresql_1  | 2023-06-27 07:07:52.889 UTC [2549] LOG:  database system was shut down at 2023-06-27 07:07:52 UTC
postgresql_1  | 2023-06-27 07:07:52.892 UTC [1] LOG:  database system is ready to accept connections

Gitlab is then working fine for us, no data loss, and the admin page is reporting the correct version:

image

+2
davidglassborow on Jun 27, 2023

Good to know @davidglassborow and @sachilles, as the README.md and sample docker-compose.yml (https://github.com/sameersbn/docker-gitlab/blob/master/docker-compose.yml) are still mentioning image: sameersbn/postgresql:12-20200524

So we can “simply” replace sameersbn/postgresql:12-20200524 by sameersbn/postgresql:latest, or is still in testing ? Sticking to 15.11.x in the mean time.

+2
omueller on Jun 27, 2023

Since major versions of PostgreSQL might be related to breaking changes I’d strongly suggest to migrate from major version to major version, i.e. 12 then 13 then 14 then 15.

+1
sachilles on Jul 14, 2023

Hello everyone, I was wondering that can I use use sameersbn/postgres:14-20230628 for my gitlab? since it released but not updated in gitlab-docker-compose file!

+1
pejmanS21 on Jul 1, 2023

Yeah I had to dump the old db, import it into the new version and then start everything and execute the db:migrate rake task manually. No idea why it skipped over one of the migrations but it lead to error 500 on load. Afterwards all seems fine. Maybe that’s just an issue with my installation.

+1
Kiina on Jun 27, 2023

Before merging this PR https://github.com/sameersbn/docker-postgresql/pull/158, you can use this zuojiazi/postgresql image instead of sameersbn/postgresql

+1
Aoiujz on Jun 26, 2023
Read more comments on GitHub
← docker-gitlab: Gitlab 11.0.1 does no longer work with Mysql
docker-gitlab: LFS does not transfer files. file already closed →