postgres: Problem setting environment variables through docker-compose

Yeah, the problem is not the existing images on your host, but it stems from the volume (or bind mount directory) being already initialized after your first start. The postgres user, and database creation only happens on the first start (ie, /var/lib/postgresql/data must not already contain database files).

Here is an example flow

1. declare volume for my image via compose: ./data/postgres:/var/lib/postgresql/data
2. set env in yaml: POSTGRES_PASSWORD=test
3. start containers docker-compose up -d
4. decide to add database to postgres env POSTGRES_DB=test_db or change the password
5. restart my containers docker-compose up -d
6. database test_db does not exist or password is unchanged 😢
   1. db files must be removed in order for postres entrypoint to re-initialize: docker-compose stop; sudo rm -rf ./data/postgres/
   2. now it can be restarted docker-compose up -d 😃

If you do not declare a volume mount point, then the VOLUME declared in the postgres image will apply and docker will create and manage the directory independent of the life-cycle of the container. This gets more complicated when using compose, since it will keep the volume to re-use later even when you docker-compose rm -f all of your running containers.

An example without a bind mounted volume:

1. say we do steps 2-5 above
2. we still have the same problem of the test_db database not existing (or the password not changing) since the data from the volume still exists 😞
   1. so, get rid of all containers and their volumes docker-compose rm -fv
   2. or docker-compose rm -fv postgres to get rid of just the postres service and its volumes
   3. now we can start up a new postgres container with a new empty volume docker-compose up -d 😃

You can see what volumes you have on your host by doing a docker volume ls (bind mounts will not show up in this list). There is currently no easy way to see what containers are attached to a volume, so you would have to docker inspect the container to see which ones are attached to it.

If you want to clean up all local volumes that are not attached to containers (WARNING this could delete important data 😲): docker volume ls | awk '$1 == "local" { print $2 }' | xargs --no-run-if-empty docker volume rm. On my development machine I usually precede this by removing stopped containers.

Also, in case it is not obvious, do not delete your postres data directory or volume if you have important data stored there. 😱

I still don’t get it but I am using the following docker-compose.yml and it works perfectly fine. I was in a similar problem that you have mentioned and the trick was to just remove the previous docker image and data dir and re-run the container from scratch.

version: '2'
services:
  postgres:
    image: postgres:9.3
    ports:
    - "5432:5432"
    environment:
      - POSTGRES_USER=test
      - POSTGRES_PASSWORD=test
      - POSTGRES_DB=test_db
    volumes:
      - ./data/postgres:/var/lib/postgresql/data

I do not know for sure if it was the root cause, but the syntax used to specify the test_db environment variable appears to be incorrect and will not set POSTGRES_DB:

  environment:
    - POSTGRES_DB:test_db 

You can either specify it as a hash (no -):

  environment:
    POSTGRES_DB:test_db 

Or as an array, -, w/ = assignment rather than key:value

  environment:
    - POSTGRES_DB=test_db 

I suspect this is happening because you have a previous image of this in your docker where you ran docker-compose up without providing the envs and hence it is not able to overwrite. Check and delete postgres image by doing docker ps -a and docker rm <image> respectively.

FYI, was helping a teammate with this issue today. It turns out they had a local installation of PostgreSQL running, and that was causing the issue. Once we stopped the (brew packaged) service with brew services stop postgresql, docker-compose worked as expected, setting the correct user and database name from the environment variables.

Had this issue when postgres data volume was in /var/… where my user had no access right. After moving it to home directory, it works.

Had the same issue. In my case it was incorrectly defined volume:

I had volume definition in docker-compose.yml: volumes: postgres-data:

I defined the volume for postgres like this: volumes: - '~/docker-volumes/postgres-data:/var/lib/postgresql/data' while I should define it like below: volumes: - 'postgres-data:/var/lib/postgresql/data'

Because of the first version the volume was not removed during docker volume rm and because of that I couldn’t update password with POSTGRES_PASSWORD, even after doing docker system prune -a (care with this command as it removes everything docker-related) 😃

Had the exact same problem. Removed all images and deleted the data directory as @mrafayaleem suggested - works!

For me adding --renew-anon-volumes did the trick (for now).

encounter same issue, @auraz your suggestion works perfectly! many thanks