compose: env variables from .env file are not set in docker compose file

There’s an obvious confusion here between:

• .env file which is used for variable substitution in the docker-compose.yaml file, and should resolve when you run docker-compose config
• env_file element in a service definition, which is just setting the environment file to send to docker engine when starting container, as a definition of runtime environment. Compose will not parse this file which is opaque to him.

So there’s nothing wrong here.

the question that i’d have to y’all is from which working directory are invoking Compose. because i encounter the same when i have one of the subfolders beside of a Compose project as a working directory.

Thanks for replying @ps23. Can I also know which OS you ran on… I’m running in Mac OS 10.14.3

tried above with .env file which resides in the same folder as .yml file, but still not populating

docker-compose -f postgreServer-test.yml config

WARNING: The POSTGRES_VERSION variable is not set. Defaulting to a blank string.
WARNING: The POSTGRES_USER variable is not set. Defaulting to a blank string.
WARNING: The POSTGRES_PASSWORD variable is not set. Defaulting to a blank string.
WARNING: The POSTGRES_DB variable is not set. Defaulting to a blank string.
services:
  testpg:
    environment:
      POSTGRES_DB: ''
      POSTGRES_PASSWORD: ''
      POSTGRES_USER: ''
    expose:
    - 5432
    image: 'testpgimage:'
    ports:
    - published: 5432
      target: 5432
version: '3.7'

+1

LSB Version: :core-4.1-amd64:core-4.1-noarch Distributor ID: Fedora Description: Fedora release 29 (Twenty Nine) Release: 29 Codename: TwentyNine

Docker version 18.09.5, build e8ff056

docker-compose version 1.22.0, build f46880f

# .env-bug
VALUE=not-null

#docker-compose.yml
version: "3.7"
services:
  cli:
    image: debian:stretch-slim
    env_file: .env-bug
    environment:
      - VALUE=$VALUE
    command: echo -e "VALUE=$VALUE VALUE=${VALUE}"

it is run with

docker-compose --file docker-compose.yml up

however

docker-compose --file docker-compose.yml config

WARNING: The VALUE variable is not set. Defaulting to a blank string. services: cli: command: echo -e “VALUE= VALUE=” environment: VALUE: ‘’ image: debian:stretch-slim version: '3.7

The following pulls the environment variables into the container that is created. It does NOT substitute the placeholders in the yml file.

env_file:
          - POSTGRES_SERVER.env

Steps to correct your approach

postgreServer-test.yml

version: "3.7"

services:
    testpg:
        image: testpgimage:${POSTGRES_VERSION}
        ports:
            - 5432:5432
        environment:
            - POSTGRES_USER=${POSTGRES_USER}
            - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
            - POSTGRES_DB=${POSTGRES_DB}
        expose:
            - 5432

.env

POSTGRES_VERSION=9.6
POSTGRES_USER=testuser
POSTGRES_PASSWORD=testPWD
POSTGRES_DB=testdb

docker-compose -f postgreServer-test.yml config

services:
  testpg:
    environment:
      POSTGRES_DB: testdb
      POSTGRES_PASSWORD: testPWD
      POSTGRES_USER: testuser
    expose:
    - 5432
    image: testpgimage:9.6
    ports:
    - published: 5432
      target: 5432
version: '3.7'

Maybe this sounds stupid and its obvious. But i had this similar issue until i sorted the variables in the .env file and docker-compose.yml in the same order.

I saw this diff on last version if this may help.

$ docker-compose --version
docker-compose version 1.28.5, build c4eb3a1

I’ve 2 same docker compose-files

$ cp docker-compose.yml  docker/docker-compose.yml
$ cat docker-compose.yml

compose content:

version: '3.7'

services:
  MyMongo:
    image: mongo:3.4.20
    container_name: mymongo
    environment:
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: ${MY_VAR}

I’ve following env file:

$ cat env/myenv.env
MY_VAR=toto

– the following didn’t work:

$ docker-compose --env-file ./env/myenv.env -f ./docker/docker-compose.yml config
The MY_VAR variable is not set. Defaulting to a blank string.

this works:

docker-compose --env-file /C/WORK/perso/tmp/dock/env/myenv.env -f ./docker/docker-compose.yml config
docker-compose --env-file ./env/myenv.env -f ./docker-compose.yml config
docker-compose --env-file ./env/myenv.env config
cd docker && docker-compose --env-file ../env/myenv.env config

source <PATH>/.env
docker-compose up

or

env $(cat <PATH>/.env | xargs) docker-compose up

So why is docker-compose not sourcing the .env files relatively to the passed docker-compose.yml? The command env $(cat <PATH>/.env | xargs) does its job but it makes the script less slim and has an imho unnecessary repetition in it, e.g.:

for DIR in "${DIRS[@]}"; do
    env $(cat "$DIR/.env" | xargs) docker-compose -f "$DIR/docker-compose.yml" up -d
done

compared to

for DIR in "${DIRS[@]}"; do
    docker-compose -f "$DIR/docker-compose.yml" up -d
done

I know I can also do

HOMEDIR=$(pwd)
for DIR in "${DIRS[@]}"; do
    cd $DIR
    docker-compose up -d
    cd $HOMEDIR
done

but that feels even worse 😞

Hmm I had similar issue, as it turned out I called docker-compose up from the service directory 😅 so I guess you should call it from the docker-compose.yml directory level

Nevermind, this one did the trick: eval $(egrep -v '^#' ./db/.env | xargs) docker-compose config

Thanks!

I have similar issue. The difference for me is that docker-compose --file docker-compose.yml up does not work either. But the

export VALUE=non-null
docker-compose -f docker-compose.yml up                                                                                                
Recreating yoba_cli_1_9b9c55ff203c ... done
Attaching to yoba_cli_1_9b9c55ff203c
cli_1_9b9c55ff203c | VALUE=non-null VALUE=non-null
yoba_cli_1_9b9c55ff203c exited with code 0

works.

The package versions are:

% docker --version
Docker version 18.09.7, build 2d0083d657f82c47044c8d3948ba434b622fe2fd
% docker-compose --version
docker-compose version 1.23.1, build b02f130

OS:

% nixos-version 
19.03.git.35841f8 (Koi)
# nixpkgs version is 35841f87afa7faaeb3fd0f558f9226ebdae0b954

@ps23 Thanks for your reply!

However, it seems compose can’t read the env variables set. I created a run.sh file at the root of my project and the result is this:

run.sh
#!/bin/bash

source ./db/.env
# env $(cat ./db/.env | xargs)

echo ${CONTAINER_DB_NAME}
echo ${DB_PATH}


docker-compose config

exit 0

and this is the output:

db
./../data/db
WARNING: The CONTAINER_DB_NAME variable is not set. Defaulting to a blank string.
WARNING: The DB_PATH variable is not set. Defaulting to a blank string.
services:
  db:
    container_name: ''
    environment:
      MYSQL_DATABASE: ''
      MYSQL_PASSWORD: ''
      MYSQL_ROOT_PASSWORD: ''
      MYSQL_USER: ''
    image: mariadb:latest
    restart: unless-stopped
    volumes:
    - .:/var/lib/mysql:rw
version: '3.0'

The variable content is printed, but it’s not passed down to compose. I’m running this in a mac os, BTW.

@mcnesium Because you can pass in multiple yml files from different locations on a single compose command