compose: docker-compose ups a bad build target with multiple compose files

This issue has been automatically marked as not stale anymore due to the recent activity.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

I had problems with this in github actions, but no in my local. I added DOCKER_BUILDKIT: 1 thinking that docker-compose it’s just a wrapper of docker, and it worked.

@DracotMolver you might be missing DOCKER_BUILDKIT=1

As of Docker 18.09 you can use BuildKit. One of the benefits is skipping unused stages, so all you should need to do is build with DOCKER_BUILDKIT=1 docker build -t my-app --target prod

https://stackoverflow.com/a/55320725/368144

@eweidner For me, rebuilding does not fix it, and I’m not running multiple compose files at once. Do you think it’s a different issue?

It seems to keep the build.target from whichever file you ran docker-compose build on.

For example, say my docker-compose.yml specifies target: dev and my docker-compose.prod.yml specifies target: bin. If I now run docker-compose -f docker-compose.prod.yml build and then docker-compose up, it starts the bin stage.

In my case @bog-h i only work with one docker compose file. I’m using version 3.7

This is happening to me but with a different configuration. It’s creating dangling images because when I target to prod it creates the dev stage as well, and this stage was previously created.

My docker file (I removed some info):

FROM node:12 as base

WORKDIR /server

COPY package.json ./

RUN npm install

COPY ./ ./

# ------------------------

FROM base as dev

ENV NODE_ENV=development
ENV APP_HOST="0.0.0.0"

EXPOSE 3000

CMD ["npm", "run", "dev"]

# ------------------------

FROM base as prod

ENV NODE_ENV=production
ENV PORT=5000

EXPOSE 5000

CMD ["npm", "start"]

My docker-compose (I removed some info too):

version: "3.7"
services:
  server:
    build:
      context: ./server
      dockerfile: Dockerfile
      target: dev
    volumes:
      - ./server:/server
      - /server/node_modules
    container_name: newlit_server
    depends_on:
      - db
    ports:
      - 3000:3000
      - 5000:5000

  db:
    image: mongo
    container_name: newlist_db
    ports:
      - 27017:27017
    volumes:
      - mongodb_data_container:/data/db

volumes:
  mongodb_data_container:

If I run docker-compose up for the very first time. It will pull the node image and then will use it as a base for the dev stage. The container is up and running fine.

base stage … I skipped all the instalation of npm dev stage

But then I face 2 situations when I change the target:

• If I change the target value to prod, the docker-compose up doesn’t rebuild the image based on that stage on the Dockerfile. Instead, it’s setting up my container like it was in development mode (devstage). Doesn’t docker-compose up build (rebuild) the image if it doesn’t exist (prod stage doesn’t exist yet) and then run the container?.

rebuild dev stage

• When I run docker-compose build to avoid the step above. It rebuild the base image from cache, perfect!. The dev stage now is dangle because the prod stage has the same label (Am I right why the dangle exist?). But when I see the terminal, it is building the dev stage and then the prod. It make sense If I think docker-compose build runs through all the Dockerfile. If that the case, it is inevitable don’t go through the dev stage.

images after all of that

Now. If I change again to dev and rebuild the image with docker-compose up server --build. It dangled the prev image (prod stage) and create again the dev from scratch, not form the cache. so know i have more dangle images, and this cycle will never ends.

What I finally ended up doing was:

• Remove target from docker-compose.yml
• Create two images running docker build ./server -t server-dev --target=dev and docker build ./server -t server-prod --target=prod
• Using these images on docker-compose.yml when I need them.

...
services:
  server:
    # for productions change it to server-prod
    image: sever-dev
...

There’s a few different things going on in this issue:

1. If target isn’t doing anything at all, it’s likely that you’re not building with BuildKit. Refer to @rafawhs’s comment.
2. There is an issue where, when the target is changed in a compose file, such as:

services:
    foo:
        build:
            context: .
            target: devel

then running compose up then changing the Compose file to:

services:
    foo:
        build:
            context: .
            target: ci

and running compose up again, the image won’t be rebuilt according to the new target. A quick fix to this is running compose up --build, to force image rebuilding. This same issue is what was seen in the original issue, but with overwriting the target attribute of the build section. This doesn’t actually have anything to do with multiple Compose files, but with how/when images are built. After Compose has built the image for the service (with whatever target was defined at the time), the image won’t be rebuilt in consequent ups, unless specified.

Please comment (feel free to @ me) if there are other issues other than these.

Same issue here running docker-compose.yml with a docker-compose.override.yml file.

It worked with BuildKit enabled, as sugested by @infinito84

More about BuildKit: What is BuildKit? - Introducing BuildKit

Hello Guys! I’m glad that it is a common problem because I started to lose my mind 😄

Looks like docker-compose override is ignoring build: target argument. I thought the problem is in the cache, but not!

If you will specify a target (ex debug or base) in the main docker-compose.yaml file then everything will work as expected in depends on your target build: target: debug|base

But if you will specify the target in an override file (ex: docker-compose.debug.yaml) then this parameter is ignored!