compose: Don't build the same thing twice

Even though Docker caches layers and the second build is fast, it’s not necessary for Compose to build the same thing twice. As far as I can tell, if two services in the yaml have exactly the same string, you can be sure they will be exactly the same build:

21:02 ~/docker/test $ cat fig.yml 
svc1:
    build: oog
svc2:
    build: oog
21:02 ~/docker/test $ fig build
Building svc2...
 ---> e72ac664f4f0
Successfully built e72ac664f4f0
Building svc1...
 ---> e72ac664f4f0
Successfully built e72ac664f4f0

Naturally, the image built is the same for both services.

There’s also the question of the race condition if someone edits the Dockerfile while fig is already building. This is a bit contrived, but does work:

$ cat fig.yml
svc1:
    build: oog
svc2:
    build: oog
$ diff oog/Dockerfile[12]
3c3
<  && echo hi

---
>  && echo bye
$ ( cd oog;ln -sf Dockerfile{1,} ); fig build --no-cache & ( cd oog;sleep 2;ln -sf Dockerfile{2,} ); fg
[1] 1374
Building svc2...
 ---> e72ac664f4f0
Step 1 : RUN sleep 5  && echo hi
 ---> Running in dd0c813779ec
fig build --no-cache
hi
 ---> 9f1aed6d839c
Removing intermediate container dd0c813779ec
Successfully built 9f1aed6d839c
Building svc1...
 ---> e72ac664f4f0
Step 1 : RUN sleep 5  && echo bye
 ---> Running in 3c7d54f6145a
bye
 ---> 02ece1cf8f17
Removing intermediate container 3c7d54f6145a
Successfully built 02ece1cf8f17

Contrived or not, if it didn’t attempt to run the same build twice, it wouldn’t have divergent images.

About this issue

Original URL
State: closed
Created 9 years ago
Reactions: 45
Comments: 35

Most upvoted comments

I’ve managed this issue in the following way:

version: "3"
services:
  build:
    image: single_image_app
    build:
      context: .
      dockerfile: test-Dockerfile
  svc1:
    image: single_image_app
    depends_on:
      - build
  svc2:
    image: single_image_app
    depends_on:
      - build

+15

pmatv on Apr 14, 2020

I’m trying to use compose to launch multiple containers from the same image but with a different command. Running docker-compose up causes the same image to be built a couple of times, which can take a long time depending on the number of containers I’m launching.

Right now, my workaround is to build and tag the image separately, and then reference it from the compose configuration file. However, this requires other developers on my team to be aware that just running docker-compose up does not work unless you have previously built and tagged the image.

I think this could be addressed in compose if there was a way to reference built images within the docker-compose.yml file.

etcinit on Feb 23, 2016

As of Compose 1.6.0 you can use both image and build together to do just that. You can optionally build the image outside of compose, and up will just use the image you built, but if someone else runs build, it will do the right thing and build the same tag.

dnephin on Feb 23, 2016

I’ve managed this issue in the following way:

version: "3"
services:
  build:
    image: single_image_app
    build:
      context: .
      dockerfile: test-Dockerfile
  svc1:
    image: single_image_app
    depends_on:
      - build
  svc2:
    image: single_image_app
    depends_on:
      - build

I’m on compose v2.12.1, for some reasons I had to add pull_policy: never to all services, so that they don’t try to pull from docker hub.

jimmygchen on Dec 9, 2022

doesn’t work now =(

terion-name on Oct 25, 2022

Both docker-compose in latest releases and Compose V2 use BuildKit as a builder backend, and this one manage building images in parallel with deduplication for equivalent Dockerfiles in compose services.

ndeloof on Jun 17, 2021

I recommend using Procfile with overmind / hivemind / foreman

elquimista on Oct 3, 2018

just write a separate docker-compose file specially for build for example:

21:02 ~/docker/test $ cat fig.yml 
svc1:
    image: xxxxx
svc2:
    image: xxxxx

21:02 ~/docker/test $ cat fig-build.yml 
svc1:
    image: xxxxx
    build: oog

21:02 ~/docker/test $ docker-compose -f fig-build.yml build
...........

21:02 ~/docker/test $ docker-compose -f fig.yml up

zsyh on Aug 9, 2018

What about using partial images?

Generate an image with all common factors and build on top of it.

This may also be resolved together with #1896, though I believe we actually need actually more decoupling of building and running (Only-build/partial images, which would allow devs to use “FROM” to build on top of it), while also allowing for granular control during run-time, which in the end could amount to 2 separate, albeit somewhat related, issues. Depends a lot on the design.

My suggestion would be for the Compose-File to allow two new keys, images and commands. Relating to this issue, images would only build and serve as basis for the services. The key commands would be related to #1896.

frnco on Mar 22, 2017

Here is the current link for docker-compose file reference build

Glideh on Dec 5, 2016