compose: docker-compose up doesn't pull down latest image if the image exists locally

Imagine that git didn’t have pull because git fetch && git merge origin/master is functionally identical.

Instead of (or in addition to) adding --pull to the cli, what about adding something on the service definition in the docker-compose file?, e.g.

version: '2'

services:

  postgres:
    image: postgres
    container_name: postgres
    pull: true
    ports:
     - '5432:5432'

This way if there’s a service I don’t care about being latest and one I do, docker-compose won’t waste time downloading images I’m not interested in

No, I’m good. This is an open-source project, if you disagree with the conservative approach we take when adding features, you’re welcome to fork it.

As I mentioned earlier, docker-compose pull && docker-compose up is functionally identical. There’s no good reason to add another flag for it.

Is there a reason why docker-compose pull && docker-compose up is impractical?

I came here looking for this feature because we use it in our production cluster of Kubernetes. There the tag is “imagePullPolicy” and it can be set to “IfNotPresent”, “Always”, or “Never.” Something similar for a compose environment would be nice.

Adding a pull: true tag could be useful for example if some of the images you use in your compose file are in your cache. docker-compose pull pull all of the images in your compose file, and that pull will fail if these images are in your cache but not in the repository.

Here is better reason: && is synchronous. But docker-compose has built in great support for parallel executor which optimizes this stuff. docker-compose up --pull --build could start building image and run it as soon as it’s pulled, instead of waiting for all images to be pulled and only then start building

One scenario in which docker-compose pull && docker-compose up becomes impractical is when you’re using multiple docker-compose files. You can easily end up with a command like docker-compose -f docker-compose.test.yml pull && docker-compose -f docker-compose.test.yml up.

+1

+1

The fact that this “closed” issue has remained so active does suggest that it has not been handled well.

Unfortunately, I’ve found that responses to issue posts on some GitHub repos are not very helpful. The tone is often terse, and suggests that feedback is less than welcome.

Some have pointed out here that this is an open-source project, and (at least most of us) aren’t paying customers. However, it’s also worth noting that submitting issue reports constitutes a significant time investment, even more so if you participate in the issue’s resolution.

A user or developer, having spent time troubleshooting a problem, and found a workaround, will then decide if they want to spend additional time to report it. An unreceptive response from maintainers will likely result in them opting not to bother the next time around.

The software is not really “FREE”, in the “free beer” sense. as we’re all trying to participate in making it better. Having people willing to test your software and provide feedback is a valuable resource. Those with the requisite programming skills are even willing to contribute code, but they want some kind of indication that their contributions are welcome before spending time on it.

Obviously, comments that simply complain about a problem and demand that it be fixed aren’t useful, but the majority aren’t doing that, and comments like “it’s open source, just fork it” are equally useless.

+1

@shin- Why was “–build” implemented? Is it different than docker-compose build && docker-compose up? Just trying to understand the philosophical different between --build (which got added) and --pull (which has been deemed redundant). Understanding the thought process might help me remember how things work. AND if the issue is opened, I’m happy to submit a PR. @everybodyelse… is it really the “spirit of opensource” that if you don’t like something you fork? I thought the spirit of opensource was “if you don’t like something you a) help contribute to the requirements if that’s where you are, b) help contribute to the code if you can” and that you only forked if your requirements were very clearly something only you would benefit from. ie I thought we benefit most when we share - but I’m happy to be educated here.

I ran into this same issue and holy crap, a bunch of whiners in here. This is FREE open source software. Give the maintainers a break. I’m sure they have much more important things to work on than this. If anyone needs this so badly, why don’t you open a PR? If the code is clean with minimal risk, I don’t see why they wouldn’t accept it.

One of the main reasons we use docker-compose is the ability to place a docker-compose.yaml file in to a repository and have reproducible output when a developer pulls the repository and runs docker-compose up [service].

We use several services within our docker-compose files that perform tasks like running codegen and running a tool to dereference a JSON schema in to a single file. Making sure these tools are up-to-date is critical especially if we update our codegen image to fix some common critical issue seen across all projects.

Having the ability to place:

services:
  codegen:
    image: myimage:latest
    pull: Always

would retain our ability to have a developer reliably run docker-compose up and get expected results, rather than having to supplement every single repository with documentation to remind developers to run a chain of commands or scripts to pull the latest availabe images before starting the apps.

It’s not about “the functionality already exists to do this by running these commands”, it’s a better user experience.

Imagine when someone suggested adding --stop to docker-compose rm that the answer was "what’s wrong with docker-compose stop myapp && docker-compose rm myapp, or if someone had requested the implementation of docker-compose down they were just asked why is docker-compose stop myapp && docker-compose rm -v myapp && docker-compose images -q myapp | xargs docker rmi ... impractical?

The reality is that the comment “it’s open source, create your own fork”, is just not realistic. The overhead of maintaining your own fork and keeping it up to date with the latest changes from the main repository makes it rarely worth the investment. A much better approach is to petition the maintainers and provide proper reasons as to why the feature is important.

Sadly that will always result in issues like this one with some dissatisfaction. I think the main issue here is that there doesn’t seem to have been a proper case put forward as to why this is a bad idea. The argument

“We should respect @shin-‘s response on this, and believe there are good reasons for not implementing it.”

just isn’t going to satisfy people. The community need to see the “good reasons”.

@shin is assuming one’s using for ourselves on a very rare basis with rare updating Dockerfiles. But if you update a Docker image every day that developers will use, it will rapidly lead to development problems if just one developer forgets to pull the image which he has to do every day (especially those who are not really familiar with docker, and that’s actually not their business to know and remember it). Disaster programmed. Is it a big problem to just add this option to the docker-compose.yml ? I mean, it will not change other things, it just adds functionality. …

it’s the killer reason why I can’t use docker-compose and need to write wrapper scripts with legacy docker run commands , but that’s ugly.

We have a scenario in which we develop locally and we would only want to pull some of the images from remote. The one (or more) developed locally should remain untouched. In that case we are obligated to build/pull images manually before running docker-compose up. A pull: true would be beneficial.

@shin-, please re-open or lock this issue. It’s been open for almost two years, receiving constant commentary (both intelligent and entertaining), dozens of participants, and hundreds of votes.
However, it seems clear that the compose team is not interested in pursuing this feature. So let’s not waste anyone’s time or give room for false hope that the issue will be revived if that’s not the case.

@shin- Our community mostly thinks the same keeping it unsaid because of the reason you’ve mentioned. @Wenzil is just being honest enough to say it out loud. A lot of people I know prefer to not bother and gave it up on trying to convince docker-compose to move toward its users.

You can disagree with me, but I’m disappointed that you would resort to ad hominems, @Wenzil. Our community generally holds itself to a higher standard.

Any news about this?

Two years of people trying to convince the Docker Compose maintainers that having a --pull option would be a better experience without having to run a separate command. If the docker-compose maintainers just implemented the feature, to begin with, everyone’s lives would be better. It seems to be clear that the current maintainers don’t want to add this feature for the betterment of the community.

Maybe we should just fork docker-compose and update it ourselves.

good point @lifeofguenter thank you. can compose check if image changed to a different hash and pull in that case too?

In such scenerio, I think I’d want all the environments (dev, prod) to have the same image as it possible now that devs are using the new hash while production use the older.

I’d expect --build to bring everything to latest.

@shin- I think @blockloop showed with the git example how convenient / useful the pull option is. Honestly I hope for such things it is not necessary to fork any project.

I totally understand the conservative approach but it feels like it is not even considered as an option anymore. Maybe it can be part of an upcoming version?

@shin- Stop being childish. Plenty of good reasons to implement this feature have been mentioned. At least be open to PRs…

I think most of the points for/against are the same as the ones discussed in the issue for adding --pull to docker run. They range from ux and consistency, to ease of scripting/workflow, to swarm integration (out of curiosity, what does docker-compose pull do with swarm?). I don’t think it’s a major issue, but something to consider. The same type of users who desired the feature elsewhere would likely also enjoy it here.

Another point here is that docker-compose up has a --quiet-pull option. When I was first trying to figure out if up included a pull, I assumed it did based on its presence. It goes to reason that not using --quiet-pull results in a verbose pull.

IMHO --build should pull; no need for any other flag or configuration. if you don’t want it to pull then specify image version.

I think there are two proposals being discussed here:

1 - Should this be added as a command-line option? Actually the posted issue. 2 - Should this option be added to the YML file.

I definitely agree with the latter, though @shin didn’t actually comment on that. To dismiss it with the same argument, that it’s functionally identical would be incoherent. Everything in the YML file is functionally identical with the command-line.

I can see his point in regards to the former, but I think the rationale is a little too broad. I can do just about anything on the command-line by chaining together a series of statements with &&. Let’s be clear, those are two commands, not one. The criteria should be: Is there enough demand for this that it should be possible to do it in one step instead of two? Because if it’s used often enough, the number chained commands just keeps growing. The point is, when you write a script, you want it to be as succinct as possible.

Which product is free exactly here? If I use docker-compose.exe and running docker EE, does not it really means that I’m in fact paying for product?

@orodbhen no need to argue. There is nobody listening to us here.

Trying to figure the circumstances under which this ticket was closed - would someone please elaborate? If it’s any help, I am pro adding a --pull flag to docker-compose up.

@shin- you keep asking for a reason why the && method wont cut it, and my reason is this. I use it for a “app” image for testing (Puppet PDK/Onceover). The compose file is part of the template repo so when a puppet dev (really ops folks) needs to make a new module they fork that repo. Jenkins runs the image for merge validation on that module repo (internally we have a jenkins plugin that handles the update for the jobs.) Now the folks who use this wont be docker experts, and having to tell them to do the && is an extra step that they can (and probably will) screw up. I dont see why it would be hard or what disadvantage it would create but this reasoning seems like a worthwhile reason to add it. It helps us developers send things out that require less directions and steps…

the short of it is… to protect against laziness

If someone were to submit a PR, would it stands a chance if being accepted? This is open source. I’ve been close to looking into implementing it a few times myself. I assume it could be accepted, otherwise this role be closed, right?

Please refrain from using profanities.

A pull: IfNotPresent would be nice. So it could be possible to use fallbacks like

1. use local image
2. pull if isn’t local
3. build if not able to pull

Agree on adding either a flag to the docker-compose up command or a parameter to the configuration. We use a base image with additional configuration that tends to change frequently during the development process. We want to create a fool-proof environment, where a developer can simply run docker-compose up without debugging what they don’t need to debug.

I actually hit this thread after my colleague was checking my pull request and said it broke the build. And the reason being is that the base image was simply missing a couple of packages - but got executed in the final Dockerfile. It would be a good feature, but apparently not something you used, @shin- .

We’d be happy if you introduce this feature in the new version.

@shin- what about rethinking your decision on this? I believe comments and reactions to them look self-descriptive enough.

Hi,

Any news about this issue?