compose: Docker compose pull doesn't respect local images

I’d agree with @johnharris85 on --ignore-pull-failures not feeling like the option to use. Something like --prefer-local would work - but I’d still probably prefer a concrete local: my-org/my-image:tag to make for a clearer intent.

I didn’t know you could pull specific images, so thanks for that @dnephin - the downside being, this would move knowledge about the composure of my stack outside the compose file, whereas for me a compose file is a one-stop encapsulation of my architecture.

–prefer-local (prefer newer image with same tag on local machine) would be great - than we can use the same compose file and workflow in local development and in production - and it is always possible to locally override images with ease for testing purpose

Both of those options seem kind of sub-optimal.

It’s not really a ‘failure’ to ignore, more of a separate use case.

The second option means you’re having to declare some behavior / state outside of the compose file. I.e. I have all of this automated but then want to change which images are pulled and which aren’t, I probably want that to be reflected in my compose file as a single-point-of-truth, but I have to edit the pull command instead.

A hacky workaround to use your local images is to run your own registry:

docker run -d -p 5000:5000 --name registry registry:2
docker tag 4e6a104bad19 localhost:5000/stevek/my-container
docker push localhost:5000/stevek/my-container
# edit docker-compose file to use localhost:5000/stevek/my-container
docker-compose ...

I’m facing similar issues. I use the --ignore-pull-failures option when I do a docker-compose pull. This basically ignores ALL errors. It is ok as a workaround. We do the same thing in our pipeline. I pull other production-images and start my freshly build release-candidate container locally. When the tests are ok, it’s uploaded.

Don’t close this issue. This is an automatic message by Fresh - a bot against stale bots.

Problem: docker-compose -f docker-compose.yml up -> doesn’t look locally for the image at all. I agree @johnharris85 - These options are suboptimal. Docker is sort of acting like a dependency manager or at least one dependency resolver. It pulls images from a specified/default repo then/and only then, commences it’s docker process. In the same way gradle, ant, maven, gulp, npm, etc. pull dependencies and define processes the user can run. The standard has been set for Dependency Mgr/Resolvers, look locally first (sometimes cache or something), then pull from repo. I’m trying to think of a really pressing reason why docker would want to force you to look at the repo first. Just like @antony & @decoomanj I don’t want to push any images that don’t even satisfy our lowest level of quality threshold: unit tests. Not to mention any other test suite.

Suggestion: rancher-compose offers a --pull option to insure you’re getting the latest image from the repo. It would be great if docker-compose looked for images locally first then to the repo. And if necessary there can be a --pull option to force docker to only search for images in the repo.

This kind of solution isn’t really viable:

 --ignore-pull-failures was added for this very reason.

People want to ignore that an image is local. Not necessarily every other type of error.

Developers aiming to create robust systems will program strictly and abort on the first error. This breaks with that.

In nearly all my years of programming auto-fix, force and ignore options turn out useless if you program safely. They tend to only be genuinely useful in 0.0001% of cases for safe code.

If it’s to be useful, then it should only fire when the error is non-critical (as in it has the image, only update failed). That’s or there should be a variant that does that.

It should be possible to have something such as pull --missing-only. That would at least allow a pre-fetch.

It doesn’t seem like docker stores enough information on images to know if they are local or remote.

The correct way to support it would be to add an option in the docker compose file that an image is built locally.

There’s the question of why this happens at all if docker-compose can build images (and then knows that they are local).

For me it’s because it isn’t very good at that (for example it doesn’t have a separate build section, instead it’s crammed into services), I’ve made my own tool to manage and wrap that.

This then breaks pull because docker-compose doesn’t know about them being built locally.

I can implement the functionality I need with my system. At that point docker-compose wont be used for build or pull at all for my, only service management (run, stop, up, down, start).

It would be nice though if docker compose fixed these issues at the core. It is also missing an image section where you should be able to specify images that are local and some management for private repositories.

If you have a repository section then you could specify images for those repositories (is left out maybe have it automatic, then some thought going into what happens when an image is in the global and one or more private).

repositories:
     local:
           url: ~ (maybe blank for local)
           images: [a, b, c] (simplest)
           namespace: company (works dependent on how you're tagging your images, matches company/db, can be more complex like a/b/c)

I don’t see this is a issue / bug with current functionality. pull is doing exactly that, how would it know in your original compose file that 2 images should be pulled remotely but one should be grabbed locally (seeing as they all probably exist locally right, you just want to ‘refresh’ the other two?).

Perhaps a --filter option to pull? So something like:

docker-compose.yml
version: '2'
services:
  api:
    image: mycompany/api
    labels:
      - "refresh"
  undertest:
    image: mycompany/undertest:${CIRCLE_SHA1}
  db:
    image: mycompany/db
    labels:
      - "refresh"

Then:

docker-compose pull --filter "label=refresh"

@aanand @dnephin thoughts?

To sum up, the problem is the following:

• we want to run docker-compose pull to pull external images
• we don’t want the same command to fail on local only images, because that’s not an actual error, that’s the expected behavior (you can’t pull an image not pushed on a registry)
• we don’t want to have to specify the list of images to pull in the command line, because we want all the config to stay in the docker-compose file
• we don’t want to ignore all errors, because errors can happen and we want to see them
• we don’t want to have to run a local registry, that’s useless and prone to security errors

Currently it fails because it does not know that the image is local only. Can’t we add something in the docker-compose file to let it know it is local only and it should not try to pull it? Something like this would do the job:

services:
  my_service:
    image: some_external image
  my_private_service:
    image: some_local_only_image
    pull: false

This way I am explicitly telling docker compose that I don’t want it to try to pull this image, and everyone is happy.

The second option means you’re having to declare some behavior outside of the compose file.

Yes, and in this case I think that’s correct. The behaviour that is external to the Compose file is not the behaviour of a container, but part of your project dev workflow. Compose is not a project workflow tool. It’s a tool for orchestrating containers.

The Compose file should really only include configuration for those containers. How to pull an image is not container configuration, it’s part of the workflow, and currently all of those options are commands and flags. I think this is an important distinction. Trying to mix both would make the Compose file very confusing.

The equivalent of --prefer-local is to just never run docker pull in the first place. Any missing images will be pulled before trying to run the container.

--ignore-pull-failures was added for this very reason.

You could also do docker-compose pull db api to only pull the ones you wanted.

I think this syntax is more logical.

services:
  my_service:
    image: some_external image
  my_private_service:
    image:
      name: some_local_only_image
      pull: false

Is it not simpler for there to be a reserved “repo” name for locally-built images such that something like:

service:
  my_service:
    image: localhost/myimage:devtag
  my_other_service:
    image: redis:alpine

Means that Compose knows that my_service is using an image that only exists as a local build and would never attempt to pull it?

@fgauer In my case I don’t want to pull images if there is already a local image because that can be time consuming if there are lots of images (even if all of them are already pulled), furthermore it could give errors if there’s no internet or if the connection fails.

What I did to solve it was to not use image: but build: in the docker-compose.yml file, because build pulls only when needed, and I use a Dockerfile that only defines the image I want to pull. I use build args to not have to create lots of Dockerfiles, like the following:

Dockerfile:

ARG IMAGE
ARG VERSION

FROM $IMAGE:$VERSION

docker-compose.yml:

services:
  my_service:
    build:
      context: .
      dockerfile: "path/to/Dockerfile"
      args:
        IMAGE: "alpine"
        VERSION: "latest"

Then I just run docker-compose build (and not docker-compose pull) and the problem is solved for me.

I don’t know if it will work in your case, but you could give it a try.

My workaround: DOCKER_BUILDKIT=0 docker compose -f docker-compose.yml up your-app

AFAICT pull_policy: missing introduced by the compose specification should be enough to address this issue.

Still no solution better than creating a local registry ?

A workaround for this while developing locally to save pull time:

1. Create a local docker registry like this
2. Pull the image from the remote docker registry (abc/xyz:latest)
3. Tag this image to point to the local docker registry (docker tag abc/xyz:latest localhost:5000/abc-xyz-latest)
4. Push this image to the local docker registry (docker push localhost:5000/abc-xyz-latest)
5. Use this image in your local docker-compose file

Hi chiming in here…this has had me stumped for days trying to stand up a new Docker based development environment using docker-compose and has led to much frustration and needless pain.

Coming from an environment where we have not used the image: feature in docker-compose.yml and now attempting to utilize it for CI/CD purposes for a new project we realized, after a significant amount of time, trial and error, that the docker-compose up always pulls from the repo defined in image: and does not give us a way to intuitively pull from a local image where/when we need to (i.e. local development).

We have the following goals with our setup:

1. We want to use the same set of docker-compose.yml files for CI/CD purposes in various environments, which includes local development, that should allow for a developer to make changes to the code base and immediately re-build the containers on their local machines in order to work with their changes.
2. In order to facilitate #1, in other words use the same set of scripts for various deployment environments, we use ENV variable substitutions in our docker-compose.yml files. This works fine until you throw in the local development scenario where we need to grab locally built images for development iteration.

Since the image: feature currently does not allow us to intuitively override ride it in such a way as to have it look for a local image, I believe we faced with going with one of the following options below. Please bear in mind that whereas the options below work, it was getting to this conclusion that was really quite painful due to the counterintuitive nature (at least this was my experience personally) of how the image: feature works without some kind of local override option.

1. Setup a completely separate docker-compose.yml file for local development purposes, or overlay multiple docker-compose.yml files to specify the environment’s image: tag in all environments except for local development. But with the ENV variable substitution features current working in docker-compose.yml for all of our other environments, why would we have to go here? It would be nice to substitute something in like ‘local:’ for the image name and have Docker Compose respond accordingly using ENV variables.

2. Setup a local ‘development’ repository on each developer’s local machine. We could do this, but it seems to rub against one of the fundamental advantages of using Docker for local development which is to minimize the amount of individual local environment setup for individual developers, and tracking/managing that whole circus.

docker run is a complete distinct beast actually. But yes indeed, could be implemented quickly as the structure is already defined in the compose-spec model

I said this was introduced by the compose-spec, and could be used to support this feature request, not that it’s already supported in docker compose!

@ard-i that’s not what the issue is about. The issue is about using docker-compose pull when at least one image in the docker-compose file does not exist on the remote registry but exists locally.

Bump!

With the status quo, using —ignore-pull-failures and —include-deps together doesn’t actually work… my dockerfile has “FROM: ubuntu:16.04”, but this dependency is never pulled because of the pull failure…

Of course it is pulled when I later build, but there is no way to force pull the latest version of every dependency, so it is left with the first version of the image forever…

I imagine there is a lot of out of date code in production for this reason…