compose: docker-compose build --pull fails on local image

this is still an issue. docker-compose is going to the repo instead of using the local image on my workstation

This should be re-opened. It’s definitely still problematic and frustrating. My compose has a mix of images w/ local and remote base and this makes it difficult to get the latest for remote parents.

I hit this issue today. I have a docker-compose.yml file like this:

version: "2.4"

services:

  solution:
    image: ${REGISTRY}build/lighthouse-solution:${SITECORE_VERSION}-${DEMO_VERSION}
    build:
      context: .
      target: solution
      dockerfile: .\docker\images\windows\demo-solution\Dockerfile
      args:
        BUILD_IMAGE: mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-${WINDOWSSERVERCORE_VERSION}
        BASE_IMAGE:  mcr.microsoft.com/windows/nanoserver:${NANOSERVER_VERSION}
    scale: 0

  mssql:
    image: ${REGISTRY}demo/lighthouse-mssql:${SITECORE_VERSION}-${WINDOWSSERVERCORE_VERSION}-${DEMO_VERSION}
    build:
      context: .
      dockerfile: .\docker\images\windows\demo-mssql\dockerfile
      args:
        SOLUTION_IMAGE: ${REGISTRY}build/lighthouse-solution:${SITECORE_VERSION}-${DEMO_VERSION}
        BASE_IMAGE: ${REGISTRY}demo/base/lighthouse-xp0-modules-base-mssql:${SITECORE_VERSION}-${WINDOWSSERVERCORE_VERSION}-${BASE_MODULE_VERSION}
    depends_on:
      - solution

I use the solution service to build all my code once. Then, I use the resulting lighthouse-solution image as a dependency and a base image in many other service dockerfiles. We never push the lighthouse-solution image to our Docker registry as it is only used for building the other images.

When building with docker-compose build --pull, we are getting the issue described here:

ERROR: Service 'mssql' failed to build : manifest for scr.sitecore.com/build/lighthouse-solution:10.0.0-1000.0 not found: manifest unknown: manifest tagged by "10.0.0-1000.0" is not found

I resolved this problem by building my own PowerShell pull and build script:

$configuration = docker-compose config
$images = @()

# Find images to pull in the docker-compose configuration
foreach ($line in $configuration) {
  if ($line -match "(BUILD_IMAGE|BASE_IMAGE|ASSETS_IMAGE):\s*([^\s]*)") {
    $images += $Matches.2
  }
}

# Pull images
$images | Select-Object -Unique | ForEach-Object {
  $tag = $_
  docker image pull $tag
  $LASTEXITCODE -ne 0 | Where-Object { $_ } | ForEach-Object { throw "Failed." }
  Write-Host ("External image '{0}' is latest." -f $tag) -ForegroundColor Green
}

docker-compose build

I know I am using only the BUILD_IMAGE, BASE_IMAGE, and ASSETS_IMAGE build arguments for base images in my docker-compose.yml file. I also know I am not using hardcoded base images in my dockerfile files. This allows me to avoid loading the dockerfile files as the Python solution shared in this thread.

I hope this helps others facing the same issue.

Thanks @jflheureux that helped me a lot and got me started on a similar one for us to run as a prior CI build step. In my case it was Dockerfiles, so I have my script find the files, rip out the dependencies and good to go. Will share in case my version is useful for folks too.

# Find the Dockerfile/s you need to pull latest image dependencies for
$dockerfiles = Get-ChildItem -Path "$PSScriptRoot" -Filter "Dockerfile" -Recurse -File | foreach fullname

# An exclude list of any local ones you have that'll cause issues with --pull
$ourLocalImages = @("microservices-builder:latest", "something-else:latest")

# Find images needed
$images = @()
foreach ($dockerfile in $dockerfiles) {
    foreach ($line in [System.IO.File]::ReadLines($dockerfile)) {
      if ($line -match "^FROM (.*) AS .*$" -and $ourLocalImages -notcontains $Matches.1) {
        $images += $Matches.1
      }
    }
}

# Pull images
# as jflheureux was doing above

same here, would welcome having --ignore-pull-failures