docker-compose-buildkite-plugin: v2.0.0 breaking customised build step where docker-compose is not being used for that step

From comments originally at https://github.com/buildkite-plugins/docker-compose-buildkite-plugin/pull/112

@lox confirmation on https://github.com/buildkite-plugins/docker-compose-buildkite-plugin/pull/112#issuecomment-377728994

Release notes didn’t comment on breaking change and that’s the exact message (within reason) I saw by bumping to v2.0.0. Rolling back to v1.8.4 works fine.

We are having to do intermediary build injecting agent held SSH creds via build arguments in so that we can access private git repo so we don’t do a docker-compose plugin based build but obviously need to tag the metadata for future uses by docker-compose plugin.

Our build step looks like this:

  - name: ":docker: build container"
    agents:
      queue: QUEUE_HAS_A_NAME_BUT_I_WONT_TELL
    command: .buildkite/build/build.sh

And .buildkite/build/build.sh looks like this:

#!/bin/bash
set -euo pipefail

echo "--- :s3: retrieving private key (again)";
aws s3 cp s3://BUCKET_NAME/PREFIX/private_ssh_key ./

echo "--- :docker: :ruby: :construction_worker: Building app...";
docker build \
  --build-arg SSH_KEY="$(cat private_ssh_key)" \
  --build-arg SSH_KNOWN_HOSTS="$(cat /var/lib/buildkite-agent/.ssh/known_hosts)" \
  -t app .

echo "--- :docker: :ruby: Tagging & pushing image";
docker tag app $AWS_ACCOUNT.dkr.ecr.$MASTER_AWS_REGION.amazonaws.com/$REPOSITORY:$DEPLOY_VERSION
docker push $AWS_ACCOUNT.dkr.ecr.$MASTER_AWS_REGION.amazonaws.com/$REPOSITORY:$DEPLOY_VERSION

echo "--- :docker: :buildkite: Setting repository and tag";
buildkite-agent meta-data set docker-compose-plugin-built-image-tag-app $AWS_ACCOUNT.dkr.ecr.$MASTER_AWS_REGION.amazonaws.com/$REPOSITORY:$DEPLOY_VERSION

The image being built is ruby, so the SSH details are needed for the private repositories (currently hosted on Bitbucket – we are in the planning phase to shift around 100 repos to Github using Terraform). It is done in an intermediate build step that is subsequently removed.

Using v2.0.0 plugin threw the following for a subsequent step:

  - name: ":docker: :rspec:"
    command: .buildkite/tests/rspec.sh
    plugins:
      docker-compose#docker-compose#v2.0.0::
        run: app
        config: docker-compose.test.yml
        image-repository: $AWS_ACCOUNT.dkr.ecr.$MASTER_AWS_REGION.amazonaws.com/$REPOSITORY
        image-name: $DEPLOY_VERSION
    agents:
      queue: $PLATFORM-test
    timeout: 15

with the message:

# Executing "/tmp/buildkite-agent-bootstrap-hook-runner-#########"
--
  | $ /bin/bash -c /tmp/buildkite-agent-bootstrap-hook-runner-#########
  | $ buildkite-agent meta-data get docker-compose-plugin-built-image-tag-app-docker-compose.test.yml
  | 2018-04-19 00:11:02 FATAL  Failed to get meta-data: POST https://agent.buildkite.com/v3/jobs/#{UUID}/data/get: 404 No key "docker-compose-plugin-built-image-tag-ap..." found
  |  

Which then attempted to do a standard docker-compose build (but no build arguments so no dice).

Would be good to know what the key is there to attempt to further debug.