compose: Using `${foo:?Error}` construct is expanded at too early.

using ${foo:?We need foo} construct is expande/triggered too early

When using multiple docker-compose files, e.g., docker-compose -f docker-compose.yml -f docker-compose.dev.yml and the former has a

service:
  build:
    args: 
      FOO: ${foo:?Set 'foo' something}

construct, and the latter docker-compose file has for example,

service:
  build:
    args:
      FOO: iamhardcoded

Then, using docker-compose errors with:

docker-compose config
ERROR: Missing mandatory value for "build" option in service "service": set 'foo'

Context information (for bug reports)

Output of docker-compose version

docker-compose version 1.22.0, build f46880fe
docker-py version: 3.4.1
CPython version: 3.6.6
OpenSSL version: OpenSSL 1.1.0f  25 May 2017

Output of docker version

docker version
Client: Docker Engine - Community
 Version:           19.03.2
 API version:       1.40
 Go version:        go1.12.8
 Git commit:        6a30dfc
 Built:             Thu Aug 29 05:29:11 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.2
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.8
  Git commit:       6a30dfc
  Built:            Thu Aug 29 05:27:45 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Output of docker-compose config (Make sure to add the relevant -f and other flags)

docker-compose -f docker-compose.yml -f docker-compose.dev.yml config 
ERROR: Missing mandatory value for "build" option in service "service":  Set FOO !

Steps to reproduce the issue

1. Use https://gist.githubusercontent.com/hansbogert/575eb05444a90e69dad8fb0c742b9626/raw/b8738564518bf7c052e176333f6182e0a91448d2/docker-compose.yml and https://gist.githubusercontent.com/hansbogert/a47b8bde980375af13a3658f0a7b65fe/raw/939da84fa15e910deca30de0aac3a96417e31d0d/docker-compose.dev.yml as docker-compose files
2. run docker-compose -f docker-compose.yml -f docker-compose.dev.yml config

Observed result

ERROR: Missing mandatory value for "build" option in service "service":  Set FOO !

Expected result

version: '3.7'
services:
  service:
    build:
      args:
        FOO: iamhardcoded

Additional information

Ubuntu 18.04.