compose: Multiple Compose files with common anchors do not work
Using the anchor utility of YAML, we cannot use some common defined anchors (such as extension fields, x-*), to declare common configuration in the base compose file and then use them in the extra compose overrides. Docker compose throws error found undefined alias
To reproduce create fragment in docker-compose.yml:
x-function: &json-logging
driver: "json-file"
options:
max-size: "100m"
And use this anchor in a point in a docker-compose.prod.yml:
logging: *json-logging
Running docker-compose -f docker-compose.yml -f docker-compose.prod.yml config will get the error mentioned. Only solution is to duplicate information across all compose files
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Reactions: 7
- Comments: 29
This is not “working as intended”, this is a severe deficiency.
I had the same issue my solution is instead of using
docker-compose -f docker-compose.yml -f ci/docker-compose.prod.yml buildI’m usingcat docker-compose.yml ci/docker-compose.prod.yml | docker-compose -f - buildThis resolves the the aliases inside docker-compose.prod.yml correctly.
There is only 1 small tweek needed
docker-compose.yml
ci/docker-compose.prod.yml
Yes, they can’t be used across different files.
Normal, PyYAML, Py-People are very garbage cans
@mcabrams You might want to call
docker-composewith multiple YAML files like this:b.ymlwill extend/modify what was declared inb.yml.This is documented here: https://docs.docker.com/compose/reference/overview/#specifying-multiple-compose-files
@Papipo No, more like the other way around. v2 supported
extendsbut v3 did not, as the documentation specifies. After many complaints/requests (see the issue I linked to), it has been added to v3 as well. The note in the documentation that says v3 does not support it (which also links to the same issue) has not yet been removed to reflect the change.@mcabrams Have you found an alternative strategy? I had a similar plan as you, storing all of my common volumes, secrets, and configs as yaml snippets using anchors in a compose.base.yaml file with aliases in my compose-stack.yaml as in:
I hear what @gautaz is saying, but IMO without cross compose file yaml anchor support the idea of building up a compose file from multiple yaml files isn’t very useful 😞.
@gautaz Thanks – I’m familiar with and currently use extension by specifying multiple compose files. My question is whether anyone has any strategies for achieving extension via multiple compose files and reusing common configuration of services.
Typically… closed without real solution and users asking what’s on 🤔 Great ticket management …
Yeah it’s a shame it hasn’t been solved for 😦
Is this ever going to be fixed or is everyone expected to duplicate information or use the
cat file1.yml file2.yml | docker-compose -f -hack? And if we have to use the hack, how do we also use terminal input? Or do we need to use a docker-compose file generator script instead?As for those who blame it on the specification, this is a severe deficiency that needs some sort of solution, and the only ones currently available are hacks. So figure out a way to accomplish this without violating the specification, or create your own extension to the specification.
I just ran into this myself. I wanted to load a different logging configuration depending on which additional docker-compose file was specified, where I would have defined those logging configurations as yaml templates.
I solved this instead with
extends(wich is supported with compose v3 files as of docker-compose 1.27.0, despite what the documentation says). So now my main docker-compose has a template service thatextendsanother service, the filename of which is an env variable. Depending on how I set that variable, one of two logger configurations is used (loki for production, json-file for development). All of my actual services then use that template service