yq: Can't handle multiple yaml documents within one file

Given the following Kubernetes YAML configuration file:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
   name: somename
spec:
   revisionHistoryLimit: 5
   minReadySeconds: 10
   strategy:
      type: RollingUpdate
      rollingUpdate:
         maxUnavailable: 0
         maxSurge: 1
   replicas: 1
   template:
      metadata:
         labels:
            microservice: somename
      spec: 
         containers:
         - name: serviceapi
           image: myimage:0.0.6
           ports:
           - containerPort: 8080
           env:
##### ENVIRONMENT VARIABLES
            - name: somename
              valueFrom:
                configMapKeyRef:
                   name: serviceapi-environment-configuration
                   key: somekey
---
apiVersion: v1
kind: Service
metadata:
  name: somename
  labels: 
     microservice: somename
spec:
   type: NodePort
   ports:
   - port: 8080
   selector:
      microservice: somename
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: some-name
  namespace: default
data:
  someData: data

This command will work:

./yaml w -i myfile.yaml spec.template.spec.containers[0].image myimage:0.0.19

The image value will be replaced, however, the file will then change to this:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
   name: somename
spec:
   revisionHistoryLimit: 5
   minReadySeconds: 10
   strategy:
      type: RollingUpdate
      rollingUpdate:
         maxUnavailable: 0
         maxSurge: 1
   replicas: 1
   template:
      metadata:
         labels:
            microservice: somename
      spec: 
         containers:
         - name: serviceapi
           image: myimage:0.0.19
           ports:
           - containerPort: 8080
           env:
##### ENVIRONMENT VARIABLES
            - name: somename
              valueFrom:
                configMapKeyRef:
                   name: serviceapi-environment-configuration
                   key: somekey

As you can see, the Services and ConfigMap sections (denoted with ---) have disappeared.

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Reactions: 2
  • Comments: 27 (12 by maintainers)

Most upvoted comments

Gday - so I’ve added the ability to operate update commands against all documents

yq w -d'*' examples/multiple_docs.yaml user.surname farah

And you can read from a specific doc

yq r -d1  examples/multiple_docs.yaml user.surname

What I haven’t got is the ability to read from multiple

yq r -d'*'  examples/multiple_docs.yaml user.surname

Not sure if that would actually be a useful thing though?

hello @mikefarah how to use this -d option in current 4.x version ? any example will be helpful, I tried searching through the documents…

– hmm… got it… thanks for the yq. yq '(select(documentIndex == 1) | path) = value' file.yaml

go-yaml recently merged a fix to their development branch. Really looking forward to this feature for scripting with kube manifests!

Opened.

so far, it’s like you guessed, I’m adding a ‘-d’ parameter to the commands which allows you to index the documents where 0 is the first and default.

yq w  my_file.yml a.b.c cat // updates the first doc, outputs all documents
yq w -d 1 my_file.yml a.b.c cat // updates the second doc, outputs all documents

Under the hood - this is a significant change, and one of the things that will have to go is JSON output for all commands except read.

Basically (my current plan) is to have ‘read’ as the only command that outputs a single document (and you can select which one), so a JSON toggles makes perfect sense.

yq r  my_file.yml a.b.c // reads a.b.c from first doc
yq r  -d 1 my_file.yml a.b.c // reads a.b.c from second doc
yq r  my_file.yml  // reads first doc
yq r  -d 1 my_file.yml  // reads second doc

The other commands, like ‘write’ will need to output all the documents in the yaml (as you are updating a field, but still need the rest of the docs), and so it doesn’t make as much sense to support json output. If you need JSON output still for those commands, then you’ll need to pipe:

yq w -d 1 my_file.yml a.b.c cat | yq r -d1 -j -

Thoughts?