prometheus-operator: Setting podMetadata labels without creation timestamp fails from golang

What did you do?

Attempted to use the podMetadata field to attach labels to a Prometheus cluster from golang.

What did you expect to see?

The cluster get created with those labels attached.

What did you see instead? Under which circumstances?

The cluster fails to create with an error of spec.podMetadata.creationTimestamp in body must be of type string: \"null\"".

Full log line, showing the payload:

time="2019-02-12T23:00:26Z" level=error msg="Failed to deploy Prometheus" error="Prometheus.monitoring.coreos.com \"example-uswest\" is invalid: []: Invalid value: map[string]interface {}{\"apiVersion\":\"monitoring.coreos.com/v1\", \"kind\":\"Prometheus\", \"metadata\":map[string]interface {}{\"creationTimestamp\":\"2019-02-12T23:00:26Z\", \"labels\":map[string]interface {}{\"app\":\"example\"}, \"ownerReferences\":[]interface {}{map[string]interface {}{\"name\":\"example\", \"uid\":\"748b7e3a-2f19-11e9-b986-803820c5878b\", \"controller\":true, \"blockOwnerDeletion\":true, \"apiVersion\":\"example.com/v1alpha1\", \"kind\":\"ExampleCluster\"}}, \"generation\":1, \"uid\":\"fc290f8c-2f19-11e9-b986-803820c5878b\", \"name\":\"example-uswest\", \"namespace\":\"default\"}, \"spec\":map[string]interface {}{\"serviceAccountName\":\"prometheus\", \"serviceMonitorSelector\":map[string]interface {}{\"matchLabels\":map[string]interface {}{\"app\":\"example\"}}, \"podMetadata\":map[string]interface {}{\"creationTimestamp\":interface {}(nil), \"labels\":map[string]interface {}{\"monitoredApp\":\"example\"}}, \"resources\":map[string]interface {}{\"requests\":map[string]interface {}{\"memory\":\"1Gi\"}}}}: validation failure list:\nspec.podMetadata.creationTimestamp in body must be of type string: \"null\""

Environment

  • Prometheus Operator version:

      `quay.io/coreos/prometheus-operator:v0.17.0`
    
  • Kubernetes version information:

Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.1", GitCommit:"eec55b9ba98609a46fee712359c7b5b365bdd920", GitTreeState:"clean", BuildDate:"2018-12-13T10:39:04Z", GoVersion:"go1.11.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.0", GitCommit:"ddf47ac13c1a9483ea035a79cd7c10005ff21a6d", GitTreeState:"clean", BuildDate:"2018-12-03T20:56:12Z", GoVersion:"go1.11.2", Compiler:"gc", Platform:"linux/amd64"}
  • Kubernetes cluster kind:

Local cluster created using minikube.

  • Manifests:

Code that fails:

return prom.Prometheus{
                TypeMeta: metav1.TypeMeta{
                        Kind:       "Prometheus",
                        APIVersion: "monitoring.coreos.com/v1",
                },
                ObjectMeta: metav1.ObjectMeta{
                        Name:      "prometheus",
                        Labels: map[string]string{
                                "app": "example",
                        },
                },
                Spec: prom.PrometheusSpec{
                        Resources: v1.ResourceRequirements{
                                Requests: v1.ResourceList{
                                        v1.ResourceMemory: *resource.NewQuantity(1024*1024*1024, resource
.BinarySI),
                                },
                        },
                        ServiceMonitorSelector: &metav1.LabelSelector{
                                MatchLabels: map[string]string{
                                        "app": "example",
                                },
                        },
                        // validation failure list:\nspec.podMetadata.creationTimestamp in body must be of type string: \"null\""
                        PodMetadata: &metav1.ObjectMeta{
                                Labels: map[string]string{
                                        // We can't use "app", because the Prometheus operator
                                        // overrides that label with "prometheus" even if we set it
                                        // here.
                                        "monitoredApp": "example",
                                },
                                //CreationTimestamp: metav1Time,
                        },
                        ServiceAccountName: "prometheus",
                },
}

When I pass in a time of this type: https://github.com/kubernetes/apimachinery/blob/master/pkg/apis/meta/v1/time.go, everything works (and I think it might eventually get overridden anyway). I would expect this to be set by kubernetes.

  • Prometheus Operator Logs:

Nothing much shows up there. I think the request fails before it reaches the operator:

ts=2019-02-12T22:56:26.909458796Z caller=operator.go:258 component=prometheusoperator msg="connection established" cluster-version=v1.13.0
ts=2019-02-12T22:56:26.91212914Z caller=operator.go:158 component=alertmanageroperator msg="connection established" cluster-version=v1.13.0
ts=2019-02-12T22:56:26.913926233Z caller=operator.go:172 component=alertmanageroperator msg="CRD API endpoints ready"
ts=2019-02-12T22:56:26.916079722Z caller=operator.go:272 component=prometheusoperator msg="CRD API endpoints ready"

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 4
  • Comments: 23 (19 by maintainers)

Commits related to this issue

Most upvoted comments

This is still a problem as far as I am aware.

cc @pgier ^ this is something we should look into 🤔

This is still a problem, using the podMetadata field completely doesn’t work, neither for labels or annotations.