kustomize: Creation timestamp set to invalid value "null" when using kustomize v5.0.0

What happened?

I was trying to generate manifests for a cluster-api-provider project (tinkerbell/cluster-api-provider) and installed kustomize using the install_kustomize script. But the generated manifests had some spurious values for creationTimestamp

➜ kustomize version
v5.0.0
➜ kustomize build config/default | grep "creationTimestamp"
  creationTimestamp: "null"
  creationTimestamp: "null"
  creationTimestamp: "null"
  creationTimestamp: null
  creationTimestamp: null
  creationTimestamp: null

Note the quotes around null. This is causing issues when trying to parse time in a client-go applicatio.

Error: unable to convert unstructured object to apiextensions.k8s.io/v1, Kind=CustomResourceDefinition: parsing time "null" as "2006-01-02T15:04:05Z07:00": cannot parse "null" as "2006"

However I don’t see this issue in an older version of kustomize.

➜ kustomize version
{Version:kustomize/v4.5.5 GitCommit:daa3e5e2c2d3a4b8c94021a7384bfb06734bcd26 BuildDate:2022-05-20T20:25:40Z GoOs:darwin GoArch:amd64}
➜ kustomize build config/default | grep "creationTimestamp"
  creationTimestamp: null
  creationTimestamp: null
  creationTimestamp: null

What did you expect to happen?

I expect that the generated manifest should conform to types of metav1.ObjectMeta where creationTimestamp is a Time object and not string, i.e., null instead of "null".

How can we reproduce it (as minimally and precisely as possible)?

kustomization.yaml under config/crd in https://github.com/tinkerbell/cluster-api-provider-tinkerbell. Commenting out some of the CRDs to reproduce minimally

# This kustomization.yaml is not intended to be run by itself,
# since it depends on service name and namespace that are out of this kustomize package.
# It should be run by config/default
commonLabels:
  cluster.x-k8s.io/v1beta1: v1beta1
resources:
- bases/infrastructure.cluster.x-k8s.io_tinkerbellclusters.yaml
#- bases/infrastructure.cluster.x-k8s.io_tinkerbellmachines.yaml  # THIS LINE HAS BEEN COMMENTED OUT FOR SAKE OF MINIMALITY
#- bases/infrastructure.cluster.x-k8s.io_tinkerbellmachinetemplates.yaml  # THIS LINE HAS BEEN COMMENTED OUT FOR SAKE OF MINIMALITY
# +kubebuilder:scaffold:crdkustomizeresource

patchesStrategicMerge:
# [WEBHOOK] To enable webhook, uncomment all the sections with [WEBHOOK] prefix.
# patches here are for enabling the conversion webhook for each CRD
- patches/webhook_in_tinkerbellclusters.yaml
#- patches/webhook_in_tinkerbellmachines.yaml
#- patches/webhook_in_tinkerbellmachinetemplates.yaml
# +kubebuilder:scaffold:crdkustomizewebhookpatch

# [CERTMANAGER] To enable webhook, uncomment all the sections with [CERTMANAGER] prefix.
# patches here are for enabling the CA injection for each CRD
- patches/cainjection_in_tinkerbellclusters.yaml
#- patches/cainjection_in_tinkerbellmachines.yaml
#- patches/cainjection_in_tinkerbellmachinetemplates.yaml
# +kubebuilder:scaffold:crdkustomizecainjectionpatch

# the following config is for teaching kustomize how to do kustomization for CRDs.
configurations:
- kustomizeconfig.yaml

bases/infrastructure.cluster.x-k8s.io_tinkerbellclusters.yaml under config/crd in https://github.com/tinkerbell/cluster-api-provider-tinkerbell

---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.10.0
  creationTimestamp: null
  name: tinkerbellclusters.infrastructure.cluster.x-k8s.io
spec:
  group: infrastructure.cluster.x-k8s.io
  names:
    categories:
    - cluster-api
    kind: TinkerbellCluster
    listKind: TinkerbellClusterList
    plural: tinkerbellclusters
    singular: tinkerbellcluster
  scope: Namespaced
  versions:
  - additionalPrinterColumns:
    - description: Cluster to which this TinkerbellCluster belongs
      jsonPath: .metadata.labels.cluster\.x-k8s\.io/cluster-name
      name: Cluster
      type: string
    - description: TinkerbellCluster ready status
      jsonPath: .status.ready
      name: Ready
      type: string
    name: v1beta1
    schema:
      openAPIV3Schema:
        description: TinkerbellCluster is the Schema for the tinkerbellclusters API.
        properties:
          apiVersion:
            description: 'APIVersion defines the versioned schema of this representation
              of an object. Servers should convert recognized schemas to the latest
              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
            type: string
          kind:
            description: 'Kind is a string value representing the REST resource this
              object represents. Servers may infer this from the endpoint the client
              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
            type: string
          metadata:
            type: object
          spec:
            description: TinkerbellClusterSpec defines the desired state of TinkerbellCluster.
            properties:
              controlPlaneEndpoint:
                description: "ControlPlaneEndpoint is a required field by ClusterAPI
                  v1beta1. \n See https://cluster-api.sigs.k8s.io/developer/architecture/controllers/cluster.html
                  for more details."
                properties:
                  host:
                    description: The hostname on which the API server is serving.
                    type: string
                  port:
                    description: The port on which the API server is serving.
                    format: int32
                    type: integer
                required:
                - host
                - port
                type: object
              imageLookupBaseRegistry:
                default: ghcr.io/tinkerbell/cluster-api-provider-tinkerbell
                description: ImageLookupBaseRegistry is the base Registry URL that
                  is used for pulling images, if not set, the default will be to use
                  ghcr.io/tinkerbell/cluster-api-provider-tinkerbell.
                type: string
              imageLookupFormat:
                description: 'ImageLookupFormat is the URL naming format to use for
                  machine images when a machine does not specify. When set, this will
                  be used for all cluster machines unless a machine specifies a different
                  ImageLookupFormat. Supports substitutions for {{.BaseRegistry}},
                  {{.OSDistro}}, {{.OSVersion}} and {{.KubernetesVersion}} with the
                  basse URL, OS distribution, OS version, and kubernetes version,
                  respectively. BaseRegistry will be the value in ImageLookupBaseRegistry
                  or ghcr.io/tinkerbell/cluster-api-provider-tinkerbell (the default),
                  OSDistro will be the value in ImageLookupOSDistro or ubuntu (the
                  default), OSVersion will be the value in ImageLookupOSVersion or
                  default based on the OSDistro (if known), and the kubernetes version
                  as defined by the packages produced by kubernetes/release: v1.13.0,
                  v1.12.5-mybuild.1, or v1.17.3. For example, the default image format
                  of {{.BaseRegistry}}/{{.OSDistro}}-{{.OSVersion}}:{{.KubernetesVersion}}.gz
                  will attempt to pull the image from that location. See also: https://golang.org/pkg/text/template/'
                type: string
              imageLookupOSDistro:
                default: ubuntu
                description: ImageLookupOSDistro is the name of the OS distro to use
                  when fetching machine images, if not set it will default to ubuntu.
                type: string
              imageLookupOSVersion:
                description: ImageLookupOSVersion is the version of the OS distribution
                  to use when fetching machine images. If not set it will default
                  based on ImageLookupOSDistro.
                type: string
            type: object
          status:
            description: TinkerbellClusterStatus defines the observed state of TinkerbellCluster.
            properties:
              ready:
                description: Ready denotes that the cluster (infrastructure) is ready.
                type: boolean
            type: object
        type: object
    served: true
    storage: true
    subresources:
      status: {}

Expected output

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  annotations:
    cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME)
    controller-gen.kubebuilder.io/version: v0.10.0
  labels:
    cluster.x-k8s.io/v1beta1: v1beta1
  name: tinkerbellclusters.infrastructure.cluster.x-k8s.io
spec:
  conversion:
    strategy: Webhook
    webhook:
      clientConfig:
        caBundle: Cg==
        service:
          name: webhook-service
          namespace: system
          path: /convert
      conversionReviewVersions:
      - v1
      - v1beta1
  group: infrastructure.cluster.x-k8s.io
  names:
    categories:
    - cluster-api
    kind: TinkerbellCluster
    listKind: TinkerbellClusterList
    plural: tinkerbellclusters
    singular: tinkerbellcluster
  scope: Namespaced
  versions:
  - additionalPrinterColumns:
    - description: Cluster to which this TinkerbellCluster belongs
      jsonPath: .metadata.labels.cluster\.x-k8s\.io/cluster-name
      name: Cluster
      type: string
    - description: TinkerbellCluster ready status
      jsonPath: .status.ready
      name: Ready
      type: string
    name: v1beta1
    schema:
      openAPIV3Schema:
        description: TinkerbellCluster is the Schema for the tinkerbellclusters API.
        properties:
          apiVersion:
            description: 'APIVersion defines the versioned schema of this representation
              of an object. Servers should convert recognized schemas to the latest
              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
            type: string
          kind:
            description: 'Kind is a string value representing the REST resource this
              object represents. Servers may infer this from the endpoint the client
              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
            type: string
          metadata:
            type: object
          spec:
            description: TinkerbellClusterSpec defines the desired state of TinkerbellCluster.
            properties:
              controlPlaneEndpoint:
                description: "ControlPlaneEndpoint is a required field by ClusterAPI
                  v1beta1. \n See https://cluster-api.sigs.k8s.io/developer/architecture/controllers/cluster.html
                  for more details."
                properties:
                  host:
                    description: The hostname on which the API server is serving.
                    type: string
                  port:
                    description: The port on which the API server is serving.
                    format: int32
                    type: integer
                required:
                - host
                - port
                type: object
              imageLookupBaseRegistry:
                default: ghcr.io/tinkerbell/cluster-api-provider-tinkerbell
                description: ImageLookupBaseRegistry is the base Registry URL that
                  is used for pulling images, if not set, the default will be to use
                  ghcr.io/tinkerbell/cluster-api-provider-tinkerbell.
                type: string
              imageLookupFormat:
                description: 'ImageLookupFormat is the URL naming format to use for
                  machine images when a machine does not specify. When set, this will
                  be used for all cluster machines unless a machine specifies a different
                  ImageLookupFormat. Supports substitutions for {{.BaseRegistry}},
                  {{.OSDistro}}, {{.OSVersion}} and {{.KubernetesVersion}} with the
                  basse URL, OS distribution, OS version, and kubernetes version,
                  respectively. BaseRegistry will be the value in ImageLookupBaseRegistry
                  or ghcr.io/tinkerbell/cluster-api-provider-tinkerbell (the default),
                  OSDistro will be the value in ImageLookupOSDistro or ubuntu (the
                  default), OSVersion will be the value in ImageLookupOSVersion or
                  default based on the OSDistro (if known), and the kubernetes version
                  as defined by the packages produced by kubernetes/release: v1.13.0,
                  v1.12.5-mybuild.1, or v1.17.3. For example, the default image format
                  of {{.BaseRegistry}}/{{.OSDistro}}-{{.OSVersion}}:{{.KubernetesVersion}}.gz
                  will attempt to pull the image from that location. See also: https://golang.org/pkg/text/template/'
                type: string
              imageLookupOSDistro:
                default: ubuntu
                description: ImageLookupOSDistro is the name of the OS distro to use
                  when fetching machine images, if not set it will default to ubuntu.
                type: string
              imageLookupOSVersion:
                description: ImageLookupOSVersion is the version of the OS distribution
                  to use when fetching machine images. If not set it will default
                  based on ImageLookupOSDistro.
                type: string
            type: object
          status:
            description: TinkerbellClusterStatus defines the observed state of TinkerbellCluster.
            properties:
              ready:
                description: Ready denotes that the cluster (infrastructure) is ready.
                type: boolean
            type: object
        type: object
    served: true
    storage: true
    subresources:
      status: {}

Actual output

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  annotations:
    cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME)
    controller-gen.kubebuilder.io/version: v0.10.0
  creationTimestamp: "null"
  labels:
    cluster.x-k8s.io/v1beta1: v1beta1
  name: tinkerbellclusters.infrastructure.cluster.x-k8s.io
spec:
  conversion:
    strategy: Webhook
    webhook:
      clientConfig:
        caBundle: Cg==
        service:
          name: webhook-service
          namespace: system
          path: /convert
      conversionReviewVersions:
      - v1
      - v1beta1
  group: infrastructure.cluster.x-k8s.io
  names:
    categories:
    - cluster-api
    kind: TinkerbellCluster
    listKind: TinkerbellClusterList
    plural: tinkerbellclusters
    singular: tinkerbellcluster
  scope: Namespaced
  versions:
  - additionalPrinterColumns:
    - description: Cluster to which this TinkerbellCluster belongs
      jsonPath: .metadata.labels.cluster\.x-k8s\.io/cluster-name
      name: Cluster
      type: string
    - description: TinkerbellCluster ready status
      jsonPath: .status.ready
      name: Ready
      type: string
    name: v1beta1
    schema:
      openAPIV3Schema:
        description: TinkerbellCluster is the Schema for the tinkerbellclusters API.
        properties:
          apiVersion:
            description: 'APIVersion defines the versioned schema of this representation
              of an object. Servers should convert recognized schemas to the latest
              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
            type: string
          kind:
            description: 'Kind is a string value representing the REST resource this
              object represents. Servers may infer this from the endpoint the client
              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
            type: string
          metadata:
            type: object
          spec:
            description: TinkerbellClusterSpec defines the desired state of TinkerbellCluster.
            properties:
              controlPlaneEndpoint:
                description: "ControlPlaneEndpoint is a required field by ClusterAPI
                  v1beta1. \n See https://cluster-api.sigs.k8s.io/developer/architecture/controllers/cluster.html
                  for more details."
                properties:
                  host:
                    description: The hostname on which the API server is serving.
                    type: string
                  port:
                    description: The port on which the API server is serving.
                    format: int32
                    type: integer
                required:
                - host
                - port
                type: object
              imageLookupBaseRegistry:
                default: ghcr.io/tinkerbell/cluster-api-provider-tinkerbell
                description: ImageLookupBaseRegistry is the base Registry URL that
                  is used for pulling images, if not set, the default will be to use
                  ghcr.io/tinkerbell/cluster-api-provider-tinkerbell.
                type: string
              imageLookupFormat:
                description: 'ImageLookupFormat is the URL naming format to use for
                  machine images when a machine does not specify. When set, this will
                  be used for all cluster machines unless a machine specifies a different
                  ImageLookupFormat. Supports substitutions for {{.BaseRegistry}},
                  {{.OSDistro}}, {{.OSVersion}} and {{.KubernetesVersion}} with the
                  basse URL, OS distribution, OS version, and kubernetes version,
                  respectively. BaseRegistry will be the value in ImageLookupBaseRegistry
                  or ghcr.io/tinkerbell/cluster-api-provider-tinkerbell (the default),
                  OSDistro will be the value in ImageLookupOSDistro or ubuntu (the
                  default), OSVersion will be the value in ImageLookupOSVersion or
                  default based on the OSDistro (if known), and the kubernetes version
                  as defined by the packages produced by kubernetes/release: v1.13.0,
                  v1.12.5-mybuild.1, or v1.17.3. For example, the default image format
                  of {{.BaseRegistry}}/{{.OSDistro}}-{{.OSVersion}}:{{.KubernetesVersion}}.gz
                  will attempt to pull the image from that location. See also: https://golang.org/pkg/text/template/'
                type: string
              imageLookupOSDistro:
                default: ubuntu
                description: ImageLookupOSDistro is the name of the OS distro to use
                  when fetching machine images, if not set it will default to ubuntu.
                type: string
              imageLookupOSVersion:
                description: ImageLookupOSVersion is the version of the OS distribution
                  to use when fetching machine images. If not set it will default
                  based on ImageLookupOSDistro.
                type: string
            type: object
          status:
            description: TinkerbellClusterStatus defines the observed state of TinkerbellCluster.
            properties:
              ready:
                description: Ready denotes that the cluster (infrastructure) is ready.
                type: boolean
            type: object
        type: object
    served: true
    storage: true
    subresources:
      status: {}

Comparing this with the expected output, the only difference is the addition of the creationTimestamp field in metadata, with the value of "null".

Kustomize version

v5.0.0

Operating system

MacOS

About this issue

  • Original URL
  • State: open
  • Created a year ago
  • Reactions: 3
  • Comments: 18 (9 by maintainers)

Commits related to this issue

Most upvoted comments

For those that are coming from kubebuilder/operator-sdk/controller-gen, I have a PR up to remove metadata.creationTimestamp when controller-gen generates CRDs, RBAC, and webhooks: https://github.com/kubernetes-sigs/controller-tools/pull/800

+4
joelanford on Apr 21, 2023

Here’s a minimum repro I just threw together, with a fun little twist included.

dir structure is just base/ and overlays/

base/base.yaml:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: fakests
spec:
  template:
    spec:
      containers:
      - something: null
        name: someName
test-spec:
  template:
    spec:
      containers:
      - something: null
        name: someName

base/kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- base.yaml

patches:
- path: nothing-patch.yaml
  target:
    kind: StatefulSet
    name: fakests

base/nothing-patch.yaml:

apiVersion: apps/v1
kind: NotImportant
metadata:
  name: not-used

overlays/kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- ../base

patches:
- path: ../base/nothing-patch.yaml
  target:
    kind: StatefulSet

If you comment out the nothing-patch.yaml from base/kustomization.yaml, and build the overlay, no bug manifests.

If you then uncomment the nothing-patch.yaml in base/kustomization.yaml, and build the overlay, the bug manifests only for one of the two null values.

The rendered yaml if you build the base with AND without the patch in place is exactly the same.

This seems to suggest that there is some inherent, internal-to-kustomize side-effect caused by running a yaml document through a patch. The fact it seems to depend on the name of the top-level field is also utterly bizarre.

+2
jbury on Jul 8, 2023

@KnVerey I don’t know anything about how ya’ll manage issues, nor what the expected process is to escalate something like this, but I think this needs a higher priority than “backlog”. Kustomize patches, as of 5.0, have side effects that can potentially result in unpredictable breakages of previously working yamls.

I also have a sneaking suspicion that the label “triage/resolved” might be causing this to be overlooked?

To be clear, I don’t exactly expect someone to drop everything and resolve this issue in an open source tool I use, but I do think this might have fallen through the cracks unintentionally, and I’m not familiar enough with your issue triaging processes to know if that’s actually happened, or how to get us back on track here. Any guidance would be appreciated!

+1
jbury on Aug 17, 2023
Read more comments on GitHub
← kustomize: Configuration-Value nameSuffix is ignored
kustomize: edit set image does not quote tags that will become non-string values in YAML →