kubernetes: Secrets does not creating

What happened?

I am writing to report an issue with the latest version of Kubernetes in Azure, version v1.24.6. Before this version, when creating a Kubernetes cluster, the service account and secrets were automatically created and everything worked smoothly. However, with version v1.24.6, I have encountered an error when attempting to connect the cluster to Azure DevOps. Also during execution command in version v1.23.8 “kubectl create serviceaccount some_name”, security token generating automatically. But not in v1.24.6. I found only 2 secrets in new cluster v1.24.6 comparing with v1.23.8 where was more than 30 secrets. I am checking all namespaces with command “kubectl get secrets --all-namespaces”

Upon investigation, I learned that a developer made the decision to delete the feature of automatically creating the service account and secrets, which has now affected all Azure DevOps organizations. I have tried to manually create the token secret command, but I have not seen any changes in the secrets.

apiVersion: v1
kind: Secret
metadata:
  name: oke-kubeconfig-sa-token
  namespace: dev-cluster
  annotations:
    kubernetes.io/service-account.name: devops-kubernetes-sa
type: kubernetes.io/service-account-token

Kubernetes just ignore this applying. I am unsure if this is a bug or if there has been a deliberate change in the functionality. I would greatly appreciate it if you could investigate this issue and provide further information or guidance.

What did you expect to happen?

I expected that after manually creating the secret, it would appear in the list of secrets when executing the “kubectl get secrets --all-namespaces” command. However, the secret is not appearing, and I am unable to connect my Kubernetes cluster to Azure DevOps as a result.

How can we reproduce it (as minimally and precisely as possible)?

To reproduce the issue, you can follow these steps:

Create a Kubernetes cluster with version v1.24.6. I am not sure if the issue only in azure kubernetes. But as I see, microsoft reported that this is a common problem. Attempt to create a token secret . Observe that the secret does not appear in the list of secrets when executing the “kubectl get secrets --all-namespaces” command.

Anything else we need to know?

Related discussion: https://techcommunity.microsoft.com/t5/azure-devops/could-not-find-any-secrets-associated-with-the-service-account/m-p/3588722

https://learn.microsoft.com/en-us/answers/questions/1051970/aks-with-kubernetes-service-connection-returns-cou

Kubernetes version

WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.3", GitCommit:"434bfd82814af038ad94d62ebe59b133fcb50506", GitTreeState:"clean", BuildDate:"2022-10-12T10:57:26Z", GoVersion:"go1.19.2", Compiler:"gc", Platform:"windows/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.6", GitCommit:"c86d003ea699ec4bcffee10ad563a26b63561c0e", GitTreeState:"clean", BuildDate:"2022-12-17T10:31:53Z", GoVersion:"go1.18.6", Compiler:"gc", Platform:"linux/amd64"}

Cloud provider

Azure

OS version

BuildNumber  Caption                          OSArchitecture  Version
19044        Microsoft Windows 10 Enterprise  64-bit          10.0.19044

Install tools

Container runtime (CRI) and version (if applicable)

Related plugins (CNI, CSI, …) and versions (if applicable)

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 15 (9 by maintainers)

Most upvoted comments

that looks like a bug in that controller that tries to index a zero-length body even when an error occurs: https://github.com/amazon-archives/k8s-cloudwatch-adapter/blob/master/pkg/aws/util.go#L23-L29

Thanks for finding that. have added automountServiceAccountToken: true for the deployments, which moved to the panic` error.

https://gist.github.com/gowthamakanthan/c7d32dd87f69756d8e2a5865fc45be83

 k_dev_new logs -f k8s-cloudwatch-adapter-7d5fbccdd-pddpg -n amazon-custom-metrics
I0411 15:40:19.555612       1 controller.go:35] Setting up external metric event handlers
I0411 15:40:19.560141       1 controller.go:57] initializing controller
panic: runtime error: slice bounds out of range [:-1]

goroutine 1 [running]:
github.com/awslabs/k8s-cloudwatch-adapter/pkg/aws.GetLocalRegion(0x0, 0x0)
	/go/src/github.com/awslabs/k8s-cloudwatch-adapter/pkg/aws/util.go:29 +0x27e
github.com/awslabs/k8s-cloudwatch-adapter/pkg/aws.NewCloudWatchManager(...)
	/go/src/github.com/awslabs/k8s-cloudwatch-adapter/pkg/aws/client.go:22
main.(*CloudWatchAdapter).makeCloudWatchManager(...)
	/go/src/github.com/awslabs/k8s-cloudwatch-adapter/cmd/adapter/adapter.go:28
main.main()
	/go/src/github.com/awslabs/k8s-cloudwatch-adapter/cmd/adapter/adapter.go:88 +0x256

No, that description is for the secrets field inside the service account, which is not populated by default by any core kubernetes components now, and doesn’t actually drive any behavior unless the service account opts into limiting mountable secrets by setting a kubernetes.io/enforce-mountable-secrets=true annotation

I opened https://github.com/kubernetes/kubernetes/pull/117160 to drop the misleading/unused fields from the human-readable get and describe output