azure-key-vault-to-kubernetes: ErrAzureVault: Failed to get secret from Azure Key Vault

I am running from a user through which test environment was created on aks. I followed the steps mentioned in the quick start guide and below is the error I got when I describe the secret CRD.

  Type     Reason         Age                     From                     Message
  ----     ------         ----                    ----                     -------
  Warning  ErrAzureVault  7m12s (x13 over 8m12s)  azurekeyvaultcontroller  Failed to get secret for 'secret-sync' from Azure Key Vault 'akv2k8s-test'
  Warning  ErrAzureVault  4m12s (x31 over 9m12s)  azurekeyvaultcontroller  Failed to get secret for 'secret-sync' from Azure Key Vault 'testingvaultd'

More detailed:


PS C:\Users\UReddy\akv2k8s> kubectl get azurekeyvaultsecret.spv.no/secret-sync -n akv-test
NAME          VAULT           VAULT OBJECT   SECRET NAME   SYNCHED
secret-sync   testingvaultd   my-secret
PS C:\Users\UReddy\akv2k8s> kubectl describe azurekeyvaultsecret.spv.no/secret-sync -n akv-test
Name:         secret-sync
Namespace:    akv-test
Labels:       <none>
Annotations:  <none>
API Version:  spv.no/v1
Kind:         AzureKeyVaultSecret
Metadata:
  Creation Timestamp:  2021-08-05T12:38:59Z
  Generation:          3
  Managed Fields:
    API Version:  spv.no/v2beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:kubectl.kubernetes.io/last-applied-configuration:
      f:spec:
        .:
        f:output:
          .:
          f:secret:
            .:
            f:dataKey:
            f:name:
        f:vault:
          .:
          f:name:
          f:object:
            .:
            f:name:
            f:type:
    Manager:         kubectl-client-side-apply
    Operation:       Update
    Time:            2021-08-05T12:38:59Z
  Resource Version:  12022
  UID:               007f2f4c-5d59-43ee-a95d-625af8b3aee8
Spec:
  Output:
    Secret:
      Data Key:  secret-value
      Name:      my-secret-from-akv
  Vault:
    Name:  testingvaultd
    Object:
      Name:  my-secret
      Type:  secret
Events:
  Type     Reason         Age                     From                     Message
  ----     ------         ----                    ----                     -------
  Warning  ErrAzureVault  7m12s (x13 over 8m12s)  azurekeyvaultcontroller  Failed to get secret for 'secret-sync' from Azure Key Vault 'akv2k8s-test'
  Warning  ErrAzureVault  4m12s (x31 over 9m12s)  azurekeyvaultcontroller  Failed to get secret for 'secret-sync' from Azure Key Vault 'testingvaultd'
PS C:\Users\UReddy\akv2k8s>

Please suggest how to fix that?

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 30 (1 by maintainers)

Most upvoted comments

Hi, if anyone is still struggling with this, I used a managed identity. However, when you’re giving access to the vault you have to select the nodepool , not the cluster itself.

Nowhere is this documented anywhere.

Bizzaro on Mar 31, 2022

Took a while to figure out but it’s straightforward (even if not at all obvious):

# fill these in
KEYVAULT=
AKS=
AKS_RG=

az keyvault set-policy \
  -n $KEYVAULT \
  --secret-permissions get \
  --object-id $(az aks show --name $AKS -g $AKS_RG --query identityProfile.kubeletidentity.objectId)

If you’re using a managed identity then it’s the objectId of the kubelet identity instead of the SPN, which I think corresponds to the AKS agent pool. I believe this is what --attach-acr does in the background when creating the cluster as well, adding the AcrPull role onto the kubeletidentity’s obectid (not sure though).

That was the only thing required for me, it worked immediately thereafter. Hope that helps.

jacobdanovitch on Aug 25, 2021

Please check the Controller logs to obtain more information why it fails (https://akv2k8s.io/troubleshooting/controller-log/). This warning is often occur when akv2k8s do not have Get permission on the keyvault. Ensure that you have added an access policy for the keyvault giving access to the service principal or managed identity that you use.

kristeey on Aug 5, 2021