k8s-config-connector: ComputeBackendService doesn't acquire IAP configurations and defaults to OFF

Describe the bug (This is potentially also a feature request) We have previously created a ComputeBackendService using Deployment Manager (DM), and then manually turned on Identity-Aware Proxy (IAP), which autogenerates the OAuth2.0 Client ID/secret and attaches it to the ComputeBackendService. We’re migrating our GLB resources from DM to KCC, so when the new definition was applied and shows UpToDate, we expected spec.iap to default to the existing GCP configuration. However, the actual behaviour was that because we left spec.iap empty, it defaulted to not populating spec.iap and turned IAP off for the ComputeBackendService leading to downtime. And with the k8s reconciliation loop, manually turning it on gets corrected automatically again.

As a workaround, we went into GCP console page for API Services & Credentials / OAuth 2.0 Client IDs. Copied the ClientID/secret which was generated when we manually flipped the IAP slider to ON. And then added them to spec.iap, and re-applied which works. Using our Infra-as-Code workflow, this now requires: PR1 to create the ComputeBackendService, manually flip IAP on to create OAuth2.0 clientId/secret, PR2 to add spec.iap.

Alternatively, we could have workflow to: pre-create the OAuth2.0 ClientID (assuming there’s no special magic behind the scenes that binds IAP created ClientIDs to these IAP-<service-name> IDs), and PR1 create the full ComputeBackendService with spec.iap. This would be a CRD feature request if we cannot simply have spec.iap: true that does the ClientID generation and binding for us.

In order of awesomeness, the fix would be:

  1. spec.iap: true magic that creates Client and binding like the UI experience
  2. CRD for OAuth2.0 Client to pre-create and reference in spec.iap.oauth2ClientRef
  3. Acquire from GCP and not default to OFF

ConfigConnector Version 1.26.0

YAML snippets:

apiVersion: compute.cnrm.cloud.google.com/v1beta1
kind: ComputeBackendService
metadata:
  name: glb-gke-service--site-reagent
spec:
  iap:  # Had to be added for it to work!
    oauth2ClientId: <REDACTED>.apps.googleusercontent.com
    oauth2ClientSecret:
      valueFrom:
        secretKeyRef:
          key: glb-gke-service--site-reagent
          name: iap-client-secrets
  backend:
  - balancingMode: RATE
    maxRatePerEndpoint: 5
    group:
      networkEndpointGroupRef:
        external: projects/b6i-stg/zones/us-east4-a/networkEndpointGroups/<REDACTED>
  - balancingMode: RATE
    maxRatePerEndpoint: 5
    group:
      networkEndpointGroupRef:
        external: projects/b6i-stg/zones/us-east4-b/networkEndpointGroups/<REDACTED>
  - balancingMode: RATE
    maxRatePerEndpoint: 5
    group:
      networkEndpointGroupRef:
        external: projects/b6i-stg/zones/us-east4-c/networkEndpointGroups/<REDACTED>

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 3
  • Comments: 16 (7 by maintainers)

Most upvoted comments

Hey @tonybenchsci, we’re sorry to hear about your downtime. Please give us some time to look into this and let you know what we find.

Hi @tonybenchsci , thanks for the followup. Yes, you are right. I should have clarified that - fields spec.iap.oauth2ClientId and spec.iap.oauth2ClientSecretSha256 can now be defaulted to GCP values, and don’t have to be user specified anymore.

@xiaobaitusi @jcanseco Thank you for the support so far, and our team is happy to see that https://github.com/GoogleCloudPlatform/k8s-config-connector/issues/325 has been fulfilled as part of https://github.com/GoogleCloudPlatform/k8s-config-connector/releases/tag/v1.43.0

Now what would be Amazing (and close this issue) is:

  • Update ComputeBackendService to have a spec.iap.IAPIdentityAwareProxyClientRef which grabs the ID and secret
  • Provide/explain a way to bind members to role IAP-secured Web App User against ComputeBackendServices (other than using the GCP UI Console)
  • Confirmation that pre-creating OAuth 2.0 Client IDs of format IAP-{ComputeBackendService_NAME} does not conflict with (is equivalent to) flipping the IAP button to “ON” on in the GCP UI.