kubeflow: TFJobs UI doesn't work behind IAP; React APP needs support IAP?

TFJobs UI is deployed on dev.kubeflow.org.

The UI shows up behind IAP but its doesn’t work

• No TFJobs are listed
• Creating a job via the UI doesn’t work.

Looking at the developer console we see requests to

https://accounts.google.com/o/oauth2/v2/auth?client_id=235037502967-9cpmvs4ljbiqb3ojtnhnhlkkd8d562rl.apps.googleusercontent.com&response_type=code&scope=openid+email&redirect_uri=https://dev.kubeflow.org/_gcp_gatekeeper/authenticate&state=Ci1odHRwczovL2Rldi5rdWJlZmxvdy5vcmcvdGZqb2JzL2FwaS9uYW1lc3BhY2USMEFOa2F0U2dYVjdSYnlHMzVXeGFwR1gxNURTS29TNjNiYnc6MTUyMjc4MDQxODEwOA

Which suggests to me the request is hitting the loadbalancer and being directed to do auth verification to sign in and its getting rejected.

So I think one of two things is happening

1. The request is coming from the server running in K8s and incorrectly being redirected to the external loadbalncer and thus hitting IAP when it shouldn’t be
2. The request is coming from the client and the client needs to be updated to support IAP.

@wbuchwalter Do you know where the request is coming from?

You should be able to access it at https://dev.kubeflow.org/tfjobs/ui/