istio: Istio 1.4 with cert-manager 0.11.0 and SDS enabled get 404 with http01 solver

Bug description

I am having problems deploying istio 1.4 with sds and cert-manger 0.11

Expected behavior

Get Let’s encrypt certificates with http01 solver

Steps to reproduce the bug

Deploy Istio 1.4

istioctl manifest apply \
  --set values.gateways.istio-ingressgateway.sds.enabled=true \
  --set values.global.k8sIngress.enabled=true \
  --set values.global.k8sIngress.enableHttps=true \
  --set values.global.k8sIngress.gatewayName=ingressgateway

Deploy cert-manager 0.11.0

kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.11.0/cert-manager.yaml

NAME                                       READY   STATUS    RESTARTS   AGE
cert-manager-cainjector-74bb68d67c-s8nmq   1/1     Running   0          111s
cert-manager-f7f8bf74d-789qg               1/1     Running   0          111s
cert-manager-webhook-645b8bdb7-7ck68       1/1     Running   0          74s

Create Issuer

cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1alpha2
kind: Issuer
metadata:
  name: letsencrypt-staging
  namespace: istio-system
spec:
  acme:
    email: test@asdad.com
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: lets-encrypt-app-cert
    solvers:
    - selector: {}
      http01:
        ingress: {}
---
EOF

Result:

NAME                  AGE
letsencrypt-staging   40s

Status:
  Acme:
    Last Registered Email:  test@asdad.com
    Uri:                    https://acme-staging-v02.api.letsencrypt.org/acme/acct/11672446
  Conditions:
    Last Transition Time:  2019-11-28T17:36:57Z
    Message:               The ACME account was registered with the ACME server
    Reason:                ACMEAccountRegistered
    Status:                True
    Type:                  Ready
Events:                    <none>

Create certificate

cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
  name: lets-encrypt-app-cert
  namespace: istio-system
spec:
  secretName: lets-encrypt-app-cert
  commonName: lets-encrypt-app.34.77.30.109.nip.io
  dnsNames:
  - lets-encrypt-app.34.77.30.109.nip.io
  issuerRef:
    name: letsencrypt-staging
    kind: Issuer
  acme:
    config:
    - http01:
        ingressClass: istio
      domains:
      - lets-encrypt-app.34.77.30.109.nip.io
---
EOF

Result: Cert:

Status:
  Conditions:
    Last Transition Time:  2019-11-28T17:39:18Z
    Message:               Waiting for CertificateRequest "lets-encrypt-app-cert-276943243" to complete
    Reason:                InProgress
    Status:                False
    Type:                  Ready
Events:
  Type    Reason     Age   From          Message
  ----    ------     ----  ----          -------
  Normal  Requested  27s   cert-manager  Created new CertificateRequest resource "lets-encrypt-app-cert-276943243"

Order:

Events:
  Type    Reason   Age   From          Message
  ----    ------   ----  ----          -------
  Normal  Created  42s   cert-manager  Created Challenge resource "lets-encrypt-app-cert-276943243-2104889026-4276716748" for domain "lets-encrypt-app.34.77.30.109.nip.io"

Pods:

istio-system   cm-acme-http-solver-zlh8t   NodePort       10.19.251.58    <none>         8089:32231/TCP                                                                                                               4m13s

Ingress:

Events:
  Type    Reason  Age    From                     Message
  ----    ------  ----   ----                     -------
  Normal  ADD     6m19s  loadbalancer-controller  istio-system/cm-acme-http-solver-lz544
  Normal  CREATE  5m4s   loadbalancer-controller  ip: 35.244.238.184

cert-manager logs:

I1128 17:45:10.698282       1 pod.go:58] cert-manager/controller/challenges/http01/selfCheck/http01/ensurePod "level"=0 "msg"="found one existing HTTP01 solver pod" "dnsName"="lets-encrypt-app.34.77.30.109.nip.io" "related_resource_kind"="Pod" "related_resource_name"="cm-acme-http-solver-8xqd7" "related_resource_namespace"="istio-system" "resource_kind"="Challenge" "resource_name"="lets-encrypt-app-cert-276943243-2104889026-4276716748" "resource_namespace"="istio-system" "type"="http-01" 
I1128 17:45:10.698389       1 service.go:43] cert-manager/controller/challenges/http01/selfCheck/http01/ensureService "level"=0 "msg"="found one existing HTTP01 solver Service for challenge resource" "dnsName"="lets-encrypt-app.34.77.30.109.nip.io" "related_resource_kind"="Service" "related_resource_name"="cm-acme-http-solver-zlh8t" "related_resource_namespace"="istio-system" "resource_kind"="Challenge" "resource_name"="lets-encrypt-app-cert-276943243-2104889026-4276716748" "resource_namespace"="istio-system" "type"="http-01" 
I1128 17:45:10.698466       1 ingress.go:91] cert-manager/controller/challenges/http01/selfCheck/http01/ensureIngress "level"=0 "msg"="found one existing HTTP01 solver ingress" "dnsName"="lets-encrypt-app.34.77.30.109.nip.io" "related_resource_kind"="Ingress" "related_resource_name"="cm-acme-http-solver-lz544" "related_resource_namespace"="istio-system" "resource_kind"="Challenge" "resource_name"="lets-encrypt-app-cert-276943243-2104889026-4276716748" "resource_namespace"="istio-system" "type"="http-01" 
E1128 17:45:10.704109       1 sync.go:184] cert-manager/controller/challenges "msg"="propagation check failed" "error"="wrong status code '404', expected '200'" "dnsName"="lets-encrypt-app.34.77.30.109.nip.io" "resource_kind"="Challenge" "resource_name"="lets-encrypt-app-cert-276943243-2104889026-4276716748" "resource_namespace"="istio-system" "type"="http-01" 
´´´

**Version (include the output of `istioctl version --remote` and `kubectl version` and `helm version` if you used Helm)**

client version: 1.4.0
control plane version: 1.4.0
data plane version: 1.4.0 (1 proxies)

Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.3", GitCommit:"b3cbbae08ec52a7fc73d334838e18d17e8512749", GitTreeState:"clean", BuildDate:"2019-11-13T11:23:11Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.8-gke.17", GitCommit:"188432a69210ca32cafded81b4dd1c063720cac0", GitTreeState:"clean", BuildDate:"2019-11-13T20:47:11Z", GoVersion:"go1.12.11b4", Compiler:"gc", Platform:"linux/amd64"}

**How was Istio installed?**
istioctl