cert-manager: Certificate creation failure (AKS) - certificate signed by unknown authority

Describe the bug: Certificates are not created. The following errors are reported:

the certificate: Error from server: conversion webhook for &{map[apiVersion:cert-manager.io/v1alpha2 kind:Certificate metadata:map[creationTimestamp:2020-05-13T17:30:48Z generation:1 name:xxx-tls namespace:test ownerReferences:[map[apiVersion:extensions/v1beta1 blockOwnerDeletion:true controller:true kind:Ingress name:xxx-ingress uid:6d73b182-bbce-4834-aee2-414d2b3aa802]] uid:d40bc037-aef7-4139-868f-bd615a423b38] spec:map[dnsNames:[xxx.test.domain.com] issuerRef:map[group:cert-manager.io kind:ClusterIssuer name:letsencrypt-prod] secretName:xxx-tls] status:map[conditions:[map[lastTransitionTime:2020-05-13T18:55:31Z message:Waiting for CertificateRequest "xxx-tls-1403681706" to complete reason:InProgress status:False type:Ready]]]]} failed: Post https://cert-manager-webhook.cert-manager.svc:443/convert?timeout=30s: x509: certificate signed by unknown authority

cert-manager-webhook container: cert-manager 2020/05/15 14:22:58 http: TLS handshake error from 10.20.0.19:35350: remote error: tls: bad certificate

Where 10.20.0.19 is the IP of tunnelfront pod.

Expected behaviour: Certificates are created without issues.

Steps to reproduce the bug:

• deploy cert-manager using static manifests
• deploy certificate request

Anything else we need to know?: Debugging with https://cert-manager.io/docs/faq/acme/ sort of “fails” when trying to kubectl describe order... as kubectl describe certificaterequest... returns CSR contents with error (as above), but not the order ID.

The questions would be:

• how to force cert-manager to debug (display more info) regarding the certificate it does not trust?
• maybe the problem is occuring regularly and there is a known solution?
• what steps should be undertaken to debug the issue further?

Environment details::

• Kubernetes version: 1.15.10
• Cloud-provider/provisioner : Azure (AKS)
• cert-manager version: 0.14.3
• Install method: static manifests (see below) + cluster issuer (see below) + regular (not legacy)

cluster issuer:

kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
  namespace: cert-manager
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: x
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
      - dns01:
          azuredns:
            clientID: x
            clientSecretSecretRef:
              name: cert-manager-stage
              key: CLIENT_SECRET
            subscriptionID: x
            tenantID: x
            resourceGroupName: dns-stage
            hostedZoneName: x

the manifest:

  imagePullSecrets: []
  isOpenshift: false

  priorityClassName: ""
  rbac:
    create: true

  podSecurityPolicy:
    enabled: false

  logLevel: 2

  leaderElection:
    namespace: "kube-system"

replicaCount: 1

strategy: {}


image:
  repository: quay.io/jetstack/cert-manager-controller
  pullPolicy: IfNotPresent

  tag: v0.14.3

clusterResourceNamespace: ""

serviceAccount:
  create: true
  name:
  annotations: {}

extraArgs: []

extraEnv: []

resources: {}

securityContext:
  enabled: false
  fsGroup: 1001
  runAsUser: 1001

podAnnotations: {}

podLabels: {}

nodeSelector: {}

ingressShim:
  defaultIssuerName: letsencrypt-prod
  defaultIssuerKind: ClusterIssuer

prometheus:
  enabled: true
  servicemonitor:
    enabled: false
    prometheusInstance: default
    targetPort: 9402
    path: /metrics
    interval: 60s
    scrapeTimeout: 30s
    labels: {}


affinity: {}

tolerations: []

webhook:
  enabled: true
  replicaCount: 1

  strategy: {}

  podAnnotations: {}

  extraArgs: []

  resources: {}

  nodeSelector: {}

  affinity: {}

  tolerations: []

  image:
    repository: quay.io/jetstack/cert-manager-webhook
    pullPolicy: IfNotPresent
    tag: v0.14.3

  injectAPIServerCA: true

  securePort: 10250

cainjector:
  replicaCount: 1

  strategy: {}

  podAnnotations: {}

  extraArgs: []

  resources: {}

  nodeSelector: {}

  affinity: {}

  tolerations: []

  image:
    repository: quay.io/jetstack/cert-manager-cainjector
    pullPolicy: IfNotPresent
    tag: v0.14.3