kubernetes: [Failing Test] [sig-cluster-lifecycle] Upgrade and Downgrade jobs are failing

Which jobs are failing: https://k8s-testgrid.appspot.com/sig-release-master-upgrade#gce-master-new-downgrade-cluster-parallel

Which test(s) are failing: 3 tests related to cluster downgrade are failing - https://gubernator.k8s.io/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gce-master-new-downgrade-cluster-parallel/4889

Error:

Expected error:
    <*url.Error | 0xc420fc0d50>: {
        Op: "Get",
        URL: "https://35.224.152.82/apis/apps/v1/namespaces/e2e-tests-sig-apps-replicaset-upgrade-cfstf/replicasets/rs",
        Err: {
            Op: "dial",
            Net: "tcp",
            Source: nil,
            Addr: {
                IP: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 255, 255, 35, 224, 152, 82],
                Port: 443,
                Zone: "",
            },
            Err: {Syscall: "connect", Err: 0x6f},
        },
    }
    Get https://35.224.152.82/apis/apps/v1/namespaces/e2e-tests-sig-apps-replicaset-upgrade-cfstf/replicasets/rs: dial tcp 35.224.152.82:443: connect: connection refused
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/upgrades/apps/replicasets.go:95

Since when has it been failing: Have been seeing this as a flake on and off on this jobs. But it started consistently failing since 1pm PST today, 11/12

Reason for failure: Delta changes point to https://github.com/kubernetes/kubernetes/compare/7712766da...08784ad9a?

@mikedanese any chance https://github.com/kubernetes/kubernetes/pull/70842 could have broken this? This is a blocker 1.13 Beta.1 targeted for this Friday., 11/16

@justinsb @fedebongio @awly @jberkus @mortent @mariantalla

/kind failing-test /assign @mikedanese /priority critical-urgent /milestone v1.13