origin: Build failed push image to registry with 500 internal server err

[provide a description of the issue] Build failed push image to registry with 500 internal server err

Version

[provide output of the openshift version or oc version command] openshift v3.6.0+c4dd4cf kubernetes v1.6.1+5115d708d7 etcd 3.2.1 This is an origin rpm pkg installed cluster with:

origin-clients-3.6.0-1.0.c4dd4cf.x86_64
origin-master-3.6.0-1.0.c4dd4cf.x86_64
centos-release-openshift-origin36-1-1.el7.centos.noarch
origin-node-3.6.0-1.0.c4dd4cf.x86_64
origin-docker-excluder-3.6.0-1.0.c4dd4cf.noarch
tuned-profiles-origin-node-3.6.0-1.0.c4dd4cf.x86_64
origin-sdn-ovs-3.6.0-1.0.c4dd4cf.x86_64
origin-excluder-3.6.0-1.0.c4dd4cf.noarch
origin-3.6.0-1.0.c4dd4cf.x86_64

Steps To Reproduce

1. [step 1] oc new-app https://github.com/openshift/ruby-ex

Current Result

Failed push

Expected Result

Succeed

Additional Information

logs for build:

Pushing image docker-registry.default.svc:5000/haowang/ruby-ex:latest ...
Pushed 0/9 layers, 1% complete
Pushed 1/9 layers, 44% complete
Pushed 2/9 layers, 25% complete
Pushed 3/9 layers, 35% complete
Pushed 4/9 layers, 46% complete
Pushed 8/9 layers, 99% complete
Pushed 9/9 layers, 100% complete
Registry server Address: 
Registry server User Name: serviceaccount
Registry server Email: serviceaccount@example.org
Registry server Password: <<non-empty>>
error: build error: Failed to push image: received unexpected HTTP status: 500 Internal Server Error

logs for registry:

time="2017-08-30T08:48:49.811499595Z" level=debug msg="filesystem.Stat(\"/docker/registry/v2/blobs/sha256/e0/e0a2cddebe2358a47455a3c492bb92583ee1f3ca31d3aa90781d65a06472cad6/data\")" go.version=go1.7.6 http.request.host="docker-registry.default.svc:5000" http.request.id=18ff1ae8-4aee-4d54-9da0-a5d2996b5514 http.request.method=HEAD http.request.remoteaddr="10.2.6.1:56244" http.request.uri="/v2/haowang1/ruby-ex/blobs/sha256:e0a2cddebe2358a47455a3c492bb92583ee1f3ca31d3aa90781d65a06472cad6" http.request.useragent="docker/1.12.6 go/go1.7.4 kernel/3.10.0-514.2.2.el7.x86_64 os/linux arch/amd64 UpstreamClient(go-dockerclient)" instance.id=a7fc1648-ab73-4749-8d8a-8ddf2a81c2d7 openshift.auth.user="system:serviceaccount:haowang1:builder" openshift.logger=registry trace.duration=18.129µs trace.file="/openshifttmp/openshift/build-rpm-release/tito/rpmbuild-originlTBPIz/BUILD/origin-3.6.0/_output/local/go/src/github.com/openshift/origin/vendor/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/openshift/origin/vendor/github.com/docker/distribution/registry/storage/driver/base.(*Base).Stat" trace.id=6cefdbb9-3fac-4f23-95d4-f06151720343 trace.line=137 vars.digest="sha256:e0a2cddebe2358a47455a3c492bb92583ee1f3ca31d3aa90781d65a06472cad6" vars.name="haowang1/ruby-ex" 
time="2017-08-30T08:48:49.811542362Z" level=debug msg="(*remoteBlobGetterService).Stat: starting with dgst=sha256:e0a2cddebe2358a47455a3c492bb92583ee1f3ca31d3aa90781d65a06472cad6" go.version=go1.7.6 http.request.host="docker-registry.default.svc:5000" http.request.id=18ff1ae8-4aee-4d54-9da0-a5d2996b5514 http.request.method=HEAD http.request.remoteaddr="10.2.6.1:56244" http.request.uri="/v2/haowang1/ruby-ex/blobs/sha256:e0a2cddebe2358a47455a3c492bb92583ee1f3ca31d3aa90781d65a06472cad6" http.request.useragent="docker/1.12.6 go/go1.7.4 kernel/3.10.0-514.2.2.el7.x86_64 os/linux arch/amd64 UpstreamClient(go-dockerclient)" instance.id=a7fc1648-ab73-4749-8d8a-8ddf2a81c2d7 openshift.auth.user="system:serviceaccount:haowang1:builder" openshift.logger=registry vars.digest="sha256:e0a2cddebe2358a47455a3c492bb92583ee1f3ca31d3aa90781d65a06472cad6" vars.name="haowang1/ruby-ex" 
10.2.6.1 - - [30/Aug/2017:08:48:49 +0000] "HEAD /v2/haowang1/ruby-ex/blobs/sha256:e0a2cddebe2358a47455a3c492bb92583ee1f3ca31d3aa90781d65a06472cad6 HTTP/1.1" 500 456 "" "docker/1.12.6 go/go1.7.4 kernel/3.10.0-514.2.2.el7.x86_64 os/linux arch/amd64 UpstreamClient(go-dockerclient)"
time="2017-08-30T08:48:49.814083918Z" level=error msg="failed to get image stream: the server has asked for the client to provide credentials (get imageStreams ruby-ex)" go.version=go1.7.6 instance.id=a7fc1648-ab73-4749-8d8a-8ddf2a81c2d7 openshift.logger=registry 
time="2017-08-30T08:48:49.814185687Z" level=error msg="response completed with error" err.code=unknown err.detail="denied: requested access to the resource is denied" err.message="unknown error" go.version=go1.7.6 http.request.host="docker-registry.default.svc:5000" http.request.id=18ff1ae8-4aee-4d54-9da0-a5d2996b5514 http.request.method=HEAD http.request.remoteaddr="10.2.6.1:56244" http.request.uri="/v2/haowang1/ruby-ex/blobs/sha256:e0a2cddebe2358a47455a3c492bb92583ee1f3ca31d3aa90781d65a06472cad6" http.request.useragent="docker/1.12.6 go/go1.7.4 kernel/3.10.0-514.2.2.el7.x86_64 os/linux arch/amd64 UpstreamClient(go-dockerclient)" http.response.contenttype="application/json; charset=utf-8" http.response.duration=10.292634ms http.response.status=500 http.response.written=456 instance.id=a7fc1648-ab73-4749-8d8a-8ddf2a81c2d7 openshift.auth.user="system:serviceaccount:haowang1:builder" openshift.logger=registry vars.digest="sha256:e0a2cddebe2358a47455a3c492bb92583ee1f3ca31d3aa90781d65a06472cad6" vars.name="haowang1/ruby-ex" 

dc

{
    "apiVersion": "v1",
    "kind": "DeploymentConfig",
    "metadata": {
        "creationTimestamp": "2017-08-29T09:44:35Z",
        "generation": 3,
        "labels": {
            "docker-registry": "default"
        },
        "name": "docker-registry",
        "namespace": "default",
        "resourceVersion": "28136",
        "selfLink": "/oapi/v1/namespaces/default/deploymentconfigs/docker-registry",
        "uid": "ab038682-8c9e-11e7-aef4-fa163e544fa9"
    },
    "spec": {
        "replicas": 1,
        "selector": {
            "docker-registry": "default"
        },
        "strategy": {
            "activeDeadlineSeconds": 21600,
            "resources": {},
            "rollingParams": {
                "intervalSeconds": 1,
                "maxSurge": "25%",
                "maxUnavailable": "25%",
                "timeoutSeconds": 600,
                "updatePeriodSeconds": 1
            },
            "type": "Rolling"
        },
        "template": {
            "metadata": {
                "creationTimestamp": null,
                "labels": {
                    "docker-registry": "default"
                }
            },
            "spec": {
                "containers": [
                    {
                        "env": [
                            {
                                "name": "REGISTRY_HTTP_ADDR",
                                "value": ":5000"
                            },
                            {
                                "name": "REGISTRY_HTTP_NET",
                                "value": "tcp"
                            },
                            {
                                "name": "REGISTRY_HTTP_SECRET",
                                "value": "wI9VDPTdCNqetNG9+hBL0OvGqdos7SWADsk387fFPF0="
                            },
                            {
                                "name": "REGISTRY_MIDDLEWARE_REPOSITORY_OPENSHIFT_ENFORCEQUOTA",
                                "value": "false"
                            },
                            {
                                "name": "OPENSHIFT_DEFAULT_REGISTRY",
                                "value": "docker-registry.default.svc:5000"
                            },
                            {
                                "name": "REGISTRY_HTTP_TLS_KEY",
                                "value": "/etc/secrets/registry.key"
                            },
                            {
                                "name": "REGISTRY_HTTP_TLS_CERTIFICATE",
                                "value": "/etc/secrets/registry.crt"
                            }
                        ],
                        "image": "openshift/origin-docker-registry:v3.6.0",
                        "imagePullPolicy": "IfNotPresent",
                        "livenessProbe": {
                            "failureThreshold": 3,
                            "httpGet": {
                                "path": "/healthz",
                                "port": 5000,
                                "scheme": "HTTPS"
                            },
                            "initialDelaySeconds": 10,
                            "periodSeconds": 10,
                            "successThreshold": 1,
                            "timeoutSeconds": 5
                        },
                        "name": "registry",
                        "ports": [
                            {
                                "containerPort": 5000,
                                "protocol": "TCP"
                            }
                        ],
                        "readinessProbe": {
                            "failureThreshold": 3,
                            "httpGet": {
                                "path": "/healthz",
                                "port": 5000,
                                "scheme": "HTTPS"
                            },
                            "periodSeconds": 10,
                            "successThreshold": 1,
                            "timeoutSeconds": 5
                        },
                        "resources": {
                            "requests": {
                                "cpu": "100m",
                                "memory": "256Mi"
                            }
                        },
                        "securityContext": {
                            "privileged": false
                        },
                        "terminationMessagePath": "/dev/termination-log",
                        "terminationMessagePolicy": "File",
                        "volumeMounts": [
                            {
                                "mountPath": "/registry",
                                "name": "registry-storage"
                            },
                            {
                                "mountPath": "/etc/secrets",
                                "name": "registry-certificates"
                            }
                        ]
                    }
                ],
                "dnsPolicy": "ClusterFirst",
                "nodeSelector": {
                    "registry": "enabled",
                    "role": "node"
                },
                "restartPolicy": "Always",
                "schedulerName": "default-scheduler",
                "securityContext": {},
                "serviceAccount": "registry",
                "serviceAccountName": "registry",
                "terminationGracePeriodSeconds": 30,
                "volumes": [
                    {
                        "emptyDir": {},
                        "name": "registry-storage"
                    },
                    {
                        "name": "registry-certificates",
                        "secret": {
                            "defaultMode": 420,
                            "secretName": "registry-certificates"
                        }
                    }
                ]
            }
        },
        "test": false,
        "triggers": [
            {
                "type": "ConfigChange"
            }
        ]
    },
    "status": {
        "availableReplicas": 1,
        "conditions": [
            {
                "lastTransitionTime": "2017-08-30T08:41:21Z",
                "lastUpdateTime": "2017-08-30T08:41:21Z",
                "message": "replication controller \"docker-registry-2\" has failed progressing",
                "reason": "ProgressDeadlineExceeded",
                "status": "False",
                "type": "Progressing"
            },
            {
                "lastTransitionTime": "2017-08-30T08:42:27Z",
                "lastUpdateTime": "2017-08-30T08:42:27Z",
                "message": "Deployment config has minimum availability.",
                "status": "True",
                "type": "Available"
            }
        ],
        "details": {
            "causes": [
                {
                    "type": "Manual"
                }
            ],
            "message": "manual change"
        },
        "latestVersion": 2,
        "observedGeneration": 3,
        "readyReplicas": 1,
        "replicas": 1,
        "unavailableReplicas": 0,
        "updatedReplicas": 0
    }
}