kubernetes: [e2e test failure] Simple pod should support exec through kubectl proxy
This test has started failing on GKE and GCI-GKI e2e tests:
https://k8s-testgrid.appspot.com/release-master-blocking#gci-gke https://k8s-testgrid.appspot.com/release-master-blocking#gke
Example failure
https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gke/12920
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubectl/kubectl.go:457
Expected error:
<exec.CodeExitError>: {
Err: {
s: "error running &{/workspace/kubernetes/platforms/linux/amd64/kubectl [kubectl --server=https://35.188.213.188 --kubeconfig=/tmp/gke-kubecfg386178168 --server=http://127.0.0.1:45529 --namespace=e2e-tests-kubectl-nlzj2 exec nginx echo running in container] [] <nil> W0810 18:25:44.244920 4112 http.go:342] Error reading backend response: unexpected EOF\nerror: error sending request: Post http://127.0.0.1:45529/api/v1/namespaces/e2e-tests-kubectl-nlzj2/pods/nginx/exec?command=echo&command=running&command=in&command=container&container=nginx&container=nginx&stderr=true&stdout=true: unexpected EOF\n [] <nil> 0xc4211d3830 exit status 1 <nil> <nil> true [0xc4203e6828 0xc4203e6840 0xc4203e6858] [0xc4203e6828 0xc4203e6840 0xc4203e6858] [0xc4203e6838 0xc4203e6850] [0x11bf750 0x11bf750] 0xc420f2db60 <nil>}:\nCommand stdout:\n\nstderr:\nW0810 18:25:44.244920 4112 http.go:342] Error reading backend response: unexpected EOF\nerror: error sending request: Post http://127.0.0.1:45529/api/v1/namespaces/e2e-tests-kubectl-nlzj2/pods/nginx/exec?command=echo&command=running&command=in&command=container&container=nginx&container=nginx&stderr=true&stdout=true: unexpected EOF\n\nerror:\nexit status 1\n",
},
Code: 1,
}
error running &{/workspace/kubernetes/platforms/linux/amd64/kubectl [kubectl --server=https://35.188.213.188 --kubeconfig=/tmp/gke-kubecfg386178168 --server=http://127.0.0.1:45529 --namespace=e2e-tests-kubectl-nlzj2 exec nginx echo running in container] [] <nil> W0810 18:25:44.244920 4112 http.go:342] Error reading backend response: unexpected EOF
error: error sending request: Post http://127.0.0.1:45529/api/v1/namespaces/e2e-tests-kubectl-nlzj2/pods/nginx/exec?command=echo&command=running&command=in&command=container&container=nginx&container=nginx&stderr=true&stdout=true: unexpected EOF
[] <nil> 0xc4211d3830 exit status 1 <nil> <nil> true [0xc4203e6828 0xc4203e6840 0xc4203e6858] [0xc4203e6828 0xc4203e6840 0xc4203e6858] [0xc4203e6838 0xc4203e6850] [0x11bf750 0x11bf750] 0xc420f2db60 <nil>}:
Command stdout:
stderr:
W0810 18:25:44.244920 4112 http.go:342] Error reading backend response: unexpected EOF
error: error sending request: Post http://127.0.0.1:45529/api/v1/namespaces/e2e-tests-kubectl-nlzj2/pods/nginx/exec?command=echo&command=running&command=in&command=container&container=nginx&container=nginx&stderr=true&stdout=true: unexpected EOF
error:
exit status 1
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:2066```
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Comments: 20 (20 by maintainers)
Commits related to this issue
- Skip "Simple pod should support exec through kubectl proxy" test As reported in https://github.com/kubernetes/kubernetes/issues/50466, this test doesn't work in GKE because the transport layer doesn'... — committed to apelisse/kubernetes by apelisse 7 years ago
- Skip "Simple pod should support exec through kubectl proxy" test As reported in https://github.com/kubernetes/kubernetes/issues/50466, this test doesn't work in GKE because the transport layer doesn'... — committed to apelisse/kubernetes by apelisse 7 years ago
- Merge pull request #51047 from apelisse/remove-gke-test Automatic merge from submit-queue Skip "Simple pod should support exec through kubectl proxy" test As reported in https://github.com/kubernet... — committed to kubernetes/kubernetes by deleted user 7 years ago
- Merge pull request #51168 from smarterclayton/fix_transport_wrap Automatic merge from submit-queue (batch tested with PRs 50775, 51397, 51168, 51465, 51536) Allow bearer requests to be proxied by ku... — committed to ii/kubernetes by deleted user 7 years ago
#50775 is necessary but not sufficient.
copied from https://github.com/kubernetes/kubernetes/pull/50775#issuecomment-322963183
the issue is with https://github.com/kubernetes/kubernetes/blob/4bfe9b1a568be137f72cf5dfc8bc2c1f13874548/staging/src/k8s.io/apimachinery/pkg/util/proxy/upgradeaware.go#L269-L282 that does not actually use the transport to handle the request, only to dial. That means that TLS-authentication methods like client certs work, and header authentication methods like bearer tokens do not.
can recreate this by running
kubectl proxy
using bearer token authentication and not cert auth, then trykubectl exec ... --server=http://localhost:8001