kubernetes: Kubectl skew tests are failing

Which jobs are failing:

  • ci-kubernetes-e2e-gce-new-master-gci-kubectl-skew
  • ci-kubernetes-e2e-gce-stable1-beta-gci-kubectl-skew

https://storage.googleapis.com/k8s-gubernator/triage/index.html?text=Kubectl&job=skew

Which test(s) are failing:

Steadily failing tests making signal from the skew jobs unusable:

  • [sig-cli] Kubectl client Kubectl rolling-update should support rolling-update to same image [Conformance]
  • [sig-cli] Kubectl client Kubectl run rc should create an rc from an image [Conformance]
  • [sig-cli] Kubectl alpha client Kubectl run CronJob should create a CronJob
  • [sig-cli] Kubectl client Kubectl run deployment should create a deployment from an image [Conformance]
  • [sig-cli] Kubectl client Kubectl run job should create a job from an image when restart is OnFailure [Conformance]
  • [sig-cli] Kubectl client Kubectl run default should create an rc or deployment from an image [Conformance]
  • [sig-cli] Kubectl client Simple pod should support inline execution and attach
  • [sig-cli] Kubectl client Update Demo should do a rolling update of a replication controller [Conformance]
  • [sig-cli] Kubectl client Simple pod should return command exit codes

Flaking tests:

  • [sig-cli] Kubectl client Guestbook application should create and stop a working application [Conformance]
  • [sig-cli] Kubectl client Kubectl replace should update a single-container pod's image [Conformance]
  • [sig-cli] Kubectl client Kubectl patch should add annotations for pods in rc [Conformance]
  • [sig-cli] Kubectl client Simple pod should support port-forward
  • [sig-cli] Kubectl client Kubectl client-side validation should create/apply a CR with unknown fields for CRD with no validation schema
  • [sig-cli] Kubectl client Kubectl client-side validation should create/apply a valid CR for CRD with validation schema

Testgrid link:

Reason for failure:

Anything else we need to know:

The skew jobs are what tell us if we’ve broken compatibility with prior kubectl releases. I’m surprised they aren’t considered release blocking, but they should be a high priority for sig-cli to fix regardless.

/sig cli /priority critical-urgent /cc @soltysh @seans3

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 28 (25 by maintainers)

Most upvoted comments

i think the version markers / labels are worthy of a KEP where we can define what is redundant and what we need.

cc @hasheddan

I think you looked at some of these and it looked like kubectl tests were making use of deprecated functionality. Detecting/omitting tests of deprecated functionality in the release the function is removed seems reasonable.

/assign @hasheddan

I’ll work with you to clean this up in v1.21 (in relation to https://github.com/kubernetes/sig-release/issues/850). /milestone v1.21

Tagging @saschagrunert on this also because he is doing work on push-build 👍

this test:

[sig-cli] Kubectl client Update Demo should do a rolling update of a replication controller [Conformance]

was removed with the “rolling-update” command removal in 1.18: https://github.com/kubernetes/kubernetes/pull/88057/commits/932500c0151a4f0926cf07e63068248540b22f90#diff-fde2ce0a73639d19347a5ea6fe001ba6L325

a failing job such as ci-kubernetes-e2e-gce-new-master-gci-kubectl-skew: https://prow.k8s.io/view/gcs/kubernetes-jenkins/logs/ci-kubernetes-e2e-gce-new-master-gci-kubectl-skew/1290907818873851904

is defined here: https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/sig-cli/sig-cli-config.yaml#L217

stable1 e2e tests run against a stable1 gce cluster using a master kubectl binary

so this test should not be running unless something is wrong with:

      - --extract=ci/latest
      - --extract=ci/k8s-stable1 

and indeed it is:

$ curl -L https://dl.k8s.io/ci/latest.txt
v1.20.0-alpha.0.509+f7e3bcdec2e090
$ curl -L https://dl.k8s.io/ci/k8s-stable1.txt
v1.17.9-rc.0.13+fdec14cd4c84b7

so the test-suite in 1.17.9… still has the test that then calls the kubectl binary from master that no longer knows about the “rolling-update” command.

maybe the intent is to use release/stable-1:

$ curl -L https://dl.k8s.io/release/stable-1.txt
v1.18.6