kubernetes: pull-kubernetes-e2e-gce-100-performance fails

Which jobs are failing: pull-kubernetes-e2e-gce-100-performance

Since when has it been failing: Oct 3rd

Testgrid link: https://prow.k8s.io/view/gcs/kubernetes-jenkins/pr-logs/pull/83420/pull-kubernetes-e2e-gce-100-performance/1179968141170577408/

Reason for failure: This seems related:

W1004 04:13:55.503] ERROR: (gcloud.compute.routers.create) Quota 'ROUTERS' exceeded.  Limit: 10.0 globally.

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 19 (18 by maintainers)

Most upvoted comments

Thanks, Folks!

Copying what I said on Slack:

Setting the quota to 30 should be enough for now. The max parallelism (# tests running simultaneously) on this project is ~25 and each test creates a single router. We’re actively working on migrating our presubmits to boskos and hopefully will have it done within the next few weeks. Boskos will help us avoiding such issues in the future.

mm4tt on Oct 8, 2019

We are actually cleaning up routers in https://prow.k8s.io/view/gcs/kubernetes-jenkins/logs/maintenance-pull-scale-janitor/1181421358874103812/

It’s just not enough quota to fit all presubmits

krzyzacy on Oct 8, 2019

the label is declared?

harshitgaur2 on Oct 4, 2019