pipelines: ServiceException: 409 Bucket ml-pipeline-playground already exists
What happened: Runs of sample pipelines (for example https://github.com/kubeflow/pipelines/blob/0.1.40/samples/core/sequential/sequential.py) failed with single line in log
ServiceException: 409 Bucket ml-pipeline-playground already exists.
What did you expect to happen: Run without errors What steps did you take: [A clear and concise description of what the bug is.] Open kubeflow GUI, Pipelines, select sample pipeline, “Create run”, task failed with that error Anything else you would like to add: [Miscellaneous information that will assist in solving the issue.] Kubeflow, version 1.0.rc4, installed on GCP, via CLI
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 2
- Comments: 15 (6 by maintainers)
Still the same error with kubeflow 1.0
You may still see intermittent timeouts with the
gcp-downloadcomponent released in KFP 0.2.0 (included in KF 1.0). That’s because its google cloud sdk client version is too low, a fix was sent in later releases https://github.com/kubeflow/pipelines/pull/3019 (I think it went into 0.2.3).Hi all, this is a duplicate of https://github.com/kubeflow/kubeflow/issues/4803. The fix for it has been merged but hasn’t been released.
The permission error is misleading, it should tell you don’t have permission to use that bucket, because
pipeline-runnerservice account isn’t bound to any Google service account in KF 1.0 by default. (KF 1.0 now recommends using workload identity to provide GCP permissions to workloads.)Or I recommend binding your pipeline-runner kubernetes service account to probably
<cluster-name>-usergoogle service account created by Kubeflow. You can use this convenience script to do that: https://github.com/kubeflow/pipelines/blob/00a2cad3cfa81fc6b6b00b4f15a9faf5cbfe5d59/manifests/kustomize/wi-utils.sh#L29Same error with other pipelines with gcp-download