pipelines: s3 Access error for visualizations

What steps did you take:

Followed the steps from https://www.kubeflow.org/docs/aws/pipeline/#support-s3-artifact-store

What happened:

  • receive a s3 access error image

What did you expect to happen:

See a markdown in the visualizations tab

Environment:

my ml-pipeline-ui deployment: image my secrets (base64) image

I can even confirm that the ml-pipeline-ui has the correct env variables by jumping into its terminal: image

And finally these keys are valid for pulling and pushing to s3.

How did you deploy Kubeflow Pipelines (KFP)?

https://www.kubeflow.org/docs/aws/aws-e2e/

KFP version: 1.0.0

KFP SDK version:

kfp                      1.0.0
kfp-server-api           1.0.0

Anything else you would like to add:

[Miscellaneous information that will assist in solving the issue.]

/kind bug

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 18 (8 by maintainers)

Most upvoted comments

specifically for the viz, the ml-pipeline-ui pod must have access rights to the S3 bucket.

There are few ways to do it:

  • provide the access key and secret as env var to the pod (as part of the deployment)
  • (recommended) use kube2iam or similar (or even the IAM role to the node - not recommended) to give the pod the IAM credentials

I maintain a community repo for pipelines on AWS - u can refer https://github.com/e2fyi/kubeflow-aws/tree/master/pipelines on the finer details on how to config for S3.

@PatrickXYS Yeah I can do that.