argo-workflows: Cannot specify key for input artifact (without full artifact location)
Tested on 2.9.0-rc3
The official example works:
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: artifactory-repository-ref-
spec:
entrypoint: main
artifactRepositoryRef:
key: minio
templates:
- name: main
container:
image: docker/whalesay:latest
command: [sh, -c]
args: ["cowsay hello world | tee /tmp/hello_world.txt"]
outputs:
artifacts:
- name: hello_world
path: /tmp/hello_world.txt
When switching to input it fails:
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: artifactory-repository-ref-
spec:
entrypoint: main
artifactRepositoryRef:
key: minio
templates:
- name: main
container:
image: docker/whalesay:latest
command: [sh, -c]
args: ["cowsay hello world | tee /tmp/hello_world.txt"]
inputs:
artifacts:
- name: hello_world
path: /tmp/hello_world.txt
with
Failed to submit workflow: templates.entrypoint.steps[0].main templates.main-template inputs.artifacts.hello_world was not supplied
Another important point IMO. I don’t see any ways to specify the key
during workflow creation (the location of your object within S3/bucket). My understanding is that artifactRepositoryRef
can be used to setup default repositories and then can be reused within workflows specifying the location of the folder or file we want to use as inputs or outputs. Was that designed for that purpose?
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 12
- Comments: 16 (12 by maintainers)
Do 👍 to show interest.
@hadim I’m going to recategorize this as an “enhancement”. We should do more work in this area, and I’d like to asses interest.
Kubeflow Pipelines (KFP) try to separate cluster config (like artifact repository) from workflow config. If this feature is supported from argo, we could let each user manage its artifact repository in their own namespace while keeping the workflow definitions shareable.
This is one of the areas KFP hasn’t been able to achieve multi-tenancy separation: https://github.com/kubeflow/pipelines/issues/1223#issuecomment-656507073.
Some recent discussion: https://kubeflow.slack.com/archives/CE10KS9M4/p1602516358147900
This issue isn’t really to do with
artifactRepositoryRef
, it’s not supported at all. I’m going to rename this issue to reflect this.I think you must specify the key within the bucket, as well as bucket, endpoint etc. Not great I agree, but that is how it is today.