argo-workflows: downloading artifact from s3 in ui, timed out waiting for condition

Checklist:

  • I’ve included the version.
  • I’ve included reproduction steps.
  • [] I’ve included the workflow YAML.
  • I’ve included the logs.

What happened: Installed the latest 2.5.0rc7 via install.yaml on eks 1.14 and added to the install.yaml the diff shown in the output below, so the archivedLogs and s3 config are enabled (workflow-controller-configmap):

336,344d322
< data:
<   config: |
<     artifactRepository:
<       archiveLogs: true
<       s3:
<         bucket: "example-argo"
<         keyPrefix: "example"
<         endpoint: "s3.amazonaws.com"
< 

to gain ui access on localhost to the ui in kubernetes kubectl port-forward svc/argo-server 2746:2746 -n argo

run a basic hello world workflow via argo cli and the workflow completes as expected, and clicking on artifacts link in ui shows main-logs object as expected, but when you click to download the actual artfact in the ui, the browser eventually returns a “timed out waiting on condition”

What you expected to happen: I expect clicking on the link to download the requested artifact.

How to reproduce it (as minimally and precisely as possible): install.yaml with a s3 config similar to above and run any workflow, and then try and download the resulting main-log artifact.

Logs argo-server log shows:

time="2020-01-31T23:02:06Z" level=info msg="S3 Load path: artifact368826374, key: example/local-script-gd5zj/local-script-gd5zj/main.log"
time="2020-01-31T23:02:06Z" level=info msg="Creating minio client s3.amazonaws.com using IAM role"
time="2020-01-31T23:02:06Z" level=info msg="Getting from s3 (endpoint: s3.amazonaws.com, bucket: example-argo, key: example/local-script-gd5zj/local-script-gd5zj/main.log) to artifact368826374"
time="2020-01-31T23:02:06Z" level=warning msg="Failed get file: Get https://s3.amazonaws.com/example-argo/?location=: x509: certificate signed by unknown authority"

Message from the maintainers:

If you are impacted by this bug please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 3
  • Comments: 44 (34 by maintainers)

Commits related to this issue

Most upvoted comments

Fix implemented.

@alexec @tcolgate have you been able to download artifacts through the UI with rc7? If so, I’m curious why your setup might be different from @haghabozorgi and mine.

@alexec using the image from your docker file results in http named cookie not present message.

One quite way of doing this is to add the following to the argo-server deployment:

      containers:
      - volumeMounts:
        - mountPath: /etc/ssl/certs/ca-certificates.crt
          name: ssl-certs
          readOnly: true

      volumes:
      - hostPath:
          path: /etc/ssl/certs/ca-certificates.crt
          type: ""
        name: ssl-certs