clearml: S3 Connection setup with minIO: Cannot find credentials

I am trying to setup and use minIO s3 storage like so.

task = Task.init(project_name="my_project",
                     task_name="linear-regression-test",
                     tags="testing",
                     output_uri="s3://minio.api.example.io/clearml-artifacts"
                     )

and then upload and artifact like so:

    task.upload_artifact(name=model_file,
                         artifact_object=model_file,
                         delete_after_upload=True,
                         auto_pickle=True,
                         metadata=metadata)

I followed the steps in the documentation for setting up S3 Storage with minIO and added my credentials in my ~/clearml.conf like so:

aws {
        s3 {
            key: "<minio-key>"
            secret: "<minio-secret>"
            credentials: [
                {
                    host: "minio.api.example.io:9000"
                    bucket: "clearml-artifacts"
                    key: "<minio-key>"
                    secret: "<minio-secret>"
                    multipart: false
                    secure: true
                }
            ]
        } 
}

I’ve many different uris for “output_uri” similarly like in this issue, but the connection cannot be established.

This is the behavior for the various values for “output_uri” that I tried:

• output_uri = minio_url: upload to the local folder with the url as name
• output_uri = minio_url/clearml-artifacts: same as above
• output_uri = https://minio_url/clearml: POST failed
• output_uri = s3://clearml: Could not get access credentials for ‘s3://clearml-artifacts’
• output_uri = s3://minio_url/clearml: same as above.
• output_uri = s3://minio_url:9000/clearml: same as above

I also tried setting my credentials like so, which did not help:

sdk {
    aws {
        s3 {
            credentials: [
                {
		  host: "minio.api.example.io:9000"
		  bucket: "clearml-artifacts"
		  key: "<minio-key>"
		  secret: "<minio-secret>"
		  multipart: false
		  secure: true
                }
            ]
        }
    }
    development.default_output_uri: "s3://minio.api.cosmic-x.idcp.inovex.io:9000/clearml-artifacts"
}

Any suggestions?

Environment

• Server type: self hosted
• ClearML SDK Version
• ClearML Server Version: WebApp: 1.10.1-359 • Server: 1.10.1-359 • API: 2.24
• Python Version 3.8.16
• OS Linux

Related Discussion

Uploading dataset to MinIO #391