pipelines: Parsing failure when uploading a pipeline with kfp cli(or SDK)

What happened: When I tried to upload a pipeline with a gzip-ed yaml file, kfp cli returned following error after successfully uploading the pipeline

$ kfp --endpoint=<<XXXXXX>>.notebooks.googleusercontent.com pipeline upload -p test <<XXXXXX>>.tar.gz
Parser must be a string or character stream, not dict
$ echo $?
1

I debugged a little and found that the returned RPC response message from the server is a little bit odd.

{"id":"ec299e6a-b994-4b2c-9060-65ff48039e50","name":"test","parameters":[{"name":"pipeline-root","value":"pipelines/XXXXXXXX"}],"default_version":{"id":"ec299e6a-b994-4b2c-9060-65ff48039e50","name":"test","created_at":{"seconds":1576824474},"parameters":[{"name":"pipeline-root","value":"pipelines/XXXXXXXX"}],"resource_references":[{"key":{"type":3,"id":"ec299e6a-b994-4b2c-9060-65ff48039e50"},"relationship":1}]},"created_at":"2019-12-20T06:47:54Z"}

created_at field in the default_version node is not a string, but "created_at":{"seconds":1576824474} which results in a parse error.

Interesting point is that created_at at the root node is having string value. FYI $ kfp pipeline list works without any problem. 😅

What did you expect to happen: created_at field should have ISO8601 string format, and kfp CLI should success.

What steps did you take: I tested in GCP with KFP Build commit: b3171f0 for the server. Client version is 0.1.37

$ pip show kfp
Name: kfp
Version: 0.1.37

Anything else you would like to add: I actually was testing TFX which uses KFP Python SDK, and had the same problem. So it looks like a problem not only for CLI.

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 16 (15 by maintainers)

Commits related to this issue

Most upvoted comments

+1 to @jiyongjung0 's reply. Kubeflow full-fledge deployment is not picking the latest version of KFP.

Sorry, I meant kubeflow pipeline installation specifically. You can get a hint from the KFP installation guide or here. The most recent release looks like 0.3.0.