velero: Velero does not send client certificate when using custom cert option

Describe the problem/challenge you have When a self signed certificate is provided to Velero for the S3 object store with cacert option, it uses SSL with TLSv1.0 for the security handshake. TLS v1.0 is very old and the server rejects the handshake. This was done using Velero v1.4 and aws-plugin v1.1.0

Describe the solution you’d like Velero should use TLSv1.2 for SSL handshakes and connections.

Anything else you would like to add: [Miscellaneous information that will assist in solving the issue.]

Environment:

  • Velero version (use velero version):

velero version Client: Version: v1.4.0 Git commit: 5963650c9d64643daaf510ef93ac4a36b6483392 Server: Version: v1.4.0

  • Kubernetes version (use kubectl version): kubectl version Client Version: version.Info{Major:“1”, Minor:“18”, GitVersion:“v1.18.3”, GitCommit:“2e7996e3e2712684bc73f0dec0200d64eec7fe40”, GitTreeState:“clean”, BuildDate:“2020-05-20T12:52:00Z”, GoVersion:“go1.13.9”, Compiler:“gc”, Platform:“linux/amd64”} Server Version: version.Info{Major:“1”, Minor:“16”, GitVersion:“v1.16.0”, GitCommit:“2bd9643cee5b3b3a5ecbd3af49d09018f0773c77”, GitTreeState:“clean”, BuildDate:“2019-09-18T14:27:17Z”, GoVersion:“go1.12.9”, Compiler:“gc”, Platform:“linux/amd64”}

  • Kubernetes installer & version:

minikube version minikube version: v1.4.0 commit: 7969c25a98a018b94ea87d949350f3271e9d64b6

Install Velero using self signed certificate velero install --use-restic --provider aws --bucket k8s-backup-view --secret-file ./secret --cacert ./ssl_cert.pem --use-volume-snapshots=false --backup-location-config region=default,s3ForcePathStyle=“true”,s3Url=https://sv4-dell87-c3-ve02.com:3000 --plugins velero/velero-plugin-for-aws:v1.1.0

Error seen -

kubectl logs deployment/velero -n velero time=“2020-08-14T20:51:13Z” level=info msg=“setting log-level to INFO” logSource=“pkg/cmd/server/server.go:177” time=“2020-08-14T20:51:13Z” level=info msg=“Starting Velero server v1.4.0 (5963650c9d64643daaf510ef93ac4a36b6483392)” logSource=“pkg/cmd/server/server.go:179” time=“2020-08-14T20:51:13Z” level=info msg=“1 feature flags enabled []” logSource=“pkg/cmd/server/server.go:181” time=“2020-08-14T20:51:14Z” level=info msg=“registering plugin” command=/velero kind=BackupItemAction logSource=“pkg/plugin/clientmgmt/registry.go:100” name=velero.io/crd-remap-version time=“2020-08-14T20:51:14Z” level=info msg=“registering plugin” command=/velero kind=BackupItemAction logSource=“pkg/plugin/clientmgmt/registry.go:100” name=velero.io/pod time=“2020-08-14T20:51:14Z” level=info msg=“registering plugin” command=/velero kind=BackupItemAction logSource=“pkg/plugin/clientmgmt/registry.go:100” name=velero.io/pv time=“2020-08-14T20:51:14Z” level=info msg=“registering plugin” command=/velero kind=BackupItemAction logSource=“pkg/plugin/clientmgmt/registry.go:100” name=velero.io/service-account time=“2020-08-14T20:51:14Z” level=info msg=“registering plugin” command=/velero kind=RestoreItemAction logSource=“pkg/plugin/clientmgmt/registry.go:100” name=velero.io/add-pv-from-pvc time=“2020-08-14T20:51:14Z” level=info msg=“registering plugin” command=/velero kind=RestoreItemAction logSource=“pkg/plugin/clientmgmt/registry.go:100” name=velero.io/add-pvc-from-pod time=“2020-08-14T20:51:14Z” level=info msg=“registering plugin” command=/velero kind=RestoreItemAction logSource=“pkg/plugin/clientmgmt/registry.go:100” name=velero.io/change-pvc-node-selector time=“2020-08-14T20:51:14Z” level=info msg=“registering plugin” command=/velero kind=RestoreItemAction logSource=“pkg/plugin/clientmgmt/registry.go:100” name=velero.io/change-storage-class time=“2020-08-14T20:51:14Z” level=info msg=“registering plugin” command=/velero kind=RestoreItemAction logSource=“pkg/plugin/clientmgmt/registry.go:100” name=velero.io/cluster-role-bindings time=“2020-08-14T20:51:14Z” level=info msg=“registering plugin” command=/velero kind=RestoreItemAction logSource=“pkg/plugin/clientmgmt/registry.go:100” name=velero.io/crd-preserve-fields time=“2020-08-14T20:51:14Z” level=info msg=“registering plugin” command=/velero kind=RestoreItemAction logSource=“pkg/plugin/clientmgmt/registry.go:100” name=velero.io/job time=“2020-08-14T20:51:14Z” level=info msg=“registering plugin” command=/velero kind=RestoreItemAction logSource=“pkg/plugin/clientmgmt/registry.go:100” name=velero.io/pod time=“2020-08-14T20:51:14Z” level=info msg=“registering plugin” command=/velero kind=RestoreItemAction logSource=“pkg/plugin/clientmgmt/registry.go:100” name=velero.io/restic time=“2020-08-14T20:51:14Z” level=info msg=“registering plugin” command=/velero kind=RestoreItemAction logSource=“pkg/plugin/clientmgmt/registry.go:100” name=velero.io/role-bindings time=“2020-08-14T20:51:14Z” level=info msg=“registering plugin” command=/velero kind=RestoreItemAction logSource=“pkg/plugin/clientmgmt/registry.go:100” name=velero.io/service time=“2020-08-14T20:51:14Z” level=info msg=“registering plugin” command=/velero kind=RestoreItemAction logSource=“pkg/plugin/clientmgmt/registry.go:100” name=velero.io/service-account time=“2020-08-14T20:51:14Z” level=info msg=“registering plugin” command=/plugins/velero-plugin-for-aws kind=VolumeSnapshotter logSource=“pkg/plugin/clientmgmt/registry.go:100” name=velero.io/aws time=“2020-08-14T20:51:14Z” level=info msg=“registering plugin” command=/plugins/velero-plugin-for-aws kind=ObjectStore logSource=“pkg/plugin/clientmgmt/registry.go:100” name=velero.io/aws time=“2020-08-14T20:51:14Z” level=info msg=“Checking existence of namespace” logSource=“pkg/cmd/server/server.go:361” namespace=velero time=“2020-08-14T20:51:14Z” level=info msg=“Namespace exists” logSource=“pkg/cmd/server/server.go:367” namespace=velero time=“2020-08-14T20:51:16Z” level=info msg=“Checking existence of Velero custom resource definitions” logSource=“pkg/cmd/server/server.go:396” time=“2020-08-14T20:51:16Z” level=info msg=“All Velero custom resource definitions exist” logSource=“pkg/cmd/server/server.go:430” time=“2020-08-14T20:51:16Z” level=info msg=“Checking that all backup storage locations are valid” logSource=“pkg/cmd/server/server.go:437” An error occurred: some backup storage locations are invalid: backup store for location “default” is invalid: rpc error: code = Unknown desc = RequestError: send request failed caused by: Get https://sv4-dell87-c3-ve02.com:3000/k8s-backup-view?delimiter=%2F&list-type=2&prefix=: remote error: tls: alert(116)

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project’s top voted issues listed here.
Use the “reaction smiley face” up to the right of this comment to vote.

  • 👍 for “The project would be better with this feature added”
  • 👎 for “This feature will not enhance the project in a meaningful way”

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 15 (13 by maintainers)

Most upvoted comments

Opened PR-3811

I have edited the issue title. Will open a docs PR.

@ashish-amarnath I am not sure why Velero as a client did not include its certificate while the SSL handshake was done. From TLS 1.3 spec https://tools.ietf.org/html/rfc8446, verifying client certificate could be optional. Changing this on the server made it work. I am not sure if you guys want to fix this in Velero, where it would send a certificate for the handshake. It would be better to edit the issue title as well. Please let me know.