velero: Backups stuck Deleting because of "filename too long"

What steps did you take and what happened: Backups fail to delete from the cluster (stuck in Deleting state) and have to be manually removed from s3 and kubectl delete backups.velero.io <backup-name>

What did you expect to happen: Backups maintenance to occur without getting stuck.

The following information will help us better understand what’s going on: I cannot attach the debug logs due to privacy issues, but the issues is easily repeatable by backing up a filename longer than 255 characters. The backup succeeds, but the backup will not delete. I assume a restore would fail in a similar fashion. With debug logging on while deleting the below error is in the log:

error invoking delete item actions: error extracting backup: open file name too long

Anything else you would like to add: It appears that the container image is based on Ubuntu 22.04, and that has a limit of 255 characters which is getting triggered when Velero tries to unzip the backup bundle.

The error is coming from here: https://github.com/vmware-tanzu/velero/blob/main/internal/delete/delete_item_action_handler.go And from the os class here: https://github.com/golang/go/blob/a3c068c57ae3f71a7720fe68da379143bb579362/src/os/getwd.go#L57 Ubuntu 22.04 ext4 filename limit https://help.ubuntu.com/stable/ubuntu-help/files-rename.html.he#:~:text=This 255 character limit includes,and folder names where possible.

Is it possible to have a container image that supports longer filenames? If not, would it be possible to not backup files with filenames > 255 characters and skip them with an error to prevent the issue I describe?

Environment:

  • Velero version (use velero version): v1.13.2
  • Velero features (use velero client config get features): <NOT SET>
  • Kubernetes version (use kubectl version): 1.25.13
  • Kubernetes installer & version: kubeadm + Spectrocloud
  • Cloud provider or hardware configuration: VMWare ESX
  • OS (e.g. from /etc/os-release): Ubuntu 22.04.3 LTS

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project’s top voted issues listed here.
Use the “reaction smiley face” up to the right of this comment to vote.

  • 👍 for “I would like to see this bug fixed as soon as possible”
  • 👎 for “There are more important bugs to focus on right now”

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 21 (3 by maintainers)

Most upvoted comments

Based on the comment, there’s a concern about the fix (PR #8449).

@kaovilai please revise the PR to resolve the comment.

@ameer2rock try ghcr.io/kaovilai/velero:maxpathlimits-a2699e765 from #8449 and lmk if it works.