one: Backup "nil:NilClass Error preparing disk files" error

Description

Backups from POWEROFF state can’t be done because of nil:NilClass Error preparing disk files. This error was present before modifications made here.

To Reproduce

Backup options for VM 84:

BACKUP_CONFIG=[
  BACKUP_VOLATILE="NO",
  FS_FREEZE="AGENT",
  INCREMENTAL_BACKUP_ID="-1",
  KEEP_LAST="7",
  LAST_INCREMENT_ID="-1",
  MODE="INCREMENT" ]

System datastore:

DATASTORE 0 INFORMATION
ID             : 0
NAME           : system
USER           : oneadmin
GROUP          : oneadmin
CLUSTERS       : 0
TYPE           : SYSTEM
DS_MAD         : -
TM_MAD         : shared
BASE PATH      : /var/lib/one//datastores/0
DISK_TYPE      : FILE
STATE          : READY

DATASTORE CAPACITY
TOTAL:         : 759G
FREE:          : 491.4G
USED:          : 267.6G
LIMIT:         : -

PERMISSIONS
OWNER          : um-
GROUP          : u--
OTHER          : ---

DATASTORE TEMPLATE
ALLOW_ORPHANS="FORMAT"
DISK_TYPE="FILE"
DS_MIGRATE="YES"
RESTRICTED_DIRS="/"
SAFE_DIRS="/var/tmp"
SHARED="YES"
TM_MAD="shared"
TYPE="SYSTEM_DS"

IMAGES

Images datastore:

DATASTORE 1 INFORMATION
ID             : 1
NAME           : images
USER           : oneadmin
GROUP          : oneadmin
CLUSTERS       : 0
TYPE           : IMAGE
DS_MAD         : fs
TM_MAD         : shared
BASE PATH      : /var/lib/one//datastores/1
DISK_TYPE      : FILE
STATE          : READY

DATASTORE CAPACITY
TOTAL:         : 759G
FREE:          : 491.4G
USED:          : 267.6G
LIMIT:         : -

PERMISSIONS
OWNER          : um-
GROUP          : u--
OTHER          : ---

DATASTORE TEMPLATE
ALLOW_ORPHANS="FORMAT"
CLONE_TARGET="SYSTEM"
CLONE_TARGET_SSH="SYSTEM"
DISK_TYPE="FILE"
DISK_TYPE_SSH="FILE"
DS_MAD="fs"
LN_TARGET="NONE"
LN_TARGET_SSH="SYSTEM"
RESTRICTED_DIRS="/"
SAFE_DIRS="/var/tmp"
TM_MAD="shared"
TM_MAD_SYSTEM="ssh"
TYPE="IMAGE_DS"

IMAGES
0
1
2
3
4
5
6
8
20
26
29
31
35
38
41
42
43
52
53
55
56
57
58
59
61
62
63
64

Gluster “Distributed-Disperse” external storage or local “soft-RAID 5”. It was tested on an OpenNebula multi-node cluster with external storage (Gluster) and single-node cluster with local storage (soft-RAID 5).

Trying to make a backup live with --reset:

onevm backup --reset -d 100 84

The following is a snippet from the /var/log/one/84.log after failed the backup action:

# [...]
# ------------------- # Convert and flatten # ------------------- /var/tmp/one/tm/lib/backup_qcow2.rb -d "0:" -
x /var/lib/one//datastores/0/84/backup/vm.xml -p /var/lib/one//datastores/0/84  Error: undefined method `each' for nil:NilClass Error preparing disk files: un
defined method `each' for nil:NilClass  + '[' -d /var/lib/one//datastores/0/84/backup ']' + rm -rf /var/lib/one//datastores/0/66/backup + mkdir -p /var/lib/on
e//datastores/0/84/backup + echo [...]

Same error if --reset option is not specified.

Expected behavior

Backup without the nil:NilClass Error preparing disk files error.

Details

  • Affected Component: [Storage]
  • Hypervisor: [KVM]
  • Version: [6.6.1]
  • OS: Ubuntu 22.04

Additional context Add any other context about the problem here.

Progress Status

  • Code committed
  • Testing - QA
  • Documentation (Release notes - resolved issues, compatibility, known issues)

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 23 (10 by maintainers)

Commits related to this issue

Most upvoted comments

@rsmontero, under certain conditions, this is a consequences of an heavy load storage backend or a slow one. Doing a pause to let the storage flush and release the I/O, this will add some time to the backup operation, but it’s necessary sometimes. Sadly, the qemu-img bitmap command doesn’t have the –force-share capability. and the only way that i can imagine is doing this sleep, or may a more elaborated one, like here, proposed by @Franco-Sparrow.

Thanks in advance Team

First of all @Franco-Sparrow thanks for the thorough testing and details provided in the issue ❤️ 🙏

Code has been merge to 6.6 and master branches (see commit 78910010d35683864d3a479a2f0c6eb17c95229d) and will be delivered in the next maintenance release.