cloudstack: issues with live migration on centos (was: Trillian test failures)

@DaanHoogland @rhtyd this needs research.

Test Result Time (s) Test File
test_01_migrate_VM_and_root_volume Error 63.19 test_vm_life_cycle.py
test_02_migrate_VM_with_two_data_disks Error 48.09 test_vm_life_cycle.py

https://github.com/apache/cloudstack/pull/4736#issuecomment-789312916

management logs

2021-03-02 18:50:44,203 DEBUG [o.a.c.s.v.VolumeServiceImpl] (Work-Job-Executor-106:ctx-a6be1c7c job-2200/job-2201 ctx-10684199) (logid:f366d768) Failed to copy volume
com.cloud.utils.exception.CloudRuntimeException: Copy operation failed in 'StorageSystemDataMotionStrategy.copyAsync': Exception during migrate: org.libvirt.LibvirtException: internal error: unable to execute QEMU command 'migrate': this feature or command is not currently supported
	at org.apache.cloudstack.storage.motion.StorageSystemDataMotionStrategy.copyAsync(StorageSystemDataMotionStrategy.java:1901)
	at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:119)
	at org.apache.cloudstack.storage.volume.VolumeServiceImpl.migrateVolumes(VolumeServiceImpl.java:1645)
	at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.migrateVolumes(VolumeOrchestrator.java:1181)
	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateWithStorage(VirtualMachineManagerImpl.java:3000)
	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateWithStorage(VirtualMachineManagerImpl.java:5524)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
	at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5631)
	at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
2021-03-02 18:50:44,203 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-106:ctx-a6be1c7c job-2200/job-2201 ctx-10684199) (logid:f366d768) Failed to migrated vm VM[User|i-191-245-VM] along with its volumes. com.cloud.utils.exception.CloudRuntimeException: Copy operation failed in 'StorageSystemDataMotionStrategy.copyAsync': Exception during migrate: org.libvirt.LibvirtException: internal error: unable to execute QEMU command 'migrate': this feature or command is not currently supported
ISSUE TYPE
  • Bug Report
  • Improvement Request
  • Enhancement Request
  • Feature Idea
  • Documentation Report
  • Other
COMPONENT NAME

CLOUDSTACK VERSION

CONFIGURATION
OS / ENVIRONMENT
SUMMARY
STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 23 (23 by maintainers)

Commits related to this issue

Most upvoted comments

For the purpose of this issue, just ask @davidjumani to revert the change (done by whoever…) that made these test suddenly run, instead of being skipped, as they had been for lastbfew years.

@andrijapanicsb we can ignore CentOS6, which has reached EOL, in the current release/context of CloudStack.

Given the limitations should we be promoting using Ubuntu now where things work out of the box (and say update the QIG guide?) and many CloudStack users notably Leaseweb, PCExtreme are already major users of Ubuntu/KVM with CloudStack? cc @wido @GabrielBrascher @weizhouapache

I’ve started a thread on dev/user ML to discuss the use of qemu-kvm-ev with CloudStack (CentOS7, 8).

I have no opinion about CentOS as we indeed use Ubuntu on all our servers.