one: VM disk monitoring not working as expected
Description
After enabling the workaround of #4791 something is working but it looks like the sent monitoring data from the remote hosts is not processed, despite the Successfully monitored VM: $VM_ID log message
To Reproduce install opennebula and a remote KVM host instantiate a VM in the default ssh SYSTEM datatore check the VM disk size
Expected behavior The VM disks size should be updated
Details
- Affected Component: [Monitring]
- Hypervisor: [KVM]
- Version: [development]
Additional context As far as I understand there are two ways to feed VM disk stats to OpenNebula:
- on the frontend node - TM_MAD/monitor when called by for SYSTEM datastore which is not tagged for “at the host” monitoring (Cant twist my mind to name it “tagged for local monitoring”…)
- On the hosts - TM_MAD/monitor_ds called by the remote monitoring probe
the monitoring message should be
VM = [ ID = $VM_ID, MONITOR = "<base64>
DISK_SIZE=[ID=$DISK_ID,SIZE=$DISK_SIZE]
SNAPSHOT_SIZE = [ID=$SNAP_ID, DISK_ID=$DISK_ID, SIZE=$SNAP_SIZE]
</base64>" ]
The message is logged as successfully monitored by monitors:
Fri May 22 10:41:02 2020 [Z0][HMM][I]: Successfully monitored VM: 3
But in the VMs the disks sizes are 0:
[root@one512 ssh]# onevm show 3
VIRTUAL MACHINE 3 INFORMATION
ID : 3
NAME : Ttylinux - KVM-3
USER : oneadmin
GROUP : oneadmin
STATE : ACTIVE
LCM_STATE : RUNNING
LOCK : None
RESCHED : No
HOST : vs04
CLUSTER ID : 0
CLUSTER : default
START TIME : 05/20 16:15:55
END TIME : -
DEPLOY ID : 6d8f347e-fbce-4d26-9df5-d455d6a2c11b
VIRTUAL MACHINE MONITORING
CPU : 1.01
MEMORY : 106.1M
NETTX : 0K
NETRX : 0K
DISKRDBYTES : 13855320
DISKRDIOPS : 974
DISKWRBYTES : 596992
DISKWRIOPS : 303
ID : 3
TIMESTAMP : 1590133292
PERMISSIONS
OWNER : um-
GROUP : ---
OTHER : ---
VM DISKS
ID DATASTORE TARGET IMAGE SIZE TYPE SAVE
0 default vda Ttylinux - KVM -/200M file NO
2 system sda 1024M -/1024M swap NO
1 - hda CONTEXT -/- - -
patched the monitor_ds script to save the base64 encoded string to a temp file
echo -e "$vm_monitor" | base64 -w 0 - | tee /tmp/monitor_ssh${vm}.b64
And there is the following data provided:
[root@vs04 tmp]# cat /tmp/monitor_ssh3.b64 | base64 -d
DISK_SIZE = [ ID=0, SIZE=27]
DISK_SIZE = [ ID=1, SIZE=1]
DISK_SIZE = [ ID=2, SIZE=1]
Progress Status
- Branch created
- Code committed to development branch
- Testing - QA
- Documentation
- Release notes - resolved issues, compatibility, known issues
- Code committed to upstream release/hotfix branches
- Documentation committed to upstream release/hotfix branches
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 33 (33 by maintainers)
Commits related to this issue
- B #4793: Fix VM monitor template merging Solves monitoring of "local" disks — committed to OpenNebula/one by deleted user 4 years ago
- B #4793: Fix VM monitor template merging Solves monitoring of "local" disks — committed to OpenNebula/one by deleted user 4 years ago
- B #4793: DS monitoring sends VM info to monitord — committed to OpenNebula/one by deleted user 4 years ago
- B #4793: Merge VM monitoring with last info in DB — committed to OpenNebula/one by deleted user 4 years ago
- B #4793: Fix DS VM monitoring * Merge VM DB monitor information with previous record. * DS sends monitor data to monitord socket * Fix VM monitor template merging for "local" disks co-au... — committed to OpenNebula/one by rsmontero 4 years ago
- B #4793: Update tm_mad/shared/monitor MONITOR_ADDRESS could differ from _127.0.0.1_ in _/etc/monitord.conf_ The default netcat (NC) on centos7.8 is _nmap-ncat_ which do not accept '0' for _--wait_ ... — committed to atodorov-storpool/one by atodorov-storpool 4 years ago
- B #4793: Fix timestamp — committed to OpenNebula/one by deleted user 4 years ago
- B #4793: Fix timestamp — committed to OpenNebula/one by rsmontero 4 years ago
- B #4793: Read monitor address from monitord.conf — committed to OpenNebula/one by deleted user 4 years ago
- B #4793: Rewrite VM disks monitoring — committed to OpenNebula/one by deleted user 4 years ago
- B #4793: Read monitor address from monitord.conf — committed to OpenNebula/one by rsmontero 4 years ago
- B #4793: Move requires to function, rescue errors — committed to OpenNebula/one by deleted user 4 years ago
- B #4793: Move requires to function, rescue errors — committed to OpenNebula/one by rsmontero 4 years ago
- B #4793: Fix VM disks monitoring for LVM — committed to OpenNebula/one by deleted user 4 years ago
- B #4793: All variables upper-case, remove unused — committed to OpenNebula/one by deleted user 4 years ago
- B #4793: Fix VM disks monitoring for LVM * All variables upper-case, remove unused — committed to OpenNebula/one by deleted user 4 years ago
@feldsam Documentation for the new monitoring: http://docs.opennebula.io/5.11/deployment/open_cloud_host_setup/monitoring.html#extending-the-monitor-system
@atodorov-storpool We will address your issues with nc and MONITOR_ADDRESS soon