windows_exporter: Unable to get Hypervisor cpu load percentage

Hello,

I’m trying to monitor my hyper-V servers with WMI_Exporter but I can’t get the real cpu load, all I got are some random unitless numbers with wmi_hyperv_host_cpu_hypervisor_run_time.

I also got the powershell command to get this numbers : Get-Counter -Counter “\Hyper-V Hypervisor Logical Processor(_Total)\% Total Run Time". If anyone is interested in adding this in the exporter, it would be really great.

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 17 (8 by maintainers)

Most upvoted comments

Those query look promising:

% consumption of the physical CPU ressources on Host ServerName by VMs (On a specific Host): (sum by (vm) (rate(windows_hyperv_vm_cpu_total_run_time{instance="ServerName"}[60s]))) / ignoring(vm) group_left sum (windows_cs_logical_processors{instance="ServerName"}) / 100000

% consumption of the physical CPU ressources by the VMs (On all Hosts): (sum by (instance)(rate(windows_hyperv_vm_cpu_total_run_time{}[1m]))) / max by (instance)(windows_cs_logical_processors{}) / 100000

% consumption of the physical CPU ressources by the Hosts himself (On all Hosts): (sum by (instance)(rate(windows_hyperv_host_cpu_total_run_time{}[1m]))) / sum by (instance)(windows_cs_logical_processors{}) / 100000

PS: The 100000 is a constant, don’t ask me why…

Hey, I just tried this one (after your query @JDA88):

  1. sum(rate(windows_hyperv_vm_cpu_total_run_time{job="_windows_exporter_"}[2m])) by (instance) / sum(windows_cs_logical_processors{job="_windows_exporter_"}) by (instance) / 100000

It is is working okay.

The results are very different to:

  1. sum(rate(windows_cpu_time_total{job="_windows_exporter_",mode!~"idle"}[2m])) by (instance)) * (100 / 7.5)

I can’t test which one is more correct right now. But you can see the difference here:

screen (the first is 2., the second is 1.)

Good news, alter a lot of digging in the values I think I finally got the solution, I am able to compute those values:

  • Virtual CPU % utilisation (Total, Guest and Hypervisor space)
  • Physical CPU % utilisation (Total, Guest and Hypervisor space)

Short answer: we just need a new counter value added to hyperv.go, or is it available already? could not find it

Win32_PerfRawData_HvStats_HyperVHypervisorVirtualProcessor
Timestamp_PerfTime

UPDATE: Timestamp_PerfTime difference value is veri close to the number of second * 10000000. I’ll try to get it with the Uptime later today…

Long answer: You need all the following counters:

Timestamp_PerfTime #Not available, need to be added from HyperVHypervisorVirtualProcessor
PercentTotalRunTime #Available
PercentGuestRunTime #Available
PercentHypervisorRunTime #Available
CorePhysical #Available in windows_cs_logical_processors
CoreInVM #Can be computed?

Once you have all those values, those are the calculation that give you the final values. The “_D” stand for the difference between two ticks

VM_CPU_Usage_Total_Percent = (PercentTotalRunTime_D / Timestamp_PerfTime_D) / CoreInVM
VM_CPU_Usage_Guest_Percent = (PercentGuestRunTime_D / Timestamp_PerfTime_D) / CoreInVM
VM_CPU_Usage_Hyper_Percent = (PercentHypervisorRunTime_D / Timestamp_PerfTime_D) / CoreInVM
Physical_CPU_Usage_Total_Percent = (PercentTotalRunTime_D / Timestamp_PerfTime_D) / CorePhysical
Physical_CPU_Usage_Guest_Percent = (PercentGuestRunTime_D / Timestamp_PerfTime_D) / CorePhysical
Physical_CPU_Usage_Hyper_Percent = (PercentHypervisorRunTime_D / Timestamp_PerfTime_D) / CorePhysical

Here you quick PowerShell code that validate the calculation. It works on my Hyper-V

$CorePhysical = (Get-CimInstance Win32_ComputerSystem).NumberOfLogicalProcessors
$VL = @('Name','Timestamp_PerfTime','PercentTotalRunTime','PercentGuestRunTime','PercentHypervisorRunTime')

$OK = $False
While($True){
 $Tick2 = Get-WmiObject -Query ('SELECT {0} FROM Win32_PerfRawData_HvStats_HyperVHypervisorVirtualProcessor' -F ($VL -join ','))

 If($OK){
  #$CoreVirtual = $Tick1.Count - 1

  $Result = @{}
  ForEach($T1 in $Tick1){
   $Name = $T1.Name
   $T2 = $Tick2 | ? Name -eq $Name

   $Name = $T1.Name.Split(':')[0]

   If(!$Result.ContainsKey($Name)){
    $Result.Add($Name,(New-Object PSObject -Property ([Ordered]@{Name = $Name; Core = 0; vCPU_Total = 0; vCPU_Guest = 0; vCPU_Hyper = 0; pCPU_Total = 0; pCPU_Guest = 0;pCPU_Hyper = 0})))
   }
   
   $Timestamp_PerfTime_D       = $T2.Timestamp_PerfTime       - $T1.Timestamp_PerfTime
   $PercentTotalRunTime_D      = $T2.PercentTotalRunTime      - $T1.PercentTotalRunTime
   $PercentGuestRunTime_D      = $T2.PercentGuestRunTime      - $T1.PercentGuestRunTime
   $PercentHypervisorRunTime_D = $T2.PercentHypervisorRunTime - $T1.PercentHypervisorRunTime
  
   $Result[$Name].Core += 1
   $Result[$Name].vCPU_Total += [Math]::Round(($PercentTotalRunTime_D      / $Timestamp_PerfTime_D) * 100,2)
   $Result[$Name].vCPU_Guest += [Math]::Round(($PercentGuestRunTime_D      / $Timestamp_PerfTime_D) * 100,2)
   $Result[$Name].vCPU_Hyper += [Math]::Round(($PercentHypervisorRunTime_D / $Timestamp_PerfTime_D) * 100,2)
   $Result[$Name].pCPU_Total += [Math]::Round(($PercentTotalRunTime_D      / $Timestamp_PerfTime_D) * 100 / $CorePhysical,2) 
   $Result[$Name].pCPU_Guest += [Math]::Round(($PercentGuestRunTime_D      / $Timestamp_PerfTime_D) * 100 / $CorePhysical,2)
   $Result[$Name].pCPU_Hyper += [Math]::Round(($PercentHypervisorRunTime_D / $Timestamp_PerfTime_D) * 100 / $CorePhysical,2)
  }

  $Result.Values | Sort-Object Name | ft -AutoSize
 }
 $OK = $True
 Start-Sleep -Seconds 4
 $Tick1 = $Tick2
}

Hope this can help 😃