macpine: Unable to create an instance

Unless, QEMU’s host CPU just passes through all CPU features without the need to explicitly set flags…?

I’m not sure if this is related but, https://manpages.ubuntu.com/manpages/kinetic/en/man7/qemu-cpu-models.7.html

Libvirt supports a third way to configure CPU models known as “Host model”. This uses the QEMU “Named model” feature, automatically picking a CPU model that is similar the host CPU, and then adding extra features to approximate the host model as closely as possible. This does not guarantee the CPU family, stepping, etc will precisely match the host CPU, as they would with “Host passthrough”, but gives much of the benefit of passthrough, while making live migration safe.

I couldn’t find qemu documentation regarding host cpu model, though some other resources also pointed out that by default, the host model tries to approximate the host cpu.

Also, when I tested without setting pdpe1gb flag, the VM was created with 1gb pages support.

This is really odd because according to QEMU docs on pdpe1gb

Not included by default in any Intel CPU model.
Should be explicitly turned on for all Intel CPU models.

sysctl could work. I’ll try this option with syscall.Sysctl from the standard library.

What about using sysctl -a to determine if the system supports 1 GB pages then setting the flag accordingly?

❯ sysctl -a | grep machdep.cpu.extfeatures
machdep.cpu.extfeatures: SYSCALL XD 1GBPAGE EM64T LAHF LZCNT PREFETCHW RDTSCP TSCI

Also, when I tested without setting pdpe1gb flag, the VM was created with 1gb pages support. I am not 100% sure, but what if qemu enables it by default if the device supports 1gb pages?

Host

❯ echo "#include <mach/vm_statistics.h>" | gcc -dM -E - | grep VM_FLAGS_SUPERPAGE_SIZE_
#define VM_FLAGS_SUPERPAGE_SIZE_2MB (SUPERPAGE_SIZE_2MB<<VM_FLAGS_SUPERPAGE_SHIFT)
#define VM_FLAGS_SUPERPAGE_SIZE_ANY (SUPERPAGE_SIZE_ANY << VM_FLAGS_SUPERPAGE_SHIFT)

VM

localhost:~# cat /proc/cpuinfo | grep -i pdpe1gb
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch pti fsgsbase bmi1 avx2 smep bmi2 erms rdseed adx clflushopt xsaveopt xgetbv1 arat
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch pti fsgsbase bmi1 avx2 smep bmi2 erms rdseed adx clflushopt xsaveopt xgetbv1 arat
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch pti fsgsbase bmi1 avx2 smep bmi2 erms rdseed adx clflushopt xsaveopt xgetbv1 arat
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch pti fsgsbase bmi1 avx2 smep bmi2 erms rdseed adx clflushopt xsaveopt xgetbv1 arat

I also tested with host,pdpe1gb=off previously, which did worked. But, 1gb pages were not enabled (see above table).

@maxzinkus looking into it now.

@themisir could you run the following on your host to check if it supports supper pages (mac’s version of huge pages)

echo "#include <mach/vm_statistics.h>" | gcc -dM -E - | grep VM_FLAGS_SUPERPAGE_SIZE_

Awesome! It looks like the flag format may be incorrect in macpine, and while the commit that fixed some x64 architectures (which didn’t support 1GB pages, I guess?) broke it for x64 machines which do support it.

@idroz when you get a chance, we should figure out when 1GB pages need to be disabled (if ever any more?) and use the =off format rather than the - format to achieve it.

Thanks again @themisir for a truly excellent bug report with the additional detail.

Same question for the other working combinations in the table?

I updated the table with regarding pdpe1gb support information for each configuration. Looks like unless explicitly disabled with pdpe1gb=off flag it’s enabled by default.

I played a bit with those options.

I tried reverting changes been made using those commits:

1. 76f05e46dabd872d10fa5bc3f2444f21eed4d785
2. 8129af32b58e9baa7129d7e4a7971a5976cb9818

It worked with both host,-pdpe1gb and host values, but not with the host,-pdpe1gb=on value which is the current state.

I also tried with the host,pdpe1gb=on value, which also worked.

Flags	Works?	pdpe1gb support
host,-pdpe1gb=on	No	N/A
host	Yes	Yes
host,pdpe1gb	Yes	Yes
host,pdpe1gb=on	Yes	Yes
host,pdpe1gb=off	Yes	No

In conclusion, I am not sure how this change may affect the current user base. Also, I am not that good with qemu stuff. So, I am leaving to you guys to choose what next needs to be done.