psutil: [MacOS] psutil.cpu_freq() broken on Apple M1

I see this wasn’t really fixed in #1895 – I’m still getting:

>>> psutil.__version__
5.9.0
>>> psutil.cpu_freq()
FileNotFoundError: [Errno 2] No such file or directory (originated from sysctl(HW_CPU_FREQ))

on M1 (MBP 2021, OS 12.2.1). Let me know how I could help with testing!

I’m also interested by thix fix, as on M1, without that, no way it seems to use BabyAGI 😃 (https://github.com/oliveirabruno01/babyagi-asi)

 ✘  py[babyagi-asi]  ~/c/babyagi-asi   main   python babyagi.py
Traceback (most recent call last):
  File "/Users/xxx/Code/babyagi-asi/babyagi.py", line 1, in <module>
    import openai, prompts, consts, pinecone, os, subprocess, tiktoken, json
  File "/Users/xxx/Code/babyagi-asi/prompts.py", line 9, in <module>
    I am running on a {platform.system()} {platform.architecture()[0]} system with {round(psutil.virtual_memory().total / (1024 ** 3), 2)} GB RAM and a {psutil.cpu_freq().current/1000 if psutil.cpu_freq() else "unknown"} GHz CPU. I am using OpenAI API. I must remember to use '|' instead of '&&' or '&' in my commands if using windows' cmd or pws.
  File "/Users/xxx/Library/Python/3.9/lib/python/site-packages/psutil/__init__.py", line 1864, in cpu_freq
    ret = _psplatform.cpu_freq()
  File "/Users/xxx/Library/Python/3.9/lib/python/site-packages/psutil/_psosx.py", line 179, in cpu_freq
    curr, min_, max_ = cext.cpu_freq()
FileNotFoundError: [Errno 2] No such file or directory (originated from sysctl(HW_CPU_FREQ))

However this works: in the requirements.txt file, replace the line with that dependency, e.g

psutil==5.9.4

with

git+https://github.com/snOm3ad/psutil.git@fix-cpu-freq-apple-silicon

then pip install ...

Same on 5.9.4 on an M1 Pro with Ventura 13.2.1

>>> psutil.__version__
'5.9.4'
>>> psutil.cpu_freq()
FileNotFoundError: [Errno 2] No such file or directory (originated from sysctl(HW_CPU_FREQ))

I just took a grand tour of almost every GitHub issue regarding this, and think I’ve cracked this nut.

This comment pointing to the clockrate.hz field is relevant, but the problem is what to multiply it by. The answer appears to be the time base frequency.

On M1s, sysctl hw.tbfrequency returns 24,000,000 which when multiplied by 100 Hz gives the expected 2.4 GHz. However, that sysctl is not reliable on Intel hardware: on my Intel chip I get 1,000,000,000.

This would obviously need testing by anyone with an M1 (or M2?) but I think the general logic should be:

• check if sysctl hw.cpufrequency returns. If so, use it.
• If not, multiply sysctl hw.tbfrequency by sysctl kern.clockrate (hz).

PR is up, could someone please review it? Thank you @shoenig and @dbwiddis both your examples where useful.

And here’s a Java/JNA based solution based on this Issue/thread: https://github.com/oshi/oshi/blob/e78a0ffb32d5bc2fe6943b903ed5eeb97f949fc5/oshi-core/src/main/java/oshi/hardware/platform/mac/MacCentralProcessor.java#L322-L357

For reference here’s a simple C version we use for Go https://github.com/shoenig/go-m1cpu/blob/main/cpu.go

I see this wasn’t really fixed in #1895 – I’m still getting:

>>> psutil.__version__
5.9.0
>>> psutil.cpu_freq()
FileNotFoundError: [Errno 2] No such file or directory (originated from sysctl(HW_CPU_FREQ))

on M1 (MBP 2021, OS 12.2.1). Let me know how I could help with testing!

Still facing this issue with M2 Max and psutil 5.9.4

>>> import psutil
>>> psutil.cpu_freq()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/john/miniconda3/envs/pytorch-m2/lib/python3.8/site-packages/psutil/__init__.py", line 1864, in cpu_freq
    ret = _psplatform.cpu_freq()
  File "/Users/john/miniconda3/envs/pytorch-m2/lib/python3.8/site-packages/psutil/_psosx.py", line 179, in cpu_freq
    curr, min_, max_ = cext.cpu_freq()
FileNotFoundError: [Errno 2] No such file or directory (originated from sysctl(HW_CPU_FREQ))

@dbwiddis no problem! 👍

@dbwiddis 2.4GHz is the timebase frequency, which is not the same as CPU frequency. Apple Silicon has different speeds per cluster. I.e an M1 truly has a PCPU Frequency of 3204 MHz, and a ECPU frequency of 2064 MHz.

M2 chips can have a single core in the PCPU boost to ~3500MHz and the entire ECPU is upped to 2424Mhz.

So if timebase freq is the true goal, then 2.4Ghz may be logical to hardcode.

Otherwise, if you want the real CPU freq, you’ll need to pull data from the voltage-states properties in the IORegistry PMGR. Specifically voltage-states1-sram and ``voltage-states5-sram`. (The data there is 32bit little endian and should result in an array. The biggest value is the CPU freq.)

Hopefully that’s helpful 👍

The problem is that on my virtualized macOS I get this:

HW_CPU_FREQ: 2590000000 KERN_CLOCKRATE -> clockinfo.hz: 100

…so it appears they are 2 different things.

Fixed by https://github.com/giampaolo/psutil/pull/1895.