libs: Unexpected number of processors (on musl build)

Agree with Leo! Thank you very much Ian, you digged into this relentlessly and provided lots of information to everyone else! Great job!

I agree that as first we have document this. Even if it’s an edge case, I would write a note in the official documentation.

Also, I want to thank you’ll for the awesome job in debugging this issue. It’s really impressive.

I’ve got a path forward now. I’ll leave it to you to determine what, if anything, you want to do here. My suggestion is that, at a minimum, a known issue is documented for this so anyone else who runs into it in the future don’t waste their time. It’s an edge case to be sure, but one we know about now.

I fully agree. /cc @leogr any idea?

OK, so this is wild, check this out.

I found this article about a potential bug with musl where the codepath for _SC_NPROCESSORS_CONF actually pulls data for ONLN instead. http://www.landley.net/notes-2022.html#26-07-2022

I created this sample code to test:

#include <stdlib.h>
#include <stdio.h>
#include <sys/sysinfo.h>
#include <unistd.h>

int main(int argc, char *argv[])
{
	printf("get_nprocs_conf(): %d\nget_nprocs(): %d\nsysconf(_SC_NPROCESSORS_CONF): %ld\nsysconf(_SC_NPROCESSORS_ONLN): %ld\n",
	get_nprocs_conf(), get_nprocs(), sysconf(_SC_NPROCESSORS_CONF), sysconf(_SC_NPROCESSORS_ONLN));
	exit(EXIT_SUCCESS);
}

When I use gcc to compile and run, I get this:

get_nprocs_conf(): 8
get_nprocs(): 6
sysconf(_SC_NPROCESSORS_CONF): 8
sysconf(_SC_NPROCESSORS_ONLN): 6

When I use musl-gcc to compile and run, I get this:

get_nprocs_conf(): 6
get_nprocs(): 6
sysconf(_SC_NPROCESSORS_CONF): 6
sysconf(_SC_NPROCESSORS_ONLN): 6

You can see the issue here, where the case for JT_NPROCESSORS_CONF simply flows into the case for JT_NPROCESSORS_ONLN: https://git.musl-libc.org/cgit/musl/tree/src/conf/sysconf.c#n202

Are you using musl? If so, then I think the solution here would be to gather this information by looking at /sys/devices/system/cpu/cpuX.

That #721 was rolled into 0.33.1. Or at least it was supposed to have been. https://falco.org/blog/falco-0-33-1/

I haven’t tried it with kmod, just bpf. I’m using only bpf in my environment.