CCL: Illegal Instruction on AMD CPUs

I was trying to use the 1.0.0 version of ccl on a centOS cluster. I’m using an anaconda environment with python 3. I installed cmake and swig from using conda and then installed pyccl from pip - all on the login node of this cluster. I can import pyccl in python without a problem on this node:

(base) nbhandar@coma Fisher (naren) λ conda activate ccl3
(ccl3) nbhandar@coma Fisher (naren) λ python
Python 3.7.4 (default, Aug 13 2019, 20:35:49)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyccl
>>> quit()
(ccl3) nbhandar@coma Fisher (naren) λ lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                16
On-line CPU(s) list:   0-15
Thread(s) per core:    1
Core(s) per socket:    8
CPU socket(s):         2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 45
Stepping:              7
CPU MHz:               1999.923
BogoMIPS:              3999.44
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              20480K
NUMA node0 CPU(s):     0,2,4,6,8,10,12,14
NUMA node1 CPU(s):     1,3,5,7,9,11,13,15

However on certain compute nodes, namely the ones running AMD cpus, I get an illegal instruction error when trying to import pyccl in python, and python crashes

(base) nbhandar@compute-2-6 Fisher (naren) λ conda activate ccl3
(ccl3) nbhandar@compute-2-6 Fisher (naren) λ python
Python 3.7.4 (default, Aug 13 2019, 20:35:49)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyccl
Illegal instruction
(ccl3) nbhandar@compute-2-6 Fisher (naren) λ lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                16
On-line CPU(s) list:   0-15
Thread(s) per core:    1
Core(s) per socket:    8
CPU socket(s):         2
NUMA node(s):          4
Vendor ID:             AuthenticAMD
CPU family:            16
Model:                 9
Stepping:              1
CPU MHz:               2400.143
BogoMIPS:              4800.46
Virtualization:        AMD-V
L1d cache:             64K
L1i cache:             64K
L2 cache:              512K
L3 cache:              5118K
NUMA node0 CPU(s):     0,2,4,6
NUMA node1 CPU(s):     8,10,12,14
NUMA node2 CPU(s):     9,11,13,15
NUMA node3 CPU(s):     1,3,5,7

On the compute nodes with Intel cpus, I do not get this error:

(base) nbhandar@compute-1-17 Fisher (naren) λ conda activate ccl3
(ccl3) nbhandar@compute-1-17 Fisher (naren) λ python
Python 3.7.4 (default, Aug 13 2019, 20:35:49)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyccl
>>> quit()
(ccl3) nbhandar@compute-1-17 Fisher (naren) λ lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                16
On-line CPU(s) list:   0-15
Thread(s) per core:    1
Core(s) per socket:    8
CPU socket(s):         2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 63
Stepping:              2
CPU MHz:               2400.043
BogoMIPS:              4799.33
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              20480K
NUMA node0 CPU(s):     0,2,4,6,8,10,12,14
NUMA node1 CPU(s):     1,3,5,7,9,11,13,15

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 24 (3 by maintainers)

Most upvoted comments

Interesting, thanks for letting me know! I’ll reach to my local IT folks to ask about the SSE instructions on the AMD nodes. In the meantime, it seems like installing pyccl on the AMD nodes seems to be the way to go to ensure that I can run it on any compute node of the cluster I’m on.

@beckermr I did a fresh install of pyccl in a new conda environment on one of the amd compute nodes I mentioned and it looks like I can use this pyccl install on any node (amd or intel) without getting the ‘illegal instruction’ error. Thanks for the tip!