astropy: System crash when inplace subtracting a FITS memmap'd array
I get a reproducible system crash (grey-screen-of-death) when trying to inplace-subtract a 2D array from a memmap’d 3D cube.
The code that crashes my machine:
cubefilename='APEX_H2CO_2014_merge_high'
cube = fits.open(cubefilename+'.fits')
cont = fits.getdata(cubefilename+'_continuum.fits')
cube[0].data -= cont
In case anyone wants the data for testing, I can share it, but it’s 7.3 GB (each image is 2.2 MB) and mostly filled with NaNs.
The system-wide traceback (“Send to Apple” report):
Anonymous UUID: 2C37F3F7-4B72-A1CB-6C89-FE8BE2DE8A41
Sat Jun 7 09:45:20 2014
panic(cpu 0 caller 0xffffff801d692d30): "vm_object_compressor_pager_create(): object size 0x1d2cfe000 >= 4GB\n"@/SourceCache/xnu/xnu-2422.90.20/osfmk/vm/vm_object.c:4570
Backtrace (CPU 0), Frame : Return Address
0xffffff81f169be40 : 0xffffff801d622fa9
0xffffff81f169bec0 : 0xffffff801d692d30
0xffffff81f169bf00 : 0xffffff801d69fd23
0xffffff81f169bfb0 : 0xffffff801d6d6ff7
BSD process name corresponding to current thread: kernel_task
Mac OS version:
13C64
Kernel version:
Darwin Kernel Version 13.1.0: Thu Jan 16 19:40:37 PST 2014; root:xnu-2422.90.20~2/RELEASE_X86_64
Kernel UUID: 9FEA8EDC-B629-3ED2-A1A3-6521A1885953
Kernel slide: 0x000000001d400000
Kernel text base: 0xffffff801d600000
System model name: MacBookPro11,3 (Mac-2BD1B31983FE1663)
System uptime in nanoseconds: 319593022592
last loaded kext at 35915365643: org.pqrs.driver.KeyRemap4MacBook 9.3.0 (addr 0xffffff7f9fc7a000, size 217088)
last unloaded kext at 135656763971: com.apple.driver.AppleUSBOHCI 656.4.1 (addr 0xffffff7f9e6ec000, size 53248)
loaded kexts:
< ... snip various likely unrelated kexts .. >
com.apple.kec.pthread 1
Model: MacBookPro11,3, BootROM MBP112.0138.B02, 4 processors, Intel Core i7, 2.3 GHz, 16 GB, SMC 2.19f3
Graphics: Intel Iris Pro, Intel Iris Pro, Built-In, 1024 MB
Graphics: NVIDIA GeForce GT 750M, NVIDIA GeForce GT 750M, PCIe, 2048 MB
Memory Module: BANK 0/DIMM0, 8 GB, DDR3, 1600 MHz, 0x02FE, 0x000000000000000000000000000000000000
Memory Module: BANK 1/DIMM0, 8 GB, DDR3, 1600 MHz, 0x02FE, 0x000000000000000000000000000000000000
< ... snip various likely unrelated kexts .. >
Because the system crashes every time I try this operation, debugging has been really slow. I’m going to try it with memmap disabled and have another go… maybe it’s just reading into memory that crashes, for all I know.
About this issue
- Original URL
- State: closed
- Created 10 years ago
- Comments: 16 (16 by maintainers)
Agreed. I am no longer using the computer that suffered this sad, repeated fate.
@bsipocz : this might be a good one to close, unless someone can reproduce it on a modern version of macOS?