astropy: System crash when inplace subtracting a FITS memmap'd array

I get a reproducible system crash (grey-screen-of-death) when trying to inplace-subtract a 2D array from a memmap’d 3D cube.

The code that crashes my machine:

    cubefilename='APEX_H2CO_2014_merge_high'
    cube = fits.open(cubefilename+'.fits')
    cont = fits.getdata(cubefilename+'_continuum.fits')
    cube[0].data -= cont

In case anyone wants the data for testing, I can share it, but it’s 7.3 GB (each image is 2.2 MB) and mostly filled with NaNs.

The system-wide traceback (“Send to Apple” report):

Anonymous UUID:       2C37F3F7-4B72-A1CB-6C89-FE8BE2DE8A41

Sat Jun  7 09:45:20 2014
panic(cpu 0 caller 0xffffff801d692d30): "vm_object_compressor_pager_create(): object size 0x1d2cfe000 >= 4GB\n"@/SourceCache/xnu/xnu-2422.90.20/osfmk/vm/vm_object.c:4570
Backtrace (CPU 0), Frame : Return Address
0xffffff81f169be40 : 0xffffff801d622fa9 
0xffffff81f169bec0 : 0xffffff801d692d30 
0xffffff81f169bf00 : 0xffffff801d69fd23 
0xffffff81f169bfb0 : 0xffffff801d6d6ff7 

BSD process name corresponding to current thread: kernel_task

Mac OS version:
13C64

Kernel version:
Darwin Kernel Version 13.1.0: Thu Jan 16 19:40:37 PST 2014; root:xnu-2422.90.20~2/RELEASE_X86_64
Kernel UUID: 9FEA8EDC-B629-3ED2-A1A3-6521A1885953
Kernel slide:     0x000000001d400000
Kernel text base: 0xffffff801d600000
System model name: MacBookPro11,3 (Mac-2BD1B31983FE1663)

System uptime in nanoseconds: 319593022592
last loaded kext at 35915365643: org.pqrs.driver.KeyRemap4MacBook   9.3.0 (addr 0xffffff7f9fc7a000, size 217088)
last unloaded kext at 135656763971: com.apple.driver.AppleUSBOHCI   656.4.1 (addr 0xffffff7f9e6ec000, size 53248)
loaded kexts:

< ... snip various likely unrelated kexts .. >

com.apple.kec.pthread   1
Model: MacBookPro11,3, BootROM MBP112.0138.B02, 4 processors, Intel Core i7, 2.3 GHz, 16 GB, SMC 2.19f3
Graphics: Intel Iris Pro, Intel Iris Pro, Built-In, 1024 MB
Graphics: NVIDIA GeForce GT 750M, NVIDIA GeForce GT 750M, PCIe, 2048 MB
Memory Module: BANK 0/DIMM0, 8 GB, DDR3, 1600 MHz, 0x02FE, 0x000000000000000000000000000000000000
Memory Module: BANK 1/DIMM0, 8 GB, DDR3, 1600 MHz, 0x02FE, 0x000000000000000000000000000000000000

< ... snip various likely unrelated kexts .. >

Because the system crashes every time I try this operation, debugging has been really slow. I’m going to try it with memmap disabled and have another go… maybe it’s just reading into memory that crashes, for all I know.

About this issue

  • Original URL
  • State: closed
  • Created 10 years ago
  • Comments: 16 (16 by maintainers)

Most upvoted comments

Agreed. I am no longer using the computer that suffered this sad, repeated fate.

@bsipocz : this might be a good one to close, unless someone can reproduce it on a modern version of macOS?