cellxgene: [BUG] Cellxpose returns 400 Bas request for some requests if data is 16-bit
Describe the bug
Try saving test file after converting to fp16:
adata.X = adata.X.astype('float16')
Server normally starts when given this data (but prints some warnigns about conversion from 64bits).
However when you select gene or run differential expression, 400 appears.
Version affected: 0.17.0, within docker
In my case cellxgene first downloads file from the cloud, so 16bit format for storage is very reasonable - almost halves the space.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 17 (11 by maintainers)
Commits related to this issue
- Fix float16 support [#2379] Convert to float32 on startup unless backed, in which case error. scipy does not support complex slicing from float16 data so this is the easiest fix for now. — committed to chanzuckerberg/cellxgene by blrnw3 2 years ago
- Fix float16 support [#2379] (#2483) * Fix float16 support [#2379] Convert to float32 on startup unless backed, in which case error. scipy does not support complex slicing from float16 data so this... — committed to chanzuckerberg/cellxgene by blrnw3 2 years ago
Fixed in 1.0.1
Suggest a cast in diffex code if the set1/set2 slices are fp16, as that is where the dependency lives. Pretty sure the rest of the code base does not have this constraint.
As it is already sliced out of the main matrix, you will retain most of the memory savings (ie, the cast to 32 bit will be only for the duration of the t-test compute)
@bkmartinjr can you please check this once again? I was about to transition datasets to fp16, but I’m still seeing same issues (400 on diffexp, and expression of genes can’t be loaded). I’ve tried both csc and csr formats for X, but neither worked for me