ivadomed: convert_file_to_nifti bug for ADS SEM test data

Discussed in https://github.com/ivadomed/ivadomed/discussions/1310

^{Originally posted by chennyyuee January 25, 2024} Dear ivadomed team,

I’m trying to use ivadomed Two-class segmentation to train a model for use in AxonDeepSeg. But when I run ivadomed -c config_microscopy.json, using the downloaded tutorial dataset. I got the following error:

  File "C:\Users\ct630\ivadomed_env\lib\site-packages\ivadomed\loader\segmentation_pair.py", line 297, in read_file
    img = self.convert_file_to_nifti(filename, extension, is_gt)
  File "C:\Users\ct630\ivadomed_env\lib\site-packages\ivadomed\loader\segmentation_pair.py", line 360, in convert_file_to_nifti
    _img = np.sum(_img * (.299, .587, .114, 0), axis=-1)
ValueError: operands could not be broadcast together with shapes (744,1154,2) (4,)

Is there a way to solve this problem ?

Many thanks! chenyue

</div>

This user’s error is reproducible on my end.

It relates to a part of code that was added (possibly by @kanishk16?) in PR #1297,

https://github.com/ivadomed/ivadomed/blob/4c95d013680ccc3cc1d96a88422e023e6027bcb1/ivadomed/loader/segmentation_pair.py#L352-L360

I ran the debugger using the ADS SEM training data, and found the following for the case where there is an error:

(Pdb) _img.ndim
2
(Pdb) _img.ndim
2
(Pdb) colorspace_idx
2
(Pdb) props.is_batch
False
(Pdb) _img.shape[colorspace_idx]
*** IndexError: tuple index out of range
(Pdb) _img.shape
(954, 1280)

Looking at imageio’s documentation, it’s not clear what props.is_batch is being used for here in relation to setting colorspace_idx, @kanishk16 do you recall? The case is_batch=True I think would mean it’s a list/tuple of images, so what does that have to do with the colorspace?

If I understand the logic, for this image (954, 1280), the image is binary so _img.ndim <= colorspace_idx: # binary or gray should be triggered but isn’t being, because props.is_batch is false and then _img.ndim = colorspace_idx. So we possibly need a different colorspace_idx condition, and/or a different if/else condition.

Lastly, to avoid this in the future, we need to add test images & tests for each cases of this is/else chain, as this issue was missed due to the lack of test coverage of these lines:

About this issue

Original URL
State: open
Created 5 months ago
Reactions: 1
Comments: 16 (16 by maintainers)

Most upvoted comments

ValueError: operands could not be broadcast together with shapes (744,1154,2) (4,)

^from the user’s issue; maybe the only problem here is the missing case of greyscale images with an alpha channel, that one if definitely not covered in the current code, and it’s an odd edgecase

I’ll test it out locally

mathieuboudreau on Jan 29, 2024

@joshuacwnewton feel free to chip in with your suggestions… Debugging using the brute force solution: https://github.com/ivadomed/ivadomed/blob/4c95d013680ccc3cc1d96a88422e023e6027bcb1/ivadomed/loader/segmentation_pair.py#L322-L341

the training works as expected. Note: Idk how did I miss the typo in the above code… np.unint8 -> np.uint8

I don’t fully understand - what change did you make to get it running with image https://github.com/axondeepseg/data_axondeepseg_sem/blob/master/sub-rat6/micr/sub-rat6_sample-data15_SEM.png ?

Since this block was inside except just raised an error in the try to execute this block of code to remove my doubt regarding any potential issue with the img 😅

kanishk16 on Jan 29, 2024

Ok so that seems to be it - reverting to your original code, but adding one elif _img.shape[colorspace_idx]==2: #gray with alpha channel condition,
            # make grayscale (treats binary as 1-bit grayscale)
            colorspace_idx = 2 + int(props.is_batch)
            if _img.ndim <= colorspace_idx:  # binary or gray
                pass  # nothing to do
            elif _img.shape[colorspace_idx]==2: #gray with alpha channel
                    _img = _img[:, :, 0]
            elif _img.shape[colorspace_idx] == 3:  # RGB or gray with alpha
                    _img = np.sum(_img * (.299, .587, .114), axis=-1)
            else:  # RGBA
                # discards alpha
                _img = np.sum(_img * (.299, .587, .114, 0), axis=-1)
            if len(_img.shape) < 3:
                _img = np.expand_dims(_img, axis=-1)
resolves the issue on my end (training starts). So we just need a PR with that addition, but also adding that image to the test images that run during the CI. @kanishk16 do you agree?

Absolutely 🚀

kanishk16 on Jan 29, 2024

(Just a small heads-up: I got the ping on the previous issue, and have been reading through the replies here, and wanted to try my hand at proposing a solution to handle this issue. I hope that’s okay!)

joshuacwnewton on Jan 29, 2024

Playing around I set this,

            # make grayscale (treats binary as 1-bit grayscale)
            colorspace_idx = 3
            print(filename)
            if _img.ndim < colorspace_idx:  # binary or gray
                pass  # nothing to do
            elif len(_img.shape)==colorspace_idx:  # RGB
                _img = np.sum(_img * (.299, .587, .114), axis=-1)
            elif len(_img.shape)==colorspace_idx+1: # RGBA
                # discards alpha
                _img = np.sum(_img * (.299, .587, .114, 0), axis=-1)
            else:
                raise Exception("File dimensions exceed 4 (RGBA).")
            if len(_img.shape) < 3:
                _img = np.expand_dims(_img, axis=-1)

Which worked fine, up until one file ([data_axondeepseg_sem/sub-rat6/micr/sub-rat6_sample-data15_SEM.png] (https://github.com/axondeepseg/data_axondeepseg_sem/blob/master/sub-rat6/micr/sub-rat6_sample-data15_SEM.png)), which has a quirk in that it’s an a grayscale image but with an alpha channel, meaning it’s not covered by one of the above conditions (or the previous ones):

  File "/Users/mathieuboudreau/neuropoly/github/ivadomed/ivadomed/loader/segmentation_pair.py", line 359, in convert_file_to_nifti
    _img = np.sum(_img * (.299, .587, .114), axis=-1)
ValueError: operands could not be broadcast together with shapes (744,1154,2) (3,)

So we need a better way to handle the alpha cases

mathieuboudreau on Jan 29, 2024