albumentations: Incorrect number of masks returned after augmentation

🐛 Bug

Describe the bug Incorrect number of masks returned after augmentation. It happend when some boxes goes outside of image. Number of boxes changed but masks stays the same.

To Reproduce

from albumentations import *
from albumentations import __version__ as ver


transform_generator = Compose([
        ShiftScaleRotate(p=1.0, shift_limit=0.1, scale_limit=0.2, rotate_limit=45, border_mode=cv2.BORDER_REFLECT),
    ], bbox_params={'format': 'pascal_voc',
                    'min_area': 0,
                    'min_visibility': 0.0,
                    'label_fields': ['labels']}, p=1.0)


def albu_bug():
    print('Albu ver: {}'.format(ver))
    image = np.zeros((500, 500, 3), dtype=np.uint8)
    mask1 = np.zeros((500, 500, 3), dtype=np.uint8)
    mask2 = np.zeros((500, 500, 3), dtype=np.uint8)
    mask3 = np.zeros((500, 500, 3), dtype=np.uint8)
    mask1[0:10, 0:10] = 255
    mask2[-10:, -10:] = 255
    mask3[250:260, 250:260] = 255

    ann = dict()
    ann['image'] = image.copy()
    ann['masks'] = [mask1, mask2, mask3]
    ann['labels'] = np.array([0, 0, 0])
    ann['bboxes'] = np.array([[0, 0, 10, 10], [490, 490, 500, 500], [250, 250, 260, 260]])

    for i in range(10):
        augm = transform_generator(**ann)
        print(len(ann['masks']), len(ann['labels']), len(ann['bboxes']))
        print(len(augm['masks']), len(augm['labels']), len(augm['bboxes']))


if __name__ == '__main__':
    albu_bug()

Expected behavior Masks corrsponding to removed boxes must be also removed

Environment

  • Albumentations version (e.g., 0.1.8): 0.5.2
  • Python version (e.g., 3.7): 3.7
  • OS (e.g., Linux): Win10
  • How you installed albumentations (conda, pip, source): pip

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 2
  • Comments: 15 (6 by maintainers)

Most upvoted comments

I face the same issue as @ZFTurbo: I have situations where the masks do not match the bboxes. The image has 25 annotations, bboxes and segmentations which are converted to masks using the COCO api (coco.annToMask(). Then the image and targets are passed through a few transforms, CenterCrop, Resize, Affine… The transformed dict only contains 9 boxes, but it still containes 25 masks. When I filter out masks which only contain zeros masks = [i for i in masks if i.any()], most of the time this works. But sometimes a bbox is still there where its mask is gone after I filtered “empty” masks.

…than create some strange code to augment both bboxes and mask and check them after each transformation.

I thought thats what DualTransforms for, applying the transforms to both image and target in the same way. But it seems that bbox and mask transforms might get inconsistent.

This doesn’t seem to work, I still get a reduced set of bounding boxes out.

e.g.

transform = A.Compose([
    A.RandomCrop(width=1024, height=1024)
], bbox_params=A.BboxParams(format='pascal_voc', label_fields=['labels'], check_each_transform=False))
# succeeds
res = transform(image=image, masks=masks, bboxes=boxes, labels=labels)

# fails
assert len(res['masks']) == len(res['bboxes'])

The check is here(?)

https://github.com/albumentations-team/albumentations/blob/87b1b7d009bcff12d9cc7a482c14cac0b1300ac8/albumentations/core/composition.py#L212-L213

which calls filter:

https://github.com/albumentations-team/albumentations/blob/87b1b7d009bcff12d9cc7a482c14cac0b1300ac8/albumentations/core/composition.py#L229

However, I think filtering is actually performed again in postprocessing, which is always called: https://github.com/albumentations-team/albumentations/blob/87b1b7d009bcff12d9cc7a482c14cac0b1300ac8/albumentations/core/composition.py#L216-L219

https://github.com/albumentations-team/albumentations/blob/87b1b7d009bcff12d9cc7a482c14cac0b1300ac8/albumentations/core/utils.py#L68-L76