albumentations: Incorrect number of masks returned after augmentation
🐛 Bug
Describe the bug Incorrect number of masks returned after augmentation. It happend when some boxes goes outside of image. Number of boxes changed but masks stays the same.
To Reproduce
from albumentations import *
from albumentations import __version__ as ver
transform_generator = Compose([
ShiftScaleRotate(p=1.0, shift_limit=0.1, scale_limit=0.2, rotate_limit=45, border_mode=cv2.BORDER_REFLECT),
], bbox_params={'format': 'pascal_voc',
'min_area': 0,
'min_visibility': 0.0,
'label_fields': ['labels']}, p=1.0)
def albu_bug():
print('Albu ver: {}'.format(ver))
image = np.zeros((500, 500, 3), dtype=np.uint8)
mask1 = np.zeros((500, 500, 3), dtype=np.uint8)
mask2 = np.zeros((500, 500, 3), dtype=np.uint8)
mask3 = np.zeros((500, 500, 3), dtype=np.uint8)
mask1[0:10, 0:10] = 255
mask2[-10:, -10:] = 255
mask3[250:260, 250:260] = 255
ann = dict()
ann['image'] = image.copy()
ann['masks'] = [mask1, mask2, mask3]
ann['labels'] = np.array([0, 0, 0])
ann['bboxes'] = np.array([[0, 0, 10, 10], [490, 490, 500, 500], [250, 250, 260, 260]])
for i in range(10):
augm = transform_generator(**ann)
print(len(ann['masks']), len(ann['labels']), len(ann['bboxes']))
print(len(augm['masks']), len(augm['labels']), len(augm['bboxes']))
if __name__ == '__main__':
albu_bug()
Expected behavior Masks corrsponding to removed boxes must be also removed
Environment
- Albumentations version (e.g., 0.1.8): 0.5.2
- Python version (e.g., 3.7): 3.7
- OS (e.g., Linux): Win10
- How you installed albumentations (
conda
,pip
, source): pip
About this issue
- Original URL
- State: open
- Created 3 years ago
- Reactions: 2
- Comments: 15 (6 by maintainers)
I face the same issue as @ZFTurbo: I have situations where the masks do not match the bboxes. The image has 25 annotations, bboxes and segmentations which are converted to masks using the COCO api (coco.annToMask(). Then the image and targets are passed through a few transforms, CenterCrop, Resize, Affine… The transformed dict only contains 9 boxes, but it still containes 25 masks. When I filter out masks which only contain zeros
masks = [i for i in masks if i.any()]
, most of the time this works. But sometimes a bbox is still there where its mask is gone after I filtered “empty” masks.I thought thats what DualTransforms for, applying the transforms to both image and target in the same way. But it seems that bbox and mask transforms might get inconsistent.
This doesn’t seem to work, I still get a reduced set of bounding boxes out.
e.g.
The check is here(?)
https://github.com/albumentations-team/albumentations/blob/87b1b7d009bcff12d9cc7a482c14cac0b1300ac8/albumentations/core/composition.py#L212-L213
which calls filter:
https://github.com/albumentations-team/albumentations/blob/87b1b7d009bcff12d9cc7a482c14cac0b1300ac8/albumentations/core/composition.py#L229
However, I think filtering is actually performed again in postprocessing, which is always called: https://github.com/albumentations-team/albumentations/blob/87b1b7d009bcff12d9cc7a482c14cac0b1300ac8/albumentations/core/composition.py#L216-L219
https://github.com/albumentations-team/albumentations/blob/87b1b7d009bcff12d9cc7a482c14cac0b1300ac8/albumentations/core/utils.py#L68-L76