kornia: AugmentationSequential does not return lists of boxes

Describe the bug

When passing boxes as lists of tensors with any BBOX datakey to AugmentationSequential the resulting augmentation is not returned. Thus, the returned outputs have one fewer element than the inputs that were passed to it. This happens for both forward and inverse.

Reproduction steps

import kornia.augmentation as K
import torch

img = torch.rand((3,3,10,10))
bbox = [torch.tensor([[1,5,2,7],[0,3,9,9]]),torch.tensor([[1,5,2,7],[0,3,9,9],[0,5,8,7]]), torch.empty((0,4))]
inputs = [img, bbox]

aug = K.AugmentationSequential(K.Resize((300,300)), data_keys=['input', 'bbox_xyxy'])

transformed = aug(*inputs)

assert len(transformed) == len(inputs)
Traceback (most recent call last):             
  File "/home/miquelmr/kornia/test.py", line 12, in <module>
    assert len(transformed) == len(inputs)
AssertionError  

Expected behavior

List of boxes is returned.

Environment

Collecting environment information...
PyTorch version: 1.13.0+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.6 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.27

Python version: 3.10.8 (main, Nov 24 2022, 14:13:03) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-4.15.0-196-generic-x86_64-with-glibc2.27
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: 
GPU 0: NVIDIA TITAN X (Pascal)
GPU 1: NVIDIA TITAN X (Pascal)
GPU 2: NVIDIA TITAN Xp

Nvidia driver version: 520.61.05
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.23.5
[pip3] torch==1.13.0
[pip3] torchvision==0.14.0
[conda] numpy                     1.23.5                   pypi_0    pypi
[conda] torch                     1.13.0                   pypi_0    pypi
[conda] torchvision               0.14.0                   pypi_0    pypi

Additional context

No response

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 16 (16 by maintainers)

Most upvoted comments

Btw, I think this is already being solved this on #2117

In #2064 we are thinking of updating to have just one bbox signature, maybe this should help, we can design the new API in a way to make it easier to work on augmentations

Right! let’s continue the conversation there then.

It seems that the Boxes wrapper works just fine when passing lists of tensors up to when it is converted back to a tensor or a list of tensors here, depending on what the input was:

https://github.com/kornia/kornia/blob/370099180c97167d066d94491eda46e7b9c8b939/kornia/augmentation/container/augment.py#L275

This seems to work fine too but now outputs contains a list of tensors (if that was the format passed to the input), which gets removed from the list of outputs in L279. It is this last removal that I don’t understand the purpose of and the implications that it would have to just not do that, as it would seem to be required to support returning lists of tensors.

I agree here. I think we shall not use any obvious wrapper at all. Say, the user input a list of tensors directly and get the corresponding list of tensors. If they input a wrapped Box object, they should get the corresponding Box object.

Additionally, we should use our own Box data type as the default communication type to pass around different methods.