nni: Doubts about NAS+TypeError: tuple is not a Module subclass

Describe the issue: I am using your framework to perform a NAS experiment on my use case. While using LayerChoice I get the below error. TypeError: tuple is not a Module subclass

Apart from the above issue, I have a few more questions:

1. If I understood correctly, right now with one-shot NAS you only support LayerChoice and InputChoice, right? In my use case I would like to use input choice and also repeat. Is it possible to do that? I started to use DARTS for it because it is faster. Would you recommend switching to another NAS approach to use other mutation options?

2. If I understood correctly, these are layerwise approaches. Would it be possible to extend it to a blockwise approach as well?

Environment*: Remote Server

• NNI version: 2.6
• Training service (local|remote|pai|aml|etc):
• Client OS:Ubuntu 20.04.3 LTS
• Server OS (for remote mode only):Ubuntu 20.04.3 LTS
• Python version:Python 3.8.12
• PyTorch/TensorFlow version: Toch 1.10.0+cu111
• Is conda/virtualenv/venv used?: conda
• Is running in Docker?: No

Configuration:

• Experiment config (remember to remove secrets!): one-shot DARTS

  self.pointwise=LayerChoice([
               (nn.Conv1d(
                in_channels, out_channels, kernel_size=1, stride=1,
                dilation=dilation, bias=bias, padding=0
            ),
            nn.Conv1d(
               in_channels, out_channels, kernel_size=2, stride=1,
                dilation=dilation, bias=bias, padding=0
            ),
              nn.Identity()

            )],label='point')
        else:
            self.conv = Conv1d(
                in_channels, out_channels, kernel_size=kernel_size,
                stride=stride, padding=padding, dilation=dilation, bias=bias
            )

    def forward(self, x):
        if self.separable:
            x = self.depthwise(x)
            x = self.pointwise(x)
        else:
            x = self.conv(x)
        return x

ERROR:

File "mutatemodel.py", line 174, in __init__
    self.pointwise=LayerChoice([
  File "nni/nni/retiarii/nn/pytorch/api.py", line 95, in __init__
    self.add_module(str(i), module)
  File "python3.8/site-packages/torch/nn/modules/module.py", line 377, in add_module
    raise TypeError("{} is not a Module subclass".format(
TypeError: tuple is not a Module subclass

I look forward to your reply.

Thanks!