audio: Exporting the operator stft to ONNX opset version 9 is not supported.

Hi, I try exporting the process of feature extraction to onnx:

import torch
import torchaudio

model = torchaudio.transforms.MelSpectrogram()
x = torch.randn(1, 16000)
torch.onnx.export(model, x, 'tmp.onnx', input_names=['input'], output_names=['output'])

and get:

RuntimeError: Exporting the operator stft to ONNX opset version 9 is not supported. Please open a bug to request ONNX export support for the missing operator.

So will torchaudio add supports operators used in torchaudio.transforms module in the future? You see that exporting the process of feature extraction and the neural network together will be very convenient.

Thanks!

About this issue

Original URL
State: open
Created 4 years ago
Reactions: 1
Comments: 19 (4 by maintainers)

Commits related to this issue

Merge pull request #982 from yzs981130/fix_batch_size_comment Closes #771. Fix vague description on batch size calculation in imagenet. — committed to mpc001/audio by msaroufim 2 years ago

Most upvoted comments

While we wait for native PyTorch integration, folks can checkout https://github.com/adobe-research/convmelspec which will implement (Mel)spectrograms via 1D conv layer to both ONNX and CoreML (and as a second option CoreML MIL ops).

njb on Nov 22, 2022

PR is finally merged. https://github.com/onnx/onnx/pull/3741

averkij on May 24, 2022

Hi, is there any news on this issue? This feature is going to be important for the SpeechBrain project. Some of our contributors are working on exporting our speech processing pipelines into a microcontroller. This requires the model to be ONNX exportable/importable (@fpaissan). The main issue is that all the feature extraction relies on fft and stft.

mravanelli on Sep 10, 2022

@chenjiasheng There is actually some good news around : pytorch/pytorch#65666 and onnx/onnx#3741

NiziL on May 18, 2022

While we wait for native PyTorch integration, folks can checkout https://github.com/adobe-research/convmelspec which will implement (Mel)spectrograms via 1D conv layer to both ONNX and CoreML (and as a second option CoreML MIL ops).

very cool solution, you guys should publish it to pypi!

Xavier-i on Nov 28, 2022

These signal processing operators (STFT/FFT/IFFT) are supported in opset 17.
I was trying to export the torch module to ONNX, but I was thrown an error. Does torch need to support this even after MR is being merged in ONNX ?

import torch
import torch.nn as nn

class FeaturizeTorchFFT(nn.Module):

    def __init__(self):
        super().__init__()

    def forward(self, x):
        return torch.stft(
            input=x,
            n_fft=320,
            hop_length=160,
            win_length=320,
            window=torch.ones(320),
            center=False,
            pad_mode="reflect",
            normalized=False,
            onesided=True,
        )

self.featurizer_model = FeaturizeTorchFFT()
torch.onnx.export(
            self.featurizer_model, 
            self.sample_input, 
            # 're_onnx_model.onnx',
            str(self.output_path),
            export_params=True,
            opset_version=17,
            do_constant_folding=True,
            input_names=["x"],
            output_names=["output"],
            dynamic_axes={
                "x": {0: "batch_size", 1: "audio_length"},
            },
        )

  File "/home/kp/miniconda3/envs/gamd6-kp2/lib/python3.8/site-packages/torch/onnx/symbolic_helper.py", line 853, in _set_opset_version
    raise ValueError("Unsupported ONNX opset version: " + str(opset_version))
ValueError: Unsupported ONNX opset version: 17

stonelazy on Jun 24, 2022