whisperX: Running on short audio: KeyError: 'level_1'

Failed to align segment: no characters in this segment found in model dictionary, resorting to original... I was testing to align a audio file, but it didn’t worked and give above error. It was a plain English .wav file

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 22 (16 by maintainers)

Most upvoted comments

Ok - I believe this is happening as a result of there being only a single segment in an audio file. It works fine on longer files where whisper is returning multiple segments. When a single segment is returned, the expected indexes aren’t there.

I’m not familiar enough with pandas to understand how to resolve it right now, but if you take a single sentence audio file you may be able to reproduce.

Failed to align segment ("."): no characters in this segment found in model dictionary, resorting to original...
/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/whisperx/alignment.py:294: FutureWarning: Not prepending group keys to the result index of transform-like apply. In the future, the group keys will be included in the index, regardless of whether the applied function returns a like-indexed object.
To preserve the previous behavior, use
>>> .groupby(..., group_keys=False)
To adopt the future behavior and silence this warning, use
>>> .groupby(..., group_keys=True)
char_segments_arr = per_seg_grp.apply(lambda x: x.reset_index(drop = True)).reset_index()
Traceback (most recent call last):
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 3802, in get_loc
return self._engine.get_loc(casted_key)
File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 165, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 5745, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 5753, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'level_1'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/cog/server/worker.py", line 209, in _predict
result = self._predictor.predict(**payload)
File "predict.py", line 126, in predict
return self.generate_aligned_transcription_whisperx(transcript, audio_path)
File "predict.py", line 45, in generate_aligned_transcription_whisperx
return whisperx.align(val['segments'],
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/whisperx/alignment.py", line 384, in align
cseg['segment-text-start'] = cseg['level_1']
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/pandas/core/frame.py", line 3807, in __getitem__
indexer = self.columns.get_loc(key)
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 3804, in get_loc
raise KeyError(key) from err
KeyError: 'level_1'