speechbrain: [Bug]: Pretrain wav2vec: No such file or directory: 'wav2vec2-base/train.csv'`**

Describe the bug

I run sbatch command and I have the .py and .yaml files in the location: /home/flck/ but I have the audio data files in directory location: /l/users/flck/new_dataset/train/clean speechbrain generates a directory in location /home/flck/ because that is where I launch the sbatch command

as you can see I would like to pretrain on another dataset called in directory “new_dataset” not librispeech. The dataset consists of .flac files all in directory: /l/users/flck/new_dataset/train/clean

this is the exact command:

python /home/flck/sb_w2v_pretrain/train_sb_wav2vec2.py /home/flck/sb_w2v_pretrain/hparams/wav2vec2_base.yaml --find_unused_parameters --max_grad_norm 0.0

FileNotFoundError: [Errno 2] No such file or directory: 'wav2vec2-base/train.csv'

‘wav2vec2-base’ directory (that was generated) contains:

env.log
hyperparams.yaml
log.txt
save
train_sb_wav2vec2.py

where save directory is empty

Expected behaviour

pretraining wav2vec model on speechbrain dataset

To Reproduce

yaml file:

data_folder: /l/users/flck/peoples_speech_/train/clean output_folder: wav2vec2-base #/l/users/flck/peoples_speech_/train/wav2vec2-base save_folder: !ref <output_folder>/save #“./wav2vec2-pretrained-demo” Logging file for every N optimizer steps (many lines) train_steps_log: !ref <output_folder>/train_steps_log.txt Logging file per epoch train_stage_log: !ref <output_folder>/train_stage_log.txt

train_splits: [“train-clean-100”] dev_splits: [“dev-clean”] test_splits: [“test-clean”] train_csv: !ref <output_folder>/train.csv valid_csv: !ref <output_folder>/dev-clean.csv

Versions

transformers version: 4.29.2
Platform: Linux-5.4.204-ql-generic-12.0-19-x86_64-with-glibc2.31
Python version: 3.9.7
Huggingface_hub version: 0.14.1
Safetensors version: not installed
PyTorch version (GPU?): 1.11.0 (False)
Tensorflow version (GPU?): not installed (NA)
Flax version (CPU?/GPU?/TPU?): not installed (NA)
Jax version: not installed
JaxLib version: not installed
Using GPU in script?: <fill in>
Using distributed or parallel set-up in script?: <fill in>

Relevant log output

speechbrain.core - Beginning experiment!
speechbrain.core - Experiment folder: wav2vec2-base
speechbrain.core - Exception:
Traceback (most recent call last):
  File "/home/flck/sb_w2v_pretrain/train_sb_wav2vec2.py", line 394, in <module>
    main()
  File "/home/flck/sb_w2v_pretrain/train_sb_wav2vec2.py", line 374, in main
    train_dataset, valid_loader, train_loader_kwargs = dataio_prepare(hparams)
  File "/home/flck/sb_w2v_pretrain/train_sb_wav2vec2.py", line 258, in dataio_prepare
    train_data = sb.dataio.dataset.DynamicItemDataset.from_csv(
  File "/home/flck/.conda/envs/vcheckworthy/lib/python3.9/site-packages/speechbrain/dataio/dataset.py", line 365, in from_csv
    data = load_data_csv(csv_path, replacements)
  File "/home/flck/.conda/envs/vcheckworthy/lib/python3.9/site-packages/speechbrain/dataio/dataio.py", line 129, in load_data_csv
    with open(csv_path, newline="") as csvfile:
FileNotFoundError: [Errno 2] No such file or directory: 'wav2vec2-base/train.csv'

About this issue

Original URL
State: closed
Created a year ago
Comments: 16

Most upvoted comments

python3 …/speechbrain/recipes/LibriSpeech/self-supervised-learning/wav2vec2/train_sb_wav2vec2.py …/speechbrain/recipes/LibriSpeech/self-supervised-learning/wav2vec2/hparams/wav2vec2_base.yaml --data_folder=/local_disk/idyie/amoumen/LibriSpeech --train_splits=[“train-clean-100”]

Outputs: speechbrain.core - Beginning experiment! speechbrain.core - Experiment folder: wav2vec2-base librispeech_prepare - Data_preparation… librispeech_prepare - Creating csv lists in wav2vec2-base/train-clean-100.csv… 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▉| 28538/28539 [02:07<00:00, 223.41it/s] librispeech_prepare - wav2vec2-base/train-clean-100.csv successfully created! librispeech_prepare - Creating csv lists in wav2vec2-base/dev-clean.csv… 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▉| 2702/2703 [00:15<00:00, 173.67it/s] librispeech_prepare - wav2vec2-base/dev-clean.csv successfully created! librispeech_prepare - Creating csv lists in wav2vec2-base/test-clean.csv… 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▉| 2619/2620 [00:09<00:00, 284.91it/s] librispeech_prepare - wav2vec2-base/test-clean.csv successfully created! speechbrain.dataio.dataio - wav2vec2-base/train.csv is created. speechbrain.dataio.sampler - Batch quantisation in latent space speechbrain.dataio.sampler - DynamicBatchSampler: Generating dynamic batches speechbrain.core - Info: debug arg overridden by command line input to: False speechbrain.core - Info: debug_batches arg overridden by command line input to: 2 speechbrain.core - Info: debug_epochs arg overridden by command line input to: 2 speechbrain.core - Info: debug_persistently arg overridden by command line input to: False speechbrain.core - Info: device arg overridden by command line input to: cuda:0 speechbrain.core - Info: data_parallel_backend arg overridden by command line input to: False speechbrain.core - Info: distributed_launch arg overridden by command line input to: False speechbrain.core - Info: distributed_backend arg overridden by command line input to: nccl speechbrain.core - Info: find_unused_parameters arg overridden by command line input to: False speechbrain.core - Info: auto_mix_prec arg from hparam file is used speechbrain.core - Info: max_grad_norm arg from hparam file is used speechbrain.core - Info: optimizer_step_limit arg from hparam file is used speechbrain.core - Info: tqdm_colored_bar arg overridden by command line input to: False speechbrain.core - 90.9M trainable parameters in W2V2Brain speechbrain.utils.checkpoints - Would load a checkpoint here, but none found yet. speechbrain.utils.epoch_loop - Going into epoch 1

So it does works to only specify train-clean-100

Adel-Moumen on May 24, 2023

Please close this issue if your problem is solved (this is important for us as we have many issues to monitor)

Adel-Moumen on May 24, 2023

Yes, first follow this tutorial to grasp a better understanding of what we are doing https://colab.research.google.com/drive/1aFgzrUv3udM_gNJNUoLaHIm78QHtxdIz?usp=sharing then you will have to write your own data prepare file. You can start from the LibriSpeech Wav2vec finetuning one and then modify it to suits your needs.

Adel-Moumen on May 24, 2023

ok I will try on my side.

Adel-Moumen on May 24, 2023