IndexError: tuple index out of range on spec_augment_pytorch

[iskorini]

Any idea on how to resolve this problem?

from SpecAugment import spec_augment_pytorch as spec_augment_l
import librosa

y, sr = librosa.load(dataset_audio_path.joinpath(audio_path), sr=16000)
y, _ = librosa.effects.trim(y, top_db=15)
S = librosa.feature.melspectrogram(y, sr=sr, n_fft=2048, hop_length=512, n_mels=128)
S = librosa.power_to_db(S, ref=np.max)
S = spec_augment_l.spec_augment(mel_spectrogram=S)

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-10-9e6d8eca1293> in <module>
     14     plt.pause(0.001)  # pause a bit so that plots are updated
     15 
---> 16 inputs, classes = next(iter(train_dataloader))
     17 out = make_grid(inputs)
     18 class_names = train_dataset.classes

~\anaconda3\envs\torch\lib\site-packages\torch\utils\data\dataloader.py in __next__(self)
    343 
    344     def __next__(self):
--> 345         data = self._next_data()
    346         self._num_yielded += 1
    347         if self._dataset_kind == _DatasetKind.Iterable and \

~\anaconda3\envs\torch\lib\site-packages\torch\utils\data\dataloader.py in _next_data(self)
    383     def _next_data(self):
    384         index = self._next_index()  # may raise StopIteration
--> 385         data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
    386         if self._pin_memory:
    387             data = _utils.pin_memory.pin_memory(data)

~\anaconda3\envs\torch\lib\site-packages\torch\utils\data\_utils\fetch.py in fetch(self, possibly_batched_index)
     42     def fetch(self, possibly_batched_index):
     43         if self.auto_collation:
---> 44             data = [self.dataset[idx] for idx in possibly_batched_index]
     45         else:
     46             data = self.dataset[possibly_batched_index]

~\anaconda3\envs\torch\lib\site-packages\torch\utils\data\_utils\fetch.py in <listcomp>(.0)
     42     def fetch(self, possibly_batched_index):
     43         if self.auto_collation:
---> 44             data = [self.dataset[idx] for idx in possibly_batched_index]
     45         else:
     46             data = self.dataset[possibly_batched_index]

~\anaconda3\envs\torch\lib\site-packages\torchvision\datasets\folder.py in __getitem__(self, index)
    133         """
    134         path, target = self.samples[index]
--> 135         sample = self.loader(path)
    136         if self.transform is not None:
    137             sample = self.transform(sample)

<ipython-input-9-9aca4872e8ea> in spec_augment_loader(audio_path)
     40     S = librosa.feature.melspectrogram(y, sr=sr, n_fft=2048, hop_length=512, n_mels=128)
     41     S = librosa.power_to_db(S, ref=np.max)
---> 42     S = spec_augment_l.spec_augment(mel_spectrogram=S)
     43     fig = plt.Figure(figsize=figsize, dpi=dpi, frameon=False)
     44     fig.patch.set_visible(False)

~\anaconda3\envs\torch\lib\site-packages\SpecAugment\spec_augment_pytorch.py in spec_augment(mel_spectrogram, time_warping_para, frequency_masking_para, time_masking_para, frequency_mask_num, time_mask_num)
     87     """
     88     v = mel_spectrogram.shape[1]
---> 89     tau = mel_spectrogram.shape[2]
     90 
     91     # Step 1 : Time warping

IndexError: tuple index out of range

[DWhettam]

Encountered the same issue myself. It's expecting a 3d input [Batch x H x W], so if you input a single data point [H x W], then you will get out of range errors. The solution is to use a batch of inputs, or you can do a quick fix by adding an additional dimension to the input.

Something like this at the top of the spec_augment() function should work:

if mel_spectrogram.ndim == 2:
        mel_spectrogram = mel_spectrogram[np.newaxis, :]

[Aditya3107]

Hi any one solve this issue ? I have encountered the same.

[DWhettam]

@Aditya3107 yes - see my comment above

[Aditya3107]

@Aditya3107 yes - see my comment above

@DWhettam Hi, yes I applied your solution but still no success. I faced below problem.

`(venv) mds-student@mdsstudent-HP-Compaq-Elite-8300-CMT:~/Documents/SpecAugment-master$ python3
Python 3.8.5 (default, Aug 5 2020, 08:36:46)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.

import librosa
from SpecAugment import spec_augment_pytorch
audio, sampling_rate = librosa.load('data/61-70968-0002.wav')
mel_spectrogram = librosa.feature.melspectrogram(y = audio, sr = sampling_rate, n_mels = 256, hop_length = 128, fmax = 8000)
warped_masked_spectrogram = spec_augment_tensorflow.spec_augment(mel_spectrogram = mel_spectrogram)
Traceback (most recent call last):
File "", line 1, in
NameError: name 'spec_augment_tensorflow' is not defined
warped_masked_spectrogram = spec_augment_pytorch.spec_augment(mel_spectrogram = mel_spectrogram)
Traceback (most recent call last):
File "", line 1, in
File "/home/mds-student/Documents/SpecAugment-master/SpecAugment/spec_augment_pytorch.py", line 97, in spec_augment
warped_mel_spectrogram = time_warp(mel_spectrogram, W=time_warping_para)
File "/home/mds-student/Documents/SpecAugment-master/SpecAugment/spec_augment_pytorch.py", line 63, in time_warp
warped_spectro, dense_flows = sparse_image_warp(spec, src_pts, dest_pts)
File "/home/mds-student/.local/lib/python3.8/site-packages/specAugment/sparse_image_warp_pytorch.py", line 138, in sparse_image_warp
image = ops.convert_to_tensor(image)
NameError: name 'ops' is not defined`

If you can suggest some solution it would be helpful.

[DWhettam]

you are importing spec_augment_pytorch, but calling spec_augment_tensorflow. That's why you are getting the error:

NameError: name 'spec_augment_tensorflow' is not defined

if you are using pytorch you need to use spec_augment_pytorch

[Aditya3107]

@DWhettam thank you for the response. But I have taken pytorch only. For the particular code snippet shown above first I took tensorflow mistakenly then I corrected it with pytorch.

warped_masked_spectrogram = spec_augment_pytorch.spec_augment(mel_spectrogram = mel_spectrogram) Traceback (most recent call last): File "", line 1, in File "/home/mds-student/Documents/SpecAugment-master/SpecAugment/spec_augment_pytorch.py", line 97, in spec_augment warped_mel_spectrogram = time_warp(mel_spectrogram, W=time_warping_para) File "/home/mds-student/Documents/SpecAugment-master/SpecAugment/spec_augment_pytorch.py", line 63, in time_warp warped_spectro, dense_flows = sparse_image_warp(spec, src_pts, dest_pts) File "/home/mds-student/.local/lib/python3.8/site-packages/specAugment/sparse_image_warp_pytorch.py", line 138, in sparse_image_warp image = ops.convert_to_tensor(image) NameError: name 'ops' is not defined

[DWhettam]

It looks like you are still using the incorrect files. Read through your error message and you can see the error occurs here:
line 138, in sparse_image_warp image = ops.convert_to_tensor(image)
however, image = ops.convert_to_tensor(image) is not in the original sparse_image_warp_pytorch.py file. Have you modified the files in some way, or changed their names? I would re-download the repo and start fresh.

[mikeydevelops]

Hi, you can use the example in the tests/ folder, there it converts the spectrogram to [batch_size, time, frequency, 1]

[Aditya3107]

Hi, thanks for reply. It is working for me.