Data preprocessing issues

[tjayada]

I'm trying to reproduce the results and ran into some trouble for the data preprocessing steps.

The first issue is the missing ego4d_gaze_untracked.csv file, which luckily can easily be found your older project in the GLC data directory, but should probably be added into this repository as well.

The second issue concerns the two missing audio files.
The two files you made available for download at https://drive.google.com/drive/folders/1iZuuRiflog9AazCtLXa9PbIYg-S3vENs are the original 1 hour audios and thus the code in preprocess.py in lines 259 & 260 with

command = f'cp {os.path.join(os.path.dirname(data_path), "missing_audio/*")} {save_path}' subprocess.call(command)

are simply copying the complete 1-hour audios into the directory with the already clipped audios.
Is there a script that you have used to clip the missing audios first as well, which didn't get committed or did you maybe accidentally uploaded the complete audios instead of the clipped versions?
Of course its also possible that I made an error and in that case please let me know as well.

[BolinLai]

Hi, thank you for pointing out the mistake. I'll add ego4d_gaze_untracked.csv in this repo later. In terms of the missing audio, I use ffmpeg to merge them with the silent videos first and then you can process them in the same way as other videos.

[tjayada]

Perfect, thank you for the answer

[Dinithipurna]

Hi,
I’m trying to reproduce the results from your paper and wanted to inquire about the pre-processing of the ARIA dataset. The gaze information seems to have a different format now, possibly using yaw and pitch rather than x and y positions. Could you clarify which dataset of ARIA was used in the paper? Did you convert the yaw and pitch data to x, y coordinates using the ARIA repo code? I’m asking to verify that I’m on the right track with my approach.

Thank you!