Question about the ModelNet40 dataset

[wangchaodi1209]

Hello, I am currently reproducing your paper and would like to ask where you downloaded the ModelNet40 dataset. I found that the dataset downloaded from the repository provided in the paper has a format that does not match the format used in your test JSON file. In the paper, the dataset format is
├── ModelNet40
│ ├── airplane
│ └── bathtub
│ └── bed
│ └── ……
│ └── xbox
whereas in your JSON file, the format is
├── ModelNet40
│ ├──8192_npy
│ │ ├── …………_8192.npy

[kaykobad]

Hello, any update on this? Can you please share the pre-processed dataset so that we can reproduce the results?

Thanks

[kaykobad]

@wangchaodi1209 did you find the solution?

CC: @carboncoo

[wangchaodi1209]

Hello, any update on this? Can you please share the pre-processed dataset so that we can reproduce the results?

Yes, I have located the dataset in question. It is indeed the ObjAverse dataset, which you can find on Hugging Face at the following address:https://huggingface.co/datasets/RunsenXu/PointLLM/tree/main

[kaykobad]

Thank you @wangchaodi1209
I see the modelnet40 is in dat format. How do we map the objects with ids? Did you preprocess the dataset for this repo? Can you please share the steps or the dataset?

[wangchaodi1209]

Thank you @wangchaodi1209 I see the modelnet40 is in dat format. How do we map the objects with ids? Did you preprocess the dataset for this repo? Can you please share the steps or the dataset?

If you proceed directly with testing using their checkpoints, you need not concern yourself with the ModelNet40 dataset's dat files. Simply follow the steps outlined in the repository at PointLLM. Download the two objaverse_660K files from the Hugging Face URL I previously provided, merge them into a single tar.gz file, then extract it. The extracted file will be named ModelNet40.

I hope this proves helpful to you.

[kaykobad]

I appreciate the help. Thank you very much.

[kaykobad]

Hey @wangchaodi1209
Thank you very much for the response. I was able to run the code on MCUB dataset. Do you have any idea about how did they evaluate on Objaverse and ModelNet40 dataset for the Type-I and Type-C thing on Table 2? Basically, how to evaluate on PointCloud Language tasks?

[wangchaodi1209]

Hey @wangchaodi1209 Thank you very much for the response. I was able to run the code on MCUB dataset. Do you have any idea about how did they evaluate on Objaverse and ModelNet40 dataset for the Type-I and Type-C thing on Table 2? Basically, how to evaluate on PointCloud Language tasks?

I apologize, but since the Objaverse and ModelNet40 datasets require GPT's API for evaluation scoring, and I haven't purchased that API, I did not use these two datasets for evaluation. Additionally, did you evaluate the AVQA dataset? Could you provide me with a website or other link where I can download this dataset?

[kaykobad]

You can get the Audios and Videos of AVQA from this link: Huggingface Repo

[wangchaodi1209]

You can get the Audios and Videos of AVQA from this link: Huggingface Repo

Thank you very much. I noticed there is a file named eval_point_tasks_gpt.py in the directory /scripts/model_composition/test within the repository. This file contains the code for evaluating the two datasets mentioned above. You can try using it, but please note that this method requires access to the GPT API.

[kaykobad]

Thank you very much for pointing that out. I will try and let you know an update if I can do it.

[wangchaodi1209]

Thank you very much for pointing that out. I will try and let you know an update if I can do it.

Hello, following your instructions, I went to Hugging Face and downloaded the four files from the repository: AVQA_audios.zip, AVQA_videos.zip.001, AVQA_videos.zip.01, and AVQA_videos.zip.02. I then extracted them. However, when I extracted the specific filenames from the /modelcompose/data/test/avqa-test_mm_answer.json file for retrieval, I found only four files matched. Did you process this file at the time? Could you share how you did it?

[kaykobad]

Yes, you are right. I verified just now. Only 19 match for my case.

--- Verification Summary ---
Total audio file references checked: 16894
Found files: 19
Missing unique files: 16844

--- Verification Summary ---
Total video file references checked: 16894
Found files: 19
Missing unique files: 16844

My guess is these are training files. We need test/validation files. I do not know where to get them. If you find them, please let me know.

[wangchaodi1209]

是的，你是对的。我刚才验证了。只有 19 个与我的情况匹配。

--- Verification Summary ---
Total audio file references checked: 16894
Found files: 19
Missing unique files: 16844

--- Verification Summary ---
Total video file references checked: 16894
Found files: 19
Missing unique files: 16844

我的猜测是这些是培训文件。我们需要测试/验证文件。我不知道从哪里得到它们。如果您找到它们，请告诉我。

Currently, the only information I have is that they provide a CSV file for YouTube downloads in their published repository here. However, due to network issues, I am unable to access this data.

[kaykobad]

YouTube blocks after a certain time. I was unable to download so far.

I found another repo https://huggingface.co/datasets/gijs/avqa-processed
Can you try that? It has val split