What is the gather index?

[abhidipbhattacharyya]

This is more of a question than issue.

In model.py at line number 330 gather index has been used to re-orient text and image embedding after concatenation. I am trying to understand what is this gather index conceptually. I am trying to find the same in the paper. From the paper it seems image and text features are concatenated. So it will be helpful to know what these gather index vectors represents and how to create them for a custom dataset.

Thanks.

[2292384454]

Hello, I have the same confusion as you. I don’t know what is the role of torch.gather() after torch.cat([txt_emb, img_emb], dim=1). Have you found the answer to this question?