Does your model support multi-image input?

[BIIIIU]

Hello!

I found that your code use id 151655 and 151653 to find the starting and ending position of the image. I wonder whether it is correct to handle situations when the user input multiple images?

Looking forward to your reply!

[ZichenWen1]

Our method supports multi-image input. However, I strongly recommend that you print out the input_ids to inspect the start and end IDs of the multi-image sequences. This will help ensure that you correctly reduce the number of visual tokens.

[BIIIIU]

        if self.config.text_config.DART_config is not None:
            # HACK
            image_token_start_index = inputs.input_ids.tolist()[0].index(151655)
            image_token_end_index = inputs.input_ids.tolist()[0].index(151653)
            image_token_length = image_token_end_index - image_token_start_index
            self.config.text_config.DART_config['image_token_start_index'] = image_token_start_index
            self.config.text_config.DART_config['image_token_length'] = image_token_length

https://github.com/ZichenWen1/DART/blob/5a19452fc173017191c247848673e0bde755695a/lmms-eval/lmms_eval/models/qwen2_5_vl.py#L332C13-L338C95
这是lmms-eval中应用了DART的qwen2.5-vl代码，这种方式在处理多图像时，image_token_start_index和image_token_end_index似乎都是第一张图片的位置。对于“ text”和“ text”这两个样本，输出的图像的起始与结束index是相同的。