question about prompt in training

[cassiaaaaaa]

Thank you for the awesome work!
I have some interest to finetune on your model. But met problems about prompts. Are you following the old InternVL2 template to use [UNUSED_TOKEN_146] and [UNUSED_TOKEN_145] in the start and end of a question or an answer?

     dict(role='HUMAN',
          begin='[UNUSED_TOKEN_146]user\n', end='[UNUSED_TOKEN_145]\n'),
     dict(role='BOT', begin='[UNUSED_TOKEN_146]assistant\n',
          end='[UNUSED_TOKEN_145]\n', generate=True),

[ByungKwanLee]

Yes correct

[cassiaaaaaa]

Yes correct

Thanks a lot for answering my questions. I noticed the difference of prompt construction in the evaluation code and inference code. Then in training, when making the labels, should the [UNUSED_TOKEN_145] and [UNUSED_TOKEN_146] be marked or they take part in the loss computing?

for example :

AI assistant should give helpful and detailed answers to user after fully understanding an image. [UNUSED_TOKEN_146]user
Does the drink appear to have milk as an ingredient?
Answer the question using a single word or phrase.[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]assistant
No[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]user
Is the blender pitcher clear?[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]assistant
Yes[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]user
What brand is the blender?[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]assistant
Philips[UNUSED_TOKEN_145]

Is my format right? And is the characters in bold fond rightly the unmasked prompt?

[ByungKwanLee]

Definitely correct

[cassiaaaaaa]

Definitely correct

Thank you very much