Token level task for transliteration

[TrigonaMinima]

As the title says, is there any way to add the evaluation script for the transliteration task. I am currently working on creation of a transliteration dataset and training a neural model on the extracted data and I wanted to use this framework, but it doesn't look like transliteration is a part of it. Any work happening in this direction?

[Genius1237]

If I am understanding you correctly, are you asking for a script to train a bert/transformer model for transliteration?

If that is correct, you are looking for a seq2seq model. The tasks here just use the Bert encoder followed by a single linear classifier for each task. If you're looking for a seq2seq model you could look at something like fairseq.

[ssitaram]

Hi Shivam, as Anirudh mentioned, there is no transliteration task in GLUECoS currently, and hence we do not provide an evaluation script for it. You are welcome to contribute the transliteration dataset to GLUECoS if it is in Hindi-English.

[TrigonaMinima]

Ok. I was talking about the transliteration task in GLUECoS specifically. I have the dataset for English-to-Hindi transliteration pairs and I have trained an encoder-decoder based transliteration model. I will finalize the things and then share the details.