Project for NLP course. The goal of the project is to build a model capable of generating lyrics for disco polo songs. We were inspired by various examples of such work. We plan to collect the data from Tekstowo.pl by scraping most popular Disco Polo bands. Then we want to use pretrained language model by finetuning it on our dataset. In this way we want to learn the model to generate Disco Polo. It this won’t be enough we plan to enhance the process of generating songs. We expect many songs to be bad, so we want to create a model capable of scoring how good the generated song is, and based on this score present only songs with highest obtained score.