First of all:
This is amazing work and I am having a lot of fun with it! I use it going back and forth with a MIDI editor and was able to get some unique arrangements that might even be called music :)
Now to the issue:
I noticed that especially >2000 tokens the continued sequences start to get quite chaotic sometimes. Lots of key changes, chord progression changes and total wild drumming. Even sequences on the drums that look like they should be for a normal instrument.
I am not sure if this is related to the inferencing (memory corruption?), to llm training or the llm structure. I use the tv2o-medium and my personal fine-tuned versions of it and run the inference on a gtx4060 mobile card.
First of all:
This is amazing work and I am having a lot of fun with it! I use it going back and forth with a MIDI editor and was able to get some unique arrangements that might even be called music :)
Now to the issue:
I noticed that especially >2000 tokens the continued sequences start to get quite chaotic sometimes. Lots of key changes, chord progression changes and total wild drumming. Even sequences on the drums that look like they should be for a normal instrument.
I am not sure if this is related to the inferencing (memory corruption?), to llm training or the llm structure. I use the tv2o-medium and my personal fine-tuned versions of it and run the inference on a gtx4060 mobile card.