For now we tokenize dataset before training. We could tokenize and pad sequences by batch, using just tokenizer.
For now we tokenize dataset before training.
We could tokenize and pad sequences by batch, using just tokenizer.