Question about distillation

[heishuini]

Regarding the distillation scheme based on trajectory consistency mentioned in the paper, for this type of scheme, how does its performance on T2I-based SR tasks compare with a restoration model trained from scratch? Have such experiments been explored? Thanks for your reply!

[cswry]

Thanks for your question. We chose trajectory consistency distillation because it is simple, effective, and GPU-memory-friendly.

We also tried TwinFlow-style memory-efficient distribution matching distillation. Although it improved generation performance, the results appeared slightly over-generated and the training time was noticeably longer. Therefore, considering both quality and efficiency, we chose RCGM.

We have not yet explored VSD, DMD, or image-/latent-GAN-based methods. We plan to release our training & testing code and model weights, and welcome the community to try these directions and share the results.