Support qwen-image-edit seriers or flux2.klien series in DPO and offline DPO

[rex-29]

Hi thanks for your great work! I found the dpo training now is based on t2i task, is there any plan to support image edit task?(i2i)

===============================================
感谢大佬开源，有计划后续支持图像编辑任务（qwen，flux2）的dpo训练吗🙏

[Jayce-Ping]

Technically, all combinations of models and algorithms are supported. The examples provided are simply the configurations I have personally verified through experimentation. Given my limited time and compute resources, I rely on the community to help explore more feasible configurations, reward functions, and algorithms.

Regarding the Qwen/Flux.2 + DPO setup, I haven't had the chance to validate it yet. If anyone in the community has a verified config, please feel free to open a PR. For more details, check the examples README.

[MengHao666]

Technically, all combinations of models and algorithms are supported. The examples provided are simply the configurations I have personally verified through experimentation. Given my limited time and compute resources, I rely on the community to help explore more feasible configurations, reward functions, and algorithms.

Regarding the Qwen/Flux.2 + DPO setup, I haven't had the chance to validate it yet. If anyone in the community has a verified config, please feel free to open a PR. For more details, check the examples README.

Would u consider to support offline dpo? @Jayce-Ping
It seems current implementation only supports online dpo, that is the chosen and reject one are defined by the best and worst reward respectively of entire roll-out samples.
the original diffusion-dpo supports for offline dpo in which there are many pairs of offline chose and reject samples.

[Jayce-Ping]

I've been considering it for a while and haven't have time to implement it. It should be easy, just skip the rollout stage and fetch data directly from the dataloader. I will do it later, if there is no one openning a PR for it.

[MengHao666]

I've been considering it for a while and haven't have time to implement it. It should be easy, just skip the rollout stage and fetch data directly from the dataloader. I will do it later, if there is no one openning a PR for it.

Thanks, I am testing the supported algorithms of Flow-Factory with a close-sourced model.
I find it is very convenient for comparing different algorithms, just like the Lllam-Factory.
Thanks for your great work again!