Question about Pre-normalization

[NEUMUKI]

Hi there,
I read you paper, notice that you mention the RMSNorm normalizing function is used in your code, but I can't find any class's forward is called which has the RMSNorm normalizing function in generation. Could you tell me where to use it? is it traning only?

[liuhuadai]

Hi, please refer to https://github.com/Text-to-Audio/AudioLCM/blob/master/ldm/modules/diffusionmodules/flag_large_dit.py.

[TheSweetestGirlInTheUniverse]

Hey, may I ask if you used TxtFlagLargeDiT when training the teacher model?[at ldm/modules/diffusionmodules/flag_large_dit.py]
I couldn’t find any reference to it in the YAML or any other files.
Did I miss something?

[NEUMUKI]

same question