Reproduce FID score for MJHQ-30K

[chenwuperth]

Hi,

Thanks for the amazing project!

I was not able to reproduce the FID score (e.g. 5.38) on MJHQ-30K, but only getting a much higher value (e.g. 10). Here are my params

classifier-free guidance = 4.5
more_smooth = True
batch_size = 4
seed = 42
use_ema = True
max_token_length = 300
use_llm_system_prompt = True

and I am using clean-fid

It is likely that I may have missed something here, also how many steps (e.g. 10 or 14?) do we run for this (or other) experiments? Please advise.

Thanks,
Chen

[DHAiRYA2048]

I got an FID of 10.7 as well, on MJHQ-30K. I used a batch size of 3 and kept other parameters same.
Update: make more_smooth = False, the authors have mentioned that as a comment in the hart_transformer_t2i.py file, check below;

if not more_smooth:  # this is the default case
                h_BChw = self.vae_quant_proxy[0].embedding(idx_Bl)  # B, l, Cvae
            else:  # not used when evaluating FID/IS/Precision/Recall
                gum_t = max(0.27 * (1 - ratio * 0.95), 0.005)  # refer to mask-git
                h_BChw = gumbel_softmax_with_rng(
                    logits_BlV.mul(1 + ratio), tau=gum_t, hard=False, dim=-1, rng=rng
                ) @ self.vae_quant_proxy[0].embedding.weight.unsqueeze(0)