pretrained_model

[CatchBeliF]

Sorry for bother you again!
After you give the initialization code, we try to train with our own data. When we run the demo.py with our own initialization model, we find this question:
size mismatch for mix_model: copying a param with shape torch.Size([48, 512, 41, 91]) from checkpoint, the shape in current model is torch.Size([46, 512, 23, 31]).
size mismatch for pointwiseinference.mix_model: copying a param with shape torch.Size([48, 512, 41, 91]) from checkpoint, the shape in current model is torch.Size([46, 512, 23, 31]).
terminate called without an active exception
We guess the pretrained_model best.pth we download from your github may not mathcing with our model . Can you provide the code to generate this pretrained_model.
Thank you very much!

[AdamKortylewski]

The issue is that your initialization model is not compatible with the pretrained model. You can either not load the pretrained model and just use the initialization model, or you can train your model with the train.py script.

[CatchBeliF]

Hi,
We run our own dataset with the initialization code you provided,however, we have encountered some problems.
We have some doubts in the code of clustering. Why is it that sometimes the result of clustering is 4 categories, sometimes less than 4 categories, is our data set not rich enough or another reason?
We are so sorry to bother you again!Look forward to your reply!

[YummyWaffle]

The issue is that your initialization model is not compatible with the pretrained model. You can either not load the pretrained model and just use the initialization model, or you can train your model with the train.py script.

I'm sorry to bother you. I meet this problem also. I run initialization code and train from scratch and the size of mix_models seems not right and the score size is [1,11] which should be [1,12]

[AdamKortylewski]

I will look at this, but it may take a bit. Thanks

[YummyWaffle]

I will look at this, but it may take a bit. Thanks

thank you for you beautiful work, looking forward to your reply

[AdamKortylewski]

I cannot reproduce this error. Are you training on the pascal3d+ data?

[YummyWaffle]

I cannot reproduce this error. Are you training on the pascal3d+ data?

I run the initialize code on the dataset provided in 'Download models'.

[AdamKortylewski]

Are you computing the similarity matrices yourself? When you learn the mixture models, have you looked at the images that visualize each mixture? Do they look similar as in the papers?

[YummyWaffle]

yes i compute the similarity matrices myself. do you mean the images in cluster_images_pool5_512? the cluster results are not quite satisfactory.

[AdamKortylewski]

Can you try using the similarity matrices we provide for download?

[YummyWaffle]

that one can work, but i am trying to use compositional net for different types of dataset, so i'am trying to run this program from the very beginning.

[gspell]

I have found myself having similar difficulty as @YummyWaffle. To be clear, I have tried to run the three pretraining/initialization scripts in succession -- vMF_clustering.py, comptSimMat.py, Learn_mix_model_vMF_view.py -- for the Pascal3D+ Vehicles dataset that you have kindly put together. After doing this, if I try to train a CompositionalNet from the yielded initialization, it seems that the mixture models initialization does not have the correct size/shape: some categories only have 3 mixtures instead of the M=4 that other categories have. My understanding is that this problem arises during the spectral clustering in Learn_mix_model_vMF_view.py, as the clustering does not yield 4 clusters for each category. It seems that one could try to simply lower the spectral split threshold parameter to force 4 clusters/category, or perhaps there are other factors affecting the number of clusters in the pretraining that I don't fully understand.

Would it be possible for you to tell us what some of the "pretraining parameters" were for obtaining the initialization? The candidates that I mostly see from your provided pretraining code are:
A) number of images per category for vMF clustering (set to 1000 in the provided code)
B) number of images per category for the similarity matrix computation (set to 5000 in the code)
C) The spectral split threshold for mixture model view clustering (set to 0.1 in the code)

I'm afraid that running the pretraining code with the provided settings doesn't quite yield an initialization that allows the training code to run, but I am very interested in your work and would love to ensure that I can replicate it as I try to incorporate CompositionalNets into my own work.

Thanks!
Greg

[AdamKortylewski]

Hi Greg,

I have not experienced this issue before, but it seems that you are not the only one experiencing this issue. For what categories does the model only have three mixtures? It could happen during the ML updates (starting line 186 in Learn_mix_model_vMF_view.py) that one of the clusters is not assigned any data. You could try to skip the ML updates and just use the model directly after the spectral clustering?

Best Adam