Implement contrastive loss function.

[tlatkowski]

Implement contrastive loss function according to paper:
http://yann.lecun.com/exdb/publis/pdf/hadsell-chopra-lecun-06.pdf

[datistiquo]

Isn't it already implemented? In layers.losses there is a contrastive loss function.

But I tried, but the loss is always 0. So, indeed it seems not working, although you can hand over the loss function a an argument?

What is actually the constrastive? Since there is also contrastive_lecun.

In your function contrastive you only hand over prediction. How is from this postive and negative examples generated?

I suppose this lecun_contrastive is similiar to triplet loss and to be a contrastive loss und need postive and negative examples.

[datistiquo]

Hey @tlatkowski . Do you have an idea how I can use the loss other than MSE loss functions? I suppose I need to change the data set reader?

[tlatkowski]

Hi @datistiquo
in current version of pipeline you have to modify code for the specific model, for example, for multihead model you have to modify the line where constructor from the base class is called:

multihead-siamese-nets/models/multihead_attention.py

Line 12 in d6f2e67

BaseSiameseNet.__init__(self, max_sequence_len, vocabulary_size, main_cfg, model_cfg, mse)

just change mse to contrastive for example

you also need to import it from losses module.

In the next version i will make a change in pipeline so you will be able to change loss from the main config file.

[datistiquo]

Hi @tlatkowski

I already tried this, but I get for the loss zero.

I think you also need to change the input dataset to get pairs for the postive and negative examples at least for the contrastive_lecun function?

Why should the data set structure work for the contrastive function at all? You just handover the predction as argument. I dont see any operation where the postive and negative samples are extracted?