Repository navigation
Opposing results when comparing to transformers interpret #21
Description
- ferret version: 0.4.0
- Python version: 3.9.2
- Operating System: Linux Debian
Description
When comparing your feature attribution scores of the explanation provided by Integrated Gradients (plain) with the ones by the transformers_interpret library (MultiLabelClassificationExplainer), I get significantly different results. For example, a token may have a high score of 0.5 with transformers_interpret, but is negatively attributed with ferret.
Why could that be ?
Of course, I tested this on the same conditions for both transformers_interpret and ferret (e.g.: pretrained local multi-label BertForSequenceClassification, bert-base-german-cased tokenizer, same sample)
What I Did
- transformers_interpret:
cls_explainer = MultiLabelClassificationExplainer(model, tokenizer, custom_labels=labels)
word_attrib = cls_explainer(<SAMPLE>)
pred = cls_explainer.predicted_class_name
print(word_attrib[pred])
- ferret:
bench = Benchmark(model, tokenizer)
score = bench.score(<SAMPLE>)
metr = bench.explain(sent, target=target)[4] ### IG (plain) ###
print(metr.scores)
Activity
g8a9 commentedon Mar 29, 2023
Hi, thank you for reaching out. Can you give me the version of transformer interpret you are using and the sentence you are explaining so that I can setup a quick colab?
EDIT: also, I notice you are specifying 0.8.0 as the ferret version, but that cannot be correct. Are you using the latest v0.4.1?
racia commentedon Mar 30, 2023
Hey, thank you for your fast answer! Yes I'm sorry, the version of ferret should have stated: 0.4.0 and transformers_interpret is 0.9.6. I talked to my co-worker about this and it seems this is the case for any sample, e.g.: "Röntgenologisch wird kein V.a. eine Krankheit gestellt." where "eine" receives a -0.04 score with ferret's plain IG & 0.48 with transformers_interpret
g8a9 commentedon Mar 30, 2023
Perfect! Can you also tell me which model checkpoint you are using? (if it's a private model, I can try a similar public one if you point me to it).
In the meantime, you could try to set the normalization false in the
explainmethod. By default, we apply L1 normalization across all tokens to make attribution scores comparable across different explainers.racia commentedon Apr 3, 2023
Hey,
please excuse the long wait and thanks very much for your tip regarding the normalization, however that unfortunately doesn't solve the deviation in scores.
I was now able to reproduce the case with the publicly available nlptown/bert-base-multilingual-uncased-sentiment from the huggingface hub (where the label "5 stars" corresponds to target=4)
transformers_interpret:
ferret:
While some attributions only have a small deviation of <0.02 (e.g.: '##s') when comparing both, others can have a difference in scoring of more than 0.4 attribution points.
Please note also, that unlike earlier the explainer here is IGxInp from ferret, since I learned this is the default version from transformers_interpret too.
Thanks a lot for looking into this and happy to hear from you!
g8a9 commentedon Apr 30, 2023
Hey, sorry for the long radio silence.
In your last code example, normalization is still happening, so that might be an issue. Anyway, another different factor I see is the baseline used for computing Integrated Gradients. We are currently using a [CLS] + token count * [PAD] + [SEP] scheme and that might be different than what transformer interpret is doing.
(The choice on how to construct the baseline is not well-agreed in the community, so they might be following another strategy)