does remove CLS_token means features before the classification layer ?

[mathshangw]

if I need to extract the features before the classification layer, will I need to remove CLS_token from the Cross-covariance attention block as it has Global aggregation with class attention that has CLS_token, does that right please?

[woctezuma]

If xcit works as dino, then you can use the [CLS] token as your feature.

Reference: facebookresearch/dino#72 (comment)