Controlled depth ablation of a BERT bi-encoder across training budgets and seeds on three BEIR tasks (nfcorpus, scifact, fiqa). L3–L12 is flat within seed noise at 20K steps; 80K training degrades every depth on zero-shot transfer (−45% NDCG@10 on fiqa for L12).
nlp information-retrieval pytorch embeddings reproducibility bert neural-ir ablation-studies sentence-transformers zero-shot-retrieval bi-encoder dense-retrieval beir ms-marco depth-ablation
-
Updated
Apr 24, 2026 - Python