Learning-to-Rank with BERT in TF-Ranking

Topics: Learning-to-rank, Marc Najork, Ranking, Reranking

The paper introduces a machine learning algorithm for document ranking, employing BERT for query and document encoding and TF-Ranking (TFR) for further optimization. TF-Ranking, a TensorFlow-based library, specializes in ranking tasks, offering support for various models and integration with TensorFlow’s features. Challenges include effectively assessing document relevance, with traditional BERT approaches being less suitable than pairwise or listwise LTR algorithms. Solutions include TFR-BERT, a framework fine-tuning BERT for ranking within TF-Ranking, with experiments on the MS MARCO dataset demonstrating its efficacy. Learnings indicate TFR-BERT outperforms baselines, with integration of RoBERTa and ELECTRA and ensemble techniques further enhancing performance. The paper highlights the effectiveness of combining BERT representations with ranking losses and ensemble methods in LTR models.

• Shuguang Han, Xuanhui Wang, Michael Bendersky, Marc Najork
• TF-Ranking Team, Google Research, Mountain View, CA