Bag of Decisions Reranker
Summary
The article introduces a bag-of-decisions reranking approach for search relevance, where an LLM generates a rubric of yes/no questions to evaluate query relevance and a Jev-like reranker scores the results. It presents experimental results comparing BM25, Jev reranking, and the new bag-of-decisions method on WANDS and ESCI datasets, showing improved mean and median NDCG. The piece also discusses potential extensions and hints at an upcoming AI-powered retrieval course.