Machine learning models as a differentiable search index for directly predicting resource retrieval results
Topics: AI (Deep Learning), Indexing, Probably in use, Ranking, User Signals
This Google patent describes a novel approach to information retrieval using machine learning models called Differentiable Search Index (DSI). The system directly predicts relevant resources in response to queries by encoding all corpus information within the model’s parameters, eliminating the need for traditional index structures. The innovation uses a single machine learning model (typically a Transformer architecture) that maps queries directly to resource identifiers, significantly simplifying the retrieval process and improving efficiency.