InPars-v2: Large Language Models as Efficient Dataset Generators for Information Retrieval
Vitor Jeronymo, Luiz Bonifacio, Hugo Abonizio, Marzieh Fadaee, Roberto Lotufo, Jakub Zavrel, Rodrigo Nogueira
Abstract
Recently, InPars introduced a method to efficiently use large language models (LLMs) in information retrieval tasks: via few-shot examples, an LLM is induced to generate relevant queries for documents. These synthetic query-document pairs can then be used to train a retriever. However, InPars and, more recently, Promptagator, rely on proprietary LLMs such as GPT-3 and FLAN to generate such datasets. In this work we introduce InPars-v2, a dataset generator that uses open-source LLMs and existing powerful rerankers to select synthetic query-document pairs for training. A simple BM25 retrieval pi…
Who it is from
Maritaca AIautor: Hugo Abonizio (cofundador)
NeuralMindautor: Roberto Lotufo (cofundador e CTO)
Other papers from the same organizations
- PRISM: Transfer and Limits of Activation Interventions in Tool-Calling Language ModelsMaritaca AI
- Structural Segmentation for Brazilian Legal Document RetrievalNeuralMind
- Lumen-bench: Cross-Lingual Safety Gaps and the Say-Do Problem in Tool-Calling Large Language ModelsMaritaca AI
- Eleven Years of BRACIS: A Meta-Scientific Study of the Brazilian Conference on Intelligent SystemsMaritaca AI
- Legal Nugget Extraction for Granular Retrieval over Long Jurisprudential TextsNeuralMind
- BLUEX v2: Benchmarking LLMs on Open-Ended Questions from Brazilian University Entrance ExamsTropic-AI