Billions of Parameters Are Worth More Than In-domain Training Data: A case study in the Legal Case Entailment Task
Guilherme Moraes Rosa, Luiz Bonifacio, Vitor Jeronymo, Hugo Abonizio, Roberto Lotufo, Rodrigo Nogueira
Abstract
Recent work has shown that language models scaled to billions of parameters, such as GPT-3, perform remarkably well in zero-shot and few-shot scenarios. In this work, we experiment with zero-shot models in the legal case entailment task of the COLIEE 2022 competition. Our experiments show that scaling the number of parameters in a language model improves the F1 score of our previous zero-shot result by more than 6 points, suggesting that stronger zero-shot capability may be a characteristic of larger models, at least for this task. Our 3B-parameter zero-shot model outperforms all models, inclu…
Who it is from
Maritaca AIautor: Hugo Abonizio (cofundador)
NeuralMindautor: Roberto Lotufo (cofundador e CTO)
Other papers from the same organizations
- PRISM: Transfer and Limits of Activation Interventions in Tool-Calling Language ModelsMaritaca AI
- Structural Segmentation for Brazilian Legal Document RetrievalNeuralMind
- Lumen-bench: Cross-Lingual Safety Gaps and the Say-Do Problem in Tool-Calling Large Language ModelsMaritaca AI
- Eleven Years of BRACIS: A Meta-Scientific Study of the Brazilian Conference on Intelligent SystemsMaritaca AI
- Legal Nugget Extraction for Granular Retrieval over Long Jurisprudential TextsNeuralMind
- BLUEX v2: Benchmarking LLMs on Open-Ended Questions from Brazilian University Entrance ExamsTropic-AI