paper7Updated Nov 21, 2026 · Português
← All papers

Sabiá: Portuguese Large Language Models

Ramon Pires, Hugo Abonizio, Thales Sales Almeida, Rodrigo Nogueira

Abstract

As the capabilities of language models continue to advance, it is conceivable that "one-size-fits-all" model will remain as the main paradigm. For instance, given the vast number of languages worldwide, many of which are low-resource, the prevalent practice is to pretrain a single model on multiple languages. In this paper, we add to the growing body of evidence that challenges this practice, demonstrating that monolingual pretraining on the target language significantly improves models already extensively trained on diverse corpora. More specifically, we further pretrain GPT-J and LLaMA model…

Who it is from

Other papers from the same organizations