paper7Atualizado em 21/11/2026 · English
← Todos os papers

Sabiá: Portuguese Large Language Models

Ramon Pires, Hugo Abonizio, Thales Sales Almeida, Rodrigo Nogueira

Resumo

As the capabilities of language models continue to advance, it is conceivable that "one-size-fits-all" model will remain as the main paradigm. For instance, given the vast number of languages worldwide, many of which are low-resource, the prevalent practice is to pretrain a single model on multiple languages. In this paper, we add to the growing body of evidence that challenges this practice, demonstrating that monolingual pretraining on the target language significantly improves models already extensively trained on diverse corpora. More specifically, we further pretrain GPT-J and LLaMA model…

De quem é

Outros papers das mesmas organizações