paper7Updated Nov 21, 2026 · Português
← All papers

ptt5-v2: A Closer Look at Continued Pretraining of T5 Models for the Portuguese Language

Marcos Piau, Roberto Lotufo, Rodrigo Nogueira

Abstract

Despite advancements in Natural Language Processing (NLP) and the growing availability of pretrained models, the English language remains the primary focus of model development. Continued pretraining on language-specific corpora provides a practical solution for adapting models to other languages. However, the impact of different pretraining settings on downstream tasks remains underexplored. This work introduces $\texttt{ptt5-v2}$, investigating the continued pretraining of T5 models for Portuguese. We first develop a baseline set of settings and pretrain models with sizes up to 3B parameters…

Who it is from

Other papers from the same organizations