paper7Updated Nov 21, 2026 · Português
← All papers

Curió-Edu 7B: Examining Data Selection Impacts in LLM Continued Pretraining

Thales Sales Almeida, Rodrigo Nogueira, Hélio Pedrini

Abstract

Continued pretraining extends a language model's capabilities by further exposing it to additional data, often tailored to a specific linguistic or domain context. This strategy has emerged as an efficient alternative to full retraining when adapting general-purpose models to new settings. In this work, we investigate this paradigm through Curió 7B, a 7-billion-parameter model derived from LLaMA-2 and trained on 100 billion Portuguese tokens from the ClassiCC-PT corpus - the most extensive Portuguese-specific continued-pretraining effort above the three-billion-parameter scale to date. Beyond …

Who it is from

Other papers from the same organizations