paper7Atualizado em 21/11/2026 · English
← Todos os papers

Curió-Edu 7B: Examining Data Selection Impacts in LLM Continued Pretraining

Thales Sales Almeida, Rodrigo Nogueira, Hélio Pedrini

Resumo

Continued pretraining extends a language model's capabilities by further exposing it to additional data, often tailored to a specific linguistic or domain context. This strategy has emerged as an efficient alternative to full retraining when adapting general-purpose models to new settings. In this work, we investigate this paradigm through Curió 7B, a 7-billion-parameter model derived from LLaMA-2 and trained on 100 billion Portuguese tokens from the ClassiCC-PT corpus - the most extensive Portuguese-specific continued-pretraining effort above the three-billion-parameter scale to date. Beyond …

De quem é

Outros papers das mesmas organizações