paper7Atualizado em 21/11/2026 · English
← Todos os papers

Comparing Knowledge Injection Methods for LLMs in a Low-Resource Regime

Hugo Abonizio, Thales Almeida, Roberto Lotufo, Rodrigo Nogueira

Resumo

Large language models (LLMs) often require vast amounts of text to effectively acquire new knowledge. While continuing pre-training on large corpora or employing retrieval-augmented generation (RAG) has proven successful, updating an LLM with only a few thousand or million tokens remains challenging. In this work, we investigate the task of injecting small, unstructured information into LLMs and its relation to the catastrophic forgetting phenomenon. We use a dataset of recent news -- ensuring no overlap with the model's pre-training data -- to evaluate the knowledge acquisition by probing the…

De quem é

Outros papers das mesmas organizações