paper7Updated Nov 21, 2026 · Português
← All papers

Comparing Knowledge Injection Methods for LLMs in a Low-Resource Regime

Hugo Abonizio, Thales Almeida, Roberto Lotufo, Rodrigo Nogueira

Abstract

Large language models (LLMs) often require vast amounts of text to effectively acquire new knowledge. While continuing pre-training on large corpora or employing retrieval-augmented generation (RAG) has proven successful, updating an LLM with only a few thousand or million tokens remains challenging. In this work, we investigate the task of injecting small, unstructured information into LLMs and its relation to the catastrophic forgetting phenomenon. We use a dataset of recent news -- ensuring no overlap with the model's pre-training data -- to evaluate the knowledge acquisition by probing the…

Who it is from

Other papers from the same organizations