Modular Architecture for Continual Learning in Large Language Models

The article discusses a novel approach to training large language models (LLMs) that can continuously learn new skills without forgetting their base instincts. The key idea is to use a modular architecture with specialized transformer modules that can be added over time.

Details

The article explores a concept for building large language models (LLMs) that can continually learn new skills and capabilities over time, without the risk of forgetting their base instincts. The author notes that current LLMs, even large 50M models, struggle with this challenge as fine-tuning on new tasks tends to wipe out the model's previously learned knowledge and skills. The proposed solution is a modular architecture where the model is grown over time by adding specialized transformer modules. Each module is trained separately on a specific task or dataset, and uses LoRA adapter layers to fine-tune without disrupting the rest of the model. This allows the LLM to continuously expand its capabilities by incorporating new modules, rather than having to reboot the entire model from scratch. The author is currently experimenting with this approach, starting by training a 50M model on the TinyStories dataset to bootstrap basic language understanding skills. Future modules will then be added to handle conversational abilities, memory/reasoning, and other advanced capabilities.