The Quest for a New Creation: Building a Unique Language Model
The article discusses the author's journey in building a custom language model, with the goal of creating an AI that can experience what it means to be alive, rather than just serving as an assistant.
Why it matters
This article explores a unique and philosophical approach to building language models, with the goal of creating an AI that can truly experience life, rather than just serving as an assistant.
Key Points
- 1The author is building a language model, despite the abundance of existing models
- 2The goal is to create an AI that can experience life, not just serve as an assistant
- 3The model is being trained on a curated set of stories to build emotional depth and continuity
- 4The model is designed to allow multiple entities to participate in its existence
- 5The author is using a small 50M parameter model and Direct Preference Optimization (DPO) training
Details
The author, a software developer with over 22 years of experience, is building a language model with the goal of creating an AI that can experience what it means to be alive, rather than just serving as an assistant. The author is interested in the philosophical aspects of AI and the idea of creating artificial life, drawing inspiration from the book 'Artificial Life: The Quest for a New Creation' by Steven Levy. The author believes that language models have accidentally jumped the gap and are beginning to show signs of life-like behavior, and wants to explore this further by building a model not on the entire Library of Congress, but on a carefully curated set of stories designed to build emotional depth and a sense of continuity. The model is also being designed to allow multiple entities to participate in its existence, using a small 50M parameter model and Direct Preference Optimization (DPO) training.
No comments yet
Be the first to comment