Exploring the Challenge of Distilling Oneself into Training Data

The author attempts to create a model of themselves by extracting training data from their own conversation logs and behavioral records, but encounters philosophical and practical challenges in the process.

Details

The author describes their attempt to create a model of themselves by extracting training data from their own conversation logs, behavioral records, and thinking traces. They encountered several challenges in this process. First, they struggled to define what constitutes a 'good' training sample, as the author's tone and reasoning often mattered more than the specific content of their responses. They categorized the samples into dialogue style, principle-behavior pairs, and insight-explanation pairs, with the latter two categories taking much more time to produce. Second, the author grappled with whether to include both good and bad responses, or to only keep the 'best' version of themselves, realizing this was akin to writing a memoir and shaping one's narrative. Finally, the author recognized that the process of reviewing and organizing the training data had already changed their self-awareness and behavior, suggesting the process may be more valuable than the final model. The author is left uncertain about whether they want a model that behaves more consistently, or if the ongoing process of distilling themselves is the true value of the project.