Improving AI Agent Memory with a Four-Signal Scoring System

The article discusses the limitations of the standard Retrieval Augmented Generation (RAG) approach for AI agent memory and presents a solution called NLM (Neural Long Memory) that adds three additional signals to the scoring system.

Details

The author has been building an AI project called Pulses, where AI personalities need long-term memory across conversations. They found that the standard RAG approach, which relies solely on cosine similarity, suffers from three key issues: temporal blindness (unable to prioritize recent updates over old information), frequency blindness (unable to distinguish frequently accessed memories), and importance blindness (unable to prefer specific, factual memories over vague ones). To address these limitations, the author developed a library called NLM that adds three additional signals to the scoring system: time decay, frequency score, and importance score. The time decay factor uses an exponential function with a 90-day half-life to prioritize recent memories. The frequency score is log-normalized to prevent one very popular memory from dominating. The importance score is computed automatically using a CPU heuristic or a zero-shot classifier. Benchmark results on a dataset of 100 memories and 30 queries show that NLM outperforms RAG by a significant margin, with a 37% improvement in overall top-1 accuracy.