Engram: A New Type of AI with Agentic Reasoning

The article discusses a new AI model called Engram, which aims to address the hallucination problem in large language models (LLMs) by incorporating agentic reasoning and a vector database during training.

Details

Engram is a novel AI model that the author is developing to improve upon the shortcomings of existing LLMs, particularly their tendency to hallucinate or generate nonsensical outputs. The key ideas behind Engram include using a vector database during training and allowing for agentic reasoning at the point of training, where the model can apply this reasoning between its internal layers. The article provides a deep dive into how Engram's reasoning engine works, likening it to a panel of four judges evaluating an input text. Each attention layer in the model acts like a judge, looking for different patterns, relationships, and themes, and passing its observations to the next layer. This allows Engram to build a more sophisticated understanding of the input compared to a traditional LLM. Another important aspect is how Engram represents words as points in a 96-dimensional space, where similar words are clustered together. This spatial representation enables the attention mechanism to selectively focus on the most relevant parts of the input context when making predictions, rather than just looking at the most recent words.