Chat History & Memory

Learn how Neuron AI manage multi turn conversations.

Neuron AI has a built-in system to manage the memory of a chat session you perform with the agent.

In many Q&A applications you can have a back-and-forth conversation with the LLM, meaning the application needs some sort of "memory" of past questions and answers, and some logic for incorporating those into its current thinking.

For example, if you ask a follow-up question like "Can you elaborate on the second point?", this cannot be understood without the context of the previous message. Therefore we can't effectively perform retrieval with a question like this.

In the example below you can see how the Agent doesn't know my name initially:

$response = \NeuronAI\Agent::make()
    ->chat(
        new UserMessage("What's my name?")
    );
    
echo $response->getContent();
// I'm sorry I don't know your name. Do you want to tell me more about yourself?

Clearly the Agent doesn't have any context about me. Now I try present me in the first message, and then ask for my name:

$response = \NeuronAI\Agent::make()
    ->chat(
        new UserMessage("Hi, my name is Valerio!")
    );
    
echo $response->getContent();
// Hi Valerio, nice to meet you, how can I help you today?

$response = \NeuronAI\Agent::make()
    ->chat(
        new UserMessage("Do you remember my name?")
    );

echo $response->getContent();
// Sure, your name is Valerio!

How Chat History works

Neuron Agents take the list of messages exchanged between your application and the LLM into an object called Chat History. It's a crucial part of the framework because the chat history needs to be managed based on the context window of the underlying LLM.

It's important to send past messages back to LLM to keep the context of the conversation, but if the list of messages grows enough to exceed the context window of the model the request will be rejected by the AI provider.

Chat history automatically truncates the list of messages to never exceed the context window avoiding unexpected errors.

How to feed a previous conversation

Sometimes you already have a representation of user to assistant conversation and you need a way to feed the agent with previous messages.

You just need to pass an array of messages to the `chat()` method. This conversation will be automatically loaded into the agent memory and you can continue to iterate on it.

$response = MyAgent::make()
    ->chat([
        new Message("user", "Hi, I work for a company called Inspector.dev"),
        new Message("assistant", "Hi Valerio, how can I assist you today?"),
        new Message("user", "What's the name of the company I work for?"),
    ]);
    
echo $response->getContent();
// You work for Inspector.dev

The last message in the list will be considered the most recent.

How to register a chat history

use NeuronAI\Agent;
use NeuronAI\Providers\AIProviderInterface;
use NeuronAI\Chat\History\InMemoryChatHistory;

class MyAgent extends Agent
{
    protected function provider(): AIProviderInterface
    {
        ...
    }
    
    protected function chatHistory()
    {
        return new InMemoryChatHistory(
            contextWindow: 50000
        );
    }
}

InMemoryChatHistory is used into the agent by default. Check out below to learn more

Available Chat History Implementations

• InMemoryChatHistory
• FileChatHistory

InMemoryChatHistory

It simply store the list of messages into an array. It is kept in memory only during the current execution.

use NeuronAI\Agent;
use NeuronAI\Providers\AIProviderInterface;
use NeuronAI\Chat\History\InMemoryChatHistory;

class MyAgent extends Agent
{
    protected function provider(): AIProviderInterface
    {
        ...
    }
    
    protected function chatHistory()
    {
        return new InMemoryChatHistory(
            contextWindow: 50000
        );
    }
}

FileChatHistory

This implementation makes you able to persist the current chat to resume the conversation later in time. To create an instance of the FileChatHistory you need to pass the absolute path of the directory where you want to store conversations, and the unique key for the current conversation.

use NeuronAI\Agent;
use NeuronAI\Providers\AIProviderInterface;
use NeuronAI\Chat\History\FileChatHistory;

class MyAgent extends Agent
{
    protected function provider(): AIProviderInterface
    {
        ...
    }
    
    protected function chatHistory()
    {
        return new FileChatHistory(
            directory: '/home/app/storage/neuron',
            key: '[user-id]',
            contextWindow: 50000
        );
    }
}

The key parameter allows you to store different files to separate conversations. You can use a unique key for each user, or the ID of a thread to make multiple users able restore and move forward on the same conversation.

How to implement a new chat history

To create a new implementation of the chat history you must implement the AbstractChatHistory. It allows you to inherit several behaviors for the internal history management, so you have just to implement a couple of methods to save messages into the external storage you want to use.

abstract class AbstractChatHistory implements ChatHistoryInterface
{
    abstract protected function storeMessage(Message $message): ChatHistoryInterface;
    
    abstract public function removeOldestMessage(): ChatHistoryInterface;

    abstract protected function clear(): ChatHistoryInterface;
}

The abstract class already implement some utility methods to calculate tokens usage based on the AI provider responses and automatically cut the conversation based on the size of the context window. You just have to focus on the interaction with the underlying storage to add and clear messages.

We strongly suggest to look at other implementations like FileChatHistory to understand how to create your own.