Streaming Response

Presenting AI response to your user in real-time.

Streaming enables you to show users chunks of response text as they arrive rather than waiting for the full response. You can offer a real-time Agent conversation experience.

Agent

To stream the AI response you should use the stream() method to run the agent, instead of chat(). This method return a PHP generator that can be used to process the response as an iterable object.

use NeuronAI\Chat\Messages\UserMessage;

$response = MyAgent::make()
    ->stream(
        new UserMessage('How are you?')
    );

// Print the response chunk-by-chunk in real-time
foreach ($stream as $text) {
    usleep(200 * 1000);
    echo $text;
}

// I'm fine, thank you! How can I assist you today?

RAG

Also the RAG implementation has its pwn streaming method: streamAnswer()

use NeuronAI\Chat\Messages\UserMessage;

$response = MyChatBot::make()
    ->streamAnswer(
        new UserMessage('I want to know more about Inspector AI Bug Fix.')
    );

// Print the response chunk-by-chunk in real-time
foreach ($stream as $text) {
    usleep(200 * 1000);
    echo $text;
}

// Sure, Inspector AI Bug Fix is an agentic monitoring tool 
// that provides bug fix proposals in real-time as an error occurs 
// in your application.

Streaming & Tools

Neuron support Tools & Function calls in combination with the streaming response. You are free to provide your Agents with Tools and they will be automatically handled in the middle of the stream, to continue toward the final response.

use NeuronAI\Chat\Messages\UserMessage;
use NeuronAI\Tools\Tool;

$response = MyChatBot::make()
    ->addTool(
        Tool::make(
            'get_server_configuration',
            'retrieve the server network configuration'
        )->addProperty(...)->setCallable(...)
    )
    ->streamAnswer(
        new UserMessage('What's the IP address of the server?')
    );

// Print the response chunk-by-chunk in real-time
foreach ($stream as $text) {
    usleep(200 * 1000);
    echo $text;
}

// Let me retrieve the server configuration. 
// The IP address of the server is: 192.168.0.10

Observability

As your Agent implementation becomes more and more complex, you can get insights of all the internal steps performed by the Agent by connecting Inspector.

use NeuronAI\Chat\Messages\UserMessage;
use NeuronAI\Observability\AgentMonitoring;
use NeuronAI\Tools\Tool;

$response = MyChatBot::make()
    ->addTool(
        Tool::make(
            'get_server_configuration',
            'retrieve the server network configuration'
        )->addProperty(...)->setCallable(...)
    )
    ->obaserve(new AgentMonitoring($inspector))
    ->streamAnswer(
        new UserMessage('What's the IP address of the server?')
    );

// Print the response chunk-by-chunk in real-time
foreach ($stream as $text) {
    usleep(200 * 1000);
    echo $text;
}

// Let me retrieve the server configuration. 
// The IP address of the server is: 192.168.0.10

Learn more about Agent observability in the dedicated documentation.