Streaming

Presenting AI response to your user in real-time.

Streaming enables you to show users chunks of response text as they arrive rather than waiting for the full response. You can offer a real-time Agent conversation experience.

Agent

To stream the AI response you should use the stream() method to run the agent, instead of chat(). This method return a PHP generator that can be used to process the response as an iterable object.

use App\Neuron\MyAgent;
use NeuronAI\Chat\Messages\UserMessage;

$stream = MyAgent::make()->stream(
    new UserMessage('How are you?')
);

// Print the response chunk-by-chunk in real-time
foreach ($stream as $text) {
    echo $text;
}

// I'm fine, thank you! How can I assist you today?

RAG

Also the RAG implementation has its own streaming method: streamAnswer()

use App\Neuron\MyChatBot;
use NeuronAI\Chat\Messages\UserMessage;

$stream = MyChatBot::make()->streamAnswer(
    new UserMessage('I want to know more about Inspector AI Bug Fix.')
);

// Print the response chunk-by-chunk in real-time
foreach ($stream as $text) {
    echo $text;
}

// Sure, Inspector AI Bug Fix is an agentic monitoring tool 
// that provides bug fix proposals in real-time as an error occurs 
// in your application.

Streaming & Tools

Neuron support Tools & Function calls in combination with the streaming response. You are free to provide your Agents with Tools and they will be automatically handled in the middle of the stream, to continue toward the final response.

use App\Neuron\MyChatBot;
use NeuronAI\Chat\Messages\UserMessage;
use NeuronAI\Tools\Tool;

$stream = MyChatBot::make()
    ->addTool(
        Tool::make(
            'get_server_configuration',
            'retrieve the server network configuration'
        )->addProperty(...)->setCallable(...)
    )
    ->streamAnswer(
        new UserMessage('What's the IP address of the server?')
    );

// Print the response chunk-by-chunk in real-time
foreach ($stream as $text) {
    echo $text;
}

// Let me retrieve the server configuration. 
// The IP address of the server is: 192.168.0.10

Monitoring

To watch inside this workflow you should connect your Agent to the Inspector monitoring dashboard in order to see the tool call execution flow in real-time.

After you sign up at the link above, make sure to set the INSPECTOR_INGESTION_KEY variable in the application environment file to start monitoring:

.env

INSPECTOR_INGESTION_KEY=nwse877auxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Learn more about Agent observability in the dedicated documentation.