Messages

Unified context unit across AI providers and LLMs.

The Unified Messaging Layer

One of the major advantages of Neuron's architecture is the unified messaging layer to interact with multiple AI providers. Rather than wrestling with different response formats of OpenAI, Anthropic, Gemini, Ollama, and countless other providers, developers work with a single, elegant abstraction that handles all the complexity behind the scenes. Your code becomes independent of the specific LLM engine you use.

When you integrate Neuron Agents in your application, you're not locked into any specific provider's ecosystem or pricing model. You can seamlessly switch between providers to manage costs, environments, or take advantage of new model releases to capitalize on these improvements immediately without undertaking a major refactoring project.

The messaging layer extends beyond simple text content, providing unified interfaces for multimodal input/output (file, image, video, audio) across all supported providers, even when underlying implementations vary dramatically. This architectural decision means that your AI agents remain portable and future-proof – when new providers emerge or existing ones update their APIs, your application code remains unchanged while Neuron's messaging layer absorbs all the adaptation complexity.

What is a Message

Messages are the fundamental unit of context. They represent the input and output of models, carrying both the content and metadata needed to represent the state of a conversation when interacting with an LLM.

Messages are objects that contain:

• Role - Identifies the message type (e.g. user, assistant)

• Content Blocks - Represents the actual content of the message (like text, images, audio, files, etc.)

• Metadata - Optional fields such as additional LLM response information.

Here is an example of how to send a user message to the agent and get back the assistant message as response.

use NeuronAI\Chat\Messages\UserMssage;

$response = MyAgent::make()
    ->chat(new UserMessage("Hi, who are you?"))
    ->getMessage();

echo $response->getContent();

Content Blocks

You can think of a message’s content block as the payload of data that gets sent to the model, or generated by the model to answer your prompt. Messages can have a list of objects extending the ContentBlock interface. Neuron provides dedicated content types for text, image, file, audio, and video.

The message can contains an arbitrary list of blocks, even multiple blocks of each type. Neuron automatically maps block types in the appropriate format for each provider.

You can get and process the list of blocks into a message using getContentBlocks() method:

$response = MyAgent::make()->chat(...)->getMessage();

foreach ($response->getContentBlocks() as $block) {
    echo match($block::class) {
        ReasoningContent::class => "Reasoning: ".$block->content."\n\n",
        TextContent::class => $block->content,
        ...
        // other content blocks 
    };
}

Or just use getContent() to get all the textual contents concatenated:

$response = MyAgent::make()
    ->chat(new UserMessage("..."))
    ->getMessage();

// Get all text blocks concatenated as a single string
echo $response->getContent();

Verify Model Capabilities

Before using specific content blocks you need to verify the model capabilities to interpret the information you want to send (image, audio, video).

Text

This block represent the text part of a message. You can initialize a message with a simple string as contructor argument, or explicitly add a TextContent block:

use NeuronAI\Chat\Messages\UserMessage;
use NeuronAI\Chat\Messages\ContentBlocks\TextContent;

// Passing a string in the constructor will add the first TextContentBlock to the message
$message = new UserMessage("Hi");

// Add other text parts to the message
$message->addContent(
    new TextContent("My name is John.")
);

$message->addContent(
    new TextContent('Remeber to answer as you are a professinal concierge.')
);

// Get all text blocks concatenated
echo $message->getContent();
// Hi
// My name is John
// Remeber to answer as you are a professinal concierge.

As you can notice the final message will be a composition of multiple blocks.

Reasoning

This block will contain the resoning steps the model made before the final text response. It's automatically captured by Neuron from the model response:

// Chat with a resoning model
$response = MyAgent::make()->chat(...)->getMessage();

foreach ($response->getContentBlocks() as $block) {
    echo match($block::class) {
        ReasoningContent::class => "Reasoning: ".$block->content."\n\n",
        TextContent::class => $block->content,
        ...
        // other content blocks 
    };
}

Image

For models that support multimodality you can attach images and other type of contents, like files, audio, and video.

use NeuronAI\Chat\Messages\UserMessage;
use NeuronAI\Chat\Messages\ContentBlocks\ImageContent;

$message = new UserMessage("Describe this image");

$message->addContent(
    new ImageContent(
        source: 'https://placehold.co/600x400/EEE/31343C',
        sourceType: SourceType::URL,
        mediaType: 'image/png'
    )
);

$response = MyAgent::make()->chat($message)->getMessage();
echo $response->getContent();

File

use NeuronAI\Chat\Messages\UserMessage;
use NeuronAI\Chat\Messages\ContentBlocks\FileContent;

$message = new UserMessage("Summarize this document");

$message->addContent(
    new FileContent(
        source: base64_encode(file_get_contents(__DIR__.'/invoice.pdf')),
        sourceType: SourceType::BASE64,
        mediaType: 'application/pdf'
    )
);

$response = MyAgent::make()->chat($message)->getMessage();
echo $response->getContent();

File ID

Usually you can attach files to your message (images or documents) as URLs, or encoded in base64 format. Many provider allows you to upload files on their platform once, and reference these files with a simple ID on the message. This can unlock big saving in token consumption and can improve the model response time.

After receiveing the file ID from the provider platofrm you can add a file block to your message with SourceType::ID.

// Reference a file ID previously uploaded on the provider platform
$message = new UserMessage([
    new TextBlock('Analyze this'),
    new FileBlock("file_id_xxxx", SourceType::ID)
]);

You can do the same with Image, or Video, etc, based on your provider specifications.

Audio

use NeuronAI\Chat\Messages\UserMessage;
use NeuronAI\Chat\Messages\ContentBlocks\AudioContent;

$message = new UserMessage("Transcribe this audio");

$message->addContent(
    new FileContent(
        source: base64_encode(file_get_contents(__DIR__.'/music.mp3')),
        sourceType: SourceType::BASE64,
        mediaType: 'audio/mpeg3'
    )
);

$response = MyAgent::make()->chat($message)->getMessage();
echo $response ->getContent();

Video

use NeuronAI\Chat\Messages\UserMessage;
use NeuronAI\Chat\Messages\ContentBlocks\VideoContent;

$message = new UserMessage("Summarize the content of this lesson.");

$message->addContent(
    new VideoContent(
        source: base64_encode(file_get_contents(__DIR__.'/lesson_1.mp4')),
        sourceType: SourceType::BASE64,
        mediaType: 'video/mp4'
    )
);

$response = MyAgent::make()->chat($message)->getMessage();
echo $response->getContent();