Attachments (Documents & Images)

Attach documents and images to your message.

Most advanced LLMs can understand the content of documents and images other than simple text. With Neuron you can attach files to your messages to enrich the context provided to the Agent.

The most common use cases for documents analysis are:

• Caption and answer questions about images

• Transcribe and reason over document contents

You have two options to attach items to your messages: as an URL, or encoded in base64.

Be sure about the possible limitations of your AI provider to handle documents and images in specific formats.

Documents

URL

use App\Neuron\MyAgent;
use NeuronAI\Chat\Attachments\Document;
use NeuronAI\Chat\Messages\UserMessage;

// Ollama only support images encoded in base64
$message = (new UserMessage("Describe this document"))
    ->addAttachment(
        new Document('https://url_of/document.pdf')
    );
    
$response = MyAgent::make()->chat($message);
// The document is a contract...

Base64

use App\Neuron\MyAgent;
use NeuronAI\Chat\Attachments\AttachmentContentType;
use NeuronAI\Chat\Attachments\Document;
use NeuronAI\Chat\Messages\UserMessage;

$content = base64_encode(file_get_contents('/document.pdf'));

$message = (new UserMessage("Describe this document"))
    ->addAttachment(
        new Document(
            document: $content,
            type: AttachmentContentType::BASE64,
            mediaType: 'application/pdf'
        )
    );
    
$response = MyAgent::make()->chat($message);
// The document is a contract...

Images

URL

use App\Neuron\MyAgent;
use NeuronAI\Chat\Attachments\Image;
use NeuronAI\Chat\Messages\UserMessage;

// Ollama only support images encoded in base64
$message = (new UserMessage("Describe this image"))
    ->addAttachment(
        new Image('https://url_of/image.jpg')
    );
    
$response = MyAgent::make()->chat($message);
// The image shows...

Base64

use App\Neuron\MyAgent;
use NeuronAI\Chat\Attachments\AttachmentContentType;
use NeuronAI\Chat\Attachments\Image;
use NeuronAI\Chat\Messages\UserMessage;

$content = base64_encode(file_get_contents('/image.jpg'));

$message = (new UserMessage("Describe this image"))
    ->addAttachment(
        new Image(
            image: $content,
            type: AttachmentContentType::BASE64,
            mediaType: 'image/jpeg'
        )
    );
    
$response = MyAgent::make()->chat($message);
// The image shows...

File ID

Usually you can attach files to your message (images or documents) as URLs, or encoded in base64 format. Many provider allows you to upload files on their platform once, and reference these files with a simple ID on the message. This can unlock big savings in token consumption and can improve the model response time.

use App\Neuron\MyAgent;
use NeuronAI\Chat\Attachments\AttachmentContentType;
use NeuronAI\Chat\Attachments\Image;
use NeuronAI\Chat\Messages\UserMessage;

$message = (new UserMessage("Describe this document"))
    ->addAttachment(
        new Document(
            document: $document_id,
            type: AttachmentContentType::ID,
        )
    );
    
$response = MyAgent::make()->chat($message);
// The document is about...

Ollama limitations

Ollama only support images in base64 format, so you have to take care to convert the file content and set up the right type for attachments:

use NeuronAI\Chat\Attachments\AttachmentContentType;
use NeuronAI\Chat\Attachments\Image;
use NeuronAI\Chat\Messages\UserMessage;

// Ollama only support images encoded in base64
$message = (new UserMessage("Describe this image"))
    ->addAttachment(
        new Image(
            image: 'base64-encoded-content', 
            type: AttachmentContentType::BASE64, 
            mediaType: 'image/jpeg'
        )
    );