Attach Images

Attach images in your message for models with vision capabilities.

Most advanced LLMs can understand images other than simple text. With Neuron you can attach images to your messages to leverage the vision capability of your AI provider.

The most common use cases for Image analysis are:

• Caption and answer questions about images

• Transcribe and reason over image content

• Detect objects in an image and return bounding box coordinates for them

• Segment objects within an image

You have two options to attach images to your messages: as an URL, or encoded in base64.

Add URLs

use NeuronAI\Chat\Messages\UserMessage;
use NeuronAI\Chat\Messages\Image;

// Ollama only support images encoded in base64
$message = (new UserMessage("Describe this image"))
    ->addImage(new Image(
        image: 'https://url_of/image.jpg', 
        type: Image::TYPE_URL
    ));
    
$response = MyAgent::make()->chat($message);
// This is image shows...

Add base64 encoded images

use NeuronAI\Chat\Messages\UserMessage;
use NeuronAI\Chat\Messages\Image;

$content = base64_encode(file_get_contents('/image_path.jpg'));

$message = (new UserMessage("Describe this image"))
    ->addImage(new Image(
        image: $content, 
        type: Image::TYPE_BASE64, 
        mediaType: 'image/jpeg'
    ));
    
$response = MyAgent::make()->chat($message);
// This is image shows...

Ollama limitations

Ollama only support images in base64 format, so you have to take care to convert the image content and set up the right image type when attaching an image to the message:

use NeuronAI\Chat\Messages\UserMessage;
use NeuronAI\Chat\Messages\Image;

// Ollama only support images encoded in base64
$message = (new UserMessage("Describe this image"))
    ->addImage(new Image(
        image: 'base64-encoded-content', 
        type: Image::TYPE_BASE64, 
        mediaType: 'image/jpeg'
    ));