Structured Output

Enforce the Agent output based on the provided schema.

PREREQUISITES

This guide assumes you are already familiar with the following concepts:

• Agent

• Tool & Function Call

There are many use cases where we need Agents to understand natural language, but output in a structured format.

One common use-case is extracting data from text to insert into a database or use with some other downstream system. This guide covers how Neuron allows you to enforce structured outputs from the agent.

How to use Structured Output

The central concept is that the output structure of LLM responses needs to be represented in some way. The schema that Neuron validates against is defined by PHP type hints. Basically you have to define a class with strictly typed properties:

use NeuronAI\StructuredOutput\SchemaProperty;

class Person 
{
    #[SchemaProperty(description: 'The user name.', required: true)]
    public string $name;
    
    #[SchemaProperty(description: 'What the user love to eat.', required: false)]
    public string $preference;
}

Neuron generates the corresponding JSON schema from the PHP object to instruct the underlying model about your required data format. Then the framework parse the LLM output to extract data and returns an object instance filled with appropriate values:

// Talk to the agent requiring the structured output
$person = MyAgent::make()->structured(
    new UserMessage("I'm John and I like pizza!"),
    Person::class
);

echo $person->name.' like '.$person->preference;
// John like pizza

Default output class

You can also encapsulate the output format into the Agent implementation, so it will be the Agent standard output format. You always need to call the structured() method to require strict output.

// Encapsulate the default output format 
class MyAgent extends Agent
{
    ...

    protected function getOutputClass(): string
    {
        return Person::class;
    }
}

// Always use the structured method if you want to get structured output
$person = MyAgent::make()->structured(new UserMessage("I'm John and I like pizza"));

echo $person->name.' like '.$person->preference;
// John like pizza

Control the output generation

Neuron requires you to define rules for two layers to create the structured output class.

The first is the SchemaProperty attribute that allows you to control the JSON schema sent to the LLM to understand the required data format.

The second layer is validation, built on top of the Symfony Validation component. It will ensure data gathered from the LLM response are consistent with your requirements.

SchemaProperty

We strongly recommend to use the SchemaProperty attribute to define at least the description, to allow the LLM understand the purpose of a property, and the required flag:

use NeuronAI\StructuredOutput\SchemaProperty;

class Person 
{
    #[SchemaProperty(description: 'The user name.', required: true)]
    public string $name;
    
    #[SchemaProperty(description: 'What the user love to eat.', required: false)]
    public string $preference;
}

Validation

Neuron ability to validate data in structured PHP objects is built on top of the Symfony Validation component.

The Validation component already contains many validation rules that you can apply to the output class properties. The example below shows you how to mark the name property as required (NotBlank):

use NeuronAI\StructuredOutput\SchemaProperty;
use Symfony\Component\Validator\Constraints\NotBlank;

class Person 
{
    #[SchemaProperty(description: 'The user name.')]
    #[NotBlank]
    public string $name;
    
    #[SchemaProperty(description: 'What the user love to eat.')]
    public string $preference;
}

You can find the available validation constraints in the official Symfony component page: https://symfony.com/doc/current/validation.html#constraints

Nested Class Validation

You can construct complex output structures using other PHP object as a property type. Following the example of a the Person class we can add the address property that is another structured class.

use NeuronAI\StructuredOutput\Property;
use Symfony\Component\Validator\Constraints\NotBlank;
use Symfony\Component\Validator\Constraints\Valid;

class Person 
{
    #[SchemaProperty(description: 'The user name.', required: true)]
    #[NotBlank()]
    public string $name;
    
    #[SchemaProperty(description: 'What user love to eat.', required: true)]
    public string $preference;
    
    #[SchemaProperty(description: 'The address to complete the delivery.', required: true)]
    #[Valid]
    public Address $address;
}

Notice that you need to add the #[Valid] attribute to require validation for nested class.

In the Address definition we require only the street and zip code properties, and allow city to be empty.

class Address
{
    #[SchemaProperty(description: 'The name of the street.', required: true)]
    #[NotBlank]
    public string $street;

    #[SchemaProperty(description: 'The name of the city.', required: false)]
    public string $city;

    #[SchemaProperty(description: 'The zip code of the address.', required: true)]
    #[NotBlank]
    public string $zip;
}

Now when you ask the agent for the structured output you will get the filled instance back:

// Talk to the agent requiring the structured output
$person = MyAgent::make()->structured(
    new UserMessage("I'm John and I want a pizza at st. James Street 00560!"),
    Person::class
);

echo $person->name.' like '.$person->preference.'. Address: '.$person->address->street;
// John like pizza. Address: st.James Street

Validate Array of Objects

If you declare a property as an array Neuron assumes the list of items to be a list of string. Assume we want to add a list of tags to the Person object:

use NeuronAI\StructuredOutput\SchemaProperty;
use Symfony\Component\Validator\Constraints\NotBlank;

class Person 
{
    #[SchemaProperty(description: 'The user name.', required: true)]
    #[NotBlank()]
    public string $name;
    
    #[SchemaPropertyerty(description: 'What user love to eat.', required: true)]
    public string $preference;
    
    #[SchemaProperty(description: 'The list of tag for the user profile.', required: true)]
    public array $tags;
}

In the example above we assume that the tags property is an array of strings by default.

echo $person->tags;

/*
[
    'tag 1',
    'tag 2',
    ...
]
*/

It could be needed to populate the list of tags with a structured data type. To do this you must specify the related class in the property doc-block:

use NeuronAI\StructuredOutput\SchemaProperty;
use Symfony\Component\Validator\Constraints\NotBlank;
use Symfony\Component\Validator\Constraints\Valid;

class Person 
{
    #[SchemaProperty(description: 'The user name.', required: true)]
    #[NotBlank]
    public string $name;
    
    #[SchemaProperty(description: 'What user love to eat.', required: true)]
    public string $preference;
    
    /**
     * @var array<\App\Agent\Models\Tag>
     */
    #[SchemaProperty(description: 'The list of tag for the user profile.', required: true)]
    #[Valid]
    #[NotBlank]
    public array $tags;
}

Notice how we are using the absolute namespace (\App\Agent\Models\Tag) to inform Neuron about the class that should instantiated.

And here is the hypotetical implementation of the Tag class with it own validation rules and property info:

use NeuronAI\StructuredOutput\SchemaProperty;
use Symfony\Component\Validator\Constraints\NotBlank;

class Tag
{
    #[SchemaProperty(description: 'The name of the tag', required: true)]
    #[NotBlank]
    public string $name;
}

Max Retries

Since the LLM are not perfectly deterministic it's mandatory to have a retry mechanism in place if something is missing in the LLM response.

By default Neuron extracts and validates the data from the LLM response and if there is one or more validation errors automatically retry the request just one more time informing the LLM about what went wrong and for what properties.

You can eventually customize the number of times the agent must retry to get a correct answer from the LLM:

$person = MyAgent::make()->structured(
    messages: new UserMessage("I'm John and I like pizza!"),
    class: Person::class,
    maxRetries: 3
);

If you work with a less capable LLM consider to use a number of retries balancing the probability to get e valid answer, and the potential token consumption.

You can disable retry just passing zero. It will be a one shot attempt:

$person = MyAgent::make()->structured(
    messages: new UserMessage("I'm John and I like pizza!"),
    class: Person::class,
    maxRetries: 0
);

Observability

You can rely on the Inspector dashboard to get full visibility on the internal steps executed by the Agent to provide you with a structured output instance:

Each segment bring its own debug information to follow the agent execution in real time:

Learn how to enable observability in the next section.