Structured Output

Enforce the Agent output based on the provided schema.

PREREQUISITES

This guide assumes you are already familiar with the following concepts:

• Agent

• Tool & Function Call

There are many use cases where we need Agents to understand natural language, but output in a structured format. One common use-case is extracting data from text to insert into a database or use with some other downstream system. This guide covers how Neuron allows you to enforce structured outputs from the agent.

How to use Structured Output

The central concept is that the output structure of LLM responses needs to be represented in some way. The schema that Neuron validates against is defined by PHP type hints. Basically you have to define a class with strictly typed properties:

<?php

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\SchemaProperty;

class Person 
{
    #[SchemaProperty(description: 'The user name.', required: true)]
    public string $name;
    
    #[SchemaProperty(description: 'What the user love to eat.', required: false)]
    public string $preference;
}

Neuron generates the corresponding JSON schema from the PHP object to instruct the underlying model about your required data format. Then the agent parse the LLM output to extract data and returns an object instance filled with appropriate values:

use NeuronAI\Chat\Messages\UserMessage;

// Talk to the agent requiring the structured output
$person = MyAgent::make()->structured(
    new UserMessage("I'm John and I like pizza!"),
    Person::class
);

echo $person->name.' like '.$person->preference;
// John like pizza

Default output class

You can also encapsulate the output format into the Agent implementation, so it will be the Agent standard output format. You always need to call the structured() method to require strict output.

use NeuronAI\Chat\Messages\UserMessage;

// Encapsulate the default output format 
class MyAgent extends Agent
{
    ...

    protected function getOutputClass(): string
    {
        return Person::class;
    }
}

// Always use the structured method if you want to get structured output
$person = MyAgent::make()
    ->structured(new UserMessage("I'm John and I like pizza"));

echo $person->name.' like '.$person->preference;
// John like pizza

Control the output generation

Neuron requires you to define two layers of rules to create the structured output class.

The first is the SchemaProperty attribute that allows you to control the JSON schema sent to the LLM to understand the required data format.

The second layer is validation. Validation attributes will ensure data gathered from the LLM response are consistent with your requirements.

SchemaProperty

The SchemaProperty attribute allows you to define the JSON schema parameters of each property:

<?php

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\SchemaProperty;

class Person 
{
    #[SchemaProperty(
        description: 'The user name.',
        required: true,
        minLength: 3,
        maxLength: 255,
    )]
    public string $name;
    
    #[SchemaProperty(
        description: 'What the user love to eat.', 
        required: false,
        min: 18,
        max: 64,
    )]
    public ?int $age = null;
}

Nested Class

You can construct complex output structures using other PHP objects as a property type. Following the example of a the Person class we can add the address property typed as another structured class.

<?php

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\Property;
use Symfony\Component\Validator\Constraints\NotBlank;
use Symfony\Component\Validator\Constraints\Valid;

class Person 
{
    #[SchemaProperty(description: 'The user name.', required: true)]
    #[NotBlank]
    public string $name;
    
    #[SchemaProperty(description: 'What user love to eat.', required: true)]
    public string $preference;
    
    #[SchemaProperty(description: 'The address to complete the delivery.', required: true)]
    public Address $address;
}

In the Address definition we require only the street and zip code properties, and allow city to be empty.

<?php

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\SchemaProperty;
use NeuronAI\StructuredOutput\Validation\Rules\NotBlank;

class Address
{
    #[SchemaProperty(description: 'The name of the street.', required: true)]
    #[NotBlank]
    public string $street;

    #[SchemaProperty(description: 'The name of the city.', required: false)]
    public string $city;

    #[SchemaProperty(description: 'The zip code of the address.', required: true)]
    #[NotBlank]
    public string $zip;
}

Now when you ask the agent for the structured output you will get the filled instance back:

use NeuronAI\Chat\Messages\UserMessage;
use NeuronAI\Observability\AgentMonitoring;

// Talk to the agent requiring the structured output
$person = MyAgent::make()->structured(
    new UserMessage("I'm John and I want a pizza at st. James Street 00560!"),
    Person::class
);

echo $person->name.' like '.$person->preference.'. Address: '.$person->address->street;
// John like pizza. Address: st.James Street

Array

If you declare a property as an array, Neuron assumes the list of items to be a list of string. If you want the array to contains a list of other structured objects you can specify that using the anyOf argument:

<?php

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\SchemaProperty;
use NeuronAI\StructuredOutput\Validation\Rules\NotBlank;

class Person 
{
    #[SchemaProperty(
        description: 'The user name.', 
        required: true
    )]
    #[NotBlank]
    public string $name;
    
    #[SchemaProperty(
        description: 'The list of tag for the user profile.', 
        required: true,
        anyOf: [Tag::class]
    )]
    public array $tags;
}

And here is the hypotetical implementation of the Tag class with its own validation rules and property info:

<?php

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\SchemaProperty;
use NeuronAI\StructuredOutput\Validation\Rules\NotBlank;

class Tag
{
    #[SchemaProperty(
        description: 'The name of the tag', 
        required: true,
    )]
    #[NotBlank]
    public string $name;
}

Array with multiple types

As you can notice from the example above the anyOf argument is an array. Neuron also supports the composition of arrays with multiple object types. Just list the structured objects the array can contains and Neuron will include all of their specs in the JSON schema for the LLM.

class Report
{
    #[SchemaProperty(
        description: 'The content of the report', 
        required: true,
        anyOf: [TextBlock::class, TableBlock::class, ImageBlock::class]
    )]
    public array $content;
}

Max Retries

Since the LLM are not perfectly deterministic it's mandatory to have a retry mechanism in place if something is missing in the LLM response.

By default Neuron extracts and validates the data from the LLM response and if there is one or more validation errors automatically retry the request just one more time informing the LLM about what went wrong and for what properties.

You can eventually customize the number of times the agent must retry to get a correct answer from the LLM:

$person = MyAgent::make()->structured(
    messages: new UserMessage("I'm John and I like pizza!"),
    class: Person::class,
    maxRetries: 3
);

If you work with a less capable LLM consider to use a number of retries balancing the probability to get e valid answer, and the potential token consumption.

You can disable retry just passing zero. It will be a one shot attempt:

$person = MyAgent::make()->structured(
    messages: new UserMessage("I'm John and I like pizza!"),
    class: Person::class,
    maxRetries: 0
);

Monitoring & Debugging

To watch inside this workflow you should connect your Agent to the Inspector monitoring dashboard in order to see the tool call execution flow in real-time.

After you sign up at the link above, make sure to set the INSPECTOR_INGESTION_KEY variable in the application environment file to start monitoring:

.env

INSPECTOR_INGESTION_KEY=nwse877auxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Each segment bring its own debug information to follow the agent execution in real time:

Learn how to enable observability in the next section.

Validation Rules

Since LLMs are not-deterministic systems and the consistency of their output can be highly influenced by the quality of the context they have as input, and they can still hallucinate, we provide a set of validation rules you can add to structured class properties to instruct Neuron verify the final set of data generated by the LLM.

Validation rules allows Neuron to resend the generation request to the LLM multiple times with a detailed report of what was wrong if one or more properties are not valid, until it reaches the maxRetries value.

If you don't define any validation rule, the data extracted from the LLM response will be filled into the structured output class directly.

#[NotBlank]

The property under validation cannot be blank. It accept the allowNull flag to treat explicitly null value as empty equivalent or not.

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\Validation\Rules\NotBlank;

class Person 
{
    #[NotBlank(allowNull: false)]
    public string $name;
}

#[Length]

Determine if the length of a string match the given criteria:

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\Validation\Rules\Length;

class Person 
{
    #[Length(min: 1, max: 10)]
    public string $name;
    
    #[Length(exactly: 5)]
    public string $zip_code;
}

#[WordsCount]

Determine if the number of words in a string match the given criteria:

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\Validation\Rules\WordsCount;

class Person 
{
    #[WordsCount(exactly: 10)]
    public string $title;
    
    #[WordsCount(min: 1, max: 10)]
    public string $content;
}

#[Count]

Determine if the size of an array match the given criteria:

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\Validation\Rules\Count;

class Person 
{
    #[Count(min: 1, max: 3)]
    public array $dogs;
    
    #[Count(exactly: 1)]
    public array $children;
}

#[EqualTo] - #[NotEqualTo]

These rules have the same structure and meaning, and accept a single argument to define the value to compare against. The property under validation must be strictly equal (#[EqualTo]) or different (#[NotEqualTo]) than the reference value:

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\Validation\Rules\EqualTo;
use NeuronAI\StructuredOutput\Validation\Rules\NotEqualTo;

class Person 
{
    #[EqualTo(reference: 'Rome')]
    public string $city;
    
    #[NotEqualTo(reference: '00502')]
    public string $zip_code;
}

#[GreaterThan] - #[GreaterThanEqual]

These rules have the same structure and meaning, and accept a single argument to define the value to compare against. The property under validation must be strictly greater (#[GreaterThan]) or equal (#[GreaterThanEqual]) than the reference value:

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\Validation\Rules\GreaterThan;
use NeuronAI\StructuredOutput\Validation\Rules\GreaterThanEqual;

class Person 
{
    #[GreaterThan(reference: 17)]
    public int $age;
    
    #[GreaterThanEqual(reference: 1)]
    public int $cars;
}

#[LowerThan] - #[LowerThanEqual]

These rules have the same structure and meaning, and accept a single argument to define the value to compare against. The property under validation must be strictly lower (#[LowerThan]) or equal (#[LowerThanEqual]) than the reference value:

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\Validation\Rules\LowerThan;
use NeuronAI\StructuredOutput\Validation\Rules\LowerThanEqual;

class Person 
{
    #[LowerThan(reference: 50)]
    public int $age;
    
    #[LowerThanEqual(reference: 1)]
    public int $cars;
}

#[OutOfRange]

Determin if a number is out of the given range:

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\Validation\Rules\InRange;

class Person 
{
    #[OutOfRange(min: 18, max: 35)]
    public int $age;
    
    // The strict argument force to stay stricly out of the range limits
    #[OutOfRange(min: 48, max: 54, strict: true)]
    public int $size;
}

#[IsFalse] - #[IsTrue]

The property under validation must have exactly the boolean value defined by the rule:

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\Validation\Rules\IsFalse;
use NeuronAI\StructuredOutput\Validation\Rules\IsTrue;

class Phone
{
    #[IsFalse]
    public bool $iphone;
    
    #[IsTrue]
    public bool $refurbed;
}

#[IsNull] - #[IsNotNull]

The property under validation must respect the nullable condition defined by the rule:

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\Validation\Rules\IsNotNull;
use NeuronAI\StructuredOutput\Validation\Rules\IsNull;

class Phone
{
    #[IsNotNull]
    public string $brand;
    
    #[IsNull]
    public ?string $test;
}

#[Json]

The property under validation must contains a valid JSON string:

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\Validation\Rules\Json;

class Person
{
    #[Json]
    public string $address;
}

#[Url]

The property under validation must contains a valid URL:

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\Validation\Rules\Url;

class Person
{
    #[Url]
    public string $website;
}

#[Email]

The property under validation must contains a valid Email address:

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\Validation\Rules\Email;

class Person
{
    #[Email]
    public string $email;
}

#[IpAddress]

The property under validation must contains a valid IP address:

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\Validation\Rules\IpAddress;

class Person
{
    #[IpAddress]
    public string $ip;
}

#[ArrayOf]

The property under validation must be an array that contains all of the given type of object. Notice that you also need to add the doc-block in order to make the agent able to instance the correct class. Use the full class namespace in the doc-block.

namespace App\Neuron\Output;

use NeuronAI\StructuredOutput\Validation\Rules\ArrayOf;

class Person
{
    /**
     * @var \App\Neuron\Output\Tag[]
     */
    #[ArrayOf(Tag::class)]
    public array $tags;
}