Tools & Function Calls

Give Agents the ability to interact with your application context and services.

Tools enable Agents to go beyond generating text by facilitating interaction with your application services, or external APIs.

Think about Tools as special functions that your AI agent can use when it needs to perform specific tasks. They let you extend your Agent's capabilities by giving it access to specific functions it can call inside your code.

In the YouTubeAgent example we can define a tool to make the Agent able to retrieve the YouTube video transcription:

use NeuronAI\Agent;
use NeuronAI\SystemPrompt;
use NeuronAI\Providers\AIProviderInterface;
use NeuronAI\Providers\Anthropic\Anthropic;
use NeuronAI\Tools\Tool;
use NeuronAI\Tools\ToolProperty;

class YouTubeAgent extends Agent
{
    protected function provider(): AIProviderInterface
    {
        // return an AI provider instance (Anthropic, OpenAI, Mistral, etc.)
        return new Anthropic(
            key: 'ANTHROPIC_API_KEY',
            model: 'ANTHROPIC_MODEL',
        );
    }
    
    public function instructions(): string 
    {
        return new SystemPrompt(
            background: ["You are an AI Agent specialized in writing YouTube video summaries."],
            steps: [
                "Get the url of a YouTube video, or ask the user to provide one.",
                "Use the tools you have available to retrieve the transcription of the video.",
                "Write the summary.",
            ],
            output: [
                "Write a summary in a paragraph without using lists. Use just fluent text.",
                "After the summary add a list of three sentences as the three most important take away from the video.",
            ]
        );
    }
    
    protected function tools(): array
    {
        return [
            Tool::make(
                'get_transcription',
                'Retrieve the transcription of a youtube video.',
            )->addProperty(
                new ToolProperty(
                    name: 'video_url',
                    type: 'string',
                    description: 'The URL of the YouTube video.',
                    required: true
                )
            )->setCallable(function (string $video_url) {
                // ... do something
            })
        ];
    }
}

Let’s break down the code.

We introduced the new method tools() into the Agent. This method expects to return an array of Tool objects that the AI will be able to use if needed.

In this example we return an array of just one tool, named get_transcription.

Notice that the ToolProperty we define should match with the signature of the function you use as a callable. The callable gets the $video_url arguments, and the name of the property is exactly "video_url".

The most important thing are the names and descriptions you give to the tool and its properties. All these pieces of information will be passed to the LLM in natural language. The more explicit and clear you are, the more likely the LLM understands when, and if, it’s the case to use these tools.

Once the Agent decides to use this tool the callable function is executed. Here we can implement the logic to gather the article from the database and return the information we want to the LLM.

Neuron provides you with these clear and simple APIs and automates the rest of interactions with the LLM under the hood. Once you get the point you can immediately open to a possibility to connect basically everything you want to the Agent. Being able to execute local functions allows you to invoke any external APIs or application components.

Retrieve the YouTube video transcription

Every time I implement an agent I prefer to separate the Tool definition in the agent class, from the implementation of the callable function.

Since it requires connecting to external APIs it’s better to implement this process into a dedicated class so it will be easier to organize the code or add new features later.Instead of defining the in-line function we can create a class implementing the PHP magic method __invoke() and pass to the tool as it will behave like a function:

use GuzzleHttp\Client;

class GetTranscription
{
    protected Client $client;

    public function __construct()
    {
        $this->client = new Client([
            'base_uri' => 'https://api.supadata.ai/v1/youtube/',
            'headers' => [
                'x-api-key' => 'SUPADATA_API_KEY',
            ]
        ]);
    }

    public function __invoke(string $video_url)
    {
        $response = $this->client->get('transcript?url=' . $video_url.'&text=true');

        if ($response->getStatusCode() !== 200) {
            return "Transcription APIs error: {$response->getBody()->getContents()}";
        }

        $response = json_decode($response->getBody()->getContents(), true);

        return $response['content'];
    }
}

Notice how the __invoke() method accepts the same arguments expected by the tool callable function.

Now we can pass an instance of this class in the Tool definition:

class YouTubeAgent extends Agent
{
    ...
	
    public function tools(): array
    {
        return [
            Tool::make(
                'get_transcription',
                'Retrieve the transcription of a youtube video.',
            )->addProperty(
                new ToolProperty(
                    name: 'video_url',
                    type: 'string',
                    description: 'The URL of the YouTube video.',
                    required: true
                )
            )->setCallable(new GetTranscription())
        ];
    }
}

To retrieve the YouTube video transcription I used a very useful API called Supadata.ai.

Transcriptions are just an example. You can eventually implement other tools to make the Agent able to retrieve other video metadata to enhance its video analysis capabilities.

Perform Video Analysis

Ok! Now we provided the agent with a tool to make it able to retrieve the YouTube video transcript when it thinks it's needed to complete the task.

Since we encapsulated this feature into the agent the interaction it's really simple:

use NeuronAI\Chat\Messages\UserMessage;

$response = YouTubeAgent::make($user)
    ->chat(
        new UserMessage('What about this video: https://www.youtube.com/watch?v=WmVLcj-XKnM')
    );
    
echo $response->getContent();

/**

Based on the transcription, I'll provide a summary of this powerful environmental 
message from "Mother Nature":
This video presents a monologue from the perspective of Nature herself, 
speaking directly to humanity. In an authoritative tone, Nature reminds 
us that she has existed for 4.5 billion years—22,500 times longer than 
humans—and doesn't need people, though people depend entirely on her.

Three most important takeaways:

1. Nature has existed for billions of years without humans and will 
continue to exist regardless of human actions, but humans cannot 
survive without Nature.

2. The wellbeing of humanity is directly linked to the wellbeing of 
natural systems—oceans, soil, rivers, and forests.

3. How humans choose to act toward Nature determines humanity's future, 
not Nature's, as she is built to endure change while humans may not be.

*/

How Tools work behind the scene

When you provide the Agent with tools, Neuron sends their information to the LLM along with the user message.

Once the LLM reads the prompt and the information of the tools attached, it will decide if some tool can help to gather additional information to respond to the user prompt. In that case it will return a special response that contains the tools the LLM wants to call.

Neuron automatically manages this response for you, executing the callable of the tools the LLM decided to call, and return the result back to the LLM to get its final response.

To watch inside this workflow you should connect your Agent to the Inspector monitoring dashboard in order to see this call execution flow in real time.

In the second image below you can see all the details about the execution of the function to retrieve the content of the article:

Extend the Framework

Thanks to the Neuron AI modular architecture, Tools are just a component of the toolkit that rely on the ToolInterface interface. So you are free to create pre-packaged tool classes that implements common functionalities, and release them as external composer packages or submit a PR to our repository to have them integrated into the core toolkit.

Once you identify a common use case, pre-packaged Tools can internally define the execute method, the properties needed by the callback, description, etc.

To create a new Tool you must implement the following interface. Also take a look at the default Tool class to get some inspiration.

use NeuronAI\Tools\Tool;
use NeuronAI\Tools\ToolProperty;

class MyCustomTool extends Tool
{
    public function __construct(protected \PDO $pdo)
    {
        parent::__construct(
            'get_transcription',
            'Retrieve the transcription of a youtube video.',
        );
    
        $this->addProperty(
            new ToolProperty(
                name: 'video_url',
                type: 'string',
                description: 'The URL of the YouTube video.',
                required: true
            )
        )->setCallable(function (string $video_url) {
            // ...
        });
    }
}

The Agent implementation could be simplified as below:

class YouTubeAgent extends Agent
{
    public function provider(): AIProviderInterface
    {
        // return an AI provider instance (Anthropic, OpenAI, Mistral, etc.)
        return new Anthropic(
            key: 'ANTHROPIC_API_KEY',
            model: 'ANTHROPIC_MODEL',
        );
    }
    
    public function instructions() 
    {
        return new SystemPrompt(...);
    }
    
    public function tools(): array
    {
        return [
            MyCustomTool::make(...)
        ];
    }
}