> For the complete documentation index, see [llms.txt](https://docs.neuron-ai.dev/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.neuron-ai.dev/neuron-v4/rag/rag.md).

# Getting Started

{% hint style="info" %}

#### PREREQUISITES

This guide assumes you are already familiar with the following concepts:

* [Agent](/neuron-v4/agent/agent.md)
* [Tool & Function Call](/neuron-v4/agent/tools.md)
  {% endhint %}

{% hint style="warning" %}

#### Coding Agent Skill

Use `/neuron-rag` to teach your coding agent how to implement a complete RAG system in your application with data pipeline and retrieval strategies.

[AI-Assisted Development](/neuron-v4/overview/agentic-development.md)
{% endhint %}

Retrieval-Augmented Generation (RAG) is the process of providing references to a knowledge base outside of the LLM training data sources before generating a response.

Large Language Models (LLMs) are trained on vast volumes of data to be able to generate original output for tasks like answering questions, translating languages, and completing sentences. RAG extends the already powerful capabilities of LLMs to specific domains or an organization's internal knowledge base, all without the need to retrain the model.

It is a cost-effective approach to improving LLM output so it remains relevant, accurate, and useful also working on your own private data.

{% embed url="<https://www.youtube.com/watch?v=ymSUOIxjoeM>" %}

## Why RAG systems are relevant

Building a RAG system is the way to use the powerful LLM capabilities on your own private data. You can create applications capable of accurately answering questions about company internal documentations. Or chatbot to serve external customers on the internal rules of an organization.

If it's not about the usage of private data, you can think of RAG as a way to provide the latest research, statistics, or news to the generative models.

## How to create a RAG system

Without RAG, the LLM takes the user input and creates a response based on information it was trained on (what it already knows).

With RAG, an information retrieval component is introduced. It utilizes the user input to first pull information from a new data source. The user query and the relevant information retrieved are both given to the LLM. The LLM uses the new knowledge and its training data to create accurate responses. The following sections provide an overview of the process.

Even if it can appear a little bit complicated, don't worry, this is just to make you aware of the process. Most of these things are automatically managed by the Neuron RAG agent.

There are three most important steps to create a RAG system.

#### 1) Process external data

The external data you want to use to augment the default LLM knowledge may exist in various formats like files, database records, or long-form text.

Before being able to submit this data to the LLM you have to convert them into a specific format called "[Embeddings](https://inspector.dev/vector-store-ai-agents-beyond-the-traditional-data-storage/)".

#### 2) Retrieval

The embeddings you have generated by processing documents and data need to be stored in specific databases able to deal with this particlar format. These database are called "[Vector Store](https://inspector.dev/vector-store-ai-agents-beyond-the-traditional-data-storage/)".

Vector store are not only able to store this data, but also to perform a particular form of search: the "similarity search" between the existing data in the database an a query we provide.

#### 3) Augment the LLM prompt

Next, the RAG agent augments your input (or prompt) by adding the relevant retrieved data in the context to make the LLM aware of the custom data before generating the response.

You just need to take care of the first step "Process external data", and Neuron gives you the toolkit to make it simple. The other steps are automatically managed by the Neuron RAG agent.

## Create a RAG Agent

To create a RAG you need to attach some additional components other than the AI provider, such as a `vector store`, and an `embeddings provider`.

First, let's create the RAG class:

{% tabs %}
{% tab title="Unix" %}

```bash
vendor/bin/neuron make:rag App\\Neuron\\MyChatBot
```

{% endtab %}

{% tab title="Windows" %}

```powershell
.\vendor\bin\neuron make:rag App\Neuron\MyChatBot
```

{% endtab %}
{% endtabs %}

Here is an example of a RAG implementation:

```php
namespace App\Neuron;

use NeuronAI\Providers\AIProviderInterface;
use NeuronAI\Providers\Anthropic\Anthropic;
use NeuronAI\RAG\Embeddings\EmbeddingsProviderInterface;
use NeuronAI\RAG\Embeddings\OpenAIEmbeddingsProvider;
use NeuronAI\RAG\RAG;
use NeuronAI\RAG\VectorStore\FileVectorStore;
use NeuronAI\RAG\VectorStore\VectorStoreInterface;

class MyChatBot extends RAG
{
    protected function provider(): AIProviderInterface
    {
        return new Anthropic(
            key: 'ANTHROPIC_API_KEY',
            model: 'ANTHROPIC_MODEL',
        );
    }
    
    protected function embeddings(): EmbeddingsProviderInterface
    {
        return new OpenAIEmbeddingsProvider(
            key: 'OPENAI_API_KEY',
            model: 'OPENAI_MODEL'
        );
    }
    
    protected function vectorStore(): VectorStoreInterface
    {
        return new FileVectorStore(
            directory: __DIR__,
            name: 'demo'
        );
    }
}
```

{% hint style="warning" %}
Explore [**Data Loaders**](/neuron-v4/rag/data-loader.md) to learn how to populate the vector store with embeddings representing the knowledge you want to integrate as additional knowledge.
{% endhint %}

### Talk to the RAG agent

Imagine having previously populated the vector store with the knowledge base you want to connect to the RAG agent, and now you want to ask questions. Check out [**Data Loaders**](/neuron-v4/rag/data-loader.md) to learn about RAG data population.

To start the execution of a RAG you call the `chat()` method:

```php
use App\Neuron\MyChatBot;
use NeuronAI\Chat\Messages\UserMessage;

$message = MyChatBot::make()
    ->chat(
        new UserMessage('I want to know more about Inspector AI Bug Fix.')
    )
    ->getMessage();
    
echo $message->getContent();

// Sure, Inspector AI Bug Fix is an agentic monitoring tool 
// that provides bug fix proposals in real-time as an error occurs 
// in your application.
```

## Feed Your RAG With Documents

Once you have defined the components of your RAG system it's time to feed the vector database with embedded chunks of text.

Neuron provides you with [Data Loaders](/neuron-v4/rag/data-loader.md) to help you set up a data loading pipeline with just a few lines of code. You can see an example below. To learn more about data loader you should check out the [dedicated documentation](/neuron-v4/rag/data-loader.md):

```php
use App\Neuron\MyChatBot;
use NeuronAI\RAG\DataLoader\FileDataLoader;

MyChatBot::make()->addDocuments(
    // Use the file data loader component to load a text file into the vector store
    FileDataLoader::for(__DIR__.'/my-article.md')->getDocuments()
);
```

## RAG + Tools

The Neuron's RAG class extends the basic `\NeuronAI\Agent` class. This means that your RAG is always an agent and you can also attach tools and define system instructions in your implementation.

Imagine we want to implement an agent able to give workout tips based on the user data. Here is an example of a complete implementation:

```php
namespace App\Neuron;

use NeuronAI\Providers\AIProviderInterface;
use NeuronAI\Providers\Anthropic\Anthropic;
use NeuronAI\RAG\Embeddings\EmbeddingsProviderInterface;
use NeuronAI\RAG\Embeddings\OpenAIEmbeddingsProvider;
use NeuronAI\RAG\RAG;
use NeuronAI\RAG\VectorStore\FileVectorStore;
use NeuronAI\RAG\VectorStore\VectorStoreInterface;
use NeuronAI\Tools\Toolkits\Calculator\CalculatorToolkit;

class WorkoutTipsAgent extends RAG
{
    protected function provider(): AIProviderInterface
    {
        return new Anthropic(
            key: 'ANTHROPIC_API_KEY',
            model: 'ANTHROPIC_MODEL',
        );
    }
    
    public function instructions(): string
    {
        return (string) new SystemPrompt(
            background: ["You are an AI Agent specialized in providing workout tips."],
        );
    }
    
    protected function embeddings(): EmbeddingsProviderInterface
    {
        return new OpenAIEmbeddingsProvider(
            key: 'OPENAI_API_KEY',
            model: 'OPENAI_MODEL'
        );
    }
    
    protected function vectorStore(): VectorStoreInterface
    {
        return new FileVectorStore(
            directory: __DIR__,
            name: 'demo'
        );
    }
    
    protected function tools(): array
    {
        return [
            CalculatorToolkit::make(),
        ];
    }
}
```

In the example above we created a RAG agent that is able to give workout tips to the user. We can load into the vector store the knowledge for the specific workouts you provide, so the agent has the knowledge to provide tips based on the current workout status of the user retrieved from the database with the tool we attached.

### RAG Workflow

In the following image you can see the complete representation of the workflow Neuron runs for a RAG application. Learning this structure can help you better understand the underlying execution process to hook the system using middleware:

<figure><img src="/files/AWyyTNUJUbr1Mewp4WI1" alt=""><figcaption></figcaption></figure>

Once the `UserMessage` comes into the system, it runs in order:

* `PreProcessQueryNode`: Run the [pre-processors](/neuron-v4/rag/pre-post-processor.md#pre-processors) pipeline like `QueryTransformationPreProcessor` to reinforce the input prompt.
* `RetrieveDocumentsNode`: Execute the [retrieval strategy](/neuron-v4/rag/retrieval.md) from the vector store or external data sources
* `PostProcessDocumentsNode`: Run the [post-processors](/neuron-v4/rag/pre-post-processor.md#post-processors) pipeline like reranking
* `EnrichInstructionsNode`: Add the final documents to the system prompt of the agent
* `ChatNode`: Run the inference and collect the LLM response
* `ToolNode`: If the agent has tool attached, the model can ask their execution eventually.

Finally the RAG will return the LLM message to you.

## Documents and schemas in RAG

A vector database needs to know the type of a metadata field before it can index or compare that field correctly. Without this information, one database could store `2026` as a number while another stores it as text.

A `DocumentSchema` gives every vector store the same understanding of your documents. It belongs to the vector store because every document in one index or collection must follow the same contract.

Document loading and ingestion are covered separately in Loading documents into a RAG.

### Define the document contract

Declare fields that need a known type, are required, or will be used in filters.

```php
use NeuronAI\RAG\Schema\DocumentField;
use NeuronAI\RAG\Schema\DocumentSchema;

$schema = DocumentSchema::of(
    DocumentField::string('tenant')->required()->filterable(),
    DocumentField::string('status')->required()->filterable(),
    DocumentField::integer('published_at')->filterable(),
    DocumentField::float('price')->filterable(),
    DocumentField::boolean('published')->filterable(),
    DocumentField::strings('tags'),
);
```

Use `required()` when every document must contain a non-null value. Use `filterable()` when the field will appear in a portable filter.

| Metadata value | Declaration                       |
| -------------- | --------------------------------- |
| String         | `DocumentField::string('name')`   |
| Integer        | `DocumentField::integer('name')`  |
| Float          | `DocumentField::float('name')`    |
| Boolean        | `DocumentField::boolean('name')`  |
| String array   | `DocumentField::strings('name')`  |
| Integer array  | `DocumentField::integers('name')` |
| Float array    | `DocumentField::floats('name')`   |
| Boolean array  | `DocumentField::booleans('name')` |

Integers are also valid values for float fields. Arrays must contain values of one type.

### Attach the schema to the RAG vector store

Define the schema where the RAG defines its vector store. This keeps the data contract next to the database configuration that depends on it.

```php
use NeuronAI\RAG\Schema\DocumentField;
use NeuronAI\RAG\Schema\DocumentSchema;
use NeuronAI\RAG\VectorStore\PineconeVectorStore;
use NeuronAI\RAG\VectorStore\VectorStoreInterface;

protected function vectorStore(): VectorStoreInterface
{
    return new PineconeVectorStore(
        key: $_ENV['PINECONE_API_KEY'],
        indexUrl: $_ENV['PINECONE_INDEX_URL'],
        schema: DocumentSchema::of(
            DocumentField::string('tenant')->required()->filterable(),
            DocumentField::string('status')->required()->filterable(),
            DocumentField::integer('published_at')->filterable(),
            DocumentField::float('price')->filterable(),
            DocumentField::strings('tags'),
        ),
    );
}
```

Every built-in vector store accepts the optional `schema` argument. The same schema can be used when changing backend:

```php
use NeuronAI\RAG\VectorStore\MemoryVectorStore;

$store = new MemoryVectorStore(schema: $schema);
```

Create the schema before creating a new database index or collection. If an existing index has incompatible field mappings, recreate it and reindex its documents.

### Declare only what the database needs

A schema does not remove the flexible nature of a document. Undeclared JSON-safe metadata is still stored and returned.

Declare a field when you need at least one of these guarantees:

* every document must contain it;
* its value must have a known type;
* it must work in portable filters.

An undeclared field cannot be used in a portable filter because its database type is unknown.

`sourceType` and `sourceName` are built-in string fields. They are always filterable and must not be declared in the schema.

### Define Filters

Semantic search finds similar content. Filters decide which documents are allowed to participate. A static retrieval filter is useful when every request must stay inside one tenant, workspace, or knowledge base.

The filtered field must be declared as filterable in the vector store schema.

```php
use NeuronAI\RAG\Retrieval\RetrievalInterface;
use NeuronAI\RAG\Retrieval\SimilarityRetrieval;
use NeuronAI\RAG\VectorStore\Filter\Filter;
use NeuronAI\RAG\VectorStore\Filter\FilterGroup;

protected function retrieval(): RetrievalInterface
{
    return new SimilarityRetrieval(
        $this->resolveVectorStore(),
        $this->resolveEmbeddingsProvider(),
        filters: FilterGroup::and(
            Filter::eq('tenant', 'acme'),
            Filter::eq('status', 'published'),
        ),
    );
}
```

Filters added by middleware or another workflow step are combined with this scope using AND. They can narrow the result but cannot remove the permanent scope.

### Comparisons

Portable filters have the same meaning on every supported vector database.

```php
Filter::eq('status', 'published');
Filter::neq('status', 'draft');
Filter::in('status', ['published', 'reviewed']);
Filter::gt('price', 10);
Filter::gte('price', 10);
Filter::lt('price', 100);
Filter::lte('price', 100);
```

The following rules prevent different databases from returning different results:

* range comparisons work only with integer and float fields;
* dates should be stored as Unix timestamps when they need range filters;
* `neq()` requires a field declared with `required()`;
* array fields are validated and stored but do not support portable filters;
* all conditions in `FilterGroup::and()` must match;
* `in()` means that one field may match any value from a list.

Invalid fields, values, and operators fail before a request reaches the database.

### Search or delete with filters

Use the same filter model when working with the vector store outside the RAG retrieval flow.

```php
use NeuronAI\RAG\VectorStore\Filter\Filter;
use NeuronAI\RAG\VectorStore\Filter\FilterGroup;
use NeuronAI\RAG\VectorStore\SearchRequest;

$documents = $store->search(new SearchRequest(
    embedding: $queryEmbedding,
    filters: FilterGroup::and(
        Filter::eq('tenant', 'acme'),
        Filter::gte('published_at', 1767225600),
    ),
    topK: 8,
));

$store->delete(FilterGroup::and(
    Filter::eq('tenant', 'acme'),
    Filter::eq('sourceType', 'files'),
));
```

### Use a backend-specific filter

The portable API contains only operations with consistent behavior across databases. Use `Filter::raw()` for a feature available only in one backend.

```php
use NeuronAI\RAG\VectorStore\MeilisearchVectorStore;

$filters = FilterGroup::and(
    Filter::eq('tenant', 'acme'),
    Filter::raw(
        MeilisearchVectorStore::class,
        '_geoRadius(45.4, 9.1, 2000)',
    ),
);
```

The raw filter is tagged with its vector-store class. Another store rejects it instead of silently changing its meaning.