Awesome
Enterprise RAG Agentic Orchestrator
Part of GPT-RAG
Table of Contents
- Concepts
- Running the Orchestrator
- NL2SQL Strategies Configuration
- Evaluation
- Contributing
- Trademarks
Concepts
How the Orchestrator Works
The GPT-RAG Agentic Orchestrator is a powerful system that leverages AutoGen's group chat capabilities to facilitate collaboration among multiple specialized agents. This orchestrator is designed to handle complex tasks by coordinating the interactions of agents, each with a specific role, to produce coherent and accurate responses.
Architecture Overview
At its core, the orchestrator employs a factory pattern to create agents based on predefined strategies. The AgentStrategyFactory
is responsible for instantiating the appropriate agents according to the selected strategy, such as classic_rag
or nl2sql
. Each strategy defines a unique set of agents, their roles, and how they interact within the group chat.
Agent Creation and Strategies
The orchestrator supports various strategies to address different types of queries and data interactions. For example:
-
Classic RAG Strategy (
classic_rag
): This strategy focuses on retrieval-augmented generation, where agents work together to retrieve relevant information from a knowledge base and generate responses grounded in that information. -
NL2SQL Strategy (
nl2sql
): This strategy enables the system to translate natural language queries into SQL statements, allowing users to interact with databases using everyday language.
Each strategy defines how agents are created and configured. The creation code for each strategy involves:
-
Creating Agents: Agents are instantiated with specific roles and system messages. For instance, in the
classic_rag
strategy, the agents include:- UserProxyAgent: Acts as a proxy for the user, executing any function calls made by AssistantAgent
- AssistantAgent: Responsible for generating responses, using the conversation history summarized for context.
- ChatClosureAgent: Manages the closure of the conversation.
-
Registering Functions: Functions, which we can also refer to as tools, are registered to enable agents to perform specific tasks, such as retrieving data from a vector index or getting the current date and time. These functions are registered with the assistant agent as the caller and the user proxy agent as the executor.
-
Defining Transitions: The allowed transitions between agents are specified to control the flow of the conversation. For example, in the
classic_rag
strategy, the assistant agent can transition to either the chat closure agent or back to the user proxy agent.
Multi-Agent Group Chat
The orchestrator utilizes AutoGen's group chat pattern to manage conversations involving multiple agents. The group chat is orchestrated by a GroupChatManager
, which coordinates the interactions among agents based on the selected strategy.
<BR> Illustration of a Group Chat Round, adapted from the AutoGen repository
The process involves:
-
Initiation: The user proxy agent initiates the conversation with a user query.
-
Agent Communication: Agents communicate by sending messages to each other, following the allowed transitions. They may call registered functions to perform tasks like data retrieval or query execution.
-
Response Generation: The assistant agent synthesizes the information gathered and generates a response to the user's query.
-
Conversation Closure: The chat closure agent manages the termination of the conversation when appropriate.
Multi-Round Interactions
The orchestrator supports multiple interaction rounds, allowing agents to share insights and collaborate efficiently. The max_rounds
property in each strategy defines the maximum number of rounds permitted. This enables complex queries to be addressed thoroughly, with agents iteratively refining the response.
Customization and Extensibility
The orchestrator is highly customizable, allowing developers to define custom strategies and agent behaviors. By subclassing BaseAgentStrategy
and implementing the create_agents
method, new strategies can be created to meet specific requirements. This extensibility ensures that the orchestrator can adapt to a wide range of operational scenarios.
Selecting an Agent Strategy
The GPT-RAG Agentic Orchestrator provides a range of agent strategies to handle different types of queries and data interactions. Selecting the appropriate strategy ensures that the orchestrator operates efficiently and meets the specific needs of your application. This section outlines how to select a strategy and provides detailed descriptions of the available strategies.
How to Select a Strategy
The orchestrator selects the agent strategy based on the AUTOGEN_ORCHESTRATION_STRATEGY
environment variable. Be sure to set this variable to the name of the desired strategy. If you're running the chat client locally, set this variable in your shell environment. For deployments as a Function App, define it in the application settings.
Available Strategies
The orchestrator supports the following strategies, each tailored to specific needs:
-
classic_rag: The
classic_rag
strategy is the default mode of operation for the orchestrator. It is optimized for retrieving information from a predefined knowledge base indexed as an AI Search Index. This strategy leverages retrieval-augmented generation (RAG) techniques to fetch and synthesize information from existing documents or databases, ensuring accurate and relevant responses based on the available data. -
nl2sql: The
nl2sql
strategy enables the orchestrator to convert natural language queries into SQL statements. This allows users to interact with relational databases using everyday language, simplifying data retrieval processes without requiring users to write complex SQL queries.
Additional NL2SQL Strategies
To enhance the functionality and accuracy of SQL query generation, the orchestrator offers specialized variations of the nl2sql
strategy:
-
nl2sql_fewshot: The
nl2sql_fewshot
strategy enhances the standardnl2sql
approach by utilizing AI-driven search to identify similar past queries. This few-shot learning technique improves the accuracy and relevance of the generated SQL statements by learning from a limited set of examples, thereby refining the query translation process. -
nl2sql_fewshot_scaled: This strategy enhances
nl2sql_fewshot
by using AI Search Indexes to handle cases with numerous tables or columns. It identifies the most relevant schema elements based on the user's question, enabling precise SQL generation even in complex database environments. -
nl2sql_dual: The
nl2sql_dual
strategy introduces a dual-agent system where a second agent reviews and refines the generated SQL queries and responses. This additional layer of validation ensures higher accuracy and clarity in the translated queries, reducing the likelihood of errors and enhancing the reliability of the responses.
How to Add and Configure you Own Agent Strategies
If the available strategies don’t fully meet your requirements, you can extend the orchestrator by implementing custom strategies. This flexibility lets you adapt the orchestrator’s behavior to unique use cases and operational demands.
Define custom agent strategies by specifying distinctive agent behaviors. To create a custom strategy:
- Subclass
BaseAgentStrategy
and implement thecreate_agents
method. - Register the strategy in
AgentStrategyFactory
for environment variable selection.
Steps to Add a Custom Strategy:
-
Create the Strategy Class:
Define a new strategy by inheriting from the base strategy class and specifying the required logic.from .strategies.base_strategy import BaseAgentStrategy class CustomAgentStrategy(BaseAgentStrategy): def execute(self, query): # Implement custom strategy logic here pass
-
Update the AgentStrategyFactory:
ModifyAgentStrategyFactory
to recognize and instantiate your custom strategy.from .strategies.custom_agent_strategy import CustomAgentStrategy class AgentStrategyFactory: @staticmethod def get_strategy(strategy_type: str): # Existing strategy selections if strategy_type == 'custom': return CustomAgentStrategy() # Other strategies... else: raise ValueError(f"Unknown strategy type: {strategy_type}")
Ensure the AUTOGEN_ORCHESTRATION_STRATEGY
environment variable is correctly set to the desired strategy name, whether a predefined strategy or a custom one you’ve implemented.
[!NOTE] The name
custom
is used here as an example. You should choose a name that best represents your specific case.
Running the Orchestrator
Running the Chat Client Locally
- Set the
AUTOGEN_ORCHESTRATION_STRATEGY
environment variable to run the NL2SQL chat client locally:
Bash:
export AUTOGEN_ORCHESTRATION_STRATEGY=nl2sql
PowerShell:
$env:AUTOGEN_ORCHESTRATION_STRATEGY = "nl2sql"
-
Rename the
.env.template
file to.env
and update the variables as needed. -
Run
./chat.sh
(for Bash) or./chat.ps1
(for PowerShell) to start the client locally.
Running the Function Locally
To run the Azure Function locally, see Testing the Solution Locally in VS Code.
Cloud Deployment
Deploy the orchestrator to the cloud using the Azure Developer CLI:
azd auth login
azd env refresh
azd deploy
Ensure prerequisites, like Python 3.11, Azure Developer CLI, and Git, are installed.
NL2SQL Strategies Configuration
Configuring NL2SQL Strategies
This section provides configuration steps for the various NL2SQL strategies. These strategies convert natural language queries into SQL statements compatible with your databases.
NL2SQL Data
Data Dictionary
The Data Dictionary is essential for SQL generation, providing a structured reference for database tables and columns. If you're using the standard nl2sql
strategy, simply review and update the config/data_dictionary.json
file as needed.
[!NOTE] If you prefer, you can create a
config/data_dictionary.custom.json
file, which will override the example file inconfig/data_dictionary.json
.
If you're using the nl2sql_fewshot_scaled
strategy, the data_dictionary.json
file will not be used. In this case, you'll need to create the JSON files differently to be indexed. You can refer to the examples in gpt-rag-ingestion to see how to set up the table and column files for AI Search indexing.
Queries
If you've chosen the nl2sql_fewshot
or nl2sql_fewshot_scaled
strategy, you'll need to create example queries and index them in AI Search. For guidance on creating and indexing queries, as well as for example queries, refer to gpt-rag-ingestion.
Database Connection Setup
Set up database connections by configuring the required environment variables for each target database.
SQL Database Connection
To set up a connection to your SQL Database, follow these steps based on your authentication method.
-
Configure environment variables:
SQL_DATABASE_SERVER=my-database-server SQL_DATABASE_NAME=my-database-name
-
If using SQL Authentication, also set the following environment variable and store the user's password securely in Key Vault as a secret named
sqlDatabasePassword
:SQL_DATABASE_UID=my-username
-
If using Azure Active Directory (AAD) Authentication, do not set the
SQL_DATABASE_UID
variable. The application will use the identity associated with your environment.
-
-
Permissions: Ensure your identity has the
db_datareader
role on the database. For more details on setting up your permissions, refer to the SQL Database Setup Guide. -
Connection details in code:
- If
SQL_DATABASE_UID
is set, the code will use SQL Authentication, retrieving the password from the Key Vault. - If
SQL_DATABASE_UID
is not set, the code will default to Azure AD token-based authentication.
- If
Teradata Connection
To set up a connection to your Teradata database, follow these steps:
-
Install the Teradata SQL driver:
pip install teradatasql
-
Configure Teradata connection settings in your environment:
TD_HOST=teradata-host TD_USER=teradata-username
-
Set up the password:
- Store the Teradata password securely in Key Vault under the name
teradataPassword
.
- Store the Teradata password securely in Key Vault under the name
-
Permissions:
Ensure your Teradata user has the necessary permissions for query access.
Evaluation
An evaluation program is provided for testing the orchestrator's performance. <BR>Refer to the Evaluation Documentation for details.
Contributing
For contribution guidelines, refer to CONTRIBUTING.md.
Trademarks
This project may contain trademarks. Follow Microsoft's Trademark Guidelines for proper use.