Retrieval-Augmented Generation (RAG)
Figure 1: Basic RAG Architecture showing document processing, embedding, vector storage, and retrieval flow
Simple RAG Implementation
IntermediateThis example demonstrates how to create a basic RAG system that can answer questions based on custom documents.
import os
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_community.vectorstores import Chroma
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough
from langchain.document_loaders import TextLoader
# Set your API key
os.environ["OPENAI_API_KEY"] = "your-api-key-here"
# 1. Load documents
loader = TextLoader("./data/company_handbook.txt")
documents = loader.load()
# 2. Split documents into chunks
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
chunks = text_splitter.split_documents(documents)
# 3. Create embeddings and store in vector database
embeddings = OpenAIEmbeddings()
vectorstore = Chroma.from_documents(documents=chunks, embedding=embeddings)
retriever = vectorstore.as_retriever(search_kwargs={"k": 4})
# 4. Create prompt template
template = """You are an AI assistant helping with questions about the company handbook.
Use the following context to answer the question. If you don't know the answer,
say you don't know and don't make up information.
Context:
{context}
Question:
{question}
Answer:"""
prompt = ChatPromptTemplate.from_template(template)
# 5. Set up LLM
llm = ChatOpenAI(temperature=0)
# 6. Create the RAG pipeline
rag_chain = (
{"context": retriever, "question": RunnablePassthrough()}
| prompt
| llm
| StrOutputParser()
)
# Example usage
if __name__ == "__main__":
question = "What is the company policy on remote work?"
response = rag_chain.invoke(question)
print(response)Key Points
- Documents are split into smaller chunks to fit into context windows and improve retrieval granularity.
- The vector store (Chroma) allows semantic search of document chunks based on their embeddings.
- The retriever finds relevant document chunks based on the question's semantic meaning.
- The prompt template combines the retrieved context with the user's question.
- The LLM generates an answer based on the retrieved context.
Conversational RAG Agent
AdvancedThis example shows how to build a conversational agent that combines RAG with tool use and conversation memory.
import os
from typing import List, Dict, Any
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_community.vectorstores import Chroma
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain.schema import Document
from langchain.tools import BaseTool
from langchain.agents import AgentType, initialize_agent
from langchain.memory import ConversationBufferMemory
from langchain.chains import ConversationalRetrievalChain
from langchain.document_loaders import DirectoryLoader
# Set API key
os.environ["OPENAI_API_KEY"] = "your-api-key-here"
class DocumentSearchTool(BaseTool):
name = "document_search"
description = "Search for information in the company documents"
def __init__(self, retriever):
super().__init__()
self.retriever = retriever
def _run(self, query: str) -> str:
"""Search for information in documents."""
docs = self.retriever.get_relevant_documents(query)
if not docs:
return "No relevant information found in the documents."
# Format the retrieved information
result = "Here's what I found in the documents:\n\n"
for i, doc in enumerate(docs, 1):
result += f"Document {i}:\n{doc.page_content}\n\n"
return result
def _arun(self, query: str) -> str:
"""Async implementation would go here."""
raise NotImplementedError("DocumentSearchTool does not support async")
def build_conversational_rag_agent():
# 1. Load documents from directory
loader = DirectoryLoader("./data/company_docs/", glob="**/*.txt")
documents = loader.load()
# 2. Split documents into chunks
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
chunks = text_splitter.split_documents(documents)
# 3. Create embeddings and vector store
embeddings = OpenAIEmbeddings()
vectorstore = Chroma.from_documents(documents=chunks, embedding=embeddings)
retriever = vectorstore.as_retriever(search_kwargs={"k": 4})
# 4. Set up LLM
llm = ChatOpenAI(temperature=0, model="gpt-4-0125-preview")
# 5. Set up conversation memory
memory = ConversationBufferMemory(
memory_key="chat_history",
return_messages=True
)
# 6. Create tools
document_tool = DocumentSearchTool(retriever)
tools = [document_tool]
# 7. Initialize agent with memory
agent = initialize_agent(
tools,
llm,
agent=AgentType.OPENAI_FUNCTIONS,
verbose=True,
memory=memory,
handle_parsing_errors=True,
)
return agent
# Example usage
if __name__ == "__main__":
agent = build_conversational_rag_agent()
# First query
response = agent.invoke({"input": "What is our company's parental leave policy?"})
print(f"Response: {response['output']}\n")
# Follow-up query using memory
response = agent.invoke({"input": "How does that compare to the industry standard?"})
print(f"Response: {response['output']}")Key Points
- This implementation combines RAG with agent capabilities and conversation memory.
- The DocumentSearchTool wraps the retriever to make it available as a tool for the agent.
- ConversationBufferMemory allows the agent to reference previous exchanges in the conversation.
- The agent can choose when to use the document search tool versus when to use its own knowledge.
- Directory loading allows processing multiple documents at once from a folder.
Multi-Agent Systems
Figure 2: Multi-Agent Architecture showing agent communication and task delegation
CrewAI Multi-Agent System
AdvancedThis example demonstrates how to create a multi-agent system with different roles using CrewAI.
import os
from crewai import Agent, Task, Crew, Process
from langchain_openai import ChatOpenAI
# Set API key
os.environ["OPENAI_API_KEY"] = "your-api-key-here"
# Initialize the language model
llm = ChatOpenAI(model="gpt-4-0125-preview", temperature=0.2)
# Create agents with different roles
researcher = Agent(
role="Market Researcher",
goal="Discover actionable market insights",
backstory="""You are an expert market researcher with years of experience in
analyzing market trends and identifying opportunities. Your analyses are known
for being comprehensive and insightful.""",
verbose=True,
allow_delegation=True,
llm=llm
)
analyst = Agent(
role="Data Analyst",
goal="Transform raw data into meaningful patterns",
backstory="""You are a skilled data analyst who excels at interpreting complex datasets
and extracting valuable insights. You have a talent for visualizing data patterns
and explaining them clearly.""",
verbose=True,
allow_delegation=True,
llm=llm
)
strategist = Agent(
role="Business Strategist",
goal="Develop effective business strategies based on market insights",
backstory="""You are a business strategy expert with a track record of developing
successful strategies for various companies. You excel at connecting market insights
to actionable business plans.""",
verbose=True,
allow_delegation=False,
llm=llm
)
# Define tasks for each agent
research_task = Task(
description="""Research the current state of the electric vehicle market.
Identify key players, market trends, consumer preferences, and growth projections.
Focus on the period from 2020 to present. Compile your findings in a comprehensive report.""",
agent=researcher,
expected_output="A detailed market research report on the electric vehicle industry"
)
analysis_task = Task(
description="""Analyze the market research data to identify patterns, opportunities,
and challenges in the electric vehicle market. Identify segments showing the highest growth
potential and factors driving adoption. Create a structured analysis with key metrics.""",
agent=analyst,
expected_output="Data analysis report with visualizations and key metrics",
context=[research_task]
)
strategy_task = Task(
description="""Develop a market entry strategy for a new company looking to enter the
electric vehicle market. Based on the research and analysis, recommend product positioning,
target customer segments, and competitive differentiation strategies. Create a 2-year
strategic roadmap with key milestones.""",
agent=strategist,
expected_output="A comprehensive market entry strategy with 2-year roadmap",
context=[research_task, analysis_task]
)
# Create the crew with sequential process
market_analysis_crew = Crew(
agents=[researcher, analyst, strategist],
tasks=[research_task, analysis_task, strategy_task],
process=Process.sequential,
verbose=2
)
# Execute the crew
result = market_analysis_crew.kickoff()
print(result)Key Points
- CrewAI allows you to create specialized agents with distinct roles, goals, and backstories.
- Tasks can be connected through context, allowing output from one task to flow to another.
- The sequential process ensures tasks are completed in order, with each agent building on previous work.
- Agent delegation allows certain agents to request help from others when needed.
AutoGen Multi-Agent Conversation
AdvancedThis example shows how to create a conversational multi-agent system using Microsoft's AutoGen framework.
import os
import autogen
from autogen import Agent, AssistantAgent, UserProxyAgent, GroupChat, GroupChatManager
# Set API configuration
config_list = [
{
"model": "gpt-4",
"api_key": os.environ.get("OPENAI_API_KEY"),
}
]
# Create specialized agents
planner = AssistantAgent(
name="Planner",
system_message="""You are a task planning expert. Your role is to:
1. Break down complex tasks into manageable steps
2. Create clear, actionable plans
3. Identify dependencies between tasks
4. Coordinate different activities to achieve goals efficiently""",
llm_config={"config_list": config_list}
)
researcher = AssistantAgent(
name="Researcher",
system_message="""You are a research specialist. Your role is to:
1. Find relevant information on technical topics
2. Summarize complex concepts in an accessible way
3. Identify credible sources and methodologies
4. Answer factual questions with precise information""",
llm_config={"config_list": config_list}
)
coder = AssistantAgent(
name="Coder",
system_message="""You are a programming expert. Your role is to:
1. Write clean, efficient code based on requirements
2. Debug issues and suggest improvements
3. Explain complex technical concepts in simple terms
4. Follow best practices for software development""",
llm_config={"config_list": config_list}
)
reviewer = AssistantAgent(
name="Reviewer",
system_message="""You are a code review and quality assurance specialist. Your role is to:
1. Review code for bugs, inefficiencies, and edge cases
2. Suggest improvements for readability and maintainability
3. Ensure code meets best practices and requirements
4. Test code functionality and identify potential issues""",
llm_config={"config_list": config_list}
)
# Create user proxy agent to interact with the group
user_proxy = UserProxyAgent(
name="User",
system_message="A user who needs help building a Python web scraper for collecting product information.",
human_input_mode="ALWAYS" # For simplicity in this example, change to "TERMINATE" for real applications
)
# Create a group chat with all agents
groupchat = GroupChat(
agents=[user_proxy, planner, researcher, coder, reviewer],
messages=[],
max_round=20
)
# Create a group chat manager
manager = GroupChatManager(
groupchat=groupchat,
llm_config={"config_list": config_list}
)
# Start the conversation
user_proxy.initiate_chat(
manager,
message="""I need to build a web scraper in Python that collects product information from an e-commerce website.
The scraper should:
1. Extract product names, prices, ratings, and descriptions
2. Store the data in a CSV file
3. Be able to handle pagination
4. Respect the website's robots.txt and rate limits
Help me build this application step by step."""
)Key Points
- AutoGen enables dynamic conversations between multiple agents with specialized roles.
- The GroupChat framework manages turn-taking and facilitates agent collaboration.
- Each agent has a distinct system message that guides its behavior and expertise.
- The UserProxyAgent acts as an interface between the human user and the agent group.
- The GroupChatManager orchestrates the conversation flow and ensures productive collaboration.