RAG Agent with ChromaDB and Tavily Web Search

Overview

This repository contains a simple RAG (Retrieval‑Augmented Generation) agent that can answer user questions by searching a local knowledge base stored in ChromaDB and by performing live web searches via Tavily. The agent automatically selects the appropriate source and reports it in the answer.

The project uses the following stack:

• Python 3.10+
• LangChain 1.x – modern agent framework
• ChromaDB – local vector store
• Ollama – LLM (llama3) and embeddings (nomic-embed-text)
• Tavily – web search API
• LangGraph – (not used directly, but required by LangChain 1.x)

Folder structure

workspace/
├─ documents/          # .txt/.md files that will be loaded into Chroma
├─ chroma_db/          # persistent Chroma data (created on first run)
├─ main.py             # CLI entry point
├─ vectorstore.py      # Chroma store helpers
├─ tools.py            # Agent tools
├─ README.md
└─ requirements.txt

Setup

1. Install Ollama and pull the required models:

   ollama pull llama3
   ollama pull nomic-embed-text
   

2. Set the Tavily API key (obtain a free key from https://tavily.com):

   export TAVILY_API_KEY=your_api_key_here
   

   On Windows use set instead of export.

3. Install Python dependencies:

   pip install -r requirements.txt
   

4. Add documents you want the agent to know about into the documents/ folder. Any .txt or .md files will be automatically loaded.

Running the Agent

python main.py

You will see a prompt where you can type questions. Type exit, quit, or q to end the session.

Example interaction:

User: What are the latest developments in AI agents?
Assistant: [Web Search] - ...
Source: tavily

User: What does our lab say about LangGraph?
Assistant: [Local KB] - ...
Source: chromadb

How It Works

1. Vector Store – vectorstore.py creates a persistent Chroma collection using OllamaEmbeddings. Documents from documents/ are split with RecursiveCharacterTextSplitter and added to the store.
2. Tools – tools.py defines two tools:
   • search_local_kb – semantic search in the local vector store.
   • web_search – live web search via Tavily.
3. Agent – In main.py we create a ChatOllama LLM and pass the two tools to create_agent. A system prompt instructs the LLM to choose the correct tool. The agent returns the answer along with a source tag.

Extending

• Add more documents to documents/ and restart the CLI – the store will be updated automatically.
• Replace the LLM or embedding model by changing the ChatOllama and OllamaEmbeddings parameters.
• Add additional tools (e.g., file system access, calculator) following the same pattern.

License

MIT License.