RAG Agent with ChromaDB and Tavily

This repository contains a lightweight RAG (Retrieval‑Augmented Generation) agent that:

1. Stores local knowledge in ChromaDB using Ollama embeddings.
2. Performs semantic search over the local store.
3. Falls back to Tavily web search for up‑to‑date information.
4. Decides automatically which source to use and indicates the source in the answer.

Prerequisites

• Python 3.10+ (recommended via pyenv or conda).
• Ollama installed locally.
• A Tavily API key – set it in a .env file.

# Pull the required models
ollama pull llama3
ollama pull nomic-embed-text

Installation

pip install -r requirements.txt

Usage

# Create a .env file with your Tavily key
# TAVILY_API_KEY=YOUR_KEY

# Populate the vector store from the documents folder
python main.py

You will be presented with a prompt. Type your question and press Enter. Type exit to quit.

Project Structure

├── agent.py          # Agent definition
├── main.py           # CLI entry point
├── tools.py          # Local KB and web search tools
├── vectorstore.py    # ChromaDB helpers
├── requirements.txt
├── README.md
└── documents/        # Folder with .txt/.md files to ingest

How It Works

1. Vector Store – vectorstore.py creates a ChromaDB instance backed by OllamaEmbeddings. Documents from documents/ are chunked with RecursiveCharacterTextSplitter and added to the store.

2. Tools – tools.py exposes two LangChain tools:

   • search_local_kb – semantic search in ChromaDB.
   • web_search – web search via Tavily.

3. Agent – agent.py builds an OpenAI‑functions‑style agent that chooses between the two tools based on the user’s query. The system prompt instructs the LLM to use search_local_kb for knowledge‑base queries and web_search for recent facts. The answer always contains a source tag.

4. CLI – main.py ties everything together: it loads the vector store, creates the agent and runs an interactive chat loop.

Extending

• Replace the LLM with any other LangChain‑compatible model.
• Add more tools (e.g., database queries, file system access).
• Persist the vector store across runs – it already does this via persist_directory.

---

Happy experimenting!