RAG Agent with ChromaDB and Web Search

This project implements a Retrieval-Augmented Generation (RAG) agent that uses a local ChromaDB vector store for document retrieval and falls back to DuckDuckGo web search when the local store does not provide sufficient context.

Features

• Local Retrieval – Store and query embeddings in a persistent ChromaDB collection.
• Web Search Fallback – If local retrieval fails to find relevant context, the agent performs a DuckDuckGo search and uses the snippets.
• OpenAI Integration – Uses OpenAI embeddings (text-embedding-ada-002) and the gpt-3.5-turbo model for generation.
• CLI – Simple command line interface for ingesting documents and asking questions.

Prerequisites

• Python 3.9+
• An OpenAI API key
• (Optional) Internet access for web search

Installation

# Clone the repository
git clone https://git.brojs.ru/kuzakhmetovartur/ekzamen-rag-agent-s-chromadb-i-veb-poisk.git
cd ekzamen-rag-agent-s-chromadb-i-veb-poisk

# Create a virtual environment (recommended)
python -m venv .venv
source .venv/bin/activate   # On Windows use `.venv\Scripts\activate`

# Install dependencies
pip install -r requirements.txt

requirements.txt contains:

openai
chromadb
duckduckgo-search
beautifulsoup4
requests

Environment Variables

Variable	Description	Example
OPENAI_API_KEY	Your OpenAI API key	sk-...
CHROMA_DB_PATH	Directory where ChromaDB stores data	./chromadb
CHROMA_COLLECTION_NAME	Name of the collection	rag_collection
TOP_K	Number of top documents to retrieve	5
SIMILARITY_THRESHOLD	Minimum similarity to consider a document relevant	0.5
WEB_SEARCH_MAX_RESULTS	Max number of web snippets to fetch	3

Set them in your shell or create a .env file and load with dotenv (optional).

Usage

Ingest Documents

Place your plain text files (.txt) in a folder, then run:

python src/index.py ingest /path/to/text/files

The script will read all .txt files, split them into chunks, embed them, and store them in ChromaDB.

Ask a Question

python src/index.py ask "What is the capital of France?"

The agent will:

1. Query the local vector store for relevant passages.
2. If none are found above the similarity threshold, perform a DuckDuckGo search.
3. Combine the retrieved context into a prompt.
4. Call OpenAI’s gpt-3.5-turbo to generate an answer.

Example

$ python src/index.py ingest ./data
INFO:root:Added 12 documents to collection 'rag_collection'.

$ python src/index.py ask "Explain the theory of relativity."
Answer:
The theory of relativity, developed by Albert Einstein, consists of two parts: special relativity and general relativity. ...

Testing

Unit tests are provided in the tests/ directory. To run them:

pytest tests/

(If you don't have pytest installed, run pip install pytest.)

Troubleshooting

• No documents ingested – Ensure the folder path is correct and contains .txt files.
• OpenAI errors – Verify that OPENAI_API_KEY is set and that you have sufficient quota.
• Web search fails – Check your internet connection and that DuckDuckGo is reachable.

License

MIT License