RAG Agent with ChromaDB and Web Search

This project implements a Retrieval-Augmented Generation (RAG) agent that uses a local ChromaDB vector store for document retrieval and falls back to DuckDuckGo web search when the local store does not provide sufficient context.

Features

Local Retrieval – Store and query embeddings in a persistent ChromaDB collection.
Web Search Fallback – If local retrieval fails to find relevant context, the agent performs a DuckDuckGo search and uses the snippets.
OpenAI Integration – Uses OpenAI embeddings (text-embedding-ada-002) and the gpt-3.5-turbo model for generation.
CLI – Simple command line interface for ingesting documents and asking questions.

Prerequisites

Python 3.9+
An OpenAI API key
(Optional) Internet access for web search

Installation

# Clone the repository
git clone https://git.brojs.ru/kuzakhmetovartur/ekzamen-rag-agent-s-chromadb-i-veb-poisk.git
cd ekzamen-rag-agent-s-chromadb-i-veb-poisk

# Create a virtual environment (recommended)
python -m venv .venv
source .venv/bin/activate   # On Windows use `.venv\Scripts\activate`

# Install dependencies
pip install -r requirements.txt

requirements.txt contains:

openai
chromadb
duckduckgo-search
beautifulsoup4
requests

Environment Variables

Variable	Description	Example
`OPENAI_API_KEY`	Your OpenAI API key	`sk-...`
`CHROMA_DB_PATH`	Directory where ChromaDB stores data	`./chromadb`
`CHROMA_COLLECTION_NAME`	Name of the collection	`rag_collection`
`TOP_K`	Number of top documents to retrieve	`5`
`SIMILARITY_THRESHOLD`	Minimum similarity to consider a document relevant	`0.5`
`WEB_SEARCH_MAX_RESULTS`	Max number of web snippets to fetch	`3`

Set them in your shell or create a .env file and load with dotenv (optional).

Usage

Ingest Documents

Place your plain text files (.txt) in a folder, then run:

python src/index.py ingest /path/to/text/files

The script will read all .txt files, split them into chunks, embed them, and store them in ChromaDB.

Ask a Question

python src/index.py ask "What is the capital of France?"

The agent will:

Query the local vector store for relevant passages.
If none are found above the similarity threshold, perform a DuckDuckGo search.
Combine the retrieved context into a prompt.
Call OpenAI’s gpt-3.5-turbo to generate an answer.

Example

$ python src/index.py ingest ./data
INFO:root:Added 12 documents to collection 'rag_collection'.

$ python src/index.py ask "Explain the theory of relativity."
Answer:
The theory of relativity, developed by Albert Einstein, consists of two parts: special relativity and general relativity. ...

Testing

Unit tests are provided in the tests/ directory. To run them:

pytest tests/

(If you don't have pytest installed, run pip install pytest.)

Troubleshooting

No documents ingested – Ensure the folder path is correct and contains .txt files.
OpenAI errors – Verify that OPENAI_API_KEY is set and that you have sufficient quota.
Web search fails – Check your internet connection and that DuckDuckGo is reachable.

License

MIT License

3.3 KiB Raw Blame History Unescape Escape