109 lines
3.0 KiB
Markdown
109 lines
3.0 KiB
Markdown
# RAG Agent with ChromaDB & Tavily
|
||
|
||
This project implements a Retrieval-Augmented Generation (RAG) agent that uses **ChromaDB** for vector storage and **Tavily** for web search.
|
||
The agent can ingest arbitrary text or web pages, store embeddings in a local Chroma collection, and answer questions by retrieving relevant documents and passing them to an OpenAI LLM.
|
||
|
||
> **Important**
|
||
> The original repository used Qdrant. All references to Qdrant have been removed.
|
||
> Only ChromaDB and Tavily are used.
|
||
|
||
## Prerequisites
|
||
|
||
| Component | Version | Notes |
|
||
|-----------|---------|-------|
|
||
| Python | 3.9+ | Tested on 3.10 |
|
||
| OpenAI API | Any key | Required for embeddings and LLM |
|
||
| Tavily API | Any key | Required for web search |
|
||
|
||
Set the following environment variables before running:
|
||
|
||
```bash
|
||
export OPENAI_API_KEY="your-openai-key"
|
||
export TAVILY_API_KEY="your-tavily-key"
|
||
```
|
||
|
||
## Installation
|
||
|
||
```bash
|
||
# Clone the repository
|
||
git clone https://git.brojs.ru/kuzakhmetovartur/ekzamen-rag-agent-s-chromadb-i-veb-poisk.git
|
||
cd ekzamen-rag-agent-s-chromadb-i-veb-poisk
|
||
|
||
# Create a virtual environment (optional but recommended)
|
||
python -m venv .venv
|
||
source .venv/bin/activate # On Windows: .venv\Scripts\activate
|
||
|
||
# Install dependencies
|
||
pip install -r requirements.txt
|
||
```
|
||
|
||
`requirements.txt` contains:
|
||
|
||
```
|
||
chromadb>=0.4
|
||
tavily>=0.1
|
||
langchain>=0.0.350
|
||
openai>=1.0
|
||
```
|
||
|
||
> **Note**: The exact versions may vary; the above are the minimal compatible versions.
|
||
|
||
## Usage
|
||
|
||
The agent is a single script `src/index.py`. It supports two commands:
|
||
|
||
### 1. Ingest
|
||
|
||
```bash
|
||
python src/index.py ingest <url_or_text>
|
||
```
|
||
|
||
- If `<url_or_text>` starts with `http://` or `https://`, the script treats it as a URL, fetches the content via Tavily, and stores it.
|
||
- Otherwise, it treats the argument as raw text and stores it directly.
|
||
|
||
Example:
|
||
|
||
```bash
|
||
python src/index.py ingest https://en.wikipedia.org/wiki/OpenAI
|
||
```
|
||
|
||
### 2. Query
|
||
|
||
```bash
|
||
python src/index.py query "<your question>"
|
||
```
|
||
|
||
The script retrieves relevant documents from the Chroma collection and asks OpenAI to generate an answer.
|
||
|
||
Example:
|
||
|
||
```bash
|
||
python src/index.py query "What is OpenAI?"
|
||
```
|
||
|
||
## Project Structure
|
||
|
||
```
|
||
.
|
||
├── src
|
||
│ └── index.py # Main script
|
||
├── README.md
|
||
└── requirements.txt
|
||
```
|
||
|
||
## How It Works
|
||
|
||
1. **Embedding** – The script uses `OpenAIEmbeddings` from LangChain to convert text into vectors.
|
||
2. **Vector Store** – `Chroma` stores these vectors locally in `~/.rag_agent/chromadb`.
|
||
3. **Retrieval** – When a query is made, the nearest vectors are fetched.
|
||
4. **Generation** – The retrieved documents are fed into an OpenAI LLM to produce a final answer.
|
||
|
||
## Troubleshooting
|
||
|
||
- **No results from Tavily** – Ensure your Tavily API key is valid and that the URL is reachable.
|
||
- **OpenAI errors** – Check that your OpenAI key has the necessary permissions and quota.
|
||
- **Chroma storage issues** – The data directory is `~/.rag_agent/chromadb`. Delete it to reset the collection.
|
||
|
||
## License
|
||
|
||
This project is released under the MIT License. |