feat: solution for 'Повторный экзамен: FAQ-бот — ChromaDB + один MCP-tool'
This commit is contained in:
@@ -1,118 +1,88 @@
|
||||
# FAQ Bot – QDrant Vector Store
|
||||
# FAQ Bot with Qdrant Vector Store
|
||||
|
||||
This project implements a simple FAQ bot that uses **QDrant** as the vector store instead of ChromaDB.
|
||||
The bot can ingest a text file containing FAQ content, embed the text using OpenAI embeddings, store the embeddings in QDrant, and answer user questions by retrieving the most relevant passages.
|
||||
|
||||
## Features
|
||||
|
||||
- **Vector Store** – QDrant (via `qdrant-client`)
|
||||
- **Embeddings** – OpenAI `text-embedding-ada-002`
|
||||
- **CLI** – Ingest data, query the bot, delete the collection
|
||||
- **API** – `get_response(question: str, top_k: int = 5)` for integration with tools like MCP-tool
|
||||
This project implements a simple FAQ chatbot that uses **Qdrant** as the vector store for embeddings.
|
||||
The bot loads a set of FAQ entries, generates embeddings with OpenAI’s `text-embedding-ada-002` model, stores them in Qdrant, and answers user queries by performing a similarity search.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.9+
|
||||
- QDrant server running locally or accessible via network
|
||||
- OpenAI API key
|
||||
- A running Qdrant instance (local or remote)
|
||||
- An OpenAI API key
|
||||
|
||||
## Setup
|
||||
|
||||
1. **Clone the repository**
|
||||
|
||||
```bash
|
||||
git clone https://git.brojs.ru/kuzakhmetovartur/povtornyy-ekzamen-faq-bot-chromadb-odin.git
|
||||
cd povtornyy-ekzamen-faq-bot-chromadb-odin
|
||||
git clone https://git.brojs.ru/kuzakhmetovartur/povtornyy-ekzamen-faq-bot-qdrant.git
|
||||
cd povtornyy-ekzamen-faq-bot-qdrant
|
||||
```
|
||||
|
||||
2. **Create a virtual environment (optional but recommended)**
|
||||
2. **Create a virtual environment and install dependencies**
|
||||
|
||||
```bash
|
||||
python -m venv venv
|
||||
source venv/bin/activate # On Windows: venv\Scripts\activate
|
||||
```
|
||||
|
||||
3. **Install dependencies**
|
||||
|
||||
```bash
|
||||
source venv/bin/activate # On Windows use `venv\Scripts\activate`
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
4. **Set environment variables**
|
||||
3. **Configure environment variables**
|
||||
|
||||
Create a `.env` file in the project root or export the variables directly:
|
||||
Create a `.env` file in the project root with the following content:
|
||||
|
||||
```bash
|
||||
export OPENAI_API_KEY="your-openai-api-key"
|
||||
export QDRANT_URL="http://localhost:6333" # Adjust if your QDrant instance is elsewhere
|
||||
export QDRANT_API_KEY="" # Leave empty if no auth is required
|
||||
export QDRANT_COLLECTION="faq_collection"
|
||||
```dotenv
|
||||
# Qdrant configuration
|
||||
QDRANT_HOST=localhost
|
||||
QDRANT_PORT=6333
|
||||
QDRANT_API_KEY= # leave empty if no API key is required
|
||||
|
||||
# OpenAI configuration
|
||||
OPENAI_API_KEY=your_openai_api_key_here
|
||||
```
|
||||
|
||||
If you prefer not to use a `.env` file, you can set the variables in your shell session.
|
||||
Replace `your_openai_api_key_here` with your actual OpenAI API key.
|
||||
|
||||
## Usage
|
||||
4. **Run the bot**
|
||||
|
||||
### 1. Ingest Data
|
||||
```bash
|
||||
python src/main.py
|
||||
```
|
||||
|
||||
Prepare a plain text file (`faq.txt`) containing your FAQ content. Then run:
|
||||
The bot will ingest the FAQ data into Qdrant and then wait for user input. Type a question and press Enter to receive an answer. Type `exit` or `quit` to stop the bot.
|
||||
|
||||
```bash
|
||||
python src/index.py ingest faq.txt
|
||||
```
|
||||
## How It Works
|
||||
|
||||
The script will:
|
||||
1. **Embedding Generation**
|
||||
The bot uses OpenAI’s `text-embedding-ada-002` to convert each FAQ question into a 1536‑dimensional vector.
|
||||
|
||||
- Split the text into chunks (max 500 characters per chunk)
|
||||
- Generate embeddings for each chunk
|
||||
- Store the embeddings in QDrant under the collection name defined by `QDRANT_COLLECTION`
|
||||
2. **Vector Store**
|
||||
Qdrant stores these vectors in a collection named `faq_collection`. Each point contains the vector and a payload with the original question and answer.
|
||||
|
||||
### 2. Query the Bot
|
||||
3. **Querying**
|
||||
When a user asks a question, the bot generates an embedding for the query, performs a cosine similarity search in Qdrant, and returns the answer from the most similar FAQ entry.
|
||||
|
||||
```bash
|
||||
python src/index.py query "What is the return policy?"
|
||||
```
|
||||
## Customization
|
||||
|
||||
You can adjust the number of results returned with `--top_k`:
|
||||
- **Adding More FAQs**
|
||||
Edit the `FAQ_DATA` list in `src/main.py` to include additional question/answer pairs.
|
||||
|
||||
```bash
|
||||
python src.index.py query "What is the return policy?" --top_k 3
|
||||
```
|
||||
- **Changing the Embedding Model**
|
||||
Replace `"text-embedding-ada-002"` in `get_embedding()` with another OpenAI embedding model if desired.
|
||||
|
||||
### 3. Delete the Collection
|
||||
|
||||
> **Warning:** This will permanently delete all data in the collection.
|
||||
|
||||
```bash
|
||||
python src/index.py delete
|
||||
```
|
||||
|
||||
### 4. Integration via API
|
||||
|
||||
If you want to use the bot programmatically (e.g., from MCP-tool), import the `get_response` function:
|
||||
|
||||
```python
|
||||
from src.index import get_response
|
||||
|
||||
answer = get_response("How do I reset my password?")
|
||||
print(answer)
|
||||
```
|
||||
- **Adjusting Search Parameters**
|
||||
Modify `top_k` in `query_faq()` to return more results or change the similarity metric in `create_or_recreate_collection()`.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **QDrant Connection Errors**
|
||||
Ensure the QDrant server is running and reachable at the URL specified by `QDRANT_URL`. Check firewall settings if accessing remotely.
|
||||
- **Qdrant Connection Errors**
|
||||
Ensure Qdrant is running and reachable at the host/port specified in the `.env` file.
|
||||
|
||||
- **OpenAI API Errors**
|
||||
Verify that `OPENAI_API_KEY` is correct and has sufficient quota. Check the OpenAI dashboard for usage limits.
|
||||
- **OpenAI Rate Limits**
|
||||
If you hit rate limits, consider adding retry logic or using a different model.
|
||||
|
||||
- **Large Documents**
|
||||
The ingestion script splits documents into 500‑character chunks. Adjust `max_chunk_size` in `split_text_into_chunks` if you need larger or smaller chunks.
|
||||
- **Missing Dependencies**
|
||||
Run `pip install -r requirements.txt` again to ensure all packages are installed.
|
||||
|
||||
## License
|
||||
|
||||
This project is provided under the MIT License. Feel free to modify and extend it for your own use cases.
|
||||
|
||||
## Contact
|
||||
|
||||
For questions or support, contact Artur Kuzakhmetov at `artur@example.com`.
|
||||
This project is provided for educational purposes and is not licensed for commercial use.
|
||||
Reference in New Issue
Block a user