113 lines
2.4 KiB
Markdown
113 lines
2.4 KiB
Markdown
# Agent with RAG Memory
|
||
|
||
This repository contains a simple **Retrieval‑Augmented Generation (RAG)** agent
|
||
implemented with LangChain, FAISS for vector storage, and OpenAI embeddings
|
||
and LLM. It also provides an `auto_check_graph` function that verifies the
|
||
generated answer against a ground‑truth mapping and returns a `verdict_row`.
|
||
|
||
> **Important**
|
||
> The auto‑check graph must return a `verdict_row`. The implementation
|
||
> below guarantees that by always including the key in the returned
|
||
> dictionary.
|
||
|
||
## Features
|
||
|
||
- **RAG Agent** – Load documents, embed them, store in FAISS, and answer queries.
|
||
- **Auto‑Check Graph** – Run a query, generate an answer, compare it to a
|
||
ground‑truth answer, and return a verdict (`PASS`, `FAIL`, or `UNKNOWN`).
|
||
- **Unit Tests** – Verify that the agent and auto‑check graph work as
|
||
expected.
|
||
|
||
## Installation
|
||
|
||
```bash
|
||
# Create a virtual environment (recommended)
|
||
python -m venv .venv
|
||
source .venv/bin/activate # On Windows: .venv\Scripts\activate
|
||
|
||
# Install dependencies
|
||
pip install -r requirements.txt
|
||
```
|
||
|
||
`requirements.txt` contains:
|
||
|
||
```
|
||
langchain
|
||
openai
|
||
faiss-cpu
|
||
pytest
|
||
```
|
||
|
||
> **OpenAI API Key**
|
||
> If you want to use real embeddings and LLM, set the environment variable
|
||
> `OPENAI_API_KEY`:
|
||
|
||
```bash
|
||
export OPENAI_API_KEY="sk-..."
|
||
```
|
||
|
||
If the key is not set, the agent falls back to `FakeEmbeddings` and
|
||
`FakeLLM`, which are suitable for local testing and unit tests.
|
||
|
||
## Usage
|
||
|
||
```python
|
||
from src.index import RAGAgent, auto_check_graph
|
||
|
||
# Create agent
|
||
agent = RAGAgent()
|
||
|
||
# Add documents (e.g., from a directory)
|
||
agent.add_documents([
|
||
"The capital of France is Paris.",
|
||
"William Shakespeare wrote Hamlet."
|
||
])
|
||
|
||
# Define ground truth mapping
|
||
ground_truth = {
|
||
"What is the capital of France?": "Paris",
|
||
"Who wrote Hamlet?": "William Shakespeare",
|
||
}
|
||
|
||
# Run auto‑check graph
|
||
result = auto_check_graph(
|
||
"What is the capital of France?",
|
||
agent,
|
||
ground_truth
|
||
)
|
||
|
||
print(result)
|
||
# Output:
|
||
# {
|
||
# "verdict_row": "PASS",
|
||
# "answer": "Paris",
|
||
# "expected": "Paris"
|
||
# }
|
||
```
|
||
|
||
## Running Tests
|
||
|
||
```bash
|
||
pytest
|
||
```
|
||
|
||
The tests cover:
|
||
|
||
- Adding documents and querying.
|
||
- Auto‑check graph returning `PASS`, `FAIL`, and `UNKNOWN` verdicts.
|
||
- Handling of empty queries and missing ground‑truth.
|
||
|
||
## Project Structure
|
||
|
||
```
|
||
src/
|
||
├── index.py # Main implementation
|
||
tests/
|
||
├── test_agent.py # Unit tests
|
||
README.md
|
||
requirements.txt
|
||
```
|
||
|
||
## License
|
||
|
||
MIT License |