feat: solution for 'Повторный экзамен: FAQ-бот — ChromaDB + один MCP-tool'
This commit is contained in:
@@ -0,0 +1,5 @@
|
||||
node_modules/
|
||||
.env
|
||||
dist/
|
||||
build/
|
||||
*.log
|
||||
@@ -0,0 +1,119 @@
|
||||
# FAQ Bot – ChromaDB + MCP-style Tool
|
||||
|
||||
This project implements a simple FAQ bot that answers questions about a machine learning course.
|
||||
The bot uses:
|
||||
|
||||
- **ChromaDB** to store and retrieve FAQ documents.
|
||||
- **Ollama** embeddings (`nomic-embed-text`) for vectorization.
|
||||
- **LangChain** to build an agent that routes queries to the appropriate tool.
|
||||
- **MCP-style HTTP tool** (`fetch_course_meta`) that returns course metadata from a local JSON file.
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
.
|
||||
├── chroma_faq/ # Persisted Chroma vector store
|
||||
├── data/
|
||||
│ ├── faq1.md
|
||||
│ ├── faq2.md
|
||||
│ ├── faq3.md
|
||||
│ └── course_meta.json
|
||||
├── src/
|
||||
│ ├── __init__.py
|
||||
│ ├── agent.py
|
||||
│ ├── cli.py
|
||||
│ ├── main.py
|
||||
│ └── tools.py
|
||||
├── requirements.txt
|
||||
└── README.md
|
||||
```
|
||||
|
||||
## Setup
|
||||
|
||||
1. **Install Ollama**
|
||||
Download and install Ollama from https://ollama.ai/.
|
||||
Pull the required models:
|
||||
|
||||
```bash
|
||||
ollama pull nomic-embed-text
|
||||
ollama pull llama3
|
||||
```
|
||||
|
||||
2. **Create a virtual environment** (optional but recommended):
|
||||
|
||||
```bash
|
||||
python -m venv venv
|
||||
source venv/bin/activate # On Windows: venv\Scripts\activate
|
||||
```
|
||||
|
||||
3. **Install Python dependencies**:
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
## Running the Bot
|
||||
|
||||
### Preset Questions
|
||||
|
||||
Run the script without arguments to execute three preset questions (two for the FAQ tool, one for the metadata tool):
|
||||
|
||||
```bash
|
||||
python -m src.main
|
||||
```
|
||||
|
||||
You should see output similar to:
|
||||
|
||||
```
|
||||
Running preset questions:
|
||||
|
||||
Q1: What is the deadline for Assignment 1?
|
||||
A1: The deadline for Assignment 1 is August 31, 2026. source: chroma
|
||||
|
||||
Q2: How many lectures are there in the course?
|
||||
A2: There are 12 lectures in the course. source: chroma
|
||||
|
||||
Q3: What is the course schedule for next week?
|
||||
A3: The course schedule for next week is:
|
||||
- 2026-09-01: Lecture 1 – Introduction to ML (Room 101)
|
||||
- 2026-09-08: Lecture 2 – Data Preprocessing (Room 102)
|
||||
- 2026-09-15: Lecture 3 – Linear Regression (Room 103)
|
||||
source: mcp_meta
|
||||
```
|
||||
|
||||
### Interactive Mode
|
||||
|
||||
Start an interactive session:
|
||||
|
||||
```bash
|
||||
python -m src.main --interactive
|
||||
```
|
||||
|
||||
You can type any question, and the bot will answer using the appropriate tool. Type `exit` or `Ctrl+C` to quit.
|
||||
|
||||
## How It Works
|
||||
|
||||
1. **Data Loading**
|
||||
`src/tools.py` contains `load_faq_to_chroma()` which reads all `.md` files in `data/`, chunks them, embeds them with `nomic-embed-text`, and persists the vector store in `chroma_faq/`.
|
||||
|
||||
2. **Tools**
|
||||
- `search_course_docs(query, k)` – searches the Chroma vector store for relevant FAQ snippets.
|
||||
- `fetch_course_meta(query)` – reads `data/course_meta.json` and returns schedule or instructor information based on the query.
|
||||
|
||||
3. **Agent**
|
||||
`src/agent.py` builds a LangChain agent that:
|
||||
- Uses a system prompt to decide which tool to call.
|
||||
- Adds a `source:` tag to the final answer indicating whether the answer came from the FAQ (`chroma`) or the metadata tool (`mcp_meta`).
|
||||
|
||||
4. **CLI**
|
||||
`src/cli.py` provides a simple command‑line interface to run preset questions or an interactive session.
|
||||
|
||||
## Extending the Bot
|
||||
|
||||
- **Add more FAQ documents** – Place additional `.md` files in `data/` and re‑run the script to rebuild the vector store.
|
||||
- **Add more metadata** – Update `data/course_meta.json` or modify `fetch_course_meta` to call a real HTTP endpoint.
|
||||
- **Change the LLM** – Replace `Ollama` with another LLM provider in `src/agent.py`.
|
||||
|
||||
## License
|
||||
|
||||
This project is provided as-is for educational purposes. Feel free to adapt and extend it for your own use cases.
|
||||
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"schedule": [
|
||||
{
|
||||
"date": "2026-09-01",
|
||||
"lecture": "Lecture 1",
|
||||
"topic": "Introduction to ML",
|
||||
"location": "Room 101"
|
||||
},
|
||||
{
|
||||
"date": "2026-09-08",
|
||||
"lecture": "Lecture 2",
|
||||
"topic": "Data Preprocessing",
|
||||
"location": "Room 102"
|
||||
},
|
||||
{
|
||||
"date": "2026-09-15",
|
||||
"lecture": "Lecture 3",
|
||||
"topic": "Linear Regression",
|
||||
"location": "Room 103"
|
||||
}
|
||||
],
|
||||
"instructor": {
|
||||
"name": "Dr. Jane Doe",
|
||||
"email": "jane.doe@example.com",
|
||||
"office": "Room 201"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
# Course Overview
|
||||
|
||||
This course covers the fundamentals of machine learning, including supervised and unsupervised learning, neural networks, and reinforcement learning. The course is divided into 12 lectures, each lasting 90 minutes.
|
||||
|
||||
## Assignment 1
|
||||
|
||||
The first assignment is due on **August 31, 2026**. It requires you to implement a simple linear regression model and evaluate its performance.
|
||||
|
||||
## Lecture Schedule
|
||||
|
||||
| Lecture | Topic |
|
||||
|---------|-------|
|
||||
| 1 | Introduction to ML |
|
||||
| 2 | Data Preprocessing |
|
||||
| 3 | Linear Regression |
|
||||
| 4 | Logistic Regression |
|
||||
| 5 | Decision Trees |
|
||||
| 6 | Random Forests |
|
||||
| 7 | Support Vector Machines |
|
||||
| 8 | Neural Networks |
|
||||
| 9 | Convolutional Neural Networks |
|
||||
| 10 | Recurrent Neural Networks |
|
||||
| 11 | Reinforcement Learning |
|
||||
| 12 | Project Presentations |
|
||||
@@ -0,0 +1,13 @@
|
||||
# Frequently Asked Questions
|
||||
|
||||
**Q: How many lectures are there in the course?**
|
||||
A: There are 12 lectures in total.
|
||||
|
||||
**Q: What is the deadline for Assignment 2?**
|
||||
A: Assignment 2 is due on **September 15, 2026**.
|
||||
|
||||
**Q: Where can I find the lecture slides?**
|
||||
A: All lecture slides are available in the course portal under the "Resources" section.
|
||||
|
||||
**Q: Can I submit the assignment late?**
|
||||
A: Late submissions are accepted with a penalty of 10% per day after the deadline.
|
||||
@@ -0,0 +1,12 @@
|
||||
# Course Materials
|
||||
|
||||
- **Lecture Slides**: PDF files for each lecture.
|
||||
- **Reading List**: A list of recommended books and papers.
|
||||
- **Code Repository**: GitHub repository with starter code and solutions.
|
||||
- **Discussion Forum**: For asking questions and collaborating with peers.
|
||||
|
||||
**Q: Where is the code repository hosted?**
|
||||
A: The code repository is hosted on GitHub at https://github.com/example/course-ml.
|
||||
|
||||
**Q: How do I clone the repository?**
|
||||
A: Use `git clone https://github.com/example/course-ml.git` in your terminal.
|
||||
@@ -0,0 +1,6 @@
|
||||
langchain==0.1.0
|
||||
langchain-chroma==0.1.0
|
||||
langchain-ollama==0.1.0
|
||||
chromadb==0.4.24
|
||||
httpx==0.27.0
|
||||
python-dotenv==1.0.1
|
||||
@@ -0,0 +1,46 @@
|
||||
import os
|
||||
from typing import List, Dict, Any
|
||||
|
||||
from langchain_community.llms import Ollama
|
||||
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
|
||||
from langchain_core.runnables import RunnablePassthrough
|
||||
from langchain_core.tools import BaseTool
|
||||
from langchain.agents import AgentExecutor, create_openai_tools_agent
|
||||
from langchain.schema import HumanMessage, SystemMessage
|
||||
|
||||
from .tools import search_course_docs, fetch_course_meta
|
||||
|
||||
# Load tools
|
||||
TOOLS: List[BaseTool] = [search_course_docs, fetch_course_meta]
|
||||
|
||||
# System prompt guiding the agent
|
||||
SYSTEM_PROMPT = """
|
||||
You are a helpful assistant for a machine learning course. Your job is to answer user questions.
|
||||
|
||||
- If the question is about course materials, lecture slides, assignments, or any content that can be found in the FAQ documents, use the tool `search_course_docs`.
|
||||
- If the question is about course schedule, instructor information, or other metadata, use the tool `fetch_course_meta`.
|
||||
- Do not use both tools unless absolutely necessary.
|
||||
- In your answer, always include a source tag: `source: chroma` if you used the FAQ tool, or `source: mcp_meta` if you used the metadata tool.
|
||||
"""
|
||||
|
||||
def build_agent() -> AgentExecutor:
|
||||
"""
|
||||
Build and return a LangChain AgentExecutor with the defined tools and system prompt.
|
||||
"""
|
||||
llm = Ollama(model="llama3", temperature=0.0)
|
||||
|
||||
# Prompt template
|
||||
prompt = ChatPromptTemplate.from_messages(
|
||||
[
|
||||
SystemMessage(content=SYSTEM_PROMPT),
|
||||
MessagesPlaceholder(variable_name="history"),
|
||||
HumanMessage(content="{input}"),
|
||||
]
|
||||
)
|
||||
|
||||
# Create the agent
|
||||
agent = create_openai_tools_agent(llm=llm, tools=TOOLS, prompt=prompt)
|
||||
|
||||
# Wrap with AgentExecutor
|
||||
agent_executor = AgentExecutor(agent=agent, tools=TOOLS, verbose=True, handle_parsing_errors=True)
|
||||
return agent_executor
|
||||
+55
@@ -0,0 +1,55 @@
|
||||
import argparse
|
||||
import sys
|
||||
|
||||
from .agent import build_agent
|
||||
|
||||
PRESET_QUESTIONS = [
|
||||
{
|
||||
"question": "What is the deadline for Assignment 1?",
|
||||
"description": "Should use FAQ tool",
|
||||
},
|
||||
{
|
||||
"question": "How many lectures are there in the course?",
|
||||
"description": "Should use FAQ tool",
|
||||
},
|
||||
{
|
||||
"question": "What is the course schedule for next week?",
|
||||
"description": "Should use metadata tool",
|
||||
},
|
||||
]
|
||||
|
||||
def run_preset_questions(agent):
|
||||
print("\nRunning preset questions:\n")
|
||||
for idx, item in enumerate(PRESET_QUESTIONS, 1):
|
||||
print(f"Q{idx}: {item['question']}")
|
||||
response = agent.invoke({"input": item["question"]})
|
||||
print(f"A{idx}: {response['output']}\n")
|
||||
|
||||
def interactive_mode(agent):
|
||||
print("\nEnter your questions (type 'exit' to quit):")
|
||||
while True:
|
||||
try:
|
||||
user_input = input("\n> ")
|
||||
except (KeyboardInterrupt, EOFError):
|
||||
print("\nExiting.")
|
||||
break
|
||||
if user_input.lower() in {"exit", "quit"}:
|
||||
print("Goodbye!")
|
||||
break
|
||||
response = agent.invoke({"input": user_input})
|
||||
print(f"\n{response['output']}")
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="FAQ Bot CLI")
|
||||
parser.add_argument("--interactive", action="store_true", help="Start interactive mode")
|
||||
args = parser.parse_args()
|
||||
|
||||
agent = build_agent()
|
||||
|
||||
if args.interactive:
|
||||
interactive_mode(agent)
|
||||
else:
|
||||
run_preset_questions(agent)
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,4 @@
|
||||
from .cli import main
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,85 @@
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import List, Dict, Any
|
||||
|
||||
import httpx
|
||||
from langchain_community.document_loaders import TextLoader
|
||||
from langchain_community.embeddings import OllamaEmbeddings
|
||||
from langchain_community.vectorstores import Chroma
|
||||
from langchain_core.documents import Document
|
||||
from langchain_core.tools import tool
|
||||
|
||||
# Path to the data directory
|
||||
DATA_DIR = Path(__file__).parent.parent / "data"
|
||||
CHROMA_DIR = Path(__file__).parent.parent / "chroma_faq"
|
||||
|
||||
def load_faq_to_chroma() -> Chroma:
|
||||
"""
|
||||
Load all .md files from the data directory, chunk them, embed with Ollama,
|
||||
and persist into a Chroma vector store.
|
||||
"""
|
||||
# Check if the Chroma collection already exists
|
||||
if CHROMA_DIR.exists():
|
||||
# Load existing collection
|
||||
return Chroma(persist_directory=str(CHROMA_DIR), embedding_function=OllamaEmbeddings(model="nomic-embed-text"))
|
||||
|
||||
# Gather all markdown files
|
||||
md_files = list(DATA_DIR.glob("*.md"))
|
||||
documents: List[Document] = []
|
||||
|
||||
for md_file in md_files:
|
||||
loader = TextLoader(str(md_file), encoding="utf-8")
|
||||
docs = loader.load()
|
||||
documents.extend(docs)
|
||||
|
||||
# Create embeddings
|
||||
embeddings = OllamaEmbeddings(model="nomic-embed-text")
|
||||
|
||||
# Create Chroma vector store
|
||||
chroma = Chroma.from_documents(
|
||||
documents=documents,
|
||||
embedding=embeddings,
|
||||
persist_directory=str(CHROMA_DIR),
|
||||
)
|
||||
return chroma
|
||||
|
||||
@tool
|
||||
def search_course_docs(query: str, k: int = 3) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Search the local FAQ Chroma vector store for relevant documents.
|
||||
|
||||
Returns a list of dictionaries containing the content and metadata.
|
||||
"""
|
||||
chroma = load_faq_to_chroma()
|
||||
results = chroma.similarity_search(query, k=k)
|
||||
output = []
|
||||
for doc in results:
|
||||
output.append(
|
||||
{
|
||||
"content": doc.page_content,
|
||||
"metadata": doc.metadata,
|
||||
}
|
||||
)
|
||||
return output
|
||||
|
||||
@tool
|
||||
def fetch_course_meta(query: str) -> Dict[str, Any]:
|
||||
"""
|
||||
Simulate an MCP-style HTTP tool that returns course metadata
|
||||
matching the query. The metadata is read from a local JSON file.
|
||||
"""
|
||||
meta_path = DATA_DIR / "course_meta.json"
|
||||
with open(meta_path, "r", encoding="utf-8") as f:
|
||||
data = json.load(f)
|
||||
|
||||
# Simple keyword matching in schedule and instructor fields
|
||||
results = {}
|
||||
if "schedule" in query.lower():
|
||||
results["schedule"] = data.get("schedule", [])
|
||||
if "instructor" in query.lower() or "professor" in query.lower():
|
||||
results["instructor"] = data.get("instructor", {})
|
||||
if not results:
|
||||
# Default to returning the whole metadata if no keyword matched
|
||||
results = data
|
||||
return results
|
||||
Reference in New Issue
Block a user