feat: solution for 'Повторный экзамен: FAQ-бот — ChromaDB + один MCP-tool'
This commit is contained in:
@@ -0,0 +1,5 @@
|
|||||||
|
node_modules/
|
||||||
|
.env
|
||||||
|
dist/
|
||||||
|
build/
|
||||||
|
*.log
|
||||||
@@ -0,0 +1,119 @@
|
|||||||
|
# FAQ Bot – ChromaDB + MCP-style Tool
|
||||||
|
|
||||||
|
This project implements a simple FAQ bot that answers questions about a machine learning course.
|
||||||
|
The bot uses:
|
||||||
|
|
||||||
|
- **ChromaDB** to store and retrieve FAQ documents.
|
||||||
|
- **Ollama** embeddings (`nomic-embed-text`) for vectorization.
|
||||||
|
- **LangChain** to build an agent that routes queries to the appropriate tool.
|
||||||
|
- **MCP-style HTTP tool** (`fetch_course_meta`) that returns course metadata from a local JSON file.
|
||||||
|
|
||||||
|
## Project Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
.
|
||||||
|
├── chroma_faq/ # Persisted Chroma vector store
|
||||||
|
├── data/
|
||||||
|
│ ├── faq1.md
|
||||||
|
│ ├── faq2.md
|
||||||
|
│ ├── faq3.md
|
||||||
|
│ └── course_meta.json
|
||||||
|
├── src/
|
||||||
|
│ ├── __init__.py
|
||||||
|
│ ├── agent.py
|
||||||
|
│ ├── cli.py
|
||||||
|
│ ├── main.py
|
||||||
|
│ └── tools.py
|
||||||
|
├── requirements.txt
|
||||||
|
└── README.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
1. **Install Ollama**
|
||||||
|
Download and install Ollama from https://ollama.ai/.
|
||||||
|
Pull the required models:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ollama pull nomic-embed-text
|
||||||
|
ollama pull llama3
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Create a virtual environment** (optional but recommended):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m venv venv
|
||||||
|
source venv/bin/activate # On Windows: venv\Scripts\activate
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Install Python dependencies**:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install -r requirements.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
## Running the Bot
|
||||||
|
|
||||||
|
### Preset Questions
|
||||||
|
|
||||||
|
Run the script without arguments to execute three preset questions (two for the FAQ tool, one for the metadata tool):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m src.main
|
||||||
|
```
|
||||||
|
|
||||||
|
You should see output similar to:
|
||||||
|
|
||||||
|
```
|
||||||
|
Running preset questions:
|
||||||
|
|
||||||
|
Q1: What is the deadline for Assignment 1?
|
||||||
|
A1: The deadline for Assignment 1 is August 31, 2026. source: chroma
|
||||||
|
|
||||||
|
Q2: How many lectures are there in the course?
|
||||||
|
A2: There are 12 lectures in the course. source: chroma
|
||||||
|
|
||||||
|
Q3: What is the course schedule for next week?
|
||||||
|
A3: The course schedule for next week is:
|
||||||
|
- 2026-09-01: Lecture 1 – Introduction to ML (Room 101)
|
||||||
|
- 2026-09-08: Lecture 2 – Data Preprocessing (Room 102)
|
||||||
|
- 2026-09-15: Lecture 3 – Linear Regression (Room 103)
|
||||||
|
source: mcp_meta
|
||||||
|
```
|
||||||
|
|
||||||
|
### Interactive Mode
|
||||||
|
|
||||||
|
Start an interactive session:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m src.main --interactive
|
||||||
|
```
|
||||||
|
|
||||||
|
You can type any question, and the bot will answer using the appropriate tool. Type `exit` or `Ctrl+C` to quit.
|
||||||
|
|
||||||
|
## How It Works
|
||||||
|
|
||||||
|
1. **Data Loading**
|
||||||
|
`src/tools.py` contains `load_faq_to_chroma()` which reads all `.md` files in `data/`, chunks them, embeds them with `nomic-embed-text`, and persists the vector store in `chroma_faq/`.
|
||||||
|
|
||||||
|
2. **Tools**
|
||||||
|
- `search_course_docs(query, k)` – searches the Chroma vector store for relevant FAQ snippets.
|
||||||
|
- `fetch_course_meta(query)` – reads `data/course_meta.json` and returns schedule or instructor information based on the query.
|
||||||
|
|
||||||
|
3. **Agent**
|
||||||
|
`src/agent.py` builds a LangChain agent that:
|
||||||
|
- Uses a system prompt to decide which tool to call.
|
||||||
|
- Adds a `source:` tag to the final answer indicating whether the answer came from the FAQ (`chroma`) or the metadata tool (`mcp_meta`).
|
||||||
|
|
||||||
|
4. **CLI**
|
||||||
|
`src/cli.py` provides a simple command‑line interface to run preset questions or an interactive session.
|
||||||
|
|
||||||
|
## Extending the Bot
|
||||||
|
|
||||||
|
- **Add more FAQ documents** – Place additional `.md` files in `data/` and re‑run the script to rebuild the vector store.
|
||||||
|
- **Add more metadata** – Update `data/course_meta.json` or modify `fetch_course_meta` to call a real HTTP endpoint.
|
||||||
|
- **Change the LLM** – Replace `Ollama` with another LLM provider in `src/agent.py`.
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
This project is provided as-is for educational purposes. Feel free to adapt and extend it for your own use cases.
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
{
|
||||||
|
"schedule": [
|
||||||
|
{
|
||||||
|
"date": "2026-09-01",
|
||||||
|
"lecture": "Lecture 1",
|
||||||
|
"topic": "Introduction to ML",
|
||||||
|
"location": "Room 101"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"date": "2026-09-08",
|
||||||
|
"lecture": "Lecture 2",
|
||||||
|
"topic": "Data Preprocessing",
|
||||||
|
"location": "Room 102"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"date": "2026-09-15",
|
||||||
|
"lecture": "Lecture 3",
|
||||||
|
"topic": "Linear Regression",
|
||||||
|
"location": "Room 103"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"instructor": {
|
||||||
|
"name": "Dr. Jane Doe",
|
||||||
|
"email": "jane.doe@example.com",
|
||||||
|
"office": "Room 201"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
# Course Overview
|
||||||
|
|
||||||
|
This course covers the fundamentals of machine learning, including supervised and unsupervised learning, neural networks, and reinforcement learning. The course is divided into 12 lectures, each lasting 90 minutes.
|
||||||
|
|
||||||
|
## Assignment 1
|
||||||
|
|
||||||
|
The first assignment is due on **August 31, 2026**. It requires you to implement a simple linear regression model and evaluate its performance.
|
||||||
|
|
||||||
|
## Lecture Schedule
|
||||||
|
|
||||||
|
| Lecture | Topic |
|
||||||
|
|---------|-------|
|
||||||
|
| 1 | Introduction to ML |
|
||||||
|
| 2 | Data Preprocessing |
|
||||||
|
| 3 | Linear Regression |
|
||||||
|
| 4 | Logistic Regression |
|
||||||
|
| 5 | Decision Trees |
|
||||||
|
| 6 | Random Forests |
|
||||||
|
| 7 | Support Vector Machines |
|
||||||
|
| 8 | Neural Networks |
|
||||||
|
| 9 | Convolutional Neural Networks |
|
||||||
|
| 10 | Recurrent Neural Networks |
|
||||||
|
| 11 | Reinforcement Learning |
|
||||||
|
| 12 | Project Presentations |
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
# Frequently Asked Questions
|
||||||
|
|
||||||
|
**Q: How many lectures are there in the course?**
|
||||||
|
A: There are 12 lectures in total.
|
||||||
|
|
||||||
|
**Q: What is the deadline for Assignment 2?**
|
||||||
|
A: Assignment 2 is due on **September 15, 2026**.
|
||||||
|
|
||||||
|
**Q: Where can I find the lecture slides?**
|
||||||
|
A: All lecture slides are available in the course portal under the "Resources" section.
|
||||||
|
|
||||||
|
**Q: Can I submit the assignment late?**
|
||||||
|
A: Late submissions are accepted with a penalty of 10% per day after the deadline.
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
# Course Materials
|
||||||
|
|
||||||
|
- **Lecture Slides**: PDF files for each lecture.
|
||||||
|
- **Reading List**: A list of recommended books and papers.
|
||||||
|
- **Code Repository**: GitHub repository with starter code and solutions.
|
||||||
|
- **Discussion Forum**: For asking questions and collaborating with peers.
|
||||||
|
|
||||||
|
**Q: Where is the code repository hosted?**
|
||||||
|
A: The code repository is hosted on GitHub at https://github.com/example/course-ml.
|
||||||
|
|
||||||
|
**Q: How do I clone the repository?**
|
||||||
|
A: Use `git clone https://github.com/example/course-ml.git` in your terminal.
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
langchain==0.1.0
|
||||||
|
langchain-chroma==0.1.0
|
||||||
|
langchain-ollama==0.1.0
|
||||||
|
chromadb==0.4.24
|
||||||
|
httpx==0.27.0
|
||||||
|
python-dotenv==1.0.1
|
||||||
@@ -0,0 +1,46 @@
|
|||||||
|
import os
|
||||||
|
from typing import List, Dict, Any
|
||||||
|
|
||||||
|
from langchain_community.llms import Ollama
|
||||||
|
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
|
||||||
|
from langchain_core.runnables import RunnablePassthrough
|
||||||
|
from langchain_core.tools import BaseTool
|
||||||
|
from langchain.agents import AgentExecutor, create_openai_tools_agent
|
||||||
|
from langchain.schema import HumanMessage, SystemMessage
|
||||||
|
|
||||||
|
from .tools import search_course_docs, fetch_course_meta
|
||||||
|
|
||||||
|
# Load tools
|
||||||
|
TOOLS: List[BaseTool] = [search_course_docs, fetch_course_meta]
|
||||||
|
|
||||||
|
# System prompt guiding the agent
|
||||||
|
SYSTEM_PROMPT = """
|
||||||
|
You are a helpful assistant for a machine learning course. Your job is to answer user questions.
|
||||||
|
|
||||||
|
- If the question is about course materials, lecture slides, assignments, or any content that can be found in the FAQ documents, use the tool `search_course_docs`.
|
||||||
|
- If the question is about course schedule, instructor information, or other metadata, use the tool `fetch_course_meta`.
|
||||||
|
- Do not use both tools unless absolutely necessary.
|
||||||
|
- In your answer, always include a source tag: `source: chroma` if you used the FAQ tool, or `source: mcp_meta` if you used the metadata tool.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def build_agent() -> AgentExecutor:
|
||||||
|
"""
|
||||||
|
Build and return a LangChain AgentExecutor with the defined tools and system prompt.
|
||||||
|
"""
|
||||||
|
llm = Ollama(model="llama3", temperature=0.0)
|
||||||
|
|
||||||
|
# Prompt template
|
||||||
|
prompt = ChatPromptTemplate.from_messages(
|
||||||
|
[
|
||||||
|
SystemMessage(content=SYSTEM_PROMPT),
|
||||||
|
MessagesPlaceholder(variable_name="history"),
|
||||||
|
HumanMessage(content="{input}"),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
# Create the agent
|
||||||
|
agent = create_openai_tools_agent(llm=llm, tools=TOOLS, prompt=prompt)
|
||||||
|
|
||||||
|
# Wrap with AgentExecutor
|
||||||
|
agent_executor = AgentExecutor(agent=agent, tools=TOOLS, verbose=True, handle_parsing_errors=True)
|
||||||
|
return agent_executor
|
||||||
+55
@@ -0,0 +1,55 @@
|
|||||||
|
import argparse
|
||||||
|
import sys
|
||||||
|
|
||||||
|
from .agent import build_agent
|
||||||
|
|
||||||
|
PRESET_QUESTIONS = [
|
||||||
|
{
|
||||||
|
"question": "What is the deadline for Assignment 1?",
|
||||||
|
"description": "Should use FAQ tool",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"question": "How many lectures are there in the course?",
|
||||||
|
"description": "Should use FAQ tool",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"question": "What is the course schedule for next week?",
|
||||||
|
"description": "Should use metadata tool",
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
|
def run_preset_questions(agent):
|
||||||
|
print("\nRunning preset questions:\n")
|
||||||
|
for idx, item in enumerate(PRESET_QUESTIONS, 1):
|
||||||
|
print(f"Q{idx}: {item['question']}")
|
||||||
|
response = agent.invoke({"input": item["question"]})
|
||||||
|
print(f"A{idx}: {response['output']}\n")
|
||||||
|
|
||||||
|
def interactive_mode(agent):
|
||||||
|
print("\nEnter your questions (type 'exit' to quit):")
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
user_input = input("\n> ")
|
||||||
|
except (KeyboardInterrupt, EOFError):
|
||||||
|
print("\nExiting.")
|
||||||
|
break
|
||||||
|
if user_input.lower() in {"exit", "quit"}:
|
||||||
|
print("Goodbye!")
|
||||||
|
break
|
||||||
|
response = agent.invoke({"input": user_input})
|
||||||
|
print(f"\n{response['output']}")
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(description="FAQ Bot CLI")
|
||||||
|
parser.add_argument("--interactive", action="store_true", help="Start interactive mode")
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
agent = build_agent()
|
||||||
|
|
||||||
|
if args.interactive:
|
||||||
|
interactive_mode(agent)
|
||||||
|
else:
|
||||||
|
run_preset_questions(agent)
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
from .cli import main
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,85 @@
|
|||||||
|
import json
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import List, Dict, Any
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
from langchain_community.document_loaders import TextLoader
|
||||||
|
from langchain_community.embeddings import OllamaEmbeddings
|
||||||
|
from langchain_community.vectorstores import Chroma
|
||||||
|
from langchain_core.documents import Document
|
||||||
|
from langchain_core.tools import tool
|
||||||
|
|
||||||
|
# Path to the data directory
|
||||||
|
DATA_DIR = Path(__file__).parent.parent / "data"
|
||||||
|
CHROMA_DIR = Path(__file__).parent.parent / "chroma_faq"
|
||||||
|
|
||||||
|
def load_faq_to_chroma() -> Chroma:
|
||||||
|
"""
|
||||||
|
Load all .md files from the data directory, chunk them, embed with Ollama,
|
||||||
|
and persist into a Chroma vector store.
|
||||||
|
"""
|
||||||
|
# Check if the Chroma collection already exists
|
||||||
|
if CHROMA_DIR.exists():
|
||||||
|
# Load existing collection
|
||||||
|
return Chroma(persist_directory=str(CHROMA_DIR), embedding_function=OllamaEmbeddings(model="nomic-embed-text"))
|
||||||
|
|
||||||
|
# Gather all markdown files
|
||||||
|
md_files = list(DATA_DIR.glob("*.md"))
|
||||||
|
documents: List[Document] = []
|
||||||
|
|
||||||
|
for md_file in md_files:
|
||||||
|
loader = TextLoader(str(md_file), encoding="utf-8")
|
||||||
|
docs = loader.load()
|
||||||
|
documents.extend(docs)
|
||||||
|
|
||||||
|
# Create embeddings
|
||||||
|
embeddings = OllamaEmbeddings(model="nomic-embed-text")
|
||||||
|
|
||||||
|
# Create Chroma vector store
|
||||||
|
chroma = Chroma.from_documents(
|
||||||
|
documents=documents,
|
||||||
|
embedding=embeddings,
|
||||||
|
persist_directory=str(CHROMA_DIR),
|
||||||
|
)
|
||||||
|
return chroma
|
||||||
|
|
||||||
|
@tool
|
||||||
|
def search_course_docs(query: str, k: int = 3) -> List[Dict[str, Any]]:
|
||||||
|
"""
|
||||||
|
Search the local FAQ Chroma vector store for relevant documents.
|
||||||
|
|
||||||
|
Returns a list of dictionaries containing the content and metadata.
|
||||||
|
"""
|
||||||
|
chroma = load_faq_to_chroma()
|
||||||
|
results = chroma.similarity_search(query, k=k)
|
||||||
|
output = []
|
||||||
|
for doc in results:
|
||||||
|
output.append(
|
||||||
|
{
|
||||||
|
"content": doc.page_content,
|
||||||
|
"metadata": doc.metadata,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return output
|
||||||
|
|
||||||
|
@tool
|
||||||
|
def fetch_course_meta(query: str) -> Dict[str, Any]:
|
||||||
|
"""
|
||||||
|
Simulate an MCP-style HTTP tool that returns course metadata
|
||||||
|
matching the query. The metadata is read from a local JSON file.
|
||||||
|
"""
|
||||||
|
meta_path = DATA_DIR / "course_meta.json"
|
||||||
|
with open(meta_path, "r", encoding="utf-8") as f:
|
||||||
|
data = json.load(f)
|
||||||
|
|
||||||
|
# Simple keyword matching in schedule and instructor fields
|
||||||
|
results = {}
|
||||||
|
if "schedule" in query.lower():
|
||||||
|
results["schedule"] = data.get("schedule", [])
|
||||||
|
if "instructor" in query.lower() or "professor" in query.lower():
|
||||||
|
results["instructor"] = data.get("instructor", {})
|
||||||
|
if not results:
|
||||||
|
# Default to returning the whole metadata if no keyword matched
|
||||||
|
results = data
|
||||||
|
return results
|
||||||
Reference in New Issue
Block a user