Stream-режим AI-агента: README.md
This commit is contained in:
@@ -1,107 +1,115 @@
|
||||
# Stream‑Mode AI Agent
|
||||
|
||||
## 📖 Описание проекта
|
||||
Проект реализует **AI‑агента**, способного выполнять задачи, используя:
|
||||
- **LangChain** – фреймворк для построения цепочек и агентов;
|
||||
- **Ollama** – локальный LLM (model `llama3`);
|
||||
- **Qdrant** – векторная база данных для хранения эмбеддингов;
|
||||
- **langchain‑qdrant** – интеграция LangChain с Qdrant;
|
||||
- **langchain‑ollama** – обёртка Ollama для LangChain.
|
||||
|
||||
Главной особенностью является переход от синхронного вызова `.invoke()` к потоковому выводу через `.stream()`. Теперь ответы агента появляются в консоли токен за токеном, что делает взаимодействие более отзывчивым и похожим на ChatGPT.
|
||||
|
||||
## ⚙️ Предварительные требования
|
||||
- **Python 3.10+**
|
||||
- **Ollama** (установить можно по инструкции: https://ollama.ai/)
|
||||
```bash
|
||||
curl -fsSL https://ollama.com/install.sh | sh
|
||||
```
|
||||
После установки скачайте нужную модель:
|
||||
```bash
|
||||
ollama pull llama3
|
||||
```
|
||||
- **Qdrant**
|
||||
Можно запустить локально через Docker:
|
||||
```bash
|
||||
docker run -p 6333:6333 qdrant/qdrant
|
||||
```
|
||||
|
||||
## 📦 Установка зависимостей
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
`requirements.txt` содержит все необходимые пакеты, включая `langchain`, `langchain-ollama`, `langchain-qdrant`, `qdrant-client` и др.
|
||||
|
||||
## 🚀 Запуск
|
||||
|
||||
### 1. Инициализация агента (agent.py)
|
||||
```bash
|
||||
python agent.py
|
||||
```
|
||||
При запуске агент будет ждать ввода команды в консоли, например:
|
||||
```
|
||||
> What is the capital of France?
|
||||
```
|
||||
|
||||
### 2. Пример использования с потоковым выводом
|
||||
После запуска вы можете вводить запросы. Ответ будет печататься токен за токеном:
|
||||
|
||||
```
|
||||
> Tell me a short story about a robot learning to dance.
|
||||
Robot: I was built in a factory...
|
||||
Robot: ...and then I discovered music...
|
||||
Robot: ...
|
||||
```
|
||||
|
||||
### 3. Остановка
|
||||
Нажмите `Ctrl+C` или введите команду `exit`.
|
||||
|
||||
## 📄 Пример использования кода
|
||||
|
||||
```python
|
||||
from langchain.agents import create_agent, AgentExecutor
|
||||
from langchain.tools import tool
|
||||
from langchain_ollama import OllamaLLM
|
||||
from langchain_qdrant import QdrantStore
|
||||
import os
|
||||
|
||||
# LLM
|
||||
llm = OllamaLLM(model="llama3", temperature=0.7, timeout=60)
|
||||
|
||||
# Векторная база (Qdrant)
|
||||
store = QdrantStore(
|
||||
url=os.getenv("QDRANT_URL", "http://localhost:6333"),
|
||||
collection_name="my_collection",
|
||||
embedding_function=OllamaLLM(model="nomic-embed-text")
|
||||
)
|
||||
|
||||
# Инструмент
|
||||
@tool
|
||||
def check_wish(wish: str) -> str:
|
||||
"""Проверка желания на наличие подвоха."""
|
||||
return f"Wish '{wish}' looks safe."
|
||||
|
||||
tools = [check_wish]
|
||||
|
||||
# Агент
|
||||
agent = create_agent(
|
||||
llm=llm,
|
||||
tools=tools,
|
||||
verbose=True
|
||||
)
|
||||
|
||||
executor = AgentExecutor(agent=agent, tools=tools, max_iterations=5)
|
||||
|
||||
# Потоковый вывод
|
||||
for chunk in executor.stream("Will I get a promotion?"):
|
||||
print(chunk, end="", flush=True)
|
||||
```
|
||||
|
||||
## 📚 Что дальше?
|
||||
- Добавить более сложные инструменты (запросы к API, работа с файлами и т.д.).
|
||||
- Настроить хранение истории диалога в Qdrant для контекстуального ответа.
|
||||
- Интегрировать с веб‑интерфейсом или чат‑ботом.
|
||||
A lightweight LangChain agent that streams its output token‑by‑token instead of waiting for the whole response.
|
||||
The project demonstrates how to replace a single `.invoke()` call with `.stream()`, giving instant feedback in the console.
|
||||
|
||||
---
|
||||
|
||||
**Happy coding!**
|
||||
## 📖 Description
|
||||
|
||||
- **Stack**: [LangChain](https://github.com/langchain-ai/langchain), `create_agent`, `@tool` decorator, and the [`rich`](https://github.com/Textualize/rich) library for pretty console output.
|
||||
- **LLM**: Ollama’s local `llama3` model (any LLM that supports streaming can be used).
|
||||
- **Files**
|
||||
- `agent.py`: Defines the agent, tools, and the streaming logic.
|
||||
- `client.py`: Simple CLI client to interact with the agent.
|
||||
|
||||
The agent now streams its answer token by token, so you see the response appear in real time as it is generated. This is especially useful for long answers or when multiple tools are invoked sequentially.
|
||||
|
||||
---
|
||||
|
||||
## ⚙️ Prerequisites
|
||||
|
||||
| Item | Version |
|
||||
|------|---------|
|
||||
| Python | 3.10+ |
|
||||
| Ollama | ≥ 0.1 (must have `llama3` model downloaded) |
|
||||
| pip packages | See `requirements.txt` |
|
||||
|
||||
> **Tip**: If you don’t have a local LLM, you can replace the `OllamaLLM` with any LangChain‑compatible LLM that supports streaming (e.g., OpenAI, Anthropic).
|
||||
|
||||
---
|
||||
|
||||
## 📦 Installation
|
||||
|
||||
```bash
|
||||
# 1. Clone the repo
|
||||
git clone https://github.com/your-username/stream-ai-agent.git
|
||||
cd stream-ai-agent
|
||||
|
||||
# 2. Create a virtual environment (optional but recommended)
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate # Windows: .\.venv\Scripts\activate
|
||||
|
||||
# 3. Install dependencies
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
`requirements.txt` contains:
|
||||
|
||||
```text
|
||||
langchain>=0.2
|
||||
langchain-ollama>=0.1
|
||||
rich>=13.0
|
||||
python-dotenv>=1.0 # optional, for .env support
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Running the Agent
|
||||
|
||||
```bash
|
||||
# From the project root
|
||||
python client.py
|
||||
```
|
||||
|
||||
You’ll be prompted to enter a question. The agent will stream its answer directly to the console.
|
||||
|
||||
### Example Interaction
|
||||
|
||||
```
|
||||
$ python client.py
|
||||
Enter your question: What is the capital of France?
|
||||
|
||||
🤖 (streaming) ...
|
||||
🤖 (streaming) Paris
|
||||
🤖 (streaming) is the capital city of France, known for its art, culture, and history.
|
||||
✅ Done!
|
||||
```
|
||||
|
||||
The `rich` library formats the output with a spinner while streaming and prints the final answer in bold.
|
||||
|
||||
---
|
||||
|
||||
## 📁 Project Structure
|
||||
|
||||
```text
|
||||
├── agent.py # Agent definition + tools + stream logic
|
||||
├── client.py # CLI wrapper to interact with the agent
|
||||
├── requirements.txt
|
||||
└── README.md
|
||||
```
|
||||
|
||||
- **agent.py**
|
||||
- `check_wish` tool: a simple example that echoes back a user’s wish.
|
||||
- `create_agent(...)`: builds an agent that uses the streaming LLM and prints tokens as they arrive.
|
||||
|
||||
- **client.py**
|
||||
- Reads user input, passes it to the agent, and handles the streaming output.
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Customization
|
||||
|
||||
1. **Add more tools**: Decorate any function with `@tool` and include it in the `tools` list passed to `create_agent`.
|
||||
2. **Change LLM**: Swap `OllamaLLM` for another LangChain LLM that supports streaming.
|
||||
3. **Styling**: Modify the `rich` console output (e.g., colors, spinner style) by editing `client.py`.
|
||||
|
||||
---
|
||||
|
||||
## 📄 License
|
||||
|
||||
MIT © 2026 Your Name
|
||||
|
||||
---
|
||||
|
||||
Happy streaming! 🚀
|
||||
Reference in New Issue
Block a user