Stream-режим AI-агента: README.md
This commit is contained in:
@@ -1,19 +1,20 @@
|
||||
# Stream‑Mode AI Agent
|
||||
|
||||
A lightweight LangChain agent that streams its output token‑by‑token instead of waiting for the whole response.
|
||||
The project demonstrates how to replace a single `.invoke()` call with `.stream()`, giving instant feedback in the console.
|
||||
A lightweight LangChain agent that streams its output token by token using the `agent.stream()` API.
|
||||
The project demonstrates how to replace a single `.invoke()` call with streaming so that responses appear in real time.
|
||||
|
||||
---
|
||||
|
||||
## 📖 Description
|
||||
|
||||
- **Stack**: [LangChain](https://github.com/langchain-ai/langchain), `create_agent`, `@tool` decorator, and the [`rich`](https://github.com/Textualize/rich) library for pretty console output.
|
||||
- **LLM**: Ollama’s local `llama3` model (any LLM that supports streaming can be used).
|
||||
- **Files**
|
||||
- `agent.py`: Defines the agent, tools, and the streaming logic.
|
||||
- `client.py`: Simple CLI client to interact with the agent.
|
||||
- **Stack**:
|
||||
- `langchain` – core framework for building agents.
|
||||
- `langchain_ollama` – Ollama wrapper for local LLM inference.
|
||||
- `@tool` decorator – simple tool definition.
|
||||
- `rich.print` – pretty console output.
|
||||
|
||||
The agent now streams its answer token by token, so you see the response appear in real time as it is generated. This is especially useful for long answers or when multiple tools are invoked sequentially.
|
||||
- **Goal**:
|
||||
Replace the blocking `.invoke()` call with a streaming version so that the agent’s answer is printed as it is generated, improving user experience for long responses or multiple tool calls.
|
||||
|
||||
---
|
||||
|
||||
@@ -21,15 +22,18 @@ The agent now streams its answer token by token, so you see the response appear
|
||||
|
||||
| Item | Version |
|
||||
|------|---------|
|
||||
| Python | 3.10+ |
|
||||
| Ollama | ≥ 0.1 (must have `llama3` model downloaded) |
|
||||
| pip packages | See `requirements.txt` |
|
||||
| Python | ≥3.10 |
|
||||
| Ollama | Installed locally (see [Ollama docs](https://ollama.com)) |
|
||||
| LLM model | Any Ollama‑compatible model (e.g., `llama2`, `mistral`) |
|
||||
|
||||
> **Tip**: If you don’t have a local LLM, you can replace the `OllamaLLM` with any LangChain‑compatible LLM that supports streaming (e.g., OpenAI, Anthropic).
|
||||
> **Tip**: Ensure the chosen model is already pulled to your local Ollama instance:
|
||||
> ```bash
|
||||
> ollama pull llama2
|
||||
> ```
|
||||
|
||||
---
|
||||
|
||||
## 📦 Installation
|
||||
## 🚀 Installation
|
||||
|
||||
```bash
|
||||
# 1. Clone the repo
|
||||
@@ -38,78 +42,54 @@ cd stream-ai-agent
|
||||
|
||||
# 2. Create a virtual environment (optional but recommended)
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate # Windows: .\.venv\Scripts\activate
|
||||
source .venv/bin/activate # On Windows: .\.venv\Scripts\activate
|
||||
|
||||
# 3. Install dependencies
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
`requirements.txt` contains:
|
||||
|
||||
```text
|
||||
langchain>=0.2
|
||||
langchain-ollama>=0.1
|
||||
rich>=13.0
|
||||
python-dotenv>=1.0 # optional, for .env support
|
||||
pip install --upgrade pip
|
||||
pip install langchain langchain_ollama rich
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Running the Agent
|
||||
## 🏃♂️ Running the Agent
|
||||
|
||||
```bash
|
||||
# From the project root
|
||||
python client.py
|
||||
python solution.py
|
||||
```
|
||||
|
||||
You’ll be prompted to enter a question. The agent will stream its answer directly to the console.
|
||||
The script will:
|
||||
|
||||
### Example Interaction
|
||||
1. Connect to your local Ollama instance.
|
||||
2. Create a simple agent with one custom tool (`@tool`).
|
||||
3. Prompt the user for a question.
|
||||
4. Stream the answer directly to the console.
|
||||
|
||||
---
|
||||
|
||||
## 📋 Example Interaction
|
||||
|
||||
```
|
||||
$ python client.py
|
||||
$ python solution.py
|
||||
Enter your question: What is the capital of France?
|
||||
|
||||
🤖 (streaming) ...
|
||||
🤖 (streaming) Paris
|
||||
🤖 (streaming) is the capital city of France, known for its art, culture, and history.
|
||||
✅ Done!
|
||||
Answer:
|
||||
The capital of France is Paris. It is known for its rich history, culture, and iconic landmarks such as the Eiffel Tower, Louvre Museum, and Notre-Dame Cathedral.
|
||||
```
|
||||
|
||||
The `rich` library formats the output with a spinner while streaming and prints the final answer in bold.
|
||||
Notice how each token appears immediately after it is generated, rather than waiting for the entire response.
|
||||
|
||||
---
|
||||
|
||||
## 📁 Project Structure
|
||||
## 🔧 Customization
|
||||
|
||||
```text
|
||||
├── agent.py # Agent definition + tools + stream logic
|
||||
├── client.py # CLI wrapper to interact with the agent
|
||||
├── requirements.txt
|
||||
└── README.md
|
||||
```
|
||||
|
||||
- **agent.py**
|
||||
- `check_wish` tool: a simple example that echoes back a user’s wish.
|
||||
- `create_agent(...)`: builds an agent that uses the streaming LLM and prints tokens as they arrive.
|
||||
|
||||
- **client.py**
|
||||
- Reads user input, passes it to the agent, and handles the streaming output.
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Customization
|
||||
|
||||
1. **Add more tools**: Decorate any function with `@tool` and include it in the `tools` list passed to `create_agent`.
|
||||
2. **Change LLM**: Swap `OllamaLLM` for another LangChain LLM that supports streaming.
|
||||
3. **Styling**: Modify the `rich` console output (e.g., colors, spinner style) by editing `client.py`.
|
||||
- **Change LLM**: Edit `model_name` in `solution.py`.
|
||||
- **Add Tools**: Decorate new functions with `@tool` and include them when creating the agent.
|
||||
- **Adjust Prompt**: Modify the initial prompt or add system messages as needed.
|
||||
|
||||
---
|
||||
|
||||
## 📄 License
|
||||
|
||||
MIT © 2026 Your Name
|
||||
MIT © 2026
|
||||
|
||||
---
|
||||
|
||||
Happy streaming! 🚀
|
||||
Reference in New Issue
Block a user