115 lines
3.2 KiB
Markdown
115 lines
3.2 KiB
Markdown
# Stream‑Mode AI Agent
|
||
|
||
A lightweight LangChain agent that streams its output token‑by‑token instead of waiting for the whole response.
|
||
The project demonstrates how to replace a single `.invoke()` call with `.stream()`, giving instant feedback in the console.
|
||
|
||
---
|
||
|
||
## 📖 Description
|
||
|
||
- **Stack**: [LangChain](https://github.com/langchain-ai/langchain), `create_agent`, `@tool` decorator, and the [`rich`](https://github.com/Textualize/rich) library for pretty console output.
|
||
- **LLM**: Ollama’s local `llama3` model (any LLM that supports streaming can be used).
|
||
- **Files**
|
||
- `agent.py`: Defines the agent, tools, and the streaming logic.
|
||
- `client.py`: Simple CLI client to interact with the agent.
|
||
|
||
The agent now streams its answer token by token, so you see the response appear in real time as it is generated. This is especially useful for long answers or when multiple tools are invoked sequentially.
|
||
|
||
---
|
||
|
||
## ⚙️ Prerequisites
|
||
|
||
| Item | Version |
|
||
|------|---------|
|
||
| Python | 3.10+ |
|
||
| Ollama | ≥ 0.1 (must have `llama3` model downloaded) |
|
||
| pip packages | See `requirements.txt` |
|
||
|
||
> **Tip**: If you don’t have a local LLM, you can replace the `OllamaLLM` with any LangChain‑compatible LLM that supports streaming (e.g., OpenAI, Anthropic).
|
||
|
||
---
|
||
|
||
## 📦 Installation
|
||
|
||
```bash
|
||
# 1. Clone the repo
|
||
git clone https://github.com/your-username/stream-ai-agent.git
|
||
cd stream-ai-agent
|
||
|
||
# 2. Create a virtual environment (optional but recommended)
|
||
python -m venv .venv
|
||
source .venv/bin/activate # Windows: .\.venv\Scripts\activate
|
||
|
||
# 3. Install dependencies
|
||
pip install -r requirements.txt
|
||
```
|
||
|
||
`requirements.txt` contains:
|
||
|
||
```text
|
||
langchain>=0.2
|
||
langchain-ollama>=0.1
|
||
rich>=13.0
|
||
python-dotenv>=1.0 # optional, for .env support
|
||
```
|
||
|
||
---
|
||
|
||
## 🚀 Running the Agent
|
||
|
||
```bash
|
||
# From the project root
|
||
python client.py
|
||
```
|
||
|
||
You’ll be prompted to enter a question. The agent will stream its answer directly to the console.
|
||
|
||
### Example Interaction
|
||
|
||
```
|
||
$ python client.py
|
||
Enter your question: What is the capital of France?
|
||
|
||
🤖 (streaming) ...
|
||
🤖 (streaming) Paris
|
||
🤖 (streaming) is the capital city of France, known for its art, culture, and history.
|
||
✅ Done!
|
||
```
|
||
|
||
The `rich` library formats the output with a spinner while streaming and prints the final answer in bold.
|
||
|
||
---
|
||
|
||
## 📁 Project Structure
|
||
|
||
```text
|
||
├── agent.py # Agent definition + tools + stream logic
|
||
├── client.py # CLI wrapper to interact with the agent
|
||
├── requirements.txt
|
||
└── README.md
|
||
```
|
||
|
||
- **agent.py**
|
||
- `check_wish` tool: a simple example that echoes back a user’s wish.
|
||
- `create_agent(...)`: builds an agent that uses the streaming LLM and prints tokens as they arrive.
|
||
|
||
- **client.py**
|
||
- Reads user input, passes it to the agent, and handles the streaming output.
|
||
|
||
---
|
||
|
||
## 🛠️ Customization
|
||
|
||
1. **Add more tools**: Decorate any function with `@tool` and include it in the `tools` list passed to `create_agent`.
|
||
2. **Change LLM**: Swap `OllamaLLM` for another LangChain LLM that supports streaming.
|
||
3. **Styling**: Modify the `rich` console output (e.g., colors, spinner style) by editing `client.py`.
|
||
|
||
---
|
||
|
||
## 📄 License
|
||
|
||
MIT © 2026 Your Name
|
||
|
||
---
|
||
|
||
Happy streaming! 🚀 |