95 lines
2.4 KiB
Markdown
95 lines
2.4 KiB
Markdown
# Stream‑Mode AI Agent
|
||
|
||
A lightweight LangChain agent that streams its output token by token using the `agent.stream()` API.
|
||
The project demonstrates how to replace a single `.invoke()` call with streaming so that responses appear in real time.
|
||
|
||
---
|
||
|
||
## 📖 Description
|
||
|
||
- **Stack**:
|
||
- `langchain` – core framework for building agents.
|
||
- `langchain_ollama` – Ollama wrapper for local LLM inference.
|
||
- `@tool` decorator – simple tool definition.
|
||
- `rich.print` – pretty console output.
|
||
|
||
- **Goal**:
|
||
Replace the blocking `.invoke()` call with a streaming version so that the agent’s answer is printed as it is generated, improving user experience for long responses or multiple tool calls.
|
||
|
||
---
|
||
|
||
## ⚙️ Prerequisites
|
||
|
||
| Item | Version |
|
||
|------|---------|
|
||
| Python | ≥3.10 |
|
||
| Ollama | Installed locally (see [Ollama docs](https://ollama.com)) |
|
||
| LLM model | Any Ollama‑compatible model (e.g., `llama2`, `mistral`) |
|
||
|
||
> **Tip**: Ensure the chosen model is already pulled to your local Ollama instance:
|
||
> ```bash
|
||
> ollama pull llama2
|
||
> ```
|
||
|
||
---
|
||
|
||
## 🚀 Installation
|
||
|
||
```bash
|
||
# 1. Clone the repo
|
||
git clone https://github.com/your-username/stream-ai-agent.git
|
||
cd stream-ai-agent
|
||
|
||
# 2. Create a virtual environment (optional but recommended)
|
||
python -m venv .venv
|
||
source .venv/bin/activate # On Windows: .\.venv\Scripts\activate
|
||
|
||
# 3. Install dependencies
|
||
pip install --upgrade pip
|
||
pip install langchain langchain_ollama rich
|
||
```
|
||
|
||
---
|
||
|
||
## 🏃♂️ Running the Agent
|
||
|
||
```bash
|
||
python solution.py
|
||
```
|
||
|
||
The script will:
|
||
|
||
1. Connect to your local Ollama instance.
|
||
2. Create a simple agent with one custom tool (`@tool`).
|
||
3. Prompt the user for a question.
|
||
4. Stream the answer directly to the console.
|
||
|
||
---
|
||
|
||
## 📋 Example Interaction
|
||
|
||
```
|
||
$ python solution.py
|
||
Enter your question: What is the capital of France?
|
||
|
||
Answer:
|
||
The capital of France is Paris. It is known for its rich history, culture, and iconic landmarks such as the Eiffel Tower, Louvre Museum, and Notre-Dame Cathedral.
|
||
```
|
||
|
||
Notice how each token appears immediately after it is generated, rather than waiting for the entire response.
|
||
|
||
---
|
||
|
||
## 🔧 Customization
|
||
|
||
- **Change LLM**: Edit `model_name` in `solution.py`.
|
||
- **Add Tools**: Decorate new functions with `@tool` and include them when creating the agent.
|
||
- **Adjust Prompt**: Modify the initial prompt or add system messages as needed.
|
||
|
||
---
|
||
|
||
## 📄 License
|
||
|
||
MIT © 2026
|
||
|
||
--- |