Stream‑Mode AI Agent

A lightweight LangChain agent that streams its output token‑by‑token instead of waiting for the whole response.
The project demonstrates how to replace a single .invoke() call with .stream(), giving instant feedback in the console.

📖 Description

Stack: LangChain, create_agent, @tool decorator, and the rich library for pretty console output.
LLM: Ollama’s local llama3 model (any LLM that supports streaming can be used).
Files
- agent.py: Defines the agent, tools, and the streaming logic.
- client.py: Simple CLI client to interact with the agent.

The agent now streams its answer token by token, so you see the response appear in real time as it is generated. This is especially useful for long answers or when multiple tools are invoked sequentially.

⚙️ Prerequisites

Item	Version
Python	3.10+
Ollama	≥ 0.1 (must have `llama3` model downloaded)
pip packages	See `requirements.txt`

Tip

: If you don’t have a local LLM, you can replace the OllamaLLM with any LangChain‑compatible LLM that supports streaming (e.g., OpenAI, Anthropic).

📦 Installation

# 1. Clone the repo
git clone https://github.com/your-username/stream-ai-agent.git
cd stream-ai-agent

# 2. Create a virtual environment (optional but recommended)
python -m venv .venv
source .venv/bin/activate   # Windows: .\.venv\Scripts\activate

# 3. Install dependencies
pip install -r requirements.txt

requirements.txt contains:

langchain>=0.2
langchain-ollama>=0.1
rich>=13.0
python-dotenv>=1.0   # optional, for .env support

🚀 Running the Agent

# From the project root
python client.py

You’ll be prompted to enter a question. The agent will stream its answer directly to the console.

Example Interaction

$ python client.py
Enter your question: What is the capital of France?

🤖  (streaming) ... 
🤖  (streaming) Paris
🤖  (streaming) is the capital city of France, known for its art, culture, and history.
✅  Done!

The rich library formats the output with a spinner while streaming and prints the final answer in bold.

📁 Project Structure

├── agent.py      # Agent definition + tools + stream logic
├── client.py     # CLI wrapper to interact with the agent
├── requirements.txt
└── README.md

agent.py
- check_wish tool: a simple example that echoes back a user’s wish.
- create_agent(...): builds an agent that uses the streaming LLM and prints tokens as they arrive.
client.py
- Reads user input, passes it to the agent, and handles the streaming output.

🛠️ Customization

Add more tools: Decorate any function with @tool and include it in the tools list passed to create_agent.
Change LLM: Swap OllamaLLM for another LangChain LLM that supports streaming.
Styling: Modify the rich console output (e.g., colors, spinner style) by editing client.py.

📄 License

Happy streaming! 🚀

3.2 KiB Raw Blame History Unescape Escape