Stream‑Mode AI Agent

A lightweight LangChain agent that streams its output token by token using the agent.stream() API. The project demonstrates how to replace a single .invoke() call with streaming so that responses appear in real time.

📖 Description

Stack:
- langchain – core framework for building agents.
- langchain_ollama – Ollama wrapper for local LLM inference.
- @tool decorator – simple tool definition.
- rich.print – pretty console output.
Goal:
Replace the blocking .invoke() call with a streaming version so that the agent’s answer is printed as it is generated, improving user experience for long responses or multiple tool calls.

⚙️ Prerequisites

Item	Version
Python	≥3.10
Ollama	Installed locally (see Ollama docs)
LLM model	Any Ollama‑compatible model (e.g., `llama2`, `mistral`)

Tip

: Ensure the chosen model is already pulled to your local Ollama instance:
ollama pull llama2

🚀 Installation

# 1. Clone the repo
git clone https://github.com/your-username/stream-ai-agent.git
cd stream-ai-agent

# 2. Create a virtual environment (optional but recommended)
python -m venv .venv
source .venv/bin/activate   # On Windows: .\.venv\Scripts\activate

# 3. Install dependencies
pip install --upgrade pip
pip install langchain langchain_ollama rich

🏃‍♂️ Running the Agent

python solution.py

The script will:

Connect to your local Ollama instance.
Create a simple agent with one custom tool (@tool).
Prompt the user for a question.
Stream the answer directly to the console.

📋 Example Interaction

$ python solution.py
Enter your question: What is the capital of France?

Answer:
The capital of France is Paris. It is known for its rich history, culture, and iconic landmarks such as the Eiffel Tower, Louvre Museum, and Notre-Dame Cathedral.

Notice how each token appears immediately after it is generated, rather than waiting for the entire response.

🔧 Customization

Change LLM: Edit model_name in solution.py.
Add Tools: Decorate new functions with @tool and include them when creating the agent.
Adjust Prompt: Modify the initial prompt or add system messages as needed.

2.4 KiB Raw Permalink Blame History Unescape Escape