110 lines
3.0 KiB
Markdown
110 lines
3.0 KiB
Markdown
# Stream‑Mode AI Agent
|
||
|
||
A lightweight Python project that demonstrates how to run a LangChain agent in **streaming mode** using the `langchain-ollama` and `langgraph` libraries.
|
||
The agent is defined in `agent.py`. Instead of waiting for the entire response, it streams tokens back to the console as they are generated.
|
||
|
||
---
|
||
|
||
## Table of Contents
|
||
|
||
- [What It Does](#what-it-does)
|
||
- [Prerequisites](#prerequisites)
|
||
- [Installation](#installation)
|
||
- [Running the Agent](#running-the-agent)
|
||
- [Example Usage](#example-usage)
|
||
|
||
---
|
||
|
||
## What It Does
|
||
|
||
* Implements a simple LangChain agent that can call external tools (e.g., web search, calculator).
|
||
* Uses `ChatOllama` as the LLM backend.
|
||
* Streams the generated answer token‑by‑token to the console with `agent.stream()`.
|
||
* Works out of the box on any machine that has Ollama installed locally.
|
||
|
||
---
|
||
|
||
## Prerequisites
|
||
|
||
| Component | Minimum Version | Notes |
|
||
|-----------|-----------------|-------|
|
||
| **Python** | 3.10+ | Tested on 3.11 |
|
||
| **Ollama** | Latest stable | Must have a model (default: `llama3`) downloaded. |
|
||
| **Nomic Embed Text** | Latest | For embeddings used by the agent. |
|
||
| **Qdrant** | Optional | If you want to persist embeddings locally. |
|
||
|
||
> **Installing Ollama**
|
||
> ```bash
|
||
> curl -fsSL https://ollama.com/install.sh | sh
|
||
> ollama pull llama3 # or any other model you prefer
|
||
> ```
|
||
|
||
---
|
||
|
||
## Installation
|
||
|
||
```bash
|
||
# Clone the repo (or copy agent.py into your project)
|
||
git clone https://github.com/yourusername/stream-ai-agent.git
|
||
cd stream-ai-agent
|
||
|
||
# Create a virtual environment (optional but recommended)
|
||
python -m venv .venv
|
||
source .venv/bin/activate # Windows: .venv\Scripts\activate
|
||
|
||
# Install dependencies
|
||
pip install -r requirements.txt
|
||
```
|
||
|
||
`requirements.txt` contains:
|
||
|
||
```text
|
||
langchain>=0.2.0
|
||
langchain-ollama>=0.1.0
|
||
langgraph>=0.1.0
|
||
nomic-embed-text>=0.1.0
|
||
python-dotenv>=1.0.0
|
||
```
|
||
|
||
---
|
||
|
||
## Running the Agent
|
||
|
||
The main entry point is `agent.py`.
|
||
Run it with:
|
||
|
||
```bash
|
||
# Basic usage – prompts the agent to answer a question
|
||
python agent.py "What is the capital of France?"
|
||
```
|
||
|
||
If you want to see the streaming output in real time, simply execute the script as shown above. The console will display each token as soon as it is produced by the LLM.
|
||
|
||
---
|
||
|
||
## Example Usage
|
||
|
||
```bash
|
||
$ python agent.py "Explain quantum computing in simple terms."
|
||
🤖 Quantum computing is a type of computation that uses quantum bits (qubits) instead of classical bits...
|
||
🤖 ...to perform certain calculations much faster than traditional computers.
|
||
```
|
||
|
||
The `🤖` emoji indicates the streaming output from the LLM.
|
||
You can also pass multiple arguments or use environment variables to change the model:
|
||
|
||
```bash
|
||
OLLAMA_MODEL=llama2 python agent.py "How many moons does Mars have?"
|
||
```
|
||
|
||
---
|
||
|
||
### Customizing the Agent
|
||
|
||
* **Change the LLM** – edit `LLM_MODEL` in `agent.py`.
|
||
* **Add tools** – extend the `tools` list with any `BaseTool` subclass.
|
||
* **Persist embeddings** – configure Qdrant by setting `QDRANT_URL` and `QDRANT_API_KEY`.
|
||
|
||
---
|
||
|
||
Happy streaming! 🚀 |