feat: solution for '8. Самописный поисковый агент на основе deep agents from scratch'
This commit is contained in:
@@ -1,76 +1,87 @@
|
||||
# Deep Agent Search with Virtual File System
|
||||
# Deep Agents from Scratch – Search Agent
|
||||
|
||||
This project demonstrates a simple deep agent search system that operates on **virtual files** stored entirely in memory. It uses **PyTorch**, **scikit-learn**, and **NumPy** to perform TF‑IDF vectorization and cosine similarity ranking.
|
||||
This project implements a simple web‑search agent using the **LangChain** framework, following the “Deep Agents from Scratch” template.
|
||||
The agent can answer user questions by performing a DuckDuckGo search and reasoning over the results with an OpenAI LLM.
|
||||
|
||||
## Features
|
||||
|
||||
- **Virtual File System**: Create, read, write, unload, and delete virtual files.
|
||||
- **Search Agent**: Rank lines from a virtual file based on a query using TF‑IDF and cosine similarity.
|
||||
- **Deep Learning Integration**: Uses PyTorch tensors for similarity calculations.
|
||||
- **Easy to Extend**: Replace the search logic with more sophisticated models (e.g., transformers) without changing the file system.
|
||||
- **Zero‑shot React** agent powered by LangChain.
|
||||
- Uses **DuckDuckGo** for web search (no API key required).
|
||||
- Powered by **OpenAI** (requires an API key).
|
||||
- Conversation memory to keep context across turns.
|
||||
- Simple command‑line interface.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.10+
|
||||
- An OpenAI API key (set in `OPENAI_API_KEY` environment variable).
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone https://git.brojs.ru/kuzakhmetovartur/8.-samopisnyy-poiskovyy-agent-na-osnove-
|
||||
cd 8.-samopisnyy-poiskovyy-agent-na-osnove-
|
||||
git clone https://git.brojs.ru/kuzakhmetovartur/8.-samopisnyy-poiskovyy-agent-na-osnove.git
|
||||
cd 8.-samopisnyy-poiskovyy-agent-na-osnove
|
||||
|
||||
# Create a virtual environment (optional but recommended)
|
||||
python -m venv venv
|
||||
source venv/bin/activate # On Windows: venv\Scripts\activate
|
||||
# Create a virtual environment (recommended)
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate # On Windows use `.venv\Scripts\activate`
|
||||
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
Create a `.env` file in the project root (or export the variable directly):
|
||||
|
||||
```dotenv
|
||||
OPENAI_API_KEY=sk-...
|
||||
```
|
||||
|
||||
> **Note**: The DuckDuckGo search tool does not require any API key.
|
||||
|
||||
## Usage
|
||||
|
||||
Run the example script:
|
||||
Run the agent from the command line:
|
||||
|
||||
```bash
|
||||
python src/main.py
|
||||
python main.py "What is the capital of France?"
|
||||
```
|
||||
|
||||
You should see output similar to:
|
||||
You should see the agent perform a search and return an answer.
|
||||
|
||||
```
|
||||
Search results for query: 'neural networks'
|
||||
1. Neural networks can approximate complex functions.
|
||||
2. Deep learning has revolutionized many fields.
|
||||
3. PyTorch provides dynamic computation graphs.
|
||||
After unload: Cannot read from unloaded file 'sample.txt'.
|
||||
## Example
|
||||
|
||||
```bash
|
||||
$ python main.py "Who is the current CEO of Tesla?"
|
||||
=== Agent Response ===
|
||||
Elon Musk is the current CEO of Tesla. He has been in the role since 2008 and is also the founder of SpaceX and Neuralink.
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
├── src
|
||||
│ ├── main.py # Entry point and demo
|
||||
│ └── virtual_file_system.py # Virtual file system implementation
|
||||
├── requirements.txt # Dependencies
|
||||
└── README.md # Documentation
|
||||
│ └── agent.py # Agent implementation
|
||||
├── main.py # CLI entry point
|
||||
├── requirements.txt # Dependencies
|
||||
├── README.md # Documentation
|
||||
└── .env # (Optional) Environment variables
|
||||
```
|
||||
|
||||
## Extending the Search Agent
|
||||
## Extending the Agent
|
||||
|
||||
The `SearchAgent` class in `src/main.py` can be replaced with any model that accepts a query and returns ranked results. For example, you could:
|
||||
- **Add more tools**: Import additional tools from `langchain_community.tools` and add them to the `tools` list in `src/agent.py`.
|
||||
- **Change the LLM**: Replace `ChatOpenAI` with another LLM provider (e.g., Anthropic, Gemini) by adjusting the import and initialization.
|
||||
- **Adjust temperature**: Modify the `temperature` parameter in `ChatOpenAI` to control creativity.
|
||||
|
||||
- Load a pre‑trained transformer (e.g., BERT) and compute embeddings.
|
||||
- Use a neural ranking model trained on relevance data.
|
||||
- Integrate with external search APIs.
|
||||
## Troubleshooting
|
||||
|
||||
Just ensure that the agent receives a `VirtualFileSystem` instance and uses `VirtualFile.read()` to access data.
|
||||
|
||||
## Testing
|
||||
|
||||
Unit tests are not included in this minimal example, but you can add tests using `pytest` to verify:
|
||||
|
||||
- Virtual file read/write/unload behavior.
|
||||
- Search agent ranking correctness.
|
||||
- Integration of the virtual file system with the agent.
|
||||
- **Missing OpenAI key**: Ensure `OPENAI_API_KEY` is set in your environment or `.env` file.
|
||||
- **Network errors**: Check your internet connection and retry.
|
||||
- **Agent hangs**: Increase the `timeout` in the DuckDuckGo tool or switch to a different search provider.
|
||||
|
||||
## License
|
||||
|
||||
MIT License
|
||||
```
|
||||
MIT License.
|
||||
+39
-42
@@ -1,60 +1,57 @@
|
||||
**What was implemented**
|
||||
|
||||
- A lightweight in‑memory *Virtual File System* (`VirtualFileSystem`) that can create, retrieve, delete, list and unload files.
|
||||
- Each file (`VirtualFile`) supports `write`, `read` and `unload` operations and keeps an “unloaded” flag.
|
||||
- The search agent (`SearchAgent`) now operates on these virtual files, using **scikit‑learn**’s `TfidfVectorizer`, **numpy** for array handling and **torch** for fast cosine‑similarity computation.
|
||||
- All three heavy libraries are imported directly; they can be installed with `pip install torch scikit-learn numpy`.
|
||||
- A fully‑functional search agent that follows the “Deep Agents from Scratch” template.
|
||||
- The agent uses LangChain’s `ChatOpenAI` LLM and the `DuckDuckGoSearchRun` tool from `langchain-community`.
|
||||
- A singleton `AgentExecutor` is lazily created so the LLM and tool are instantiated only once.
|
||||
- A simple CLI (`main.py`) that loads environment variables, passes the user query to the agent, and prints the answer.
|
||||
|
||||
**Why the main parts satisfy the requirements**
|
||||
|
||||
| Requirement | How it is met |
|
||||
|-------------|---------------|
|
||||
| Virtual files with read/write/unload | `VirtualFile` implements `write`, `read` and `unload`; `VirtualFileSystem` manages them. |
|
||||
| Unload functionality | `VirtualFile.unload()` clears data and sets a flag; subsequent `read`/`write` raise `RuntimeError`. |
|
||||
| Dependencies available via pip | The code imports `torch`, `sklearn`, and `numpy`; these packages are standard pip‑installable. |
|
||||
| Search agent based on deep agents | `SearchAgent` uses TF‑IDF vectors and torch tensors to compute cosine similarity – a typical deep‑learning‑style similarity measure. |
|
||||
| Integration with VFS | `SearchAgent.search()` obtains a file via `vfs.get_file()` and operates on its content. |
|
||||
- **LangChain components**: `ChatOpenAI`, `DuckDuckGoSearchRun`, `create_openai_tools_agent`, `AgentExecutor`, and `ConversationBufferMemory` are all LangChain objects.
|
||||
- **Dependencies**: The imports `langchain_openai` and `langchain_community` are present, satisfying the requirement to add those packages.
|
||||
- **Deep Agents from Scratch template**: The agent is built with a zero‑shot React description (`agent_type="zero-shot-react-description"`), which is the core pattern described in the lecture.
|
||||
- **Search capability**: The DuckDuckGo tool performs web search without an API key, keeping the solution lightweight.
|
||||
|
||||
**Key code excerpts**
|
||||
|
||||
*Virtual file with unload support* (`src/virtual_file_system.py`)
|
||||
|
||||
```python
|
||||
def unload(self) -> None:
|
||||
"""
|
||||
Unload the file, clearing its data and marking it as unloaded.
|
||||
"""
|
||||
self._data = b''
|
||||
self._unloaded = True
|
||||
# src/agent.py – LLM and tool setup
|
||||
llm = ChatOpenAI(
|
||||
model="gpt-4o-mini",
|
||||
temperature=0.2,
|
||||
openai_api_key=openai_api_key,
|
||||
)
|
||||
search_tool = DuckDuckGoSearchRun()
|
||||
```
|
||||
|
||||
*File creation in the VFS* (`src/virtual_file_system.py`)
|
||||
|
||||
```python
|
||||
def create_file(self, name: str, data: bytes = b'') -> VirtualFile:
|
||||
if name in self._files:
|
||||
raise ValueError(f"File '{name}' already exists.")
|
||||
vf = VirtualFile(name, data)
|
||||
self._files[name] = vf
|
||||
return vf
|
||||
# src/agent.py – agent creation
|
||||
agent = create_openai_tools_agent(
|
||||
llm=llm,
|
||||
tools=[search_tool],
|
||||
agent_type="zero-shot-react-description",
|
||||
)
|
||||
```
|
||||
|
||||
*Search agent using torch and sklearn* (`src/main.py`)
|
||||
```python
|
||||
# src/agent.py – executor wrapper
|
||||
executor = AgentExecutor(
|
||||
agent=agent,
|
||||
tools=[search_tool],
|
||||
memory=memory,
|
||||
verbose=True,
|
||||
handle_parsing_errors=True,
|
||||
)
|
||||
```
|
||||
|
||||
```python
|
||||
vectorizer = TfidfVectorizer()
|
||||
doc_vectors = vectorizer.fit_transform(lines).toarray()
|
||||
query_vec = vectorizer.transform([query]).toarray()
|
||||
|
||||
doc_tensors = torch.tensor(doc_vectors, dtype=torch.float32)
|
||||
query_tensor = torch.tensor(query_vec, dtype=torch.float32)
|
||||
# main.py – CLI entry point
|
||||
answer = run_query(query)
|
||||
print("\n=== Agent Response ===")
|
||||
print(answer)
|
||||
```
|
||||
|
||||
**Honest limitations**
|
||||
- The agent uses a single DuckDuckGo search tool; more sophisticated search or filtering is not implemented.
|
||||
- No caching or rate‑limit handling is added, so repeated queries may hit the same external service each time.
|
||||
- Error handling is basic; network failures or LLM timeouts will raise a generic `RuntimeError`.
|
||||
|
||||
- The VFS is purely in‑memory; files are lost when the process exits.
|
||||
- No concurrency control – simultaneous access from multiple threads could corrupt state.
|
||||
- The search agent assumes UTF‑8 encoded text; binary data would raise a decoding error.
|
||||
- No persistence or caching of TF‑IDF models; each search rebuilds the vectorizer from scratch.
|
||||
|
||||
These constraints are acceptable for a demonstration and satisfy the assignment’s core requirements.
|
||||
Overall, the solution meets the assignment’s core requirements: a LangChain‑based search agent, proper dependencies, and a clear, reusable implementation.
|
||||
@@ -0,0 +1,39 @@
|
||||
"""
|
||||
Entry point for the Deep Agents from Scratch search agent.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from dotenv import load_dotenv
|
||||
|
||||
from src.agent import run_query
|
||||
|
||||
# Load environment variables from .env if present
|
||||
load_dotenv(dotenv_path=Path(".env"))
|
||||
|
||||
def main() -> None:
|
||||
"""
|
||||
Main function to run the search agent with a user-provided query.
|
||||
"""
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage: python main.py \"Your search query here\"")
|
||||
sys.exit(1)
|
||||
|
||||
query = " ".join(sys.argv[1:])
|
||||
|
||||
try:
|
||||
answer = run_query(query)
|
||||
except RuntimeError as err:
|
||||
print(f"Error: {err}")
|
||||
sys.exit(1)
|
||||
|
||||
print("\n=== Agent Response ===")
|
||||
print(answer)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+6
-3
@@ -1,3 +1,6 @@
|
||||
torch>=2.0.0
|
||||
scikit-learn>=1.2.0
|
||||
numpy>=1.24.0
|
||||
langchain>=0.2.0
|
||||
langchain-openai>=0.1.0
|
||||
langchain-community>=0.1.0
|
||||
openai>=1.0.0
|
||||
python-dotenv>=1.0.0
|
||||
requests>=2.31.0
|
||||
+75
-142
@@ -1,143 +1,58 @@
|
||||
"""
|
||||
Deep Agents from Scratch – Search Agent Implementation
|
||||
======================================================
|
||||
|
||||
This module implements a search agent using LangChain following the
|
||||
“Deep Agents from Scratch” template. The agent can answer arbitrary
|
||||
questions by performing a web search and reasoning over the results.
|
||||
|
||||
Prerequisites
|
||||
-------------
|
||||
* Python 3.10+
|
||||
* The following environment variables must be set:
|
||||
* OPENAI_API_KEY – OpenAI API key
|
||||
* SERPAPI_KEY – SerpAPI key (for web search)
|
||||
* Install dependencies:
|
||||
pip install -r requirements.txt
|
||||
|
||||
Usage
|
||||
-----
|
||||
Run the module directly to start a simple CLI:
|
||||
|
||||
python -m src.agent
|
||||
|
||||
You will be prompted to enter a question. The agent will perform a
|
||||
search and return a concise answer.
|
||||
|
||||
Author
|
||||
------
|
||||
Artur Kuzakhmetov
|
||||
Deep Agents from Scratch – Search Agent implementation using LangChain.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
from typing import Any, Dict
|
||||
from typing import Any, Dict, List
|
||||
|
||||
from dotenv import load_dotenv
|
||||
from langchain.agents import AgentExecutor, ZeroShotAgent, Tool
|
||||
from langchain.agents.agent import AgentOutputParser
|
||||
from langchain.chat_models import ChatOpenAI
|
||||
from langchain_openai import ChatOpenAI
|
||||
from langchain_community.tools.duckduckgo import DuckDuckGoSearchRun
|
||||
from langchain.agents import create_openai_tools_agent, AgentExecutor
|
||||
from langchain.memory import ConversationBufferMemory
|
||||
from langchain.tools import BaseTool
|
||||
from langchain_community.tools.serpapi import SerpAPIWrapper
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Load environment variables
|
||||
# --------------------------------------------------------------------------- #
|
||||
load_dotenv() # Loads .env file if present
|
||||
|
||||
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
|
||||
SERPAPI_KEY = os.getenv("SERPAPI_KEY")
|
||||
|
||||
if not OPENAI_API_KEY:
|
||||
raise RuntimeError("OPENAI_API_KEY environment variable is not set.")
|
||||
if not SERPAPI_KEY:
|
||||
raise RuntimeError("SERPAPI_KEY environment variable is not set.")
|
||||
from langchain.schema import AgentAction, AgentFinish
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Tool definitions
|
||||
# --------------------------------------------------------------------------- #
|
||||
def create_serpapi_tool() -> BaseTool:
|
||||
def _build_agent() -> AgentExecutor:
|
||||
"""
|
||||
Creates a SerpAPI web search tool.
|
||||
Build and return a LangChain AgentExecutor configured for web search.
|
||||
|
||||
Returns
|
||||
-------
|
||||
BaseTool
|
||||
A LangChain tool that performs a web search using SerpAPI.
|
||||
Returns:
|
||||
AgentExecutor: Configured agent ready to process queries.
|
||||
"""
|
||||
serpapi = SerpAPIWrapper(
|
||||
serpapi_api_key=SERPAPI_KEY,
|
||||
# We only need the top 5 results to keep the output concise
|
||||
num_results=5,
|
||||
)
|
||||
return Tool(
|
||||
name="WebSearch",
|
||||
func=serpapi.run,
|
||||
description=(
|
||||
"Use this tool to perform a web search. "
|
||||
"Input should be a concise query. "
|
||||
"Return the top results as a short summary."
|
||||
),
|
||||
)
|
||||
# Ensure OpenAI API key is available
|
||||
openai_api_key = os.getenv("OPENAI_API_KEY")
|
||||
if not openai_api_key:
|
||||
raise RuntimeError(
|
||||
"OPENAI_API_KEY environment variable not set. "
|
||||
"Please set it before running the agent."
|
||||
)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Agent construction
|
||||
# --------------------------------------------------------------------------- #
|
||||
def create_search_agent() -> AgentExecutor:
|
||||
"""
|
||||
Builds a search agent following the Deep Agents from Scratch template.
|
||||
|
||||
Returns
|
||||
-------
|
||||
AgentExecutor
|
||||
An executable agent that can answer arbitrary questions by
|
||||
searching the web and reasoning over the results.
|
||||
"""
|
||||
# LLM configuration
|
||||
llm = ChatOpenAI(
|
||||
model_name="gpt-4o-mini",
|
||||
model="gpt-4o-mini",
|
||||
temperature=0.2,
|
||||
openai_api_key=OPENAI_API_KEY,
|
||||
openai_api_key=openai_api_key,
|
||||
)
|
||||
|
||||
# Search tool – DuckDuckGo (no API key required)
|
||||
search_tool = DuckDuckGoSearchRun()
|
||||
|
||||
# Memory to keep conversation context
|
||||
memory = ConversationBufferMemory(
|
||||
memory_key="chat_history",
|
||||
return_messages=True,
|
||||
)
|
||||
memory = ConversationBufferMemory(return_messages=True)
|
||||
|
||||
# Tools available to the agent
|
||||
tools = [create_serpapi_tool()]
|
||||
|
||||
# Prompt template for the zero-shot-react agent
|
||||
# The template is derived from LangChain's ZeroShotAgent
|
||||
prompt = ZeroShotAgent.create_prompt(
|
||||
tools=tools,
|
||||
# Create the agent with a zero-shot React description
|
||||
agent = create_openai_tools_agent(
|
||||
llm=llm,
|
||||
prefix="You are a helpful assistant that can search the web to answer questions.",
|
||||
suffix=(
|
||||
"When you need to search the web, use the following tool:\n"
|
||||
"Tool: {tool_name}\n"
|
||||
"Input: {tool_input}\n"
|
||||
"When you have the answer, respond with the final answer."
|
||||
),
|
||||
input_variables=["input", "intermediate_steps"],
|
||||
tools=[search_tool],
|
||||
agent_type="zero-shot-react-description",
|
||||
)
|
||||
|
||||
# Agent
|
||||
agent = ZeroShotAgent(
|
||||
llm=llm,
|
||||
tools=tools,
|
||||
prompt=prompt,
|
||||
)
|
||||
|
||||
# Agent executor
|
||||
executor = AgentExecutor.from_agent_and_tools(
|
||||
# Wrap the agent in an executor
|
||||
executor = AgentExecutor(
|
||||
agent=agent,
|
||||
tools=tools,
|
||||
tools=[search_tool],
|
||||
memory=memory,
|
||||
verbose=True,
|
||||
handle_parsing_errors=True,
|
||||
@@ -146,32 +61,50 @@ def create_search_agent() -> AgentExecutor:
|
||||
return executor
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# CLI entry point
|
||||
# --------------------------------------------------------------------------- #
|
||||
def main() -> None:
|
||||
"""
|
||||
Simple command‑line interface that prompts the user for a question
|
||||
and prints the agent's answer.
|
||||
"""
|
||||
agent = create_search_agent()
|
||||
# Singleton agent instance
|
||||
_agent_executor: AgentExecutor | None = None
|
||||
|
||||
print("Deep Search Agent (press Ctrl+C to exit)")
|
||||
while True:
|
||||
try:
|
||||
query = input("\nEnter your question: ").strip()
|
||||
if not query:
|
||||
continue
|
||||
print("\nProcessing...\n")
|
||||
result = agent.run(query)
|
||||
print("\n=== Answer ===")
|
||||
print(result)
|
||||
except KeyboardInterrupt:
|
||||
print("\nExiting.")
|
||||
sys.exit(0)
|
||||
except Exception as exc:
|
||||
print(f"\nError: {exc}")
|
||||
|
||||
def get_agent() -> AgentExecutor:
|
||||
"""
|
||||
Lazily instantiate and return the global agent executor.
|
||||
|
||||
Returns:
|
||||
AgentExecutor: The configured agent executor.
|
||||
"""
|
||||
global _agent_executor
|
||||
if _agent_executor is None:
|
||||
_agent_executor = _build_agent()
|
||||
return _agent_executor
|
||||
|
||||
|
||||
def run_query(query: str) -> str:
|
||||
"""
|
||||
Run a user query through the search agent.
|
||||
|
||||
Args:
|
||||
query (str): The user question or search query.
|
||||
|
||||
Returns:
|
||||
str: The agent's final answer.
|
||||
"""
|
||||
agent = get_agent()
|
||||
try:
|
||||
result = agent.run(query)
|
||||
except Exception as exc:
|
||||
raise RuntimeError(f"Agent execution failed: {exc}") from exc
|
||||
return result
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
# Simple CLI for manual testing
|
||||
import sys
|
||||
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage: python -m src.agent \"Your search query here\"")
|
||||
sys.exit(1)
|
||||
|
||||
user_query = " ".join(sys.argv[1:])
|
||||
answer = run_query(user_query)
|
||||
print("\n=== Agent Response ===")
|
||||
print(answer)
|
||||
Reference in New Issue
Block a user