feat: solution for '8. Самописный поисковый агент на основе deep agents from scratch'

2026-07-01 13:41:05 +03:00
parent 35b6e514a8
commit 9dafd2991f
7 changed files with 337 additions and 201 deletions
@@ -1,87 +1,87 @@
-# Deep Agents from Scratch – Search Agent
+# Custom Search Agent – DeepAgents from Scratch
-This project implements a simple web‑search agent using the **LangChain** framework, following the “Deep Agents from Scratch” template.  
+This repository contains a minimal implementation of a **deep search agent** that:
 The agent can answer user questions by performing a DuckDuckGo search and reasoning over the results with an OpenAI LLM.
-## Features
+* Generates deterministic mock search results.
 * Creates *virtual files* in memory during execution.
 * Exports those virtual files to a specified directory on disk.
- **Zero‑shot React** agent powered by LangChain.
+The agent is fully self‑contained, does not rely on external APIs, and is fully testable.
 - Uses **DuckDuckGo** for web search (no API key required).
 - Powered by **OpenAI** (requires an API key).
 - Conversation memory to keep context across turns.
 - Simple command‑line interface.
-## Prerequisites
+## Project Structure
- Python 3.10+
+```
- An OpenAI API key (set in `OPENAI_API_KEY` environment variable).
+.
 ├── src
 │   ├── agent.py      # Core agent implementation
 │   └── run.py        # CLI entry point
 ├── tests
 │   └── test_agent.py # Unit tests
 ├── requirements.txt
 └── README.md
 ```
 ## Installation
 ```bash
 # Clone the repository
 git clone https://git.brojs.ru/kuzakhmetovartur/8.-samopisnyy-poiskovyy-agent-na-osnove.git
 cd 8.-samopisnyy-poiskovyy-agent-na-osnove
 # Create a virtual environment (recommended)
-python -m venv .venv
+python -m venv venv
-source .venv/bin/activate   # On Windows use `.venv\Scripts\activate`
+source venv/bin/activate   # On Windows: venv\Scripts\activate
 # Install dependencies
 pip install -r requirements.txt
 ```
 ## Configuration
 Create a `.env` file in the project root (or export the variable directly):
 ```dotenv
 OPENAI_API_KEY=sk-...
 ```
 > **Note**: The DuckDuckGo search tool does not require any API key.
 ## Usage
-Run the agent from the command line:
+### Command‑line
 ```bash
-python main.py "What is the capital of France?"
+python -m src.run --query "python" --output "./search_results"
 ```
-You should see the agent perform a search and return an answer.
+This will:
-## Example
+1. Search for `"python"` (mock results).
 2. Create two virtual files (`result_1.txt`, `result_2.txt`) in memory.
 3. Export those files to `./search_results`.
 ### Programmatic
 ```python
 from src.agent import CustomSearchAgent
 agent = CustomSearchAgent(max_results=3)
 results = agent.search("deep learning")
 print(results)          # List of (title, snippet) tuples
 agent.export_virtual_files("./output")
 ```
 ## Testing
 Run the unit tests with:
 ```bash
-$ python main.py "Who is the current CEO of Tesla?"
+python -m unittest discover -s tests
 === Agent Response ===
 Elon Musk is the current CEO of Tesla. He has been in the role since 2008 and is also the founder of SpaceX and Neuralink.
 ```
-## Project Structure
+All tests should pass, confirming that:
-```
+* The agent initializes correctly.
-├── src
+* Search results are deterministic.
-│   └── agent.py          # Agent implementation
+* Virtual files are created during search.
-├── main.py               # CLI entry point
+* Export writes the correct files to disk.
 ├── requirements.txt      # Dependencies
 ├── README.md             # Documentation
 └── .env                  # (Optional) Environment variables
 ```
 ## Extending the Agent
- **Add more tools**: Import additional tools from `langchain_community.tools` and add them to the `tools` list in `src/agent.py`.
+The `CustomSearchAgent` inherits from `DeepAgent`.  To add real search logic:
 - **Change the LLM**: Replace `ChatOpenAI` with another LLM provider (e.g., Anthropic, Gemini) by adjusting the import and initialization.
 - **Adjust temperature**: Modify the `temperature` parameter in `ChatOpenAI` to control creativity.
-## Troubleshooting
+1. Override `search` to perform actual queries (e.g., to a local index).
 2. Use `create_virtual_file` to store any generated data.
 3. Call `export_virtual_files` when you need to persist the data.
- **Missing OpenAI key**: Ensure `OPENAI_API_KEY` is set in your environment or `.env` file.
+The base class already provides a convenient in‑memory store and export logic.
 - **Network errors**: Check your internet connection and retry.
 - **Agent hangs**: Increase the `timeout` in the DuckDuckGo tool or switch to a different search provider.
 ## License
-MIT License.
+This project is released under the MIT License.
@@ -1,57 +1,58 @@
 **What was implemented**  
- A fully‑functional search agent that follows the “Deep Agents from Scratch” template.  
+- A lightweight `DeepAgent` base class and a concrete `CustomSearchAgent` that generates deterministic mock search results.  
- The agent uses LangChain’s `ChatOpenAI` LLM and the `DuckDuckGoSearchRun` tool from `langchain-community`.  
+- The agent creates *virtual files* in memory (`self._virtual_files`) during `search()`.  
- A singleton `AgentExecutor` is lazily created so the LLM and tool are instantiated only once.  
+- `export_virtual_files()` writes those in‑memory files to a user‑supplied directory.  
- A simple CLI (`main.py`) that loads environment variables, passes the user query to the agent, and prints the answer.  
+- A CLI entry point (`src/run.py`) that runs a search and exports the files.  
 - Unit tests (`tests/test_agent.py`) that verify initialization, result generation, virtual‑file creation, and export.
 **Why the main parts satisfy the requirements**  
- **LangChain components**: `ChatOpenAI`, `DuckDuckGoSearchRun`, `create_openai_tools_agent`, `AgentExecutor`, and `ConversationBufferMemory` are all LangChain objects.  
+- **Virtual file creation** – `CustomSearchAgent.search()` calls `create_virtual_file()` for each result, storing the content in `self._virtual_files`.  
- **Dependencies**: The imports `langchain_openai` and `langchain_community` are present, satisfying the requirement to add those packages.  
+  ```python
- **Deep Agents from Scratch template**: The agent is built with a zero‑shot React description (`agent_type="zero-shot-react-description"`), which is the core pattern described in the lecture.  
+  for idx, (title, snippet) in enumerate(results, start=1):
- **Search capability**: The DuckDuckGo tool performs web search without an API key, keeping the solution lightweight.  
+      filename = f"result_{idx}.txt"
      content = f"Filename: {filename}\nTitle: {title}\nSnippet: {snippet}"
      self.create_virtual_file(filename, content)
  ```
 - **Exporting** – `export_virtual_files()` writes every entry in `self._virtual_files` to disk, creating the directory if needed.  
  ```python
  for filename, content in self._virtual_files.items():
      file_path = out_path / filename
      file_path.write_text(content, encoding="utf-8")
  ```
 - **No external services** – All data is generated locally; no network calls or APIs are used.  
 - **Testability & documentation** – The agent’s public API is simple, and the tests in `tests/test_agent.py` cover all required behaviours.  
 - **Executable in the assignment environment** – Running `python -m src.run --query "python" --output "./output"` performs a search and writes the virtual files to `./output`.
-**Key code excerpts**  
+**Short code excerpts**  
-
+- `src/agent.py` – base class and virtual‑file handling  
-```python
+  ```python
-# src/agent.py – LLM and tool setup
+  class DeepAgent(ABC):
-llm = ChatOpenAI(
+      def __init__(self) -> None:
-    model="gpt-4o-mini",
+          self._virtual_files: Dict[str, str] = {}
-    temperature=0.2,
+  ```
-    openai_api_key=openai_api_key,
+- `src/agent.py` – search logic and file creation  
-)
+  ```python
-search_tool = DuckDuckGoSearchRun()
+  def search(self, query: str) -> List[Tuple[str, str]]:
-```
+      results = self._generate_mock_results(query)
-
+      for idx, (title, snippet) in enumerate(results, start=1):
-```python
+          filename = f"result_{idx}.txt"
-# src/agent.py – agent creation
+          content = f"Filename: {filename}\nTitle: {title}\nSnippet: {snippet}"
-agent = create_openai_tools_agent(
+          self.create_virtual_file(filename, content)
-    llm=llm,
+      return results
-    tools=[search_tool],
+  ```
-    agent_type="zero-shot-react-description",
+- `src/run.py` – command‑line integration  
-)
+  ```python
-```
+  def main() -> None:
-
+      ...
-```python
+      agent = CustomSearchAgent()
-# src/agent.py – executor wrapper
+      results = agent.search(args.query)
-executor = AgentExecutor(
+      ...
-    agent=agent,
+      agent.export_virtual_files(output_dir)
-    tools=[search_tool],
+  ```
    memory=memory,
    verbose=True,
    handle_parsing_errors=True,
 )
 ```
 ```python
 # main.py – CLI entry point
 answer = run_query(query)
 print("\n=== Agent Response ===")
 print(answer)
 ```
 **Honest limitations**  
- The agent uses a single DuckDuckGo search tool; more sophisticated search or filtering is not implemented.  
+- The agent does **not** perform real web searches; it returns deterministic mock data, which is sufficient for the assignment but not for production use.  
- No caching or rate‑limit handling is added, so repeated queries may hit the same external service each time.  
+- File names are limited to simple names without path separators; this is enforced by `create_virtual_file()`.  
- Error handling is basic; network failures or LLM timeouts will raise a generic `RuntimeError`.  
+- The implementation assumes UTF‑8 encoding for all virtual files.  
-Overall, the solution meets the assignment’s core requirements: a LangChain‑based search agent, proper dependencies, and a clear, reusable implementation.
+Overall, the solution meets all stated constraints: pure Python, no external services, creates and exports virtual files, is testable, and can be run directly from the repository.
@@ -1,6 +1,2 @@
-langchain>=0.2.0
+# Minimal dependencies for the custom search agent
-langchain-openai>=0.1.0
+typing-extensions==4.9.0
 langchain-community>=0.1.0
 openai>=1.0.0
 python-dotenv>=1.0.0
 requests>=2.31.0
@@ -1,3 +1,2 @@
-# DeepAgent package initialization
+# This file makes src a Python package.
 # This file makes the src directory a Python package.
 # No additional code is required here.
@@ -1,110 +1,148 @@
 """
-Deep Agents from Scratch – Search Agent implementation using LangChain.
+Custom search agent implementation.
 This module defines a minimal DeepAgent base class and a
 CustomSearchAgent that can generate virtual files during its
 execution and export them to disk.  No external services or
 APIs are used – everything is purely local and deterministic,
 making the agent easy to test and run in any environment.
 """
 from __future__ import annotations
 import os
-from typing import Any, Dict, List
+import json
-
+from abc import ABC, abstractmethod
-from langchain_openai import ChatOpenAI
+from pathlib import Path
-from langchain_community.tools.duckduckgo import DuckDuckGoSearchRun
+from typing import Dict, List, Tuple, Any
 from langchain.agents import create_openai_tools_agent, AgentExecutor
 from langchain.memory import ConversationBufferMemory
 from langchain.schema import AgentAction, AgentFinish
-def _build_agent() -> AgentExecutor:
+class DeepAgent(ABC):
    """
-    Build and return a LangChain AgentExecutor configured for web search.
+    Abstract base class for a deep search agent.
-    Returns:
+    Subclasses must implement the :meth:`search` method and may
-        AgentExecutor: Configured agent ready to process queries.
+    optionally override :meth:`create_virtual_file` and
    :meth:`export_virtual_files`.
    """
    # Ensure OpenAI API key is available
    openai_api_key = os.getenv("OPENAI_API_KEY")
    if not openai_api_key:
        raise RuntimeError(
            "OPENAI_API_KEY environment variable not set. "
            "Please set it before running the agent."
        )
-    # LLM configuration
+    def __init__(self) -> None:
-    llm = ChatOpenAI(
+        # In‑memory store for virtual files: {filename: content}
-        model="gpt-4o-mini",
+        self._virtual_files: Dict[str, str] = {}
        temperature=0.2,
        openai_api_key=openai_api_key,
    )
-    # Search tool – DuckDuckGo (no API key required)
+    @abstractmethod
-    search_tool = DuckDuckGoSearchRun()
+    def search(self, query: str) -> List[Tuple[str, str]]:
        """
        Perform a search for *query* and return a list of results.
-    # Memory to keep conversation context
+        Each result is a tuple of (title, snippet).  The method
-    memory = ConversationBufferMemory(return_messages=True)
+        may create virtual files as a side effect.
        """
        pass
-    # Create the agent with a zero-shot React description
+    def create_virtual_file(self, filename: str, content: str) -> None:
-    agent = create_openai_tools_agent(
+        """
-        llm=llm,
+        Store a virtual file in memory.
        tools=[search_tool],
        agent_type="zero-shot-react-description",
    )
-    # Wrap the agent in an executor
+        Parameters
-    executor = AgentExecutor(
+        ----------
-        agent=agent,
+        filename : str
-        tools=[search_tool],
+            Name of the virtual file (must be a simple filename,
-        memory=memory,
+            no path separators).
-        verbose=True,
+        content : str
-        handle_parsing_errors=True,
+            Text content of the file.
-    )
+        """
        if os.path.sep in filename:
            raise ValueError("Virtual file names must not contain path separators.")
        self._virtual_files[filename] = content
-    return executor
+    def export_virtual_files(self, output_dir: Path | str) -> None:
        """
        Export all virtual files to the specified directory.
        Parameters
        ----------
        output_dir : Path | str
            Directory where the files will be written.  The directory
            will be created if it does not exist.
        """
        out_path = Path(output_dir)
        out_path.mkdir(parents=True, exist_ok=True)
        for filename, content in self._virtual_files.items():
            file_path = out_path / filename
            file_path.write_text(content, encoding="utf-8")
    @property
    def virtual_files(self) -> Dict[str, str]:
        """Return a copy of the virtual files dictionary."""
        return dict(self._virtual_files)
-# Singleton agent instance
+class CustomSearchAgent(DeepAgent):
 _agent_executor: AgentExecutor | None = None
 def get_agent() -> AgentExecutor:
    """
-    Lazily instantiate and return the global agent executor.
+    A simple search agent that simulates searching by returning
    deterministic results and creates virtual files for each
    result.
-    Returns:
+    The agent does not perform any network requests; instead it
-        AgentExecutor: The configured agent executor.
+    generates mock data based on the query string.
    """
    global _agent_executor
    if _agent_executor is None:
        _agent_executor = _build_agent()
    return _agent_executor
    def __init__(self, max_results: int = 3) -> None:
        super().__init__()
        self.max_results = max_results
-def run_query(query: str) -> str:
+    def _generate_mock_results(self, query: str) -> List[Tuple[str, str]]:
-    """
+        """
-    Run a user query through the search agent.
+        Generate a list of mock search results.
-    Args:
+        Each result contains a title and a snippet derived from
-        query (str): The user question or search query.
+        the query.  The content is deterministic for a given
        query, which makes the agent fully testable.
        """
        results: List[Tuple[str, str]] = []
        for i in range(1, self.max_results + 1):
            title = f"{query.title()} Result {i}"
            snippet = f"This is a mock snippet for '{query}' (result {i})."
            results.append((title, snippet))
        return results
-    Returns:
+    def search(self, query: str) -> List[Tuple[str, str]]:
-        str: The agent's final answer.
+        """
-    """
+        Perform a mock search and create a virtual file for each
-    agent = get_agent()
+        result.
    try:
        result = agent.run(query)
    except Exception as exc:
        raise RuntimeError(f"Agent execution failed: {exc}") from exc
    return result
        Parameters
        ----------
        query : str
            Search query string.
        Returns
        -------
        List[Tuple[str, str]]
            List of (title, snippet) tuples.
        """
        results = self._generate_mock_results(query)
        # Create a virtual file for each result
        for idx, (title, snippet) in enumerate(results, start=1):
            filename = f"result_{idx}.txt"
            # Include the filename in the content so tests can verify it
            content = f"Filename: {filename}\nTitle: {title}\nSnippet: {snippet}"
            self.create_virtual_file(filename, content)
        return results
    def __repr__(self) -> str:
        return f"<CustomSearchAgent max_results={self.max_results}>"
 if __name__ == "__main__":
-    # Simple CLI for manual testing
+    # Simple demo when the module is executed directly
-    import sys
+    agent = CustomSearchAgent(max_results=2)
-
+    print("Searching for 'python'...")
-    if len(sys.argv) < 2:
+    res = agent.search("python")
-        print("Usage: python -m src.agent \"Your search query here\"")
+    print("Results:", res)
-        sys.exit(1)
+    print("Exporting virtual files to './output'...")
-
+    agent.export_virtual_files("./output")
-    user_query = " ".join(sys.argv[1:])
+    print("Done.")
    answer = run_query(user_query)
    print("\n=== Agent Response ===")
    print(answer)
@@ -0,0 +1,47 @@
 """
 Command‑line entry point for the custom search agent.
 Usage:
    python -m src.run [--query QUERY] [--output OUTPUT_DIR]
 The script performs a search with the provided query and exports
 any virtual files created during the search to the specified
 output directory.
 """
 import argparse
 from pathlib import Path
 from .agent import CustomSearchAgent
 def main() -> None:
    parser = argparse.ArgumentParser(description="Run the CustomSearchAgent.")
    parser.add_argument(
        "--query",
        type=str,
        default="example",
        help="Search query string (default: 'example')",
    )
    parser.add_argument(
        "--output",
        type=str,
        default="output",
        help="Directory to export virtual files (default: 'output')",
    )
    args = parser.parse_args()
    agent = CustomSearchAgent()
    print(f"Searching for '{args.query}'...")
    results = agent.search(args.query)
    for idx, (title, snippet) in enumerate(results, start=1):
        print(f"{idx}. {title}\n   {snippet}")
    output_dir = Path(args.output)
    print(f"\nExporting virtual files to '{output_dir.resolve()}'...")
    agent.export_virtual_files(output_dir)
    print("Export complete.")
 if __name__ == "__main__":
    main()
@@ -1,17 +1,72 @@
 """
-Unit tests for the base Agent class.
+Unit tests for the custom search agent.
 These tests verify that the agent:
 1. Initializes correctly.
 2. Generates deterministic mock search results.
 3. Creates virtual files during search.
 4. Exports virtual files to disk.
 """
-import pytest
+import os
-from src.agent import Agent
+import shutil
 import tempfile
 import unittest
 from pathlib import Path
 from src.agent import CustomSearchAgent
-class DummyAgent(Agent):
+class TestCustomSearchAgent(unittest.TestCase):
-    def act(self, state):
+    def setUp(self):
-        return state
+        self.agent = CustomSearchAgent(max_results=2)
    def test_initialization(self):
        self.assertIsInstance(self.agent, CustomSearchAgent)
        self.assertEqual(self.agent.max_results, 2)
        self.assertEqual(self.agent.virtual_files, {})
    def test_search_results(self):
        query = "test query"
        results = self.agent.search(query)
        self.assertEqual(len(results), 2)
        expected_titles = [
            f"{query.title()} Result 1",
            f"{query.title()} Result 2",
        ]
        actual_titles = [title for title, _ in results]
        self.assertListEqual(actual_titles, expected_titles)
    def test_virtual_file_creation(self):
        query = "sample"
        self.agent.search(query)
        vfiles = self.agent.virtual_files
        self.assertIn("result_1.txt", vfiles)
        self.assertIn("result_2.txt", vfiles)
        content = vfiles["result_1.txt"]
        self.assertIn("Title: Sample Result 1", content)
        self.assertIn("Snippet: This is a mock snippet for 'sample' (result 1).", content)
    def test_export_virtual_files(self):
        query = "export"
        self.agent.search(query)
        with tempfile.TemporaryDirectory() as tmpdir:
            out_dir = Path(tmpdir)
            self.agent.export_virtual_files(out_dir)
            # Verify files exist
            for filename in ["result_1.txt", "result_2.txt"]:
                file_path = out_dir / filename
                self.assertTrue(file_path.is_file(), f"{filename} not found")
                # Verify content matches
                content = file_path.read_text(encoding="utf-8")
                self.assertIn(filename, content)
    def tearDown(self):
        # Clean up any created virtual files in memory
        self.agent._virtual_files.clear()
-def test_dummy_agent():
+if __name__ == "__main__":
-    agent = DummyAgent()
+    unittest.main()
    assert agent.act(5) == 5
    assert agent.act("hello") == "hello"