RAG Agent with ChromaDB and Web Search

This project implements a Retrieval-Augmented Generation (RAG) agent that uses ChromaDB as the vector database and the OpenAI API to generate responses based on retrieved documents. It also includes a simple websearch component that fetches content from specified URLs for indexing.

Features

  • Vector Store: Uses ChromaDB to store embeddings of text chunks.
  • OpenAI Integration: Generates answers using GPT3.5Turbo.
  • Web Search: Fetches and parses HTML pages, splits them into manageable chunks.
  • Command Line Interface: Ask questions interactively.

Prerequisites

  • Node.js v18+ (supports native ES modules and node-fetch v2).
  • An OpenAI API key.

Setup

  1. Clone the repository (or copy the files into a directory).

  2. Install dependencies

    npm install
    
  3. Configure environment

    Create a .env file in the project root (or edit the existing one) and add your OpenAI API key:

    OPENAI_API_KEY=your_api_key_here
    
  4. Run the agent

    npm start
    

    The script will:

    • Fetch and index the example URLs.
    • Prompt you to enter questions.
    • Display answers generated by the RAG agent.

Customization

  • Adding URLs: Edit the urls array in src/index.js to index different web pages.
  • Chunk Size: Adjust the size parameter in chunkText inside src/webSearch.js if you need larger or smaller chunks.
  • Model Parameters: Modify temperature, max tokens, or model name in src/agent.js.

Notes

  • The implementation strictly uses ChromaDB as the vector database; no other vector DBs are used.
  • All dependencies are declared in package.json and can be installed via npm install.
  • The OpenAI API key is loaded securely from the .env file using dotenv.

License

MIT License

Enjoy building with RAG!

S
Description
BroJS: Экзамен: RAG-агент с ChromaDB и веб-поиском
Readme 101 KiB
Languages
Python 82.6%
JavaScript 16.6%
Dockerfile 0.8%