Human‑in‑the‑Loop (HITL) middleware is a lightweight Python framework that lets you build conversational agents on top of Qdrant vector store and Ollama LLMs.
The project demonstrates how to:

Store and retrieve documents with Qdrant.
Embed text using the nomic-embed-text model from Ollama.
Generate responses with the llama3 chat model via LangChain.
Wrap everything in a simple HTTP client that can be used by external services or UI front‑ends.

The core logic lives in two files:

File	Purpose
agent.py	Implements the LangChain pipeline: embedding → vector search → LLM generation.
client.py	Exposes a minimal FastAPI server that accepts user queries and returns responses from `agent.py`.

📦 Prerequisites

Component	Minimum Version	Notes
Python	3.10+	Tested on 3.12
pip	–	Use the system package manager or `pipx`
Ollama	Latest	Install from https://ollama.ai/
Qdrant	1.7+	Run locally (`docker run -p 6333:6333 qdrant/qdrant`) or use a managed instance

Important:
The Ollama image must expose the llama3 and nomic-embed-text models.
ollama pull llama3
ollama pull nomic-embed-text
Qdrant should be reachable at http://localhost:6333 (or set via the QDRANT_URL env var).

⚙️ Installation

# Clone the repo
git clone https://github.com/your-org/hitl-middleware.git
cd hitl-middleware

# Create a virtual environment (optional but recommended)
python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

requirements.txt contains:

langchain==0.2.*
langchain-ollama==0.1.*
langchain-qdrant==0.1.*
fastapi==0.*   # for client.py
uvicorn==0.*   # ASGI server
python-dotenv==1.*

🚀 Running the Project

1️⃣ Start Qdrant (if not already running)

docker run -d --name qdrant \
  -p 6333:6333 \
  qdrant/qdrant

Make sure the container is healthy before proceeding.

2️⃣ Run the Agent

The agent can be executed as a script or imported into other code.
It will automatically load embeddings, connect to Qdrant, and expose a process_query function.

python agent.py

The script prints a simple “Agent ready” message and waits for input if run directly.

3️⃣ Run the Client (FastAPI)

The client exposes an HTTP endpoint /query.
It forwards incoming requests to the agent and returns the LLM response.

uvicorn client:app --host 0.0.0.0 --port 8000

You should see:

INFO:     Started server process [12345]
...
INFO:     Application startup complete.

Now you can send requests to http://localhost:8000/query.

📄 Example Usage

Using the HTTP API

curl -X POST http://localhost:8000/query \
  -H "Content-Type: application/json" \
  -d '{"question": "What is the capital of France?"}'

Response

{
  "answer": "The capital of France is Paris."
}

Using the Agent Directly (Python)

from agent import process_query

response = process_query("Explain quantum computing in simple terms.")
print(response)
# Output: "Quantum computing uses qubits..."

🔧 Configuration

All configuration values can be overridden via environment variables or a .env file placed at the project root.

Variable	Default	Description
`QDRANT_URL`	`http://localhost:6333`	Qdrant endpoint
`OLLAMA_HOST`	`http://localhost:11434`	Ollama API host
`LLM_MODEL`	`llama3`	Chat model name
`EMBEDDING_MODEL`	`nomic-embed-text`	Embedding model name

Example .env:

QDRANT_URL=http://qdrant:6333
OLLAMA_HOST=http://ollama:11434
LLM_MODEL=llama3
EMBEDDING_MODEL=nomic-embed-text

📚 Further Reading

Feel free to extend the middleware with custom prompts, retrieval strategies, or additional LLMs. Happy hacking!

README.md Unescape Escape

Human‑in‑the‑Loop Middleware