Log Parser CLI
Overview
This project provides a simple command‑line tool that parses raw HTTP log lines into typed API events using Pydantic v2 and LangChain structured output. The script accepts a string or a file containing log lines, parses each line into one of two event types (HttpOkEvent or HttpErrorEvent), and prints a summary table.
Installation
# Create a virtual environment (recommended)
python -m venv venv
source venv/bin/activate
# Install dependencies
pip install -r requirements.txt
The requirements.txt contains the following packages:
langchain-corelangchain-openaipydanticpython-dotenv
Environment Variables
The tool uses OpenAI (or any compatible LLM) for structured parsing. Set the following variables in a .env file or your environment:
OPENAI_API_KEY=your_api_key
OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_MODEL=gpt-3.5-turbo
If you are using a local LLM (e.g., Ollama or LM Studio), set:
OPENAI_BASE_URL=http://localhost:11434/v1
OPENAI_API_KEY=ollama
OPENAI_MODEL=llama3
Usage
# Parse a file containing log lines
python agent.py --input logs.txt
# Or provide raw log string directly
python agent.py --input "200 /api/v1/home\n404 /api/v1/notfound error: Not Found"
# If no input is provided, a sample log is used
python agent.py
Sample Log
The repository contains a sample log you can use for testing:
200 /api/v1/users
404 /api/v1/items error: Not Found
500 /api/v1/orders error: Internal Server Error
Output
The script prints a table with columns: Kind, Status, Path, and Error (if applicable). Example:
Kind Status Path Error
----- ------ -------------------- --------------------------------
ok 200 /api/v1/users
error 404 /api/v1/items Not Found
error 500 /api/v1/orders Internal Server Error
Extending
You can extend the Pydantic models or the parsing logic by editing agent.py. The LLM prompt is generated automatically using the PydanticOutputParser schema instructions.
Feel free to open issues or pull requests if you encounter bugs or have suggestions.