Log Parser CLI

Overview

This project provides a simple command‑line tool that parses raw HTTP log lines into typed API events using Pydantic v2 and LangChain structured output. The script accepts a string or a file containing log lines, parses each line into one of two event types (HttpOkEvent or HttpErrorEvent), and prints a summary table.

Installation

# Create a virtual environment (recommended)
python -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

The requirements.txt contains the following packages:

langchain-core
langchain-openai
pydantic
python-dotenv

Environment Variables

The tool uses OpenAI (or any compatible LLM) for structured parsing. Set the following variables in a .env file or your environment:

OPENAI_API_KEY=your_api_key
OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_MODEL=gpt-3.5-turbo

If you are using a local LLM (e.g., Ollama or LM Studio), set:

OPENAI_BASE_URL=http://localhost:11434/v1
OPENAI_API_KEY=ollama
OPENAI_MODEL=llama3

Usage

# Parse a file containing log lines
python agent.py --input logs.txt

# Or provide raw log string directly
python agent.py --input "200 /api/v1/home\n404 /api/v1/notfound error: Not Found"

# If no input is provided, a sample log is used
python agent.py

Sample Log

The repository contains a sample log you can use for testing:

200 /api/v1/users
404 /api/v1/items error: Not Found
500 /api/v1/orders error: Internal Server Error

Output

The script prints a table with columns: Kind, Status, Path, and Error (if applicable). Example:

Kind  Status Path                Error                          
----- ------ -------------------- --------------------------------
ok    200    /api/v1/users         
error 404    /api/v1/items         Not Found                     
error 500    /api/v1/orders        Internal Server Error

Extending

You can extend the Pydantic models or the parsing logic by editing agent.py. The LLM prompt is generated automatically using the PydanticOutputParser schema instructions.

Feel free to open issues or pull requests if you encounter bugs or have suggestions.

2.2 KiB Raw Blame History Unescape Escape