Structured Log Parser

This project demonstrates how to parse raw log lines into typed events using Pydantic v2 and LangChain's structured output.
It supports two event types:

Event	Fields
HttpOkEvent	kind="ok", status=200, path, duration_ms
HttpErrorEvent	kind="error", status (4xx/5xx), path, error_message

The parser uses a LangChain prompt to convert each log line into a JSON object that matches one of the schemas. The output is then validated with Pydantic.

Features

• Discriminated Union: ApiEvent is a union of HttpOkEvent and HttpErrorEvent with a kind discriminator.
• Structured Output: Uses LangChain's PydanticOutputParser to enforce schema.
• Batch Parsing: Handles multiple log lines in a single run.
• CLI: Accepts example logs, a file, or custom text and prints a table of parsed events.

Installation

# Clone the repo
git clone https://github.com/yourusername/structured-log-parser.git
cd structured-log-parser

# Create a virtual environment (optional but recommended)
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

Usage

1. Using the built‑in example

python -m src.main --example

2. Parsing a log file

python -m src.main --file path/to/log.txt

3. Parsing custom text

python -m src.main --text "2026-08-31 12:00:01 INFO /api/users 200 123ms"

The output will be a Markdown‑style table:

| kind | path          | status | duration_ms / error_message |
|------|---------------|--------|-----------------------------|
| ok   | /api/users    | 200    | 123 ms                      |
| error| /api/orders   | 404    | Not Found                   |
| error| /api/payments | 500    | Internal Server Error       |
| ok   | /api/products | 200    | 45 ms                       |

Environment Variables

The project uses OpenAI's API. Set the following variable before running:

export OPENAI_API_KEY="sk-..."

Alternatively, create a .env file in the project root:

OPENAI_API_KEY=sk-...

The python-dotenv package will load it automatically.

Project Structure

src/
├── __init__.py
├── main.py          # CLI entry point
├── cli.py           # Argument parsing and table rendering
├── parser.py        # Log parsing logic
└── models.py        # Pydantic event models

License

MIT License