85 lines
2.2 KiB
Markdown
85 lines
2.2 KiB
Markdown
# Log Parser CLI
|
||
|
||
## Overview
|
||
|
||
This project provides a simple command‑line tool that parses raw HTTP log lines into typed API events using **Pydantic v2** and **LangChain** structured output. The script accepts a string or a file containing log lines, parses each line into one of two event types (`HttpOkEvent` or `HttpErrorEvent`), and prints a summary table.
|
||
|
||
## Installation
|
||
|
||
```bash
|
||
# Create a virtual environment (recommended)
|
||
python -m venv venv
|
||
source venv/bin/activate
|
||
|
||
# Install dependencies
|
||
pip install -r requirements.txt
|
||
```
|
||
|
||
The `requirements.txt` contains the following packages:
|
||
|
||
- `langchain-core`
|
||
- `langchain-openai`
|
||
- `pydantic`
|
||
- `python-dotenv`
|
||
|
||
## Environment Variables
|
||
|
||
The tool uses OpenAI (or any compatible LLM) for structured parsing. Set the following variables in a `.env` file or your environment:
|
||
|
||
```env
|
||
OPENAI_API_KEY=your_api_key
|
||
OPENAI_BASE_URL=https://api.openai.com/v1
|
||
OPENAI_MODEL=gpt-3.5-turbo
|
||
```
|
||
|
||
If you are using a local LLM (e.g., Ollama or LM Studio), set:
|
||
|
||
```env
|
||
OPENAI_BASE_URL=http://localhost:11434/v1
|
||
OPENAI_API_KEY=ollama
|
||
OPENAI_MODEL=llama3
|
||
```
|
||
|
||
## Usage
|
||
|
||
```bash
|
||
# Parse a file containing log lines
|
||
python agent.py --input logs.txt
|
||
|
||
# Or provide raw log string directly
|
||
python agent.py --input "200 /api/v1/home\n404 /api/v1/notfound error: Not Found"
|
||
|
||
# If no input is provided, a sample log is used
|
||
python agent.py
|
||
```
|
||
|
||
## Sample Log
|
||
|
||
The repository contains a sample log you can use for testing:
|
||
|
||
```
|
||
200 /api/v1/users
|
||
404 /api/v1/items error: Not Found
|
||
500 /api/v1/orders error: Internal Server Error
|
||
```
|
||
|
||
## Output
|
||
|
||
The script prints a table with columns: `Kind`, `Status`, `Path`, and `Error` (if applicable). Example:
|
||
|
||
```
|
||
Kind Status Path Error
|
||
----- ------ -------------------- --------------------------------
|
||
ok 200 /api/v1/users
|
||
error 404 /api/v1/items Not Found
|
||
error 500 /api/v1/orders Internal Server Error
|
||
```
|
||
|
||
## Extending
|
||
|
||
You can extend the Pydantic models or the parsing logic by editing `agent.py`. The LLM prompt is generated automatically using the `PydanticOutputParser` schema instructions.
|
||
|
||
---
|
||
|
||
Feel free to open issues or pull requests if you encounter bugs or have suggestions.
|