| title | Log Pipeline Debugger |
|---|---|
| colorFrom | green |
| colorTo | blue |
| sdk | docker |
| app_port | 7860 |
LogPipelineEnv is a high-fidelity OpenEnv simulation that challenges LLM agents to act as Site Reliability Engineers (SRE). The environment focuses on a common production pain point: fixing broken log parsing pipelines in distributed systems.
In modern cloud architectures, unstructured logs are the primary source of truth for observability. However, log formats are brittle—a minor upstream service change can break a regex parser, causing data loss in monitoring dashboards and alerting systems.
This environment simulates a production incident where a parser has failed. The agent must analyze raw logs, interpret Python tracebacks from failed attempts, and synthesize a robust, corrected parsing function that can handle inconsistencies and edge cases.
The agent receives a structured observation containing:
- raw_logs_sample: A list of unstructured log lines from the live stream.
- current_parser_code: The Python source code of the parser currently failing in production.
- current_error_traceback: The specific Python exception traceback generated by the current parser.
The agent interacts using a simple JSON-based action:
- action_type: Either update_parser or noop.
- new_parser_code: The full Python source for a corrected parse(line: str) -> dict function.
The environment provides a continuous reward signal:
- Accuracy (80%): Percentage of lines correctly parsed from a hidden test set.
- Robustness (20%): A bonus for handling malformed lines gracefully instead of crashing.
- Penalties: Points are deducted for syntax errors, timeouts, or infinite loops.
The environment features three distinct levels: Easy (Standard Access Logs), Medium (Mixed App Logs), and Hard (Corrupted Distributed Traces).
Install core dependencies via pip, set your HF_TOKEN, and run inference.py to evaluate the agent.
This environment is containerized and compatible with Hugging Face Spaces using the Docker SDK.