📋 Day 02 — Log Analyzer & Report Generator

30 Days of DevOps | Day 2/30 — Linux & Shell Scripting

A Bash-powered log analysis engine that parses any .log or .txt file, extracts error patterns, IP addresses, HTTP status codes, keyword frequencies, and busiest hours — then auto-generates a dark-themed HTML report and a plain text report.

---

📸 What Gets Generated

Terminal output:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  File: app.log
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  Total lines  : 50
  Errors       : 12
  Warnings     : 7
  Info         : 20

  Keyword Frequency:
  ERROR                : 12
  WARNING              : 7
  TIMEOUT              : 2
  REFUSED              : 3
  FAILED               : 4

Plus two report files per run:

• reports/report_YYYY-MM-DD_HH-MM-SS.html — dark-themed dashboard
• reports/report_YYYY-MM-DD_HH-MM-SS.txt — plain text for piping/email

---

📁 Project Structure

day-02-log-analyzer/
├── scripts/
│   └── log_analyzer.sh       # Main analysis engine
├── config/
│   └── analyzer.conf         # Keywords, paths, retention settings
├── sample-logs/
│   ├── app.log               # Sample application log
│   ├── nginx-access.log      # Sample Nginx access log
│   └── syslog.log            # Sample Linux syslog
├── reports/                  # Auto-created, gitignored
├── tests/
│   └── test_analyzer.sh      # 10 automated tests
├── docs/
│   └── architecture.md
├── .gitignore
└── README.md

---

⚙️ Features

Feature	Description
Multi-file analysis	Analyzes every .log/.txt in a directory
Keyword frequency	Configurable list of keywords counted per file
IP extraction	Top IPs parsed from any log with IPv4 addresses
HTTP status codes	Parses Nginx/Apache combined log format
Busiest hours	Extracts HH timestamps and ranks by frequency
Recent errors	Shows last 5 error lines per file
HTML report	Dark-themed dashboard with tables and color badges
Text report	Plain text for cron email or archiving
Log retention	Auto-deletes oldest reports beyond configured limit
Cron-ready	Pass any path as argument or set default in config

---

🚀 Quick Start

1. Clone the repo

git clone https://github.com/YOUR_USERNAME/day-02-log-analyzer.git
cd day-02-log-analyzer
chmod +x scripts/log_analyzer.sh

2. Run on sample logs (built-in)

bash scripts/log_analyzer.sh

3. Run on a real log directory

# Nginx logs
bash scripts/log_analyzer.sh /var/log/nginx

# A single file
bash scripts/log_analyzer.sh /var/log/syslog

# Apache
bash scripts/log_analyzer.sh /var/log/apache2

4. Open the HTML report

# The path is printed at the end of every run:
#   🌐 HTML  : reports/report_2026-06-01_10-30-00.html
xdg-open reports/report_*.html     # Linux
open reports/report_*.html         # Mac

On a remote server, copy to your local machine:

scp user@server:/path/to/day-02-log-analyzer/reports/report_*.html .

5. Run tests

bash tests/test_analyzer.sh

---

⏰ Automate with Cron

Daily report at 7am, saved automatically:

crontab -e

Add:

0 7 * * * /bin/bash /path/to/day-02-log-analyzer/scripts/log_analyzer.sh /var/log >> /var/log/log_analyzer_cron.log 2>&1

---

🔧 Configuration (config/analyzer.conf)

# Default log path when no argument is given
DEFAULT_LOG_PATH="sample-logs"

# Keywords counted in every file
KEYWORDS=(
  "ERROR" "WARNING" "CRITICAL" "FATAL"
  "EXCEPTION" "TIMEOUT" "REFUSED" "FAILED"
  "UNAUTHORIZED" "NOT FOUND"
)

# Keep only the last N reports
REPORT_RETENTION=10

---

🛠️ Requirements

Requirement	Notes
OS	Linux / macOS
Shell	Bash 4.0+
Commands	grep, awk, sed, sort, uniq, find, wc — all standard
Optional	Browser to open HTML report

---

📊 Understanding the HTML Report

The HTML report opens in any browser — no server needed:

• Summary cards — total files, lines, errors, warnings at a glance
• Per-file section — health badge (Clean / Has Errors / Critical)
• Keyword table — count of each configured keyword
• IP table — most frequent source IPs (useful for detecting attacks)
• HTTP codes — 200/400/500 breakdown for web server logs
• Busiest hours — which hour of the day had most log activity
• Recent errors — last 5 error lines with monospace rendering

---

🤝 Part of 30 Days of DevOps

Day	Topic
Day 1	✅ System Health Monitor
Day 2	✅ Log Analyzer & Report Generator (this project)
Day 3	Automated Backup Script with Rotation
...	Docker, CI/CD, Kubernetes, Terraform

---

📄 License

MIT License — free to use, modify, and distribute.