Load Balancer in Python (aiohttp)

A simple HTTP proxy and load balancer with weighted round-robin, sticky sessions, retries, health checks, circuit breaker, backpressure, and connection pooling, built with aiohttp.

---

Project Structure

load-balancer/
├── app/
│   ├── app.py                 # entry point
│   ├── backends.py            # backend server model
│   ├── balancer.py            # weighted round-robin logic
│   ├── proxy.py               # request forwarding & retries
│   ├── health.py              # health check loop
│   ├── config.py              # settings
│   ├── logging_conf.py        # logging configuration
│   └── tests/
│       ├── __init__.py
│       ├── conftest.py
│       ├── test_backend.py
│       ├── test_balancer.py
│       └── test_proxy_integration.py
├── mock/
│   ├── backend_1.py           # mock server on port 9001
│   └── backend_2.py           # mock server on port 9002
├── requirements.txt
├── README.md
└── main.py

---

Features

• Weighted round-robin load balancing
• Sticky sessions (cookies)
• Health checks
• Retries & request timeouts
• Circuit breaker for failing backends
• Backpressure control
• Connection pooling
• Request logging

---

Quick Start

1. Dependencies

python -m venv .venv
source .venv/bin/activate  
pip install -r requirements.txt

2. Start mock backends

python mock/backend_1.py &
python mock/backend_2.py &

Backends listen on ports 9001 and 9002:

• GET / → {"server": "backend1", "message": "ok"}
• GET /health → {"status": "alive"}

3. Start the load balancer

python main.py

Listens on http://localhost:8080

4. Test it

curl http://localhost:8080/

---

Running Tests

pytest -v

Tests cover:

• Circuit breaker logic
• Weighted round-robin selection
• Proxy forwarding
• Sticky sessions

---

Load Testing

python app/tests/load_test.py

Sends concurrent requests and reports average/max latency.

---

Configuration

Edit app/config.py:

• REQUEST_TIMEOUT — Backend request timeout
• RETRIES — Retry attempts
• HEALTH_CHECK_INTERVAL — Health check frequency
• CIRCUIT_BREAKER_THRESHOLD — Failures before circuit opens
• CIRCUIT_BREAKER_TIMEOUT — Duration backend stays disabled
• MAX_INFLIGHT_REQUESTS — Concurrent request limit
• STICKY_COOKIE — Session cookie name

---

Key Components

• WeightedRoundRobinBalancer (app/balancer.py) — Selects backends by weight
• Proxy (app/proxy.py) — Forwards requests with retries & backpressure
• Health checks (app/health.py) — Monitors backend availability
• Circuit breaker — Disables unhealthy backends