Python Multi-threaded HTTP Server

This project is a multi-threaded HTTP server built from scratch in Python using socket programming. It's designed to handle multiple concurrent clients, serve static and binary files for GET requests, process JSON data for POST requests, and implement key features of the HTTP/1.1 protocol, including connection persistence and security validations.

---

Features

• Multi-threading: Utilizes a configurable thread pool to handle concurrent client connections efficiently.
• Connection Queuing: Manages incoming connections in a queue when the thread pool is saturated.
• HTTP/1.1 Protocol Support:
  • Handles GET and POST methods.
  • Implements persistent connections (Connection: keep-alive).
  • Supports connection timeouts and max requests per connection.
• Static File Serving:
  • Serves HTML files from a resources directory.
  • Renders index.html for the root path (/).
• Binary Content Delivery:
  • Serves images (.png, .jpg) and text files (.txt) as downloadable binary streams (application/octet-stream).
  • Uses Content-Disposition header to prompt file downloads.
• JSON API Endpoint:
  • A POST /upload endpoint that accepts application/json data.
  • Validates JSON and saves it to a uniquely named file in resources/uploads/.
• Security Hardening:
  • Path Traversal Protection: Prevents access to files outside the designated resources directory.
  • Host Header Validation: Rejects requests with missing or mismatched Host headers.
• Comprehensive Logging: Provides detailed, timestamped logs for server events, requests, responses, security checks, and thread pool status.
• Robust Error Handling: Returns appropriate HTTP error codes (e.g., 400, 403, 404, 405, 415, 500, 503).

---

Requirements

• Python 3.6+
• No external libraries are required.

---

Installation

1. Clone the repository:

   git clone <your-repository-url>
   cd python-http-server

---

Project Structure

python-http-server/
├── server.py             \# The main server implementation
├── resources/              \# Root directory for serving files
│   ├── index.html
│   ├── about.html
│   ├── contact.html
│   ├── sample.txt
│   ├── logo.png
│   ├── large_photo1.png
│   ├── large_photo2.png
│   ├── photo.jpg
│   ├── photo1.jpg
│   └── uploads/          \# Directory for POST request uploads (auto-created)
└── README.md

---

Usage

You can run the server with default settings or provide command-line arguments to customize its behavior.

To run with default settings (127.0.0.1:8080, 10 threads):

python server.py

To run with custom settings: The server accepts up to three command-line arguments:

1. Port: The port number to listen on.
2. Host: The host address to bind to (0.0.0.0 to listen on all interfaces).
3. Max Threads: The maximum number of threads in the thread pool.

# Syntax: python server.py [PORT] [HOST] [MAX_THREADS]
# Example: Run on port 8000 on all interfaces with 20 threads
python server.py 8000 0.0.0.0 20

---

API Endpoints

Method	Endpoint	Description	Success Response
GET	/	Serves the resources/index.html file.	200 OK (HTML content)
GET	/[filename].html	Serves any HTML file from the resources directory.	200 OK (HTML content)
GET	/[filename].[png/jpg/txt]	Serves a binary file and prompts a download.	200 OK (Binary data)
POST	/upload	Accepts JSON data and saves it to a new file in resources/uploads/.	201 Created (JSON)

---

Testing

You can use a web browser or a command-line tool like curl to test the server.

Basic GET Requests

# Get the index page (connects to default 127.0.0.1:8080)
curl -v http://127.0.0.1:8080/

# Download a PNG file (output saved to my_logo.png)
curl -v http://127.0.0.1:8080/logo.png -o my_logo.png

# Request a non-existent file (should get 404)
curl -v http://127.0.0.1:8080/nonexistent.html

POST JSON Request

# Send a JSON payload to the /upload endpoint
curl -v -X POST http://127.0.0.1:8080/upload -H "Content-Type: application/json" -d "{\"username\": \"test\", \"data\": [1, 2, 3]}"

# Expected response:
# {
#     "status": "success",
#     "message": "File created successfully",
#     "filepath": "/uploads/upload_YYYYMMDD_HHMMSS_xxxxxx.json"
# }

Security Tests

# Path Traversal attempt (should get 403 Forbidden)
# NOTE: Browser and curl both sanitise the URL before making the request, so pls test this via Postman 
curl -v http://127.0.0.1:8080/../../../../etc/passwd

# Request with mismatched Host header (should get 403 Forbidden)
curl -v http://127.0.0.1:8080/ -H "Host: evil.com"

# Request without Host header (should get 400 Bad Request)
# Note: curl often adds Host header automatically. This is best tested programmatically.
# Example using netcat:
printf "GET / HTTP/1.1\r\n\r\n" | nc 127.0.0.1 8080

Concurrency Test

Open multiple terminal windows and run download commands for large files simultaneously to test the thread pool.

# Terminal 1
curl http://127.0.0.1:8080/large_file.png -o file1.png

# Terminal 2
curl http://127.0.0.1:8080/large_file.png -o file2.png

Check the server logs to see different threads handling the connections.

---

Configuration

Parameter	Command-line Argument	Default Value	Description
Host	2nd argument	127.0.0.1	The network interface the server binds to.
Port	1st argument	8080	The port the server listens on.
Max Threads	3rd argument	10	The size of the worker thread pool.

---

Security Features

Path Traversal Protection

The server validates all file paths requested via GET. It canonicalizes the requested path and ensures that the resulting absolute path is located within the resources directory. Any request attempting to access files outside this directory (e.g., using ../) will be rejected with a 403 Forbidden error.

Host Header Validation

As per HTTP/1.1, the Host header is mandatory. This server enforces this rule:

1. Requests without a Host header are rejected with a 400 Bad Request.
2. Requests where the Host header does not match the server's own address (e.g., 127.0.0.1:8080) are rejected with a 403 Forbidden. This prevents certain DNS rebinding attacks and ensures requests are intentionally directed at this server.

---

Monitoring

The server provides real-time logging to the console. Key logged events include:

• Server startup and configuration.
• New client connections, including IP and port.
• Request details (GET /path HTTP/1.1).
• Security validation results (Host header checks, path traversal blocks).
• File serving details (filename and size).
• Final response status and bytes transferred.
• Connection status (keep-alive, close, timeout).
• Thread pool status (e.g., 8/10 active).
• Warnings when the thread pool is saturated and connections are queued.
• All errors (I/O, parsing, etc.).

---

Development

Adding New File Types

To support a new file type for download:

1. Open server.py.
2. Add a new entry to the MIME_TYPES dictionary. For a downloadable file, use application/octet-stream. For a file to be rendered by the browser, use its specific MIME type.

   MIME_TYPES = {
       # ... existing types
       '.pdf': 'application/octet-stream', # For download
       '.css': 'text/css; charset=utf-8', # For rendering
   }

Adding New Endpoints

To add a new API endpoint (e.g., PUT /update):

1. In the handle_request method, add a new condition:

   # ...
   elif method == 'PUT':
       response = self.handle_put(path, headers, body)
   # ...

2. Implement the corresponding handler method (handle_put) within the HTTPServer class, similar to handle_get and handle_post.

---

Troubleshooting

• "Address already in use" Error: This means another process is using the specified port. Either stop the other process or run the server on a different port.
• Permission Denied Error: You may not have permission to bind to the specified port (e.g., ports below 1024 often require root privileges). Try a higher port number (>1024).
• Files Not Found (404): Ensure the resources directory exists in the same location as server.py and that the requested files are inside it.
• Connection Refused: Make sure the server is running and you are connecting to the correct host and port.

---

Known Limitations

• HTTP/2.0 and HTTPS not supported: This is a simple HTTP/1.1 server. It does not implement encryption (HTTPS) or newer versions of the protocol.
• Performance: While multi-threaded, a Python-based server with this architecture may not be suitable for high-performance, production-level loads compared to optimized solutions like Nginx or Apache.
• Header Parsing: The header parser is basic and assumes a simple Key: Value format. It may not handle all edge cases of the HTTP specification.