PDF2HTML Converter - Backend API

A robust Python Flask API that converts PDF documents to HTML format using AWS Textract and advanced OCR technology.

🌟 Features

PDF Processing: Convert PDF documents to structured HTML
AWS Integration: Utilizes AWS S3 for storage and Textract for OCR
Table Detection: Advanced table extraction and HTML conversion
Memory Optimization: Efficient processing for large documents
CORS Support: Cross-origin requests enabled for web applications
Error Handling: Comprehensive error management and logging
File Cleanup: Automatic temporary file management
RESTful API: Simple and intuitive API endpoints

🚀 Live API

Base URL: https://backend-pdf2html.onrender.com

Note: The service is currently configured with usage limits to manage AWS costs. Contact cho.yoonho023@gmail.com for access.

🛠️ Technology Stack

Framework: Flask (Python)
Document Processing: PyMuPDF (fitz) for PDF handling
Cloud Services:
- AWS S3 for file storage
- AWS Textract for OCR and document analysis
Server: Gunicorn WSGI server
Deployment: Render.com
Dependencies: See requirements.txt

📋 Prerequisites

Python 3.9 or higher
AWS Account with S3 and Textract access
AWS IAM credentials with appropriate permissions

🔧 Installation & Setup

1. Clone the repository

git clone https://github.com/0GhOsTO/backend-pdf2html.git
cd backend-pdf2html

2. Install dependencies

pip install -r requirements.txt

3. Environment Variables

Create a .env file or set the following environment variables:

AWS_ACCESS_KEY_ID=your_aws_access_key
AWS_SECRET_ACCESS_KEY=your_aws_secret_key
AWS_REGION=your_aws_region
BUCKET_NAME=your_s3_bucket_name

4. AWS Setup Requirements

S3 Bucket Configuration

Create an S3 bucket for temporary file storage
Configure appropriate access permissions

IAM User Permissions

Your IAM user needs the following policies:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": "arn:aws:s3:::your-bucket-name/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "textract:AnalyzeDocument"
            ],
            "Resource": "*"
        }
    ]
}

5. Run the Application

Development

python app.py

Production (with Gunicorn)

gunicorn -c gunicorn.conf.py app:app

📁 Project Structure

backend-pdf2html/
├── app.py                 # Main Flask application
├── requirements.txt       # Python dependencies
├── gunicorn.conf.py      # Gunicorn configuration
├── uploads/              # Temporary file storage (auto-created)
├── pages/                # Generated images (auto-created)
└── README.md             # This file

🔌 API Endpoints

POST `/upload`

Converts a PDF file to HTML format.

Request

Method: POST
Content-Type: multipart/form-data
Body: PDF file with key file
File Size Limit: 10MB maximum
Page Limit: 10 pages maximum per PDF

Response

{
    "job_id": "uuid-string",
    "html": "converted-html-content"
}

Error Response

{
    "error": "Error description"
}

Example Usage

cURL

curl -X POST \
  https://backend-pdf2html.onrender.com/upload \
  -F "file=@document.pdf"

JavaScript (Fetch)

const formData = new FormData();
formData.append('file', pdfFile);

const response = await fetch('https://backend-pdf2html.onrender.com/upload', {
    method: 'POST',
    body: formData
});

const result = await response.json();

Python (requests)

import requests

with open('document.pdf', 'rb') as f:
    files = {'file': f}
    response = requests.post(
        'https://backend-pdf2html.onrender.com/upload',
        files=files
    )
    result = response.json()

⚙️ Configuration

Memory Optimization

The application includes several optimizations for memory-constrained environments:

Page Limit: Processes maximum 10 pages per PDF
DPI Reduction: Uses 150 DPI instead of 200 for smaller image sizes
File Cleanup: Automatic cleanup of temporary files
Gunicorn Settings: Optimized worker configuration

File Processing Limits

Maximum File Size: 10MB per PDF
Supported Format: PDF files only
Page Limit: 10 pages per document
Image Quality: 150 DPI for optimal performance

🔄 Processing Workflow

File Upload: PDF file received via POST request
Validation: Check file type, size, and page count
PDF Conversion: Convert PDF pages to images using PyMuPDF
S3 Upload: Upload images to AWS S3 bucket
OCR Processing: Analyze images using AWS Textract
Data Extraction: Extract text, tables, and document structure
HTML Generation: Convert extracted data to structured HTML
Cleanup: Remove temporary files and images from S3 and local storage
Response: Return HTML content to client

🚧 Development

Local Development

# Install dependencies
pip install -r requirements.txt

# Set environment variables
export AWS_ACCESS_KEY_ID=your_key
export AWS_SECRET_ACCESS_KEY=your_secret
export AWS_REGION=us-east-1
export BUCKET_NAME=your-bucket

# Run development server
python app.py

Testing

# Test with sample PDF
curl -X POST http://localhost:5000/upload \
  -F "file=@sample.pdf"

🐛 Error Handling

The API handles various error scenarios:

Missing File: Returns 400 if no file is uploaded
File Size: Returns 400 if file exceeds 10MB limit
Invalid Format: Returns 400 if file is not a PDF
AWS Errors: Returns 500 for AWS service issues
Memory Errors: Returns 500 for memory-related issues (SIGKILL)
Processing Errors: Returns 500 for document processing failures
Permission Errors: Returns 500 for AWS permission issues

🔒 Security Considerations

File Validation: Only accepts PDF files with proper validation
CORS Configuration: Restricted to frontend domain (frontend-pdf2html.vercel.app)
AWS Credentials: Secure environment variable storage
Temporary Files: Automatic cleanup to prevent storage issues
File Size Limits: Prevents resource exhaustion attacks
Input Sanitization: Secure filename handling

📊 Performance & AWS Cost Management

Current Optimizations

Memory Management: Efficient image processing and cleanup
Worker Configuration: Single worker to reduce memory usage (Render free tier)
Request Timeouts: Extended timeouts for large file processing
File Size Limits: Prevents excessive resource usage

AWS Cost Management

Page Limits: Reduces Textract API costs (~$50 per 1,000 pages)
Image Cleanup: Minimizes S3 storage costs
Processing Efficiency: Optimized API usage patterns
Usage Monitoring: Service limits to control costs

Performance Metrics

Average Processing Time: 10-30 seconds per PDF
Memory Usage: ~400MB peak during processing
Textract Costs: ~$0.05 per page analyzed

🚀 Deployment

Render.com Deployment

Connect your GitHub repository to Render
Set environment variables in Render dashboard
Configure build and start commands:
- Build Command: pip install -r requirements.txt
- Start Command: gunicorn -c gunicorn.conf.py app:app

Environment Variables on Render

AWS_ACCESS_KEY_ID=your_access_key
AWS_SECRET_ACCESS_KEY=your_secret_key
AWS_REGION=us-east-1
BUCKET_NAME=your-bucket-name

Resource Requirements

Memory: Minimum 512MB (free tier), recommended 1GB+
CPU: Single core sufficient for current load
Storage: Minimal (temporary files only)

🔄 Recent Updates

✅ Added memory optimization for large files
✅ Implemented automatic file cleanup
✅ Enhanced error handling and logging
✅ Added file size validation (10MB limit)
✅ Optimized Gunicorn configuration
✅ Improved AWS integration and error handling
✅ Fixed SIGKILL memory issues
✅ Added page limit restrictions (10 pages max)

🚀 Future Enhancements

Async processing for large files using Celery
Redis queue for job management and status tracking
Additional file format support (DOCX, images)
Enhanced table detection algorithms
API rate limiting and authentication
Webhook support for processing notifications
Batch processing capabilities
Custom HTML styling options

🤝 Contributing

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Commit your changes (git commit -m 'Add amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request

Development Guidelines

Follow PEP 8 Python style guidelines
Add tests for new features
Update documentation for API changes
Test with various PDF formats

📞 Contact

Andrew Cho

Email: cho.yoonho023@gmail.com
GitHub: @0GhOsTO
Frontend Demo: frontend-pdf2html.vercel.app

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

AWS Textract for advanced document analysis and OCR capabilities
PyMuPDF for efficient PDF processing and image extraction
Flask community for excellent web framework documentation
Render.com for reliable and affordable hosting platform

Name		Name	Last commit message	Last commit date
README.md		README.md
app.py		app.py
pdf2htmlmock.gif		pdf2htmlmock.gif
requirements.txt		requirements.txt

Latest commit

History

Folders and files

Repository files navigation