🍽️ Purdue Dining Hall Menu Scraper & Web App

A comprehensive tool for scraping, analyzing, and viewing nutrition information from Purdue University dining halls. Features both a command-line interface for data collection and a beautiful web application for interactive browsing and filtering.

✨ Features

CLI Scraper (main.py)

• 📊 Scrape menu data from all Purdue dining courts via API
• 💾 Persistent SQLite database for storing and updating menu items
• 📈 Multiple ranking systems:
  • Lowest calories
  • Real food items (excludes condiments/seasonings)
  • Diet score: Protein / (Fat + Carbs × 0.25)
• 📁 CSV exports organized by meal type (Breakfast, Lunch, Dinner, Late Lunch)
• 🔄 Incremental updates - Only updates changed nutrition values
• ⚡ Concurrent fetching for faster data collection

Web Application (api.py)

• 🌐 Interactive web interface at http://localhost:5000
• 🔍 Advanced filtering:
  • Filter by calories, protein, fat, carbs, fiber, sodium, cholesterol
  • Search by item name, dining hall, or meal
  • Filter "real food" items only
• 📊 Multiple sorting options:
  • Lowest calories
  • Real food (lowest calories)
  • Best diet score
• 🏛️ Filter by dining hall (Wiley, Earhart, Hillenbrand, Windsor, Ford)
• ⏰ Filter by meal time (Breakfast, Brunch, Lunch, Late Lunch, Dinner)
• 📱 Responsive design - Works on desktop, tablet, and mobile
• 💾 5-minute caching to reduce API calls

🚀 Quick Start

Prerequisites

• Python 3.7+
• Internet connection (to access Purdue's dining API)

Installation

1. Clone the repository

   git clone <your-repo-url>
   cd WescraperMenuScript

2. Install dependencies

   pip install -r requirements.txt

Usage

Run the Web App

python api.py

Then open your browser to http://localhost:5000

Run the CLI Scraper

python main.py

This will:

• Scrape today's menu from all dining courts
• Update the SQLite database (purdue_menu.db)
• Generate CSV files in the output/ directory

📁 Project Structure

WescraperMenuScript/
├── api.py                 # Flask web application server
├── main.py                # CLI scraper entry point
├── requirements.txt       # Python dependencies
├── README.md             # This file
├── .gitignore            # Git ignore rules
│
├── core/                 # Core functionality modules
│   ├── __init__.py
│   ├── config.py        # Configuration constants
│   ├── database.py      # SQLite database operations
│   ├── parsers.py       # Nutrition data parsing
│   ├── scrapers.py      # API/HTML scraping logic
│   └── utils.py         # Filtering, sorting, utility functions
│
├── templates/            # Flask templates
│   └── index.html       # Main web app page
│
├── static/              # Static web assets
│   ├── css/
│   │   └── style.css   # Web app styling
│   └── js/
│       └── app.js      # Frontend JavaScript
│
├── output/              # Generated CSV files (gitignored)
├── debug_json/          # Debug API responses (gitignored)
└── purdue_menu.db       # SQLite database (gitignored)

🔧 Configuration

Core Settings (core/config.py)

• DINING_COURTS: List of dining halls to scrape
• MEAL_TIMES: Available meal times
• MAX_CONCURRENT_WORKERS: Thread pool size for concurrent requests
• SAVE_DEBUG_JSON: Save debug JSON files (default: False)

Web App Settings (api.py)

• CACHE_DURATION: Menu data cache duration in seconds (default: 300)
• Port: Default 5000 (change in api.py if needed)

📊 Data Models

Menu Item Fields

• Item_ID: Unique identifier
• Name: Food item name
• Court: Dining hall name
• Meal: Meal time (Breakfast, Lunch, Dinner, etc.)
• Station: Serving station name
• Serving Size: Serving size with gram conversion
• Calories, Total Fat, Saturated Fat, Trans Fat
• Cholesterol, Sodium
• Total Carbohydrate, Dietary Fiber, Total Sugars
• Protein
• Protein_Fat_Ratio: Calculated diet score

🎯 Use Cases

1. Find Low-Calorie Options: Sort by calories to find the healthiest options
2. High Protein Meals: Use diet score to find protein-rich, low-fat meals
3. Dining Hall Comparison: Filter by court to compare options across halls
4. Meal Planning: Export CSV data for meal planning and tracking
5. Nutrition Tracking: Use the database to track nutrition over time

🛠️ Development

Running Tests

The scraper can be tested by running:

python main.py

Database Management

The SQLite database is automatically created and managed. Database operations are handled in core/database.py.

Adding New Features

• New sorting methods: Add functions in core/utils.py
• New filters: Extend filter_nutrition_data() in core/utils.py
• UI improvements: Edit templates/index.html and static/css/style.css
• API endpoints: Add routes in api.py

🐛 Troubleshooting

Port Already in Use

If port 5000 is already in use:

• Wait 10-30 seconds for the port to be released, or
• Kill the process: taskkill /PID <PID> /F (Windows) or kill <PID> (Unix)
• Or change the port in api.py

No Data Loading

• Verify the date is valid (API typically only has current day data)
• Check internet connection
• Check browser console for errors

Module Not Found

pip install -r requirements.txt

Server Won't Shut Down

Press Ctrl+C once. If it doesn't respond, use taskkill (Windows) or kill (Unix) to force stop.

📝 Notes

• API Limitations: Purdue's dining API typically only provides data for the current date
• Data Freshness: Menu data is cached for 5 minutes in the web app
• Database: The SQLite database (purdue_menu.db) stores historical data and is automatically updated
• Real Food Filter: Excludes condiments, sauces, seasonings, fruits, and vegetables from certain rankings

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

📄 License

This project is for educational purposes. Please respect Purdue University's terms of service when using their dining API.

🙏 Acknowledgments

• Purdue University Dining Services for providing the menu API
• Built with Flask, BeautifulSoup4, and SQLite

---

Made for Purdue students who care about nutrition! 🎓