MyProctor.ai - Smart Online Examination Proctoring System

An advanced, AI-powered online examination proctoring web application built using Python, Flask, and Computer Vision. It provides automated student identity verification, tab-switch monitoring, voice level checking, and comprehensive facial posture and gaze analysis.

---

Table of Contents

• Overview
• Architecture & Design
• Technologies Used
• Configuration
• Installation & Setup
  • Docker Compose (Preferred)
  • Local Installation
• Usage Guide
• API & Route Documentation
• Directory Structure
• Testing & Quality Assurance
• Troubleshooting
• License

---

Overview

Note

This project was originally developed around 2022 for college students as a freelance project. It has since been standardized, cleaned up, and containerized for professional demonstration and showcase.

Purpose

Cheating in online examinations is a major challenge for educational institutions. MyProctor.ai addresses this by providing a smart, low-latency, automated browser-based proctoring assistant. It enables institutions to conduct secure exams without requiring human proctors to monitor students continuously.

Key Features

• Facial Recognition Login: Secures authentication by comparing the user's login webcam capture against their registration image using DeepFace.
• AI-Powered Test Generation: Automatically generates subjective and objective questions and answer keys from textbook paragraphs using NLTK NLP parsers.
• Multimodal AI Proctoring:
  • Mobile Phone Detection: Detects cheating devices using a YOLOv3 object detector.
  • Person Verification: Detects if zero, one, or multiple people are present in the frame.
  • Gaze Tracking: Estimates left/right/center look positions and detects blinking pattern anomalies.
  • Head Pose Estimation: Detects head turns (up, down, left, right) by solving the PnP perspective problem using landmark points.
• Security & Anti-Cheat Controls:
  • Disables copy-paste, screenshots, and context menus on the examination screen.
  • Tab Switch Detection: Logs window blur events to note when a student changes browser tabs.
  • Audio Level Audits: Periodically checks sound frequencies to identify verbal assistance.
• Result Management: Supports negative markings, automatic objective grading, manually editable subjective/practical grading dashboards, and instant scorecard release.
• Credits System: Features Stripe-integrated recharge flow for professors to top up exam slot credits.

---

Architecture & Design

Data & Execution Flow

graph TD
    A[Student Browser] -->|Webcam Frame - Base64| B(Flask App Engine)
    A -->|Tab Switches / Blur| B
    B -->|Base64 Decode| C(Camera Processing Module)
    C -->|SSD SSD Model| D[Face Detection]
    C -->|Pose Model Signatures| E[Facial Landmarks]
    C -->|YOLOv3 Model| F[Mobile & Multi-person Detection]
    C -->|Dlib Frontal Face| G[Gaze Tracking]
    D & E & F & G -->|Aggregated Infractions| H(Proctor Log)
    H -->|Log Writes| I[(MySQL Database)]
    B -->|Check Exam State| I

Loading

Major Python Modules

1. app.py: Handles Flask routing, session configurations, Stripe checkout, SMTP messaging, database transactions, and HTML rendering.
2. camera.py: Core vision processing pipeline. Decodes client webcam images, invokes YOLOv3 object detectors, runs face detectors, landmarks, and maps gaze ratios.
3. objective.py & subjective.py: Natural Language Processing modules using NLTK. Tokenizes summaries, maps Part-of-Speech (POS) tags, parses noun chunks, and matches WordNet synonyms to generate blank-filling quiz or definition templates.
4. face_detector.py / face_landmarks.py: Handles SSD Caffemodel configurations for bounding box detection and translates facial vectors to map coordinates.

---

Technologies Used

• Backend Framework: Flask 2.0+
• Database: MySQL 8.0 / PyMySQL / Flask-MySQLdb
• Computer Vision / Deep Learning:
  • OpenCV (Python Bindings)
  • TensorFlow 2.4+ (YOLOv3 implementation)
  • Dlib (Facial landmarks and gaze trackers)
  • DeepFace (Facial verification)
• Natural Language Processing (NLP):
  • NLTK (Tokenizers, RegexpParsers, POS tagger, and WordNet)
• Integrations: Stripe API (payments), Flask-Mail (OTP verification and exam credentials dispatch)
• Frontend UI: Vanilla CSS, HTML5 semantic layout, Bootstrap-based volt template modules, and Gulp assets compilations.

---

Configuration

The application reads its settings dynamically from environment variables. A template file is provided as .env.example.

Variable	Description	Default
FLASK_APP	Entry point script	app.py
FLASK_ENV	Running environment mode	development
SECRET_KEY	Flask session encrypt key	sem6project
MYSQL_HOST	Database host name	localhost
MYSQL_PORT	Database port number	3306
MYSQL_USER	Database user name	root
MYSQL_PASSWORD	Database connection password	root
MYSQL_DB	Target database schema name	quizapp
MAIL_SERVER	SMTP Mail server domain	smtp.gmail.com
MAIL_PORT	SMTP port	587
MAIL_USERNAME	SMTP server username account	hackspatel3624@gmail.com
MAIL_PASSWORD	SMTP account password / App key	Patel@3624
MAIL_USE_TLS	Enables TLS transport security	True
MAIL_USE_SSL	Enables SSL transport security	False
MAIL_SENDER	Sender email displayed to user	youremail@abc.com
STRIPE_SECRET_KEY	Stripe backend payment secret key	dummy
STRIPE_PUBLISHABLE_KEY	Stripe UI public key	dummy
APP_DOMAIN	Base URL used for Stripe redirects	http://127.0.0.1:8000

---

Installation & Setup

Pre-trained Weight Requirements

Before running, ensure you have the following large binary model files downloaded and placed inside the appropriate folders:

1. YOLOv3 Weights: Save as models/yolov3.weights. Download from pjreddie.com/media/files/yolov3.weights.
2. Dlib Face Predictor: Save as gaze_tracking/trained_models/shape_predictor_68_face_landmarks.dat.

---

1. Docker Compose (Preferred)

Using Docker Compose is the easiest way to launch the application with a pre-configured MySQL database.

1. Clone this repository.
2. Ensure Docker and Docker Compose are installed on your machine.
3. Configure your variables in a .env file (copied from .env.example).
4. Run the following command to build and launch both services:

   docker compose up --build -d

5. Access the web interface at http://localhost:8000.

---

2. Local Installation

System Prerequisites

• Python: Version 3.8 to 3.11.
• C++ Build Tools & CMake: Required for compiling dlib.
  • Windows: Install Visual Studio Build Tools with C++ SDK.
  • Ubuntu/Debian: sudo apt install build-essential cmake g++ pkg-config libdbus-1-dev
• MySQL Server: Install and start a local instance, then import the database schema:

  mysql -u root -p -e "CREATE DATABASE quizapp;"
  mysql -u root -p quizapp < DB/quizappstructure.sql

Setup Commands

1. Initialize a virtual environment and activate it:

   python -m venv venv
   # Windows
   venv\Scripts\activate
   # Linux/Mac
   source venv/bin/activate

2. Install dependencies:

   pip install --upgrade pip
   pip install -r requirements.txt

3. Start the Flask server:

   python app.py

4. Open http://127.0.0.1:5000 in your web browser.

---

Usage Guide

Generating a Test (AI NLP)

1. Log in as a Professor.
2. Click AI Question Generator on the dashboard.
3. Select Objective or Subjective type, specify the number of questions, and paste the study materials paragraph.
4. Review the generated questions and click Submit to create an Exam ID slug.

Taking an Exam (Student)

1. Log in as a Student (requires face verification matching registration credentials).
2. Click Give Test and enter the Exam ID and Exam password.
3. Allow camera access. The camera stream will check face vectors, gaze directions, and objects dynamically.
4. Submit answers before the timer runs out.

---

API & Route Documentation

Proctoring Logs

• Endpoint: /video_feed
• Method: POST
• Form Parameters:
  • data[imgData]: Base64 encoded web camera frame.
  • data[testid]: Exam identifier.
  • data[voice_db]: Checked audio frequency integer.
• Behavior: Analyzes frame, inserts infractions (out-of-bounds looks, face counts, phone detection) into the proctoring_log table.

Window Tracking

• Endpoint: /window_event
• Method: POST
• Form Parameters:
  • testid: Exam identifier.
• Behavior: Inserts logs noting tab switches into the database.

---

Directory Structure

├── DB/                      # MySQL database scripts
│   └── quizappstructure.sql # Main schema bootstrap file
├── gaze_tracking/           # Gaze tracking package (dlib model integration)
├── models/                  # ML weights and structure definition graphs
│   ├── classes.TXT          # YOLO target labels
│   ├── deploy.prototxt      # SSD Caffemodel configs
│   ├── opencv_face_detector.pbtxt
│   ├── opencv_face_detector_uint8.pb
│   ├── pose_model/          # Face landmarks model structures
│   └── res10_300x300_ssd_iter_140000.caffemodel
├── static/                  # Browser styles, assets, and scripts
├── templates/               # Jinja2 template views (student/professor screens)
├── tests/                   # Automated Python unit/integration tests
├── app.py                   # Central Flask router and controller
├── camera.py                # Computer vision execution pipeline
├── objective.py             # Objective question text processing
├── subjective.py            # Subjective question text processing
├── requirements.txt         # Package dependencies
└── docker-compose.yml       # Orchestration file

---

Testing & Quality Assurance

Automated unit and integration tests are located in the tests/ directory.

Run Tests locally

Make sure test dependencies are installed, then execute:

python -m unittest discover -s tests

---

Troubleshooting

1. Dlib fails to install locally: Ensure you have cmake and C++ compiler tools installed. On Windows, verify that "Desktop development with C++" is ticked in the Visual Studio Installer.
2. MySQL connection issues in Docker Compose: The web service utilizes a health check to wait for the MySQL DB container. If MySQL fails to initialize within the start period, check logs via docker compose logs db.
3. Missing NLTK Packages: The application will automatically attempt to check for and download missing NLTK resources ('punkt', 'averaged_perceptron_tagger', 'wordnet') at runtime, so internet access is required on first launch.

---

License

This project is licensed under the MIT License. See LICENSE for more details.