This project implements a complete data engineering and analytics pipeline designed to analyze how study habits, stress levels, and digital learning tools influence student productivity and academic outcomes.
The system integrates multiple datasets, performs automated ETL processing, stores structured data in a PostgreSQL data warehouse, and produces analytical insights through Tableau dashboards.
The project demonstrates a production-style analytics architecture combining data engineering, SQL analytics, and business intelligence visualization.
The primary goal of this project is to build a data-driven analytics platform capable of identifying behavioral patterns in student learning.
The platform analyzes relationships between:
- Study hours ( related toi how many hours)
- Study consistency
- Stress levels
- Digital tool usage
- AI tool adoption
- Academic performance indicators
The final system enables behavioral analytics and educational insights that help identify:
- High engagement students
- Students at academic risk
- Impact of digital tools on productivity
- Stress-productivity relationships
- Study behavior trends over time
CSV / Survey Datasets ↓ Python ETL Pipelines ↓ PostgreSQL Data Warehouse ↓ SQL Analytical Views ↓ Tableau Dashboard
The architecture follows a modern analytics pipeline structure, similar to those used in real-world data teams.
The platform focuses on student behavior rather than only final grades, allowing deeper understanding of:
- learning habits
- productivity fluctuations
- stress impact
- digital tool usage
The platform integrates multiple datasets:
| Dataset | Purpose |
|---|---|
| Student Study Survey | Weekly study behavior |
| Student Performance Dataset | Academic performance indicators |
| Student Stress Dataset | Psychological and lifestyle indicators |
The project implements a modular ETL pipeline that performs:
- Data ingestion
- Data cleaning
- Schema alignment
- Data validation
- Warehouse loading
| Layer | Technology |
|---|---|
| Data Processing | Python |
| Data Analysis | Pandas |
| ETL Pipeline | Custom Python Scripts |
| Database | PostgreSQL |
| SQL Analytics | PostgreSQL Views |
| Visualization | Tableau |
| Version Control | Git + GitHub |
| CI/CD | GitHub Actions |
| Environment Management | Python venv |
The architecture follows a modern analytics pipeline structure, similar to those used in real-world data teams.
The platform focuses on student behavior rather than only final grades, allowing deeper understanding of:
- learning habits
- productivity fluctuations
- stress impact
- digital tool usage
The platform integrates multiple datasets:
| Dataset | Purpose |
|---|---|
| Student Study Survey | Weekly study behavior |
| Student Performance Dataset | Academic performance indicators |
| Student Stress Dataset | Psychological and lifestyle indicators |
The project implements a modular ETL pipeline that performs:
- Data ingestion
- Data cleaning
- Schema alignment
- Data validation
- Warehouse loading
| Layer | Technology |
|---|---|
| Data Processing | Python |
| Data Analysis | Pandas |
| ETL Pipeline | Custom Python Scripts |
| Database | PostgreSQL |
| SQL Analytics | PostgreSQL Views |
| Visualization | Tableau |
| Version Control | Git + GitHub |
| CI/CD | GitHub Actions |
| Environment Management | Python venv |
study_tools_survey │ ├── backend │ │ ├── etl │ │ baseline_etl.py │ │ weekly_etl.py │ │ student_performance_loader.py │ │ stress_loader.py │ │ utils.py │ │ │ ├── pipeline │ │ orchestrator.py │ │ │ ├── sql │ │ schema.sql │ │ analytics_views.sql │ │ refresh_views.sql │ │ │ ├── run_pipeline.py │ └── requirements.txt │ ├── data │ ├── layer2_student_performance │ │ student-mat.csv │ │ student-por.csv │ │ │ └── layer3_stress_dataset │ stress_students.csv │ ├── dashboards │ study_behavior_dashboard.twb │ ├── .github │ workflows │ pipeline.yml │ └── README.md
The PostgreSQL warehouse contains multiple relational tables designed for behavioral analytics.
| Table | Description |
|---|---|
| baseline_cohorts | Student baseline academic context |
| weekly_observations | Weekly behavioral survey responses |
| student_performance | External academic performance dataset |
| student_stress_context | Student lifestyle and stress dataset |
| cohort_weekly_metrics | Derived analytics table |
baseline_cohorts │ └── cohort_key │ ▼ weekly_observations
student_performance student_stress_context
The schema allows:
- cohort-level analysis
- weekly behavioral tracking
- multi-dataset correlation
The ETL pipeline is implemented in Python and consists of several modular scripts.
Processes baseline survey responses and loads them into:
Processes weekly behavioral survey responses and loads them into:
Loads external academic performance dataset.
Loads psychological and lifestyle dataset.
To simplify execution, the entire ETL workflow can be executed using a single command.
The orchestrator performs:
Baseline ETL
Weekly ETL
Student performance dataset load
Stress dataset load
SQL analytics refresh
The analytics layer generates behavioral insights through SQL views.
CASE WHEN stress_level >= 4 AND total_hours_this_week < 15 THEN 'HIGH_RISK'
WHEN stress_level >= 3 THEN 'MEDIUM_RISK'
ELSE 'LOW_RISK' END
| View | Purpose |
|---|---|
| engagement_metrics | Student engagement score |
| academic_risk_analysis | Risk classification |
| tool_adoption_analysis | Digital tool usage patterns |
| stress_behavior_analysis | Stress behavior relationships |
The final analytics results are visualized using Tableau.
The dashboard includes:
- Total Students
- Average Study Hours
- Average Productivity
- Average Stress
- Study Hours vs Productivity
- Stress vs Productivity
- Academic Risk Distribution
- Digital Tool Adoption by Program
- Weekly Engagement Trends
Users can filter results by:
- Course program
- University type
- State
- Risk category
The project supports automated execution of the ETL pipeline.
Automation features include:
- Pipeline orchestration script
- Automated SQL view refresh
- Scheduled pipeline execution
- Logging support
The repository includes a GitHub Actions workflow that validates the pipeline.
The CI pipeline performs:
Install dependencies
Run ETL scripts
Validate database connections
Execute analytics SQL
This ensures the pipeline remains stable during development.
Baseline ETL completed. Accepted rows: 89 Rejected rows: 13
Weekly ETL completed. Accepted rows: 241 Rejected rows: 19
Loading student performance datasets... Inserted 1044 rows.
Loading stress dataset... Inserted 2000 rows.
REFRESH MATERIALIZED VIEW
Potential future improvements include:
- Real-time streaming ingestion
- Airflow pipeline orchestration
- Predictive modeling for academic risk
- Machine learning models for performance prediction
- API layer using FastAPI
- Cloud deployment
This project demonstrates skills in:
- Data Engineering
- SQL Analytics
- Data Warehousing
- ETL Pipeline Design
- Business Intelligence
- Behavioral Data Analysis
This system implements a full analytics pipeline that transforms raw behavioral datasets into actionable educational insights.
The platform demonstrates how data engineering pipelines and business intelligence tools can be applied to analyze student learning behavior and productivity trends.
Jayesh Kalla Rao
Computer Science and Engineering
Amrita Vishwa Vidyapeetham
✔ Data Engineering Pipeline Complete
✔ PostgreSQL Warehouse Implemented
✔ Analytical SQL Layer Complete
✔ Tableau Dashboard Functional
✔ Automation Implemented
✔ CI/CD Pipeline Integrated
Project ready for demonstration and portfolio use.
- JAY-cloudbuster (Lead Developer)
- Kanishthika11 (Data Engineer)
- Mayukasri06 (Documentation & Data)
- Srijak1202 (Testing & QA)
- suhas107 (DevOps & CI/CD)