Skip to content

feat: optimize analytics via MongoDB aggregations and Redis caching#1306

Open
PremSahith wants to merge 1 commit into
JiyaBatra:mainfrom
PremSahith:feat/1297-optimize-analytics
Open

feat: optimize analytics via MongoDB aggregations and Redis caching#1306
PremSahith wants to merge 1 commit into
JiyaBatra:mainfrom
PremSahith:feat/1297-optimize-analytics

Conversation

@PremSahith

Copy link
Copy Markdown
Contributor

🛠️ Related Issue

Closes: #1297

📌 Description

This PR significantly optimizes the application's heavy data-processing endpoints by offloading complex operations from the Node.js event loop to MongoDB's Aggregation Engine. It also introduces a Redis caching layer for the most expensive, high-traffic endpoints (leaderboards) to prevent database strain and minimize latency at scale.

✨ Changes Made

  • Database Indexing: Added compound indexes to the Progress, Analytics, and ExamResult models to allow MongoDB to efficiently $match and $sort data before aggregating it.
  • Leaderboard Aggregation & Caching: Refactored progressController.js to use a $facet aggregation, allowing the system to compute the paginated leaderboard and total counts in a single DB query. Integrated a 10-minute Redis caching layer.
  • Exam Analytics Optimization: Refactored examController.js's getMyResults to compute user attempts, best scores, and pass counts entirely in the database. Added a new /api/exam/leaderboard route with Redis caching to compute top scores globally or per-course.
  • Analytics Aggregation Pipeline: Replaced expensive synchronous JavaScript loops in analyticsController.js (including buildWeeklyStats) with a powerful MongoDB $facet aggregation that computes weekly buckets, point totals, and event filtering natively in the DB layer.

📷 Screenshots

N/A - This is a purely backend performance and architectural optimization.

🧪 Type of Change

  • Bug fix
  • New feature
  • UI/UX improvement
  • Documentation update

✔️ Checklist

  • My code follows project style guidelines
  • I have tested my changes
  • I have linked the related issue
  • No unnecessary files are included

🚨 Notes for Contributors

  • This PR introduces a Redis dependency. Ensure that a Redis server is running locally on port 6379, or configure the connection string via REDIS_URL in the .env file. (The application will gracefully fallback to direct database queries if Redis is unavailable).

@vercel

vercel Bot commented Jun 21, 2026

Copy link
Copy Markdown

@PremSahith is attempting to deploy a commit to the jiya's projects Team on Vercel.

A member of the Team first needs to authorize it.

@netlify

netlify Bot commented Jun 21, 2026

Copy link
Copy Markdown

Deploy Preview for codevibeforyou canceled.

Name Link
🔨 Latest commit 6661bfb
🔍 Latest deploy log https://app.netlify.com/projects/codevibeforyou/deploys/6a37b426d7bc6c000874ccfa

@PremSahith

Copy link
Copy Markdown
Contributor Author

@JiyaBatra please review and merge this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Optimize Analytics & Leaderboard via MongoDB Aggregation Pipelines

1 participant