This project simulates a real-world enterprise data migration and modernization strategy. It extracts transactional data from a simulated "On-Premise" environment (hosted on AWS EC2), performs heavy distributed processing using a Hadoop/Spark cluster, and ultimately serves the data via a Cloud-Native, serverless architecture to optimize costs .

Build a data warehouse from scratch, including full load, daily incremental load, design schema, SCD Type 1 and 2.

A Data Warehousing project for retail sales using dimension modelling best practices with SCD type 2 on AWS Redshift. Utilizing AWS Lambda, Glue Workflows and Python Shell jobs to create and automate an ELT pipeline where batch data coming into S3 is loaded onto Redshift and necessary transformations are performed to meet requirements.

End-to-end Azure Databricks retail data engineering project using Medallion Architecture (Bronze, Silver, Gold). Implements Auto Loader, Unity Catalog, Delta Lake, SCD Type 1 & 2 dimensions, and Fact Orders for analytics-ready star schema modeling.

Production-grade ELT pipeline classifying VIX volatility regimes. Features incremental models, SCD-2 snapshots, recursive CTEs, and Slim CI/CD. Stack: dbt, DuckDB/MotherDuck, Python.

Enterprise Data Warehouse with Star Schema, SCD Type 2, ETL pipelines, and multi-currency analytics (SQL Server + Python)

End-to-End Retail Lakehouse on Microsoft Fabric | Medallion Architecture | PySpark | SCD Type-2 | DirectLake

Greetings and welcome to my profile.

Enterprise 5-layer Medallion pipeline on AWS | 88% ETL improvement (50 min → 6 min) | AWS Glue · PySpark · Redshift · Athena

Working with SCD Type (Change Data Capture) and need a Data Vault model to test Azure Data Factory v2? - This Code with Help!

In this project we'll create real time healthcare patient data pipeline as data source and use arious services and tools like Azure Eventhubs, Azure Databricks, Delta lake and synapse analytics. also, implement medallion architecture, schema evolution and create facts and dimension tables and connect the cleaned and transformed data to PowerBI.

End-to-end data engineering project using AWS S3, Snowflake, and dbt to implement Medallion Architecture with SCD Type 1 & Type 2 logic on Walmart sales data, followed by analytical visualizations using Seaborn and Plotly.

End-to-end retail analytics platform on Snowflake + dbt | Staging → Marts · SCD Type 2 · 45% query time reduction · GitHub Actions CI/CD

End-to-end data lakehouse on Azure Databricks — Medallion Architecture, Star Schema, SCD Type 2, CI/CD, Data Quality Framework

A production-grade Modern Data Stack (MDS) implementation featuring automated ELT, SCD Type 2 history tracking, and CI/CD quality guardrails using Dagster, dbt Core, DuckDB, and Soda.

Design and implement a full ELT data pipeline using Snowflake and S3, featuring star schema modelling, SCD Type 1 & 2 handling, and incremental load automation

Enterprise data warehouse for ICE Entertainment — 5-layer architecture with MySQL star schema, Pentaho ETL pipelines, multi-currency USD conversion, and SCD Type 2 across 500 stores and 10 countries

E-commerce analytics pipeline using snowflake and dbt - staging, intermediate, marts, scd2 snapshots

End-to-end data engineering project using AWS S3, Snowflake, and dbt to implement Medallion Architecture with SCD Type 1 & Type 2 logic on Walmart sales data, followed by analytical visualizations using Seaborn and Plotly.

Production-grade data lakehouse on Databricks — PySpark · Delta Lake · Spark Streaming · MLflow · 19.4M real NYC taxi trips