Mohamed Taha Abo Heiba — Data Engineer.
I built this project to apply what I've learned in data warehousing — from working with raw data to producing business-ready insights. It demonstrates an end-to-end data pipeline including data ingestion, transformation, and modeling, following industry best practices.
- ETL processes
- Data modeling
The data architecture follows Medallion Architecture with Bronze, Silver, and Gold layers:
- Bronze Layer: Stores raw data as-is from the source systems. Data is ingested from CSV files into a SQL Server database.
- Silver Layer: Data cleansing, standardization, and normalization to prepare data for analysis.
- Gold Layer: Business-ready data modeled into a star schema for reporting and analytics.
- Data Architecture: Designing a Modern Data Warehouse using Medallion Architecture Bronze, Silver, and Gold layers.
- ETL Pipelines: Extracting, transforming, and loading data from source systems into the warehouse.
- Data Modeling: Developing fact and dimension tables optimized for analytical queries.
- SQL Server — Data storage, transformation, and processing
- SQL — ETL pipelines and data modeling logic
- Draw.io — Architecture and data model documentation
Develop a modern data warehouse using SQL Server to consolidate sales data, enabling analytical reporting and informed decision-making.
- Data Sources: Import data from two source systems (ERP and CRM) provided as CSV files.
- Data Quality: Cleanse and resolve data quality issues before analysis.
- Integration: Combine both sources into a single, user-friendly data model designed for analytical queries.
- Scope: Focus on the latest dataset only; historization of data is not required.
- Documentation: Provide clear documentation of the data model to support both business stakeholders and analytics teams.
Mohamed Taha Abo Heiba