End-to-end data analysis project: raw messy data → cleaned dataset → SQL business-question queries → Python EDA & visualizations → interactive Power BI dashboard. Built to demonstrate the full Data Analyst workflow, not just a single tool.
Note on the data: This uses a synthetically generated dataset (
scripts/generate_data.py) structured like a typical retail transactions export (Orders, Region, Category, Sales, Profit, Discount, etc.) — chosen so I could inject realistic messiness (missing values, duplicate rows, inconsistent date formats, outliers) on purpose and show how I handle each one. It is not scraped or copied from any existing dataset.
A retail company wants to understand: which regions/categories drive profit vs. just revenue, how discounting affects margin, and where operational inefficiencies (shipping delays, loss-making orders) are hiding in the data — to inform pricing and inventory decisions.
| Layer | Tools |
|---|---|
| Data cleaning | Python, pandas, numpy |
| Analysis | SQL (SQLite), pandas |
| Visualization | matplotlib |
| Dashboard | Power BI |
retail-sales-analysis/
├── data/
│ ├── raw_sales_data.csv # Raw (intentionally messy) export
│ └── cleaned_sales_data.csv # Output of data_cleaning.py
├── scripts/
│ ├── generate_data.py # Generates the raw synthetic dataset
│ ├── data_cleaning.py # Cleans raw -> cleaned CSV
│ ├── eda_visualization.py # Produces the 5 charts in visuals/
│ └── sql_analysis.py # Loads data into SQLite, runs sql/queries.sql
├── sql/
│ └── queries.sql # 8 business-question SQL queries
├── visuals/ # PNG charts generated by matplotlib
├── powerbi/
│ ├── PowerBI_Dashboard_Guide.md # Step-by-step build guide
│ └── Retail_Sales_Dashboard.pbix # (add after building — see guide)
├── requirements.txt
└── README.md
git clone https://github.com/<your-username>/retail-sales-analysis.git
cd retail-sales-analysis
pip install -r requirements.txtA raw dataset is already included in data/raw_sales_data.csv, but you can
regenerate a fresh one:
python3 scripts/generate_data.pypython3 scripts/data_cleaning.pyThis prints each cleaning step as it runs (duplicates removed, missing
values handled, outliers capped) and writes data/cleaned_sales_data.csv.
python3 scripts/sql_analysis.pyLoads the cleaned CSV into a local SQLite database (data/sales.db) and
runs all 8 queries from sql/queries.sql, printing the results.
python3 scripts/eda_visualization.pySaves 5 charts to visuals/ and prints the key numeric insights.
Follow powerbi/PowerBI_Dashboard_Guide.md — loads the same cleaned CSV
into Power BI Desktop and walks through building 5 visuals, 3 KPI cards,
DAX measures, and slicers.
- Overall profit margin sits around 7.3%, with North the top region by total sales.
- Discount level is negatively correlated (-0.30) with profit margin — heavier discounting measurably erodes profitability, not just intuitively.
- Technology sub-categories (Phones, Copiers, Machines) drive the most revenue but carry some of the largest loss-making outlier orders — worth a pricing/discount policy review.
- A small number of orders are outright loss-making even after cleaning, flagged directly by SQL query #5 for follow-up.
- Handling realistic messy data (mixed date formats, duplicates, nulls,
outliers) with documented, justified cleaning decisions — not just
dropna()everywhere. - Writing business-question-driven SQL, not just
SELECT *. - Turning analysis into visual, interpretable charts with clear takeaways.
- Building an interactive BI dashboard a non-technical stakeholder could actually use.
- Swap the synthetic dataset for a real one (e.g., your own college placement data, a Kaggle e-commerce dataset) — the scripts are written generically enough to adapt with minor column-name changes.
- Add a forecasting model (e.g., simple linear trend or Prophet) on the monthly sales series.
- Deploy the Power BI report to Power BI Service and share a public link instead of just a screenshot.
Author: Rizvan Khan Contact: rizwankhan59153@gmail.com