A behavioral-analytics and rigorous A/A/B testing pipeline auditing user conversion funnels and validating mobile app typography changes for FoodFlow — combining event log ingestion with non-parametric proportion tests and Bonferroni correction to drive UX decisions with statistical certainty, not assumptions[cite: 4].
This project audits FoodFlow's user acquisition and conversion funnel, validates baseline traffic splitting through an A/A test, and rigorously evaluates whether an experimental font design change meaningfully improves conversion or poses risks to business operations[cite: 4]. Every finding below is backed by executed statistical tests (Mann-Whitney U) and verified behavioral aggregations from SQL/event-extracted logs[cite: 4].
-
Phase 1 (Data Sanitization & Temporal Alignment): Ingested tab-delimited behavioral logs (
logs_exp_us.csv), normalized schemas, converted epoch timestamps, and applied defensive filtering for August 2019 — removing only 2,828 events (1.16%) and 17 users (0.23%) to preserve 98.84% data integrity[cite: 4]. -
Phase 2 (Funnel Analysis): Reconstructed the core navigation sequence (
MainScreen$\rightarrow$ OffersScreen$\rightarrow$ CartScreen$\rightarrow$ PaymentScreen), identifying the steepest drop-off between the Main Screen and Offers Screen (61.91% retention) with a global conversion baseline of 47.70%[cite: 4]. -
Phase 3 (Integrity Validation / A/A Test): Validated statistical equivalence between control groups 246 and 247 using Mann-Whitney U proportion tests across all milestones (all p-values > 0.05,
significant = False), confirming zero assignment bias[cite: 4]. -
Phase 4 (Experimental Evaluation / A/B Test): Contrasted Variant 248 against controls under a strict Bonferroni correction (
$\alpha = 0.0025$ ) to prevent false positives across multiple comparisons[cite: 4]. All p-values remained well above threshold, confirming total statistical neutrality[cite: 4]. - Phase 5 (Consolidation & Visualization): Summarized multi-variant p-value distributions into a professional significance heatmap for stakeholder review[cite: 4].
- The primary friction point is early discovery, not checkout: Nearly 38% of users abandon the app between the main screen and product offers (
OffersScreenAppearretention: 61.91%), whereas checkout conversion is exceptionally robust (81.30% cart retention, 94.78% payment retention)[cite: 4]. - Experiment infrastructure is rock-solid: The A/A test proved absolute parity between control groups 246 and 247, protecting product leadership from false positives[cite: 4].
- Typography changes are statistically neutral: Variant 248 neither harms operational integrity nor boosts conversion rates, freeing the product team to focus resources on higher-leverage UX improvements[cite: 4].
| 🎨 UX & Product Strategy | 🧪 Experimentation Governance | 📊 Funnel Optimization |
|---|---|---|
| Halt the rollout of Variant 248 typography changes as they provide no measurable lift to conversion[cite: 4]. | Mandate Bonferroni or Holm-Bonferroni corrections for all multi-variant mobile app experiments[cite: 4]. | Redesign the transition between the Main Screen and Offers Screen to reduce the 38% drop-off rate[cite: 4]. |
| Redirect development bandwidth toward optimizing product discovery and offer presentation[cite: 4]. | Maintain rigorous A/A testing protocols as a mandatory gate before deploying future UI variants[cite: 4]. | Investigate user friction points on the offers page through qualitative session recordings or surveys[cite: 4]. |
| Strategic Initiative | Primary Target KPI | Statistical Basis |
|---|---|---|
| Funnel Discovery Optimization | Main Screen |
Steepest drop-off identified at 61.91% retention across 7,419 baseline users[cite: 4] |
| Rigorous Experimentation Gate | Family-wise error rate control[cite: 4] | Bonferroni-adjusted |
| Typography & UI Governance | Conversion rate stability[cite: 4] | Variant 248 p-values range from 0.18 to 0.76 (statistically neutral vs. control)[cite: 4] |
- Repository Slug:
foodflow-user-behavior-aab-test[cite: 4] - Primary Goal: Equip FoodFlow's product and UX teams with a statistically validated audit of user conversion funnels and typography experiment results[cite: 4].
- Key Achievements:
- Funnel Bottleneck Identification: Pinpointed the exact drop-off point where 38% of traffic abandons the navigation path prior to reviewing offers[cite: 4].
- Rigorous A/A/B Validation: Confirmed zero control group bias and proved the absolute neutrality of Variant 248 using non-parametric Mann-Whitney U tests under strict Bonferroni correction[cite: 4].
- Language: Python 3.12[cite: 4]
- Data Source: Behavioral event logs (
logs_exp_us.csv)[cite: 4] - Data Processing: pandas, numpy[cite: 4]
- Statistical Inference: scipy.stats (Mann-Whitney U test, Bonferroni correction)[cite: 4]
- Data Visualization: plotly.express, matplotlib, seaborn[cite: 4]
- Environment: Jupyter Notebook[cite: 4]
foodflow-user-behavior-aab-test/ ├── foodflow-user-behavior-aab-test.ipynb # Full analysis pipeline (executed, outputs included) ├── logs_exp_us.csv # Raw behavioral event logs (tab-delimited) ├── requirements.txt # Reproducible environment dependencies ├── README.md └── .gitignore # Excludes venv/ and generated artifacts
# Clone the repository
git clone [https://github.com/CarlosACrespoS/foodflow-user-behavior-aab-test](https://github.com/CarlosACrespoS/foodflow-user-behavior-aab-test)
# Navigate to the project directory
cd foodflow-user-behavior-aab-test
# Create and activate a virtual environment (recommended)
python -m venv venv_foodflow
source venv_foodflow/bin/activate # Windows: venv_foodflow\Scripts\activate
# Install required dependencies
pip install -r requirements.txt
# Launch the notebook
jupyter notebook foodflow-user-behavior-aab-test.ipynb