PrepAdvisor-PFN

An intelligent Auto-ML tool that combines Automated Exploratory Data Analysis (EDA) and Classical Machine Learning pipelines with cutting-edge Tabular Foundation Models (like TabPFN). It features Zero-Shot prediction, intelligent Deep Imputation, dynamic Algorithm Ranking, and an automated head-to-head Benchmarker for any tabular dataset.

Features

• Automated EDA: Data types, missing values, correlations, outliers, skewness, problem type detection
• Preprocessing Recommendations: Missing value strategies, outlier handling, scaling, encoding, transformations
• Algorithm Ranking: Meta-feature based ranking of 10+ ML algorithms with rationale
• Feature Engineering: Polynomial features, ratios, binning, aggregations, rare category grouping
• Feature Selection: Variance threshold, correlation filtering, target correlation, mutual information, permutation importance, RFE
• CLI Tool: One-command analysis for any CSV file

Installation

git clone https://github.com/yourusername/prep_advisor.git
cd prep_advisor/preprocessing_advisor
pip install -e .

Or install dependencies manually:

pip install pandas numpy scikit-learn scipy statsmodels category-encoders fancyimpute jinja2 pyyaml loguru tqdm

Quick Start

CLI (Recommended)

# With target column (classification/regression)
python run_advisor.py your_data.csv --target target_column

# Without target (clustering)
python run_advisor.py your_data.csv

# Custom top-K algorithms
python run_advisor.py your_data.csv --target target_column --top-k 3

Advanced Features (Foundational Models)

We now support cutting-edge Tabular Foundation Models (like TabPFN) and advanced Machine Learning techniques natively in the CLI!

1. The "Zero-Shot Instant Predictor" (--auto-predict)

Instead of just telling you how to preprocess and which algorithm to use, the tool uses TabPFN to solve the problem for you instantly.

• How it works: Under the hood, we load TabPFN. Because TabPFN requires zero preprocessing (no scaling, no imputation, no hyperparameter tuning), we simply pass the raw dataframe into it.
• The Output: The tool instantly prints out the zero-shot accuracy in seconds, giving you a massive head start.

python run_advisor.py your_data.csv --target target_column --auto-predict

2. Foundation Model vs. Classical ML Benchmarker (--benchmark)

People always wonder: "Is a Tabular Foundation Model actually better than XGBoost on my specific dataset?"

• How it works: We take the top recommended classical algorithm (e.g., Random Forest or LightGBM), automatically apply the preprocessing pipeline (scaling, encoding, imputing), and train it. Then, we run a Foundation Model (TabPFN) on the raw data.
• The Output: We print a leaderboard showing whether the Foundation Model beat the classical approach on your specific CSV.

python run_advisor.py your_data.csv --target target_column --benchmark

# You can also control the train/test split size! (e.g. 70/30 split)
python run_advisor.py your_data.csv --target target_column --benchmark --test-size 0.3

3. "Deep" Zero-Shot Data Imputation (--auto-impute)

Right now, tools suggest basic imputation methods (like median or knn). We use Foundation/ML Models for cutting-edge data cleaning.

• How it works: If a column has missing values, we treat that column as the "target". We use a fast foundational/ML model to predict the missing values based on all the other columns in the dataset in seconds. (Pass --target to prevent target leakage!)
• The Output: Automatically saves the perfectly clean dataset to your_data_imputed.csv.

python run_advisor.py your_data.csv --target target_column --auto-impute

Python API

from preprocessor.eda import DataLoader, DataStats
from preprocessor.preprocessing import (
    MissingHandler, OutlierHandler, ScalingHandler, EncodingHandler, TransformHandler
)
from preprocessor.algorithms import MetaFeatureExtractor, AlgorithmRanker
from preprocessor.feature_engineering import FeatureSuggestor, FeatureSelector

# Load data
df = DataLoader.load("your_data.csv")

# Full EDA
stats = DataStats(df, target="target_column")
stats.print_summary()

# Get preprocessing recommendations
print("Missing:", MissingHandler(df).suggest_strategies())
oh = OutlierHandler(df)
oh.detect_outliers()
print("Outliers:", oh.suggest_strategy())

# Get algorithm recommendations
meta = MetaFeatureExtractor(df, target="target_column").extract_all()
ranked = AlgorithmRanker(meta).rank(top_k=5)
for r in ranked:
    print(f"{r['name']}: {r['score']}/100 - {r['rationale'][0]}")

# Feature engineering suggestions
fs = FeatureSuggestor(df, target="target_column")
for cat, sugg in fs.suggest_all().items():
    if sugg:
        print(f"{cat}: {len(sugg)} suggestions")

Sample Output (Titanic Dataset)

Running on the included titanic.csv:

$ python run_advisor.py titanic.csv --target Survived

EDA Report

============================================================
DATA SUMMARY REPORT
============================================================
Rows: 891, Columns: 12
Numeric columns
Numeric columns: 7, Categorical: 5
Duplicate rows: 0
Constant columns: []

--- Missing Data ---
  Age: 19.87% missing
  Cabin: 77.10% missing
  Embarked: 0.22% missing
Columns with >80% missing: []

--- Problem Type ---
  Type: binary_classification
  Target: Survived, unique values: 2
  Imbalance ratio: 1.61

--- Correlation ---
  No high correlations (>0.8) detected.

--- Outliers (IQR) ---
  Age: 11 outliers
  SibSp: 46 outliers
  Parch: 213 outliers
  Fare: 116 outliers
============================================================

Missing Value Strategies

{'Age': 'knn', 'Cabin': 'missing_category', 'Embarked': 'mode'}

Outlier Strategies

{'PassengerId': 'keep', 'Survived': 'keep', 'Pclass': 'keep', 'Age': 'cap', 'SibSp': 'remove', 'Parch': 'remove', 'Fare': 'remove'}

Feature Engineering Suggestions

  polynomial: 5 suggestions
  ratio: 1 suggestions
  binning: 4 suggestions
  categorical_encodings: 5 suggestions
  rare_grouping: 3 suggestions

Top Algorithm Recommendations

Algorithm	Score	Primary Rationale
Random Forest	70/100	Categorical features – tree models handle them well
Logistic Regression	68/100	Low dimensionality – linear models work well
SVM	68/100	SVM works well for moderate-sized datasets
Decision Tree	65/100	Categorical features – tree models handle them well
KNN	63/100	Small to medium dataset – KNN is fast and effective

🧠 How AlgorithmRanker Works (The "Senior Data Scientist" Logic)

Why did we build AlgorithmRanker? Because there is no "one perfect algorithm for everything."

Imagine handing a Senior Data Scientist a new CSV file. They will immediately ask:

1. "How many rows is it?"
2. "How many columns?"
3. "Are there a lot of missing values or categorical text columns?"

Based only on the answers to those questions (which we call meta-features), a Senior Data Scientist instantly knows which algorithms will work and which will fail. For example, if the data has 5 million rows, they know KNN will crash the computer's memory. If it has only 200 rows, they know XGBoost might overfit, so they prefer a simpler model.

The AlgorithmRanker automates this intuition. When you upload a dataset:

1. It gives every ML algorithm a baseline score of 50.
2. It analyzes the meta-features of your CSV.
3. It adds points (+) if the dataset's characteristics match what the algorithm is famously good at.
4. It subtracts points (-) if the dataset has characteristics that would cause the algorithm to fail, overfit, or crash.

At the end, it sorts the scores and gives you the top recommendations, saving you hours of trial and error!

Example: Scoring Tabular Foundation Models (TabPFN) TabPFN is evaluated under this exact same logic:

• The Strengths: TabPFN was trained on a supercomputer to instantly solve small datasets (under 1,000 rows) zero-shot. If you upload a small CSV, the ranker adds +40 points, bumping it to #1.
• The Weaknesses: TabPFN's transformer architecture breaks if you give it more than ~10,000 rows. If you upload a massive CSV, the ranker subtracts -50 points, dropping it to the bottom of the list so you don't run out of memory.

Deep Imputation Output (--auto-impute)

Running python run_advisor.py titanic.csv --target Survived --auto-impute uses TabPFN and Random Forest to intelligently predict and fill missing values:

=== RUNNING DEEP IMPUTATION ===
Executing intelligent deep imputation strategies on missing values...
INFO     | preprocessor.preprocessing.missing:impute - Using Deep Imputation (ML/TabPFN) on 'Age'...
INFO     | preprocessor.preprocessing.missing:impute - Using Deep Imputation (ML/TabPFN) on 'Cabin'...
Success! Saved fully imputed dataset to: titanic_imputed.csv

--- Missing Values After Imputation ---
Age      0
Cabin    0

Sample of Imputed Ages (Original vs Predicted):
Row 5: Original Age = nan, Predicted Age = 33.9
Row 17: Original Age = nan, Predicted Age = 39.7
Row 19: Original Age = nan, Predicted Age = 22.9
Row 26: Original Age = nan, Predicted Age = 37.1
Row 28: Original Age = nan, Predicted Age = 22.4

Auto-Benchmarker Output (--benchmark)

Running python run_advisor.py titanic_imputed.csv --target Survived --benchmark builds a full pipeline and tests it against TabPFN head-to-head:

[Benchmarker] Starting head-to-head comparison...
-> Splitting dataset: 80% train (712 rows), 20% test (179 rows)

-> Training Classical Model (Random Forest) with preprocessing pipeline...
      - Numeric columns (6): ['PassengerId', 'Pclass', 'Age', 'SibSp', 'Parch', 'Fare']
        -> Applied: SimpleImputer(median) + StandardScaler
      - Categorical columns (5): ['Name', 'Sex', 'Ticket', 'Cabin', 'Embarked']
        -> Applied: SimpleImputer('missing') + OrdinalEncoder

-> Training Foundation Model (TabPFN)...
      - Bypassing heavy preprocessing pipelines (no scaling or imputation).
      - Applying lightweight OrdinalEncoder to string columns (TabPFN requirement).

========================================
🏆 BENCHMARK LEADERBOARD 🏆
========================================
1. TabPFN               | Acc: 82.68% | Time: 21.71s
2. Random Forest        | Acc: 81.56% | Time: 0.25s
========================================

💡 Why Benchmarking Matters (The Importance of Zero-Shot)

The benchmarker exists to answer a critical modern question: "Is a Tabular Foundation Model actually better than a traditional algorithm on my specific dataset?"

Foundation Models like TabPFN are incredible because they require zero setup. They bypass scaling, imputation, and feature engineering, giving you a massive head start. However, this benchmarker proves that a properly preprocessed classical algorithm (like a Random Forest with a full Scikit-Learn pipeline) can still pull ahead on certain datasets!

By running --benchmark, you get undeniable proof of whether the cutting-edge zero-shot model or the classical ML pipeline is the best fit for your exact CSV.

⚙️ How the Benchmarker Works Under the Hood

When you run --benchmark, the code isn't just guessing these numbers—it is doing the hard work of training both models from scratch in real-time!

Powered by the preprocessor/pipeline/benchmarker.py file, it prints exactly what it is doing at every step:

1. The Train/Test Split: It takes your CSV and splits it so a portion of the rows are used for training and the rest are hidden away for testing (default is 80/20, but you can change this with --test-size).
2. The Classical Pipeline:
   • It automatically builds a full Scikit-Learn Pipeline object.
   • For numeric columns, it applies a SimpleImputer(median) and a StandardScaler.
   • For categorical text columns, it applies a SimpleImputer('missing') and an OrdinalEncoder so the math won't crash.
   • It trains the top recommended classical model (e.g., Random Forest) on the training split and evaluates on the hidden test set.
3. The Foundation Model (TabPFN):
   • It takes the exact same training data, but it bypasses the entire preprocessing pipeline. No standard scaling, no heavy imputation. It just feeds the raw data directly into the pre-trained transformer and evaluates on the hidden test set.

Because you have this Benchmarker, your tool is no longer just giving "advice" on a whiteboard—it's actually proving its advice mathematically on your exact dataset!

Module Overview

Module	Purpose
preprocessor.eda.DataLoader	Load CSV, JSON, Parquet, Excel, SQL
preprocessor.eda.DataStats	Comprehensive statistics & data quality report
preprocessor.preprocessing.MissingHandler	Missing value strategies & imputation
preprocessor.preprocessing.OutlierHandler	IQR/Z-score/IsolationForest detection & handling
preprocessor.preprocessing.ScalingHandler	Algorithm-aware scaling (Standard/MinMax/Robust/Power)
preprocessor.preprocessing.EncodingHandler	Label/OneHot/Target/Frequency/Binary encoding
preprocessor.preprocessing.TransformHandler	Log/sqrt/Box-Cox/Yeo-Johnson + duplicate removal
preprocessor.algorithms.MetaFeatureExtractor	Dataset meta-features for algorithm selection
preprocessor.algorithms.AlgorithmRanker	Rank 10+ algorithms with rationale
preprocessor.feature_engineering.FeatureSuggestor	Polynomial, ratio, binning, aggregation, encoding suggestions
preprocessor.feature_engineering.FeatureSelector	Variance, correlation, MI, permutation, RFE selection

Supported Algorithms

Classification: Logistic Regression, Decision Tree, Random Forest, XGBoost, LightGBM, SVM, KNN, Naive Bayes, Neural Network

Regression: Linear/Ridge/Lasso, Decision Tree, Random Forest, XGBoost, LightGBM, SVM, KNN, Neural Network

Clustering: K-Means, DBSCAN, Hierarchical, Gaussian Mixture

Project Structure

preprocessing_advisor/
├── preprocessor/
│   ├── core/              # Logging utilities
│   ├── eda/               # Data loading & statistics
│   ├── preprocessing/     # Missing, outliers, scaling, encoding, transforms
│   ├── algorithms/        # Meta-features & algorithm ranking
│   ├── feature_engineering/ # Suggestions, selection, interactions
│   ├── pipeline/          # Pipeline builder (planned)
│   └── explainability/    # SHAP/LIME (planned)
├── examples/
│   └── demo.ipynb
├── tests/
├── test_all.py            # Full pipeline integration test
├── setup.py
├── requirements.txt
└── README.md

Running Tests

cd preprocessing_advisor
python test_all.py

Expected output: All tests completed successfully!