Netrunner Template: Straight from Cyberpunk

Welcome to netrunner-template, a structured, modular pipeline designed to help you efficiently load, clean, inspect, and analyze datasets—especially tabular ones like CSV files—without diving deep into repetitive data-wrangling boilerplate.

Whether you're working on exploratory data analysis, preparing data for modeling, or automating data pipelines, this template provides a clean and configurable foundation.

Quickstart Guide

1. Clone the project and install dependencies

git clone https://github.com/AestheticVoyager/netrunner-template.git
cd netrunner-template
pip install -r requirements.txt

2. Add your dataset

Place your dataset in ./data/raw/. For example:

./data/raw/house_prices.csv

3. Run the example pipeline

python house_price_example.py

You’ll see logs about loading, inspecting, cleaning, and auditing the dataset.

---

What This Project Does

• Load data from CSV (local or remote)

• Inspect structure, memory usage, and missingness

• Clean:

• Handle missing values

• Remove duplicates

• Convert types (numeric/dates)

• Detect (and optionally remove) outliers

• Audit and log all steps

• Fully configurable via Python dataclasses

---

Project Structure

netrunner-template/
│
├── data/
│   └── raw/                  # Place your dataset files here
│
├── pipeline/
│   ├── __init__.py
│   ├── data_loader.py        # Handles dataset loading
│   ├── data_cleaning.py      # Full-featured cleaning pipeline
│   ├── utils.py              # Shared utilities and logger
│
├── house_price_example.py    # Included example you can run
├── README.md
└── requirements.txt

---

Features and Configuration

Cleaning Configuration

All cleaning behavior is controlled via the CleanConfig dataclass:

from data_cleaning import CleanConfig

cfg = CleanConfig(
    missing_threshold=0.5,
    missing_strategy="auto",  # auto | drop_row | drop_column | impute
    impute_num_strategy="median",  # mean | median | mode | constant
    impute_cat_strategy="mode",    # mode | constant
    parse_dates=True,
    outlier_method="iqr",     # iqr | zscore
    remove_outliers=True,
    verbose=True
)

You can use this config to enable/disable logs, switch cleaning strategies, and control outlier thresholds.

---

Example Use Case

Here’s a minimal version of house_price_example.py to show how to use the pipeline:

from pipeline.data_loader import load_csv
from pipeline.data_cleaning import clean_pipeline, CleanConfig

print("[INFO] === House Price Analysis ===")

# Step 1: Load raw data
df = load_csv("./data/raw/house_prices.csv")

# Step 2: Set up config
cfg = CleanConfig(remove_outliers=True, verbose=True)

# Step 3: Clean it
df_cleaned, audit = clean_pipeline(df, cfg)

# Step 4: Done!
print("[SUCCESS] Cleaning finished.")
print("Initial shape:", audit['initial_shape'])
print("Final shape:  ", audit['final_shape'])
print("Outliers removed:", audit['outliers_detected'])

---

Extending the Pipeline

Want to customize it?

• Add new preprocessing steps to data_cleaning.py
• Add ML model training in a new pipeline/modeling.py file
• Chain multiple CSVs together in data_loader.py
• Add visualization tools (e.g., matplotlib, seaborn)

You can structure additional stages into the pipeline/ directory to keep things modular.

---

Testing & Debugging

• Enable verbose=True in CleanConfig to get detailed logs
• Logs will show dataset shape, column types, memory usage, and cleaning actions
• Use inspect_dataframe(df) to view structure at any point in your scripts

---