DABs Demo

Databricks Asset Bundle demo with an ETL pipeline, an ML model, an orchestrating Job, and a Streamlit app.

Layout

dabs-demo/
├── databricks.yml            # bundle config + dev/prod targets
├── resources/
│   ├── job.yml               # ETL + ML job
│   └── app.yml               # Streamlit app
└── src/
    ├── notebooks/
    │   ├── 01_etl_nyc_taxi.py        # NYC taxi -> Delta
    │   └── 02_ml_fare_predictor.py   # GBM fare model
    └── app/
        ├── app.py            # Streamlit UI
        ├── app.yaml          # app runtime config
        └── requirements.txt

Targets

Target	Mode	Catalog	Schema
dev (default)	development	timstanton_stable	dabs_demo_dev
prod	production	energy	dabs_demo_prod

Dev uses the DEFAULT CLI profile; prod uses azure-field-eng.

Common commands

# Validate
databricks bundle validate
databricks bundle validate -t prod

# Deploy
databricks bundle deploy           # dev (default)
databricks bundle deploy -t prod

# Run the pipeline
databricks bundle run nyc_taxi_pipeline

# Start the app (apps require an explicit run after deploy)
databricks bundle run taxi_dashboard
databricks apps logs dabs-demo-taxi-dev