Spark + Iceberg batch processing job

[kiyeonjeon21]

Priority: P1

The batch-processing.env recipe starts Spark, Iceberg, Trino, and Jupyter but has no actual Spark job.

What to do

• Create stacks/processing/spark/jobs/etl_to_iceberg.py — PySpark reads from PostgreSQL, writes Iceberg tables
• Configure Spark to use Iceberg REST catalog
• Create Jupyter notebook 04-spark-iceberg.ipynb demonstrating the workflow

Acceptance criteria

• spark-submit runs the ETL job without errors
• At least 2 Iceberg tables created in MinIO
• Trino can query the Spark-created tables
• Jupyter notebook demonstrates full workflow