Add Federate and Govern Iceberg Tables Using Snowpark Connect quickstart#3410
Conversation
sfc-gh-ncherukuri
temporarily deployed
to
staging
GitHub Actions
Inactive
👋 Helpful Information
|
🔗 Staging Preview ReadyQuickstart:
Generated by GitHub Actions |
…x location .... Generated with [Cortex Code](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code) Co-Authored-By: Cortex Code <noreply@snowflake.com>
.... Generated with [Cortex Code](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code) Co-Authored-By: Cortex Code <noreply@snowflake.com>
…rg tables Two separate tables with headers, Policy column added, key insight callout explaining independence of Snowflake and Databricks UC governance. .... Generated with [Cortex Code](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code) Co-Authored-By: Cortex Code <noreply@snowflake.com>
sfc-gh-ncherukuri
force-pushed
the
add-iceberg-scos-assets
branch
from
7eba864 to
13aa488
Compare
sfc-gh-ncherukuri
temporarily deployed
to
staging
GitHub Actions
Inactive
👋 Helpful Information
|
🔗 Staging Preview ReadyQuickstart:
Generated by GitHub Actions |
sfc-gh-ridasafdar
temporarily deployed
to
staging
GitHub Actions
Inactive
👋 Helpful Information
|
🔗 Staging Preview ReadyQuickstart:
Generated by GitHub Actions |
2 participants
Summary
Adds new quickstart: Federate and Govern Iceberg Tables Using Snowpark Connect for Apache Spark
By the end of this quickstart, a business user opens Snowsight → Cortex Analyst, asks "Which orders are HIGH risk?" and gets a governed, AI-enriched answer drawn from Iceberg tables across two separate catalogs — with Horizon masking and row access policies enforced end-to-end, including on AI-generated columns.
What You Build
Step 1 (
01,02) — Snowflake-managed Iceberg tables with Horizon column masking + row access policies; Snowpark Connect governance demoStep 2 (
04,05) — Databricks UC Iceberg tables federated via Catalog-Linked Database; Snowpark Connect queries with live role-based masking; federation setup fully idempotent (tested 2 consecutive runs)Step 3 (
06,07) — Cortex AI enrichment pipeline: Snowpark Connect reads both catalogs, Cortex classifies risk + generates ops notes, writes to SF-managed Iceberg; Cortex Analyst semantic view for natural language querying with governance enforced per roleGovernance contrast (
03) — Snowpark Connect (governed) vs external Spark via Horizon IRC (raw Parquet, policies bypassed)Assets (7 files, sequential 01–07)
01_sf_iceberg_catalog_setup.sql02_sf_iceberg_demo.ipynb03_databricks_rw_sf_iceberg.py04_databricks_create_uc_tables.py05_databricks_federation_demo.ipynb06_cortex_ai_pipeline.ipynb07_ai_pipeline.ipynbTest plan
05tested 2 consecutive full runs — zero errors06tested 2 consecutive full runs — zero errors.... Generated with Cortex Code