Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ jobs:
git config --global --add safe.directory "$GITHUB_WORKSPACE"

- name: "🔒 Auth: Azure login with OIDC"
uses: azure/login@7ddb5af1ef8758cf1353cf3b42f940aee27ba21c # v3.0.2
uses: azure/login@a641126d1b8aa4d1fa005f4f92df94a3a4c4c906 # v3.1.0
with:
client-id: ${{ secrets.APP_ID }}
tenant-id: ${{ secrets.TENANT_ID }}
Expand Down
24 changes: 1 addition & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -624,7 +624,7 @@ duplicate effects.
| `workspace_name` | string | — | Optional default workspace for cross-workspace 4-part naming. When set and the lakehouse has schemas enabled, all relations without a model-level `workspace_name` will be rendered with this workspace prefix. Ignored for non-schema lakehouses. Exposed as `target.workspace_name` in Jinja. |
| `quote_identifiers` | bool | `false` | When `true`, backtick-quotes table identifiers so Fabric Spark preserves their casing instead of folding to lowercase. Requires `spark_config.conf` `{ "spark.sql.caseSensitive": "true" }` to take effect (the adapter warns if it's missing). Session-wide — also affects column resolution. See [Case-sensitive identifiers](#case-sensitive-identifiers). |
| `auto_optimize` | bool | `true` | Run `OPTIMIZE` on Delta relations after every `table`, `incremental` and `snapshot` build. Override per model with `config(auto_optimize=false)`, or disable everywhere with the `DBT_FABRICSPARK_SKIP_OPTIMIZE` environment variable. See [Automatic OPTIMIZE](#automatic-optimize). |
| `enable_experimental_unstable` | bool | `false` | Opt into experimental native streaming tables. Requires an unquoted boolean; ordinary profiles need no changes. See [Experimental native streaming](#experimental-native-streaming). |
| `enable_experimental_unstable` | bool | `false` | Opt into experimental features that may undergo breaking changes or get removed anytime |
| `threads` | int | `1` | Number of threads for parallel execution |
| **Authentication** | | | |
| `authentication` | string | `CLI` | Auth method: `CLI`, `SPN`, or `fabric_notebook` |
Expand Down Expand Up @@ -975,28 +975,6 @@ SELECT ...
- **No temp views as sources** — The SELECT query can reference tables and other MLVs, but not temporary views.
- **Schedule is per-lakehouse** — One active schedule per lakehouse lineage, not per MLV.

### Experimental native streaming

Requires `method: session`, Delta, and a compatible runtime implementing native
`CREATE STREAMING TABLE` / `SHOW STREAMING TABLES` with AvailableNow execution
and run-identity/status metadata. Stock Spark compatibility is not assumed.
Set `enable_experimental_unstable: true` in the active `profiles.yml` output;
the default is `false`. Unsupported profiles fail before hooks or DDL.

```sql
{{ config(materialized='streaming_table', file_format='delta') }}
select *
from stream {{ source('raw', 'events') }}
with ('maxFilesPerTrigger' = '8')
```

Source options stay in `WITH (...)`. CREATE is submitted once; post-hooks and
downstream models wait for the exact run to stop, inactive and failure-free,
on the same Spark context. Missing/replaced identities, failures, disconnects
and timeouts are errors. Polling uses `statement_timeout` (zero becomes one hour)
and `poll_statement_wait`. A fresh checkpoint reads the initial snapshot;
later runs resume it. Query changes fail by default; `--full-refresh` is rejected.

## FAQs

### No support for Python Models
Expand Down
168 changes: 84 additions & 84 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading