Summary
fabric__get_incremental_microbatch_sql always performs a delete-then-insert sequence, even when unique_key is configured on the model. Fabric DW supports native MERGE, which is atomic and avoids the intermediate "empty rows" window that delete-then-insert exposes to concurrent readers.
Evidence (HEAD 0de2190, v1.10.0)
dbt/include/fabric/macros/materializations/models/incremental/merge.sql: the fabric__get_incremental_microbatch_sql macro emits a delete-by-microbatch-predicate followed by an insert, with no branch on unique_key.
User impact
- For models with
unique_key configured, what should be one atomic MERGE becomes two statements.
- Concurrent readers see a window where the deleted rows are gone but the inserts have not yet landed. On a typical Fabric DW with snapshot isolation this is bounded but observable.
- More warehouse work per run than necessary.
Suggested fix
When unique_key is present, dispatch to get_incremental_merge_sql so the operation runs as a single MERGE:
{% macro fabric__get_incremental_microbatch_sql(arg_dict) %}
{% if arg_dict["unique_key"] %}
{% do return(adapter.dispatch('get_incremental_merge_sql', 'dbt')(arg_dict)) %}
{% endif %}
...
{% endmacro %}
A PR with the change is linked from this issue.
Summary
fabric__get_incremental_microbatch_sqlalways performs a delete-then-insert sequence, even whenunique_keyis configured on the model. Fabric DW supports nativeMERGE, which is atomic and avoids the intermediate "empty rows" window that delete-then-insert exposes to concurrent readers.Evidence (HEAD
0de2190, v1.10.0)dbt/include/fabric/macros/materializations/models/incremental/merge.sql: thefabric__get_incremental_microbatch_sqlmacro emits a delete-by-microbatch-predicate followed by an insert, with no branch onunique_key.User impact
unique_keyconfigured, what should be one atomic MERGE becomes two statements.Suggested fix
When
unique_keyis present, dispatch toget_incremental_merge_sqlso the operation runs as a single MERGE:A PR with the change is linked from this issue.