fix: ref() and source() in Python models

[akmalsoliev]

fix: ref() and source() in Python models to use dbt-core's resolved functions

The previous implementation constructed table references manually. When calling source("my_source_aliase", "my_table") it would literally pass those string into spark.table resulting in a call spark.table("my_source_aliase"."my_table").

This fix will resolve this issue with reutilization of

def ref(*args, **kwargs):
    refs = {"my_ref_table": "my_schema.my_ref_table"}
    key = '.'.join(args)
    version = kwargs.get("v") or kwargs.get("version")
    if version:
        key += f".v{version}"
    dbt_load_df_function = kwargs.get("dbt_load_df_function")
    return dbt_load_df_function(refs[key])

def source(*args, dbt_load_df_function):
    sources = {"my_source_aliase.my_source_table": "my_source_schema.my_source_table"}
    key = '.'.join(args)
    return dbt_load_df_function(sources[key])

NOTE: that instead of having my_source_aliase it points to my_source_schema, which would solve the issue of an incorrect pointer with spark.table, outputting spark.table("my_source_schema"."my_source_table")

This took me ages to figure out, was so confused on why there was source declaration.

resolves #

Description

Checklist

• I have signed the CLA
• I have run this code in development and it appears to resolve the stated issue
• This PR includes tests, or tests are not required/relevant for this PR
• I have updated the CHANGELOG.md and added information about my change to the "dbt-glue next" section.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[akmalsoliev]

I believe this also fixes the issue: #635

[yotahk]

Thanks for the fix. I believe delegating to dbt-core's resolved ref/source functions (as defined in py_script_postfix) is the right approach.

One small ask: could you add an entry to CHANGELOG.md for this change & complete the PR checklist?

[akmalsoliev]

Thanks for the fix. I believe delegating to dbt-core's resolved ref/source functions (as defined in py_script_postfix) is the right approach.

One small ask: could you add an entry to CHANGELOG.md for this change & complete the PR checklist?

Hey @yotahk,

You'll find the updates, thanks to Claude was able to understand how in the god's name to create test.

[yotahk]

Merged, thank you for the contribution!