[BUG] The notebooks/procedures kernel dies when using log_artifact from experiment tracking

[chriss1245]

I am using experiment tracking for my experiments.

    tracker = ExperimentTracking(session=session)
    tracker.set_experiment(experiment_name=EXP_NAME, database_name=DB_NAME, schema_name=SCHEMA_NAME)

    # 5. EJECUTAR EL RUN Y LOGGEAR ABSOLUTAMENTE TODO
    print("🏃‍♂️ Iniciando el Run y enviando datos a Snowflake...")
    with tracker.start_run(run_name="prueba_full_stack_13") as run:
        
        # A) Loggear Parámetros
        tracker.log_params({"max_iter": 100, "solver": "lbfgs"})
        print("   ✅ Parámetros registrados.")
        
        # B) Loggear Métricas estáticas
        tracker.log_metrics({"auc_falso": 0.90, "precision_falsa": 0.91})
        print("   ✅ Métricas estáticas registradas.")

        # C) Loggear Métricas con Steps (Simulando un entrenamiento por épocas)
        for paso in range(10):
            tracker.log_metric(key="loss_falsa", value=(1.0 - paso*0.1), step=paso)
        print("   ✅ Métricas vectoriales (steps) registradas.")

        # D) Loggear el Artefacto (El CSV que creamos antes)
        print("REading csv")
        import pandas as pd
        df = pd.read_csv(ruta_artefacto)
        print("   ✅ Éxito al leer el artefacto")
       

        tracker.log_artifact(local_path=ruta_artefacto, artifact_path="reportes/importancia_falsa")

When it comes to log the artifact, i get the following exception:

The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "Cell [cell11]", line 72, in <module>
tracker.log_artifact(local_path=ruta_artefacto, artifact_path="reportes/importancia_falsa")
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/ml/experiment/experiment_tracking.py", line 447, in log_artifact
self._sql_client.put_artifact(
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/ml/_internal/telemetry.py", line 611, in wrap
return ctx.run(execute_func_with_statement_params)
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/ml/_internal/telemetry.py", line 576, in execute_func_with_statement_params
result = func(*args, **kwargs)
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/ml/experiment/_client/experiment_tracking_sql_client.py", line 122, in put_artifact
return self._session.file.put(
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/snowpark/file_operation.py", line 131, in put
cursor._upload(local_file_name, stage_location, options)
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/connector/cursor.py", line 1857, in _upload
file_transfer_agent.execute()
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/connector/file_transfer_agent.py", line 448, in execute
self.transfer(self._file_metadata)
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/connector/file_transfer_agent.py", line 692, in transfer
raise exception_caught_in_work
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/connector/file_transfer_agent.py", line 628, in function_and_callback_wrapper
work(*args, **kwargs),
File "_stored_proc_storage_client.py", line 173, in upload_chunk
File "_stored_proc_storage_client.py", line 143, in upload
File "_sfstream.py", line 227, in init
SystemError: <built-in function open_stream> returned a result with an exception set

Few seconds after this error, the kernel dies and I get this message:

Notebook session ended due to error: com.snowflake.common.SnowflakeSQLException: SQL execution internal error:
Processing aborted due to error 300002:2522016384; incident 5776370.

I was tracing a bit the issue. And it seems that the registry uses the snowflake session put file method with a special path:

def put_artifact(
        self,
        *,
        experiment_name: sql_identifier.SqlIdentifier,
        run_name: sql_identifier.SqlIdentifier,
        artifact_path: str,
        file_path: str,
        auto_compress: bool = False,
    ) -> file_operation.PutResult:
        return self._session.file.put(
            local_file_name=file_path,
            stage_location=self._build_snow_uri(experiment_name, run_name, artifact_path),
            overwrite=True,
            auto_compress=auto_compress,
        )[0]

I tried to run this by hand

stage_path = f"snow://experiment/{EXP_NAME}/versions/FOOLISH_JELLYFISH_1"
session.file.put(str(ruta_artefacto), stage_path, auto_compress=False, overwrite=True)

And got the error SystemError: <built-in function open_stream> returned a result with an exception set

File "Cell [cell21]", line 2, in <module>
    session.file.put(str(ruta_artefacto), stage_path, auto_compress=False, overwrite=True)
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/snowpark/file_operation.py", line 131, in put
    cursor._upload(local_file_name, stage_location, options)
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/connector/cursor.py", line 1857, in _upload
    file_transfer_agent.execute()
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/connector/file_transfer_agent.py", line 448, in execute
    self.transfer(self._file_metadata)
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/connector/file_transfer_agent.py", line 692, in transfer
    raise exception_caught_in_work
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/connector/file_transfer_agent.py", line 628, in function_and_callback_wrapper
    work(*args, **kwargs),
File "_stored_proc_storage_client.py", line 173, in upload_chunk
File "_stored_proc_storage_client.py", line 143, in upload
File "_sfstream.py", line 227, in __init__

I also tried to run it using the SQL API

session.sql(f"PUT file://{ruta_artefacto} {stage_path} AUTO_COMPRESS=FALSE OVERWRITE=TRUE").collect()

And got this error SnowparkSQLException: (1304): 01c2e94c-0208-8dd3-0215-66019a7b3277: 090236 (42601): Stored procedure execution error: Unsupported statement type 'PUT_FILES'.

File "Cell [cell22]", line 1, in <module>
    session.sql(f"PUT file://{ruta_artefacto} {stage_path} AUTO_COMPRESS=FALSE OVERWRITE=TRUE").collect()
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/snowpark/_internal/telemetry.py", line 295, in wrap
    result = func(*args, **kwargs)
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/snowpark/_internal/utils.py", line 1148, in call_wrapper
    return func(*args, **kwargs)
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/snowpark/dataframe.py", line 774, in collect
    return self._internal_collect_with_tag_no_telemetry(
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/snowpark/dataframe.py", line 847, in _internal_collect_with_tag_no_telemetry
    return self._session._conn.execute(
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/snowpark/_internal/server_connection.py", line 648, in execute
    result_set, result_meta = self.get_result_set(
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/snowpark/_internal/analyzer/snowflake_plan.py", line 426, in wrap
    raise ne.with_traceback(tb) from None
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/snowpark/_internal/analyzer/snowflake_plan.py", line 179, in wrap
    return func(*args, **kwargs)
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/snowpark/_internal/server_connection.py", line 770, in get_result_set
    result = self.run_query(
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/snowpark/_internal/server_connection.py", line 137, in wrap
    raise ex
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/snowpark/_internal/server_connection.py", line 131, in wrap
    return func(*args, **kwargs)
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/snowpark/_internal/server_connection.py", line 544, in run_query
    raise ex
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/snowpark/_internal/server_connection.py", line 529, in run_query
    results_cursor = self.execute_and_notify_query_listener(
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/snowpark/_internal/server_connection.py", line 459, in execute_and_notify_query_listener
    raise ex
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/snowpark/_internal/server_connection.py", line 450, in execute_and_notify_query_listener
    results_cursor = self._cursor.execute(query, **kwargs)
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/connector/cursor.py", line 1142, in execute
    Error.errorhandler_wrapper(self.connection, self, error_class, errvalue)
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/connector/errors.py", line 286, in errorhandler_wrapper
    handed_over = Error.hand_to_other_handler(
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/connector/errors.py", line 341, in hand_to_other_handler
    cursor.errorhandler(connection, cursor, error_class, error_value)
File "/usr/lib/python_udf/6b365976d6a0e3e7a1cccaa98eccadd724df624f04ef9ad079cc8f719ae935b5/lib/python3.10/site-packages/snowflake/connector/errors.py", line 217, in default_errorhandler
    raise error_class(

NOTE: For some reason, this works when i run this experiment in my local machine. I do not have any issue when running from local. But when i run it inside the snowflake notebook we have this failure.

NOTE: Additionally, we have the same issue in the procedures.

My packages in the warehouse and in local are:

snowflake-connector-python==3.18.0
snowflake-ml-python==1.29.0
snowflake-snowpark-python==1.44.0
snowflake-telemetry-python==0.7.1
snowflake.core==1.11.0

[sfc-gh-yuzhao]

Closing this issue as the fix is in Snowflake 10.8.0. You can check your Snowflake version with CALL CURRENT_VERSION();

[chriss1245]

wow thanks a lot. it works wonders