Skip to content

Enable client_session_keep_alive on Snowflake hook to survive long tasks#324

Merged
JuFrei merged 2 commits into
masterfrom
feat/snowflake-keep-alive
May 29, 2026
Merged

Enable client_session_keep_alive on Snowflake hook to survive long tasks#324
JuFrei merged 2 commits into
masterfrom
feat/snowflake-keep-alive

Conversation

@JuFrei

@JuFrei JuFrei commented May 28, 2026

Copy link
Copy Markdown
Contributor

Summary

Long-running ELT tasks using key-pair (JWT) auth on the EWAH Snowflake hook fail mid-task once the task runs longer than ~60 minutes — Snowflake caps JWT validity at 60 min and the connector does not refresh the JWT on its own, so the connection opened at task start dies before the task's final write.

Real-world trigger: Amazon Brand Analytics Search Query Performance loads where Amazon's report-ready polling stretches a task to 1–4 hours, and the closing `COPY INTO` then hits `JWT token is invalid` / `Authentication token has expired`.

Changes

  • `ewah/hooks/snowflake.py`: pass `client_session_keep_alive=True` to `snowflake.connector.connect`. The connector then runs a background heartbeat (~every 15 min, connector-clamped minimum) that refreshes the JWT and resets the session's idle timer for the lifetime of the connection.
  • No effect on short-lived tasks or password-auth setups.

Long-running ELT tasks (e.g. Amazon Brand Analytics Search Query
Performance loads that take 1-4 hours) fail when using key-pair (JWT)
auth because the JWT signed at connect-time expires 60 minutes later.

Setting client_session_keep_alive=True makes the snowflake-connector
send periodic /session/heartbeat requests that refresh the JWT in the
background, keeping the session valid for the lifetime of the task.

Verified end-to-end: heartbeat fires after ~15 min (connector-enforced
minimum interval); 4-hour Snowflake-writing tasks now complete without
the previous "JWT token is invalid" / "Authentication token expired"
failures.
@JuFrei JuFrei requested a review from lpillmann May 28, 2026 10:08
Comment thread ewah/hooks/snowflake.py Outdated
@gemma-claude-assistant

Copy link
Copy Markdown
Contributor

The fix is correct and safe — client_session_keep_alive=True is the right lever for surviving long-running tasks with key-pair auth, and the placement inside the connection guard is correct. One style issue with the comment block.


Style

  • The 7-line comment block violates CLAUDE.md's explicit "one short line max" rule for code comments. Condense to one line (see inline suggestion). Two factual inaccuracies in the comment are also noted inline: client_session_keep_alive keeps the server-side session alive — it doesn't refresh the JWT itself — and the heartbeat default is 3600 s (60 min), not 15 min (15 min is the minimum floor for the frequency setting).

You can request a new review by commenting @claude review.

@gemma-claude-assistant

Copy link
Copy Markdown
Contributor

Claude review cost: $0.2650 · 8 turns · 168s · eu.anthropic.claude-sonnet-4-6

Co-authored-by: gemma-claude-assistant[bot] <262789552+gemma-claude-assistant[bot]@users.noreply.github.com>
@gemma-claude-assistant

Copy link
Copy Markdown
Contributor

Looks good — this is a small, targeted, correct fix for the JWT session-expiry problem on long-running tasks.


You can request a new review by commenting @claude review.

@gemma-claude-assistant

Copy link
Copy Markdown
Contributor

Claude review cost: $0.2433 · 7 turns · 151s · eu.anthropic.claude-sonnet-4-6

@JuFrei JuFrei requested a review from soltanianalytics May 29, 2026 12:45
@JuFrei JuFrei merged commit 8652e3a into master May 29, 2026
1 check passed
@JuFrei JuFrei mentioned this pull request May 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants