Skip to content

Quiet variable lookup logging and run pool reconcile daily - #1

Open
timkpaine wants to merge 1 commit into
mainfrom
tkp/hf
Open

Quiet variable lookup logging and run pool reconcile daily#1
timkpaine wants to merge 1 commit into
mainfrom
tkp/hf

Conversation

@timkpaine

Copy link
Copy Markdown
Member

Two unrelated papercuts found while bringing an MWAA environment up on Airflow 3.

Offline DAG generation has no usable metastore, so resolving a Variable password always fails and falls back to UNRESOLVED_PASSWORD. That is expected and already handled, but Airflow logs a full traceback at ERROR for every secrets backend it fails to read before falling through, so every generate run emits a wall of noise. Host.hook now raises the airflow.models.variable logger to CRITICAL for the duration of the lookup it already tolerates.

The pool manager DAG defaulted to */5 * * * *. Centrally configured pools change about as often as the config is deployed, so reconciling every five minutes is mostly wasted scheduler work. Default is now @daily; anything that already pins dag.schedule is unaffected, since the validators only backfill fields that were not explicitly set.

Signed-off-by: Tim Paine <3105306+timkpaine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant