fix(observability): make the engine's log routing correct and race-free

[aparajon]

Why this matters

The apply log stream is the account of a schema change an operator actually reads — from the CLI, and from the summary comment a failed apply folds onto its PR. The engine's own lines are most of that account: copy progress, checksum results, the fatal line that explains a failure.

Two things stood between those lines and the stream.

The process log level decided for both consumers. The filter that routes engine lines serves the apply log stream and the process logs, but answered Enabled for only the second — and slog skips building the record entirely when a handler reports false. So the routing in Handle was never reached on a deployment that runs its own logs above info, and every apply recorded nothing from the engine. A setting about stdout verbosity quietly emptied the surface operators triage from.

                       before                          after

Info line   ──> Enabled? handler only ──> dropped   ──> Enabled? either consumer
                                                          ├──> apply log stream
                                                          └──> process logs, only
                                                               at their own level

The callback itself was read unsynchronized. The filter reached the engine's log callback and debug toggle through raw pointers while engine methods wrote them under the engine mutex. A drive that unwires its apply-log callback as it hands the engine over raced with a runner still logging, and the callback's check-then-call window could dereference a cleared slot.

What it does

Each consumer decides for itself. A record is built when either wants it; it is routed to the apply log stream whenever an apply is being driven, and passed to the process logs only at the level that deployment configured. That second half is load-bearing — widening Enabled alone would fix the silence by printing every driven apply's info lines to a deployment's stdout, trading lost signal for unasked-for volume. An apply cannot raise a deployment's log volume, and a quiet deployment cannot silence an apply.

The callback and debug toggle move into atomic slots, and the filter loads the callback once per record and calls through that copy. Atomics rather than the engine mutex: the filter runs on the Spirit runner's logging path, where taking the engine lock would invite reentrancy against the engine methods that already hold it.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR fixes an observability gap in the Spirit engine slog filter so engine INFO+ lines are always recorded into the apply log stream even when the deployment’s process log level would otherwise filter them out, while still preserving the deployment’s configured verbosity for stdout/stderr logs.

Changes:

• Adjust spiritLogFilter.Enabled to return true when either the apply log stream or the process logger would consume the record.
• Prevent “apply-log-only” records from being emitted to the process logs when the handler’s level would reject them.
• Expand unit tests to distinguish apply-log routing vs process-log emission.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
pkg/engine/spirit/logger.go	Updates slog handler filtering so apply-log routing is independent of process log level and avoids increasing deployment log volume.
pkg/engine/spirit/logger_test.go	Adds tests covering routing below process log level and the “no callback installed” case; updates logger setup helper.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.