fix(log): route production logs directly to the systemd journal#199
Merged
Conversation
env_logger emits PRIORITY via an in-band "<N>" text prefix at the start of
each line. journald's SyslogLevelPrefix parser only accepts "<N>" at byte
0, so any perturbation of the byte stream before journald sees it defeats
the parser. journald then falls back to the stream's default priority
(ERROR on our unit config). This trips base_test.sh's whitelist check in
omnect-os/ci/tests (journalctl -b -p err) with unrelated debug messages.
Replace env_logger with systemd-journal-logger for production builds.
PRIORITY becomes a structured journal field, not a text prefix, so the
class of failure becomes impossible. Keep env_logger under the mock
feature so cargo test / cargo run --features mock output remains visible
in the test harness.
MESSAGE loses its "{target}: " prefix; the target lives in the structured
TARGET= field. Whitelist entries in omnect-os are updated in the
companion PR.
Move logger setup from main.rs to a dedicated logging module.
Signed-off-by: Jan Zachmann <50990105+JanZachmann@users.noreply.github.com>
Signed-off-by: Jan Zachmann <50990105+JanZachmann@users.noreply.github.com>
Logger installation can fail on systems without a journal socket or on double-init. Propagate the error through main() so the binary exits cleanly with a stderr message that still carries a syslog priority prefix. Signed-off-by: Jan Zachmann <50990105+JanZachmann@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Routes production logging directly to journald using systemd-journal-logger, while keeping an env_logger-based stdout logger for the mock feature, and centralizes logger setup into a dedicated module.
Changes:
- Move logger initialization out of
main.rsintosrc/logging.rs. - Use journald-native logging for non-
mockbuilds; keepenv_loggerformock. - Update dependencies/features and bump crate version to
0.42.2.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
src/main.rs |
Switches startup to call logging::init() and handles init failure via stderr. |
src/logging.rs |
Introduces feature-gated logger initialization for mock (stdout) vs production (journald). |
src/lib.rs |
Exposes the new logging module. |
Cargo.toml |
Adds env_filter + systemd-journal-logger, makes env_logger optional, wires mock feature. |
Cargo.lock |
Locks new dependency additions and version bump. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
mlilien
reviewed
Apr 15, 2026
… features - Non-mock logger init now reads RUST_LOG, falling back to default_filter() only when the env var is absent or empty; matches the mock path's behavior. - systemd-journal-logger is now an optional dependency, pulled in only by bootloader_grub / bootloader_uboot. Mock builds no longer compile it. Signed-off-by: Jan Zachmann <50990105+JanZachmann@users.noreply.github.com>
mlilien
reviewed
Apr 16, 2026
When using the systemd-journal-logger, debug and trace records now
include the log target in the MESSAGE body ("{target}: {args}"), matching
the old env_logger format for these levels. Info/warn/error records keep
the bare "{args}" shape, consistent with the platform convention and the
existing syslog whitelist patterns.
This gives operators module-path context in interactive debug sessions
(journalctl -p debug) without requiring -o verbose or jq, while keeping
production-level messages clean.
The mock/env_logger path is unchanged — it always includes the target at
all levels.
Signed-off-by: Jan Zachmann <50990105+JanZachmann@users.noreply.github.com>
mlilien
approved these changes
Apr 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
env_loggerwithsystemd-journal-loggerfor production builds; keepenv_loggerfor themockfeature only.MESSAGEbodies lose the inline"{target}: "prefix; the target now lives in the structuredTARGET=journal field.main.rsintosrc/logging.rs.Reason
base_test.shinomnect-os/ci/tests/lib_test.shrunsjournalctl -b -q -p errand checks every PRIORITY≤3 line against a per-platform whitelist. It has been tripping on ods debug messages like:Mechanism
env_loggerencodes severity as an in-band"<N>"text prefix at the start of each line. journald'sSyslogLevelPrefixparser only accepts"<N>"at byte 0 — if anything precedes it, parsing fails silently and the line falls back to the stream's default priority, which on our unit config is ERROR. So adebug!()call lands at PRIORITY=3 and gets flagged.We didn't identify the exact byte-perturbation source. The class of failure is inherent to in-band textual prefixes: it cannot be ruled out as long as severity travels through the MESSAGE byte stream.
Why
systemd-journal-loggersystemd-journal-loggersends records via the native sd-journal socket protocol.PRIORITYbecomes a structured field alongsideMESSAGE,SYSLOG_IDENTIFIER, andTARGET. It is not text, not parsed from a stream, and cannot be corrupted by concurrent writes.Advantages
journalctl -p erronly returns actual errors.TARGET=,CODE_FILE=,CODE_LINE=,CODE_MODULE=become queryable (journalctl TARGET=omnect_device_service::twin::network).RUST_LOGand the samewarn,azure_iot_sdk=debug,...defaults are preserved viaenv_filter(env_logger's filter half, published as a standalone crate).cargo test --features mockandcargo run --features mockstill log to stdout throughenv_loggerwith the<N>{args}format, so test harness output stays readable.mockfeature now impliesdep:env_logger.