logging feature

[hamano]

Hi, I’m watching this repository with great interest.
In my current project, I’m using WinSW, but if logging functionality is supported here, I might consider switching and contributing to the development.
What do you think about supporting log4rs, similar to how WinSW plans to support log4net?
I’d love to hear about your plans.
Thank you.

[hamano]

I've finished reading src/pkg/logs.rs, so I'd like to offer some additional comments.

In many service applications, it is necessary for end users to be able to customize the logging configuration.
WinSW used XML to define its logging setup, but any changes required re-registering the service, which made it difficult for end users to make modifications.
Ideally, we envision a setup where something like log4rs's logging_config.yaml is placed under ProgramData, allowing end users to easily customize logging.

[ferama]

Thanks for your interest in WSW.
When I think about WSW I think about something "similar (but way less feature reach)" to systemD. Ideally I don't want WSW to handle log management. WSW should focus solely on managing services and their lifecycle, nothing more, so that it remains as simple as possible.

For that use case, we can rely on other tools such as Promtail, Vector.dev or Fluent Bit to move and manage logs where they should be.

[hamano]

Thank you for your reply.
In my view, logging(log forward) is the responsibility of the supervisor (or, in other words, the parent process).
In fact, systemd receives the stdout and stderr of child processes and forwards them to the journald.

The most important point is that end users have a wide variety of requirements when it comes to logging — such as log file names, rotation intervals, and timestamp formats.
Application developers and supervisor developers should not have to spend their resources accommodating all of these demands.
As you may know, many issues related to logging have occurred in WinNW.

The purpose of my proposal is to offload the cost of around logging to an external module by integrating a powerful logging system like log4rs.

Promtail, Vector.dev or Fluent Bit

The supervisor (wsw) needs to have the ability to forward logs to these log aggregators via inter-process communication.
Tracking written log files would be too costly.

[ferama]

You’re right systemd forwards logs to a separate component (journald). But it’s journald’s responsibility and not systemd one to handle indexing, retention, storage usage, and so on.

The supervisor (wsw) needs to have the ability to forward logs to these log aggregators via inter-process communication.
Tracking written log files would be too costly.

That’s not how it works. Promtail, for example, scrapes logs directly from files and then forwards them to Loki. It works the same way in container scenarios. Vector.dev and Fluent Bit can do the same. Why do you think is too costly?

WSW is already built around the rust tracing ecosystem. I agree with you that it requires some improvements, and I'm thinking to support at least configurable rotation and retention but still I don't like the idea to go much further.

The closest thing to the systemd → journald pair that comes to mind is wsw with tracing-loki crate → Loki, but I still don’t like the idea of tying wsw to Loki or any other aggregator.

[hamano]

Why do you think is too costly?

Compared to systemd, which simply writes logs to a Unix domain socket, tracking log files is a high cost operation. It also requires maintaining a position file to record how much of the log has already been read.
Moreover, having a process other than the one writing the logs perform log rotation carries inherent risks.

It seems that you and I share the same opinion: we want to separate the responsibility of log writing from the supervisor(wsw), and we want to avoid complexity.

There are three actors in this situation:

• supervisor developer
• application developer
• application user

The application user tends to be demanding, making all sorts of requests—such as wanting to append signature, encrypt them, or store them on Amazon S3. I don’t want to use development resources to such demands.

Since I’ve realized that log4rs isn’t very feature-rich and doesn’t support log forwarding, I’d like to propose an alternative.
This approach may be similar to the multilog from DJB’s daemontools.

For example:

wsw.exe install --name myapp --cmd "app.exe" --logger "logger.exe"

With the wsw launches app.exe, and redirects its stdout and stderr to the stdin of logger.exe.
This enables users to have complete flexibility—such as signature, encryption, and more.

Furthermore, by separating the logging into a different process, we gain the advantage that any bug related to logging will not affect the main service itself.

Would it be possible for you to accept this kind of feature addition?
Thank you.

[ferama]

This seems like a much better solution.
I think the default behavior should be to use the log files as it currently does.
If --logger is set, then the log files will be used for wsw logs only and app.exe logs will be directed to logger.exe instead.

logger.exe should also be spawned by wsw, so it must be managed the same way app.exe is managed. It must belong to the same job_object (see runner.rs) and should be monitored just like app.exe currently is.

What about the log subcommand? How should wsw.exe handle logs in this case? We need to consider this as well—perhaps simply disable it if the --logger option was used to launch the service.

A basic example logger would be a nice addition.

[hamano]

Thank you for considering it!
If possible, I would like the default logger to work using the same mechanism (as a separate process).
This would serve as a good example for users developing their own custom loggers.
Alternatively, by making --logger="wsw.exe defaultlogger" subcommand the default logger,
you can get the advantage of releasing a single binary.

That said, these details are relatively minor.
If the --logger option is accepted, there's a chance I can migrate from WinSW and participate in the development of wsw.
It might take a little time, but after investigating the migration, I will send a pull request.

[ferama]

If possible, I would like the default logger to work using the same mechanism (as a separate process).
This would serve as a good example for users developing their own custom loggers.
Alternatively, by making --logger="wsw.exe defaultlogger" subcommand the default logger,
you can get the advantage of releasing a single binary.

If you don’t need advanced logging features, it’s probably better not to spawn an unnecessary extra process. I definitely wouldn’t recommend requiring two binaries just to start using WSW