Documentation/purpose of "date" and "time" fields

[candlerb]

Is your feature request related to a problem? Please describe.

I have some issues around the "date" and "time" fields which come from syslog, either as pipe-delimited fields or as mapped JSON fields.

The fundamental question is: are these fields actually used for anything, apart from being included in alert messages? And does it matter if the format is not YYYY-MM-DD or HH:MM:SS?

Context

(1) I notice from sagan-rules/json-input.map

{"software":"syslog-ng"," ...,"time":"DATE","date":"DATE"...}

That is: both fields are being set to the same value! According to syslog-ng documentation this field contains the BSD format date/time (e.g. Jun 13 15:58:00).

It looks like it should be possible to get syslog-ng to synthesise the required date and time using value-pairs. But as I'm using rsyslog rather than syslog-ng I don't have much interest in developing this.

What I am interested in is the importance of these fields, which affects how I configure rsyslog.

rsyslog's default JSON format generates a single ISO format string (e.g. 2019-10-28T16:56:43.171226+00:00). Should I put that in DATE, and leave TIME undefined? Or something else? Does it matter?

Note: with some effort, it's also possible to get rsyslog to generate custom JSON with YYYY-MM-DD and HH:MM:SS - I have documented this here in PR #146. The question is, is this necessary and/or worthwhile?

(2) The pipe examples shows the "date" field in YYYY-MM-DD, but this appears to be inconsistent with alerts which report MM-DD-YYYY. e.g.

[**] [1:5000077:3] [OPENSSH] Attempt to login using a denied user [**]
[Classification: unsuccessful-user] [Priority: 1] [127.0.0.1]
[Alert Time: 10-28-2019 16:18:19.654404]
2019-10-28 16:18:19 192.168.2.1:514 -> 192.168.2.1:22 user notice sshd
Message:  User ubuntu not allowed because shell /etc/passwd is not executable
[Xref => http://wiki.quadrantsec.com/bin/view/Main/5000077]

(compare line 3 and line 4 above). This is not locale-dependent; the alert time format comes from CreateTimeString in src/util-time.c

It appears to be legacy from poor design choices made in Suricata / Snort

Describe the solution you'd like

• Document what function the syslog "date" and "time" fields perform

Describe alternatives you've considered

• Change date to MM-DD-YYYY for consistency? No, YYYY-MM-DD is better for many reasons

Additional context
N/A