[jira] [Created] (METRON-1639) Grok Parser does not handle missing year well in syslog rfc3164 timestamps

["Simon Elliston Ball (JIRA)" <ji...@apache.org>]

Simon Elliston Ball created METRON-1639:
-------------------------------------------

             Summary: Grok Parser does not handle missing year well in syslog rfc3164 timestamps
                 Key: METRON-1639
                 URL: https://issues.apache.org/jira/browse/METRON-1639
             Project: Metron
          Issue Type: Improvement
    Affects Versions: 0.5.0
            Reporter: Simon Elliston Ball
            Assignee: Simon Elliston Ball


The grok parser does not handle timestamp fields in rfc3164 format well, since the format omits a year from the date, the year defaults to 1970. We should either switch this to default year to current, or create a "dateFormat" config option "syslog" which runs the SyslogUtils parser used in other parsers on the captured fields for the field specified in "timestampField" config.

This capability should also reflect the timezone for the sensor, which is not currently applied to Grok parsing but is honoured in parsers like BasicASAParser. Note that it is not universally applied across all parsers, but probably should be.

"Mmm dd hh:mm:ss" is the canonical date format in rfc3164, with options to include a timezone and year. We currently handle this and variants found in the wild in
SyslogUtils::parseTimestampToEpochMillis, which also accounts for timezone based on a Clock parameter. This function assumes that any date more than 4 days in the future is in the past, which seems acceptable and consistent for our purposes and covers the possibility of year end discrepancies. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)