You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@mesos.apache.org by "David J. Palaitis" <da...@gmail.com> on 2015/03/06 03:40:01 UTC

logstash config

Anyone out there have a logstash config for Mesos log format they'd like to
share? I'm finding the date format stubbornly difficult to map to timestamp.

Re: logstash config

Posted by "David J. Palaitis" <da...@gmail.com>.

that works, but it doesn't retain the original timestamp from the log.  I
finally got it working with the following:

>>> patterns/mesos.logstash

LOGLEVEL [I,E,W,F]

MESOSTIMESTAMP %{MONTHNUM}%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}

MESOSLOG %{LOGLEVEL:loglevel}%{MESOSTIMESTAMP:timestamp} %{POSINT:threadid}
%{GREEDYDATA:file}:%{POSINT:line}] %{GREEDYDATA:msg}

>>> logstash.config

filter {

  grok {

    match => { "message" => "%{MESOSLOG}" }

  }

  mutate {

    gsub => [

      # make the logTimestamp sortable. otherwise, date filter will break.

      "timestamp", " ", ";"

    ]

  }

  date {

    match => ["timestamp", "MMdd;HH:mm:ss.SSSSSS"]

  }

}



On Fri, Mar 6, 2015 at 2:11 AM, Gurvinder Singh <gu...@uninett.no>
wrote:

> This is the config we use for mesos logs
>
> MESOSTIMESTAMP %{MONTHNUM}%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}
>
> MESOSLOG
> %{DATA:loglevel}%{MESOSTIMESTAMP:timestamp}\s+%{POSINT:pid}\s+%{NOTSPACE:class}:
>
> in logstash config
>
> if [type] == "mesos" {
>     grok {
>       patterns_dir => ["path to your patterns dir"]
>       match        => ["message", "%{MESOSLOG}"]
>     }
> }
>
> - Gurvinder
> On 03/06/2015 03:40 AM, David J. Palaitis wrote:
> > Anyone out there have a logstash config for Mesos log format they'd like
> > to share? I'm finding the date format stubbornly difficult to map to
> > timestamp.
> >
>
>

Re: logstash config

Posted by Gurvinder Singh <gu...@uninett.no>.

This is the config we use for mesos logs

MESOSTIMESTAMP %{MONTHNUM}%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}

MESOSLOG %{DATA:loglevel}%{MESOSTIMESTAMP:timestamp}\s+%{POSINT:pid}\s+%{NOTSPACE:class}:

in logstash config

if [type] == "mesos" {
    grok {
      patterns_dir => ["path to your patterns dir"]
      match        => ["message", "%{MESOSLOG}"]
    }
}

- Gurvinder
On 03/06/2015 03:40 AM, David J. Palaitis wrote:
> Anyone out there have a logstash config for Mesos log format they'd like
> to share? I'm finding the date format stubbornly difficult to map to
> timestamp.
>