You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by Charles Allen <ch...@metamarkets.com> on 2017/01/24 17:20:14 UTC
best practices for log rotation
Anyone have good hints for best practices for log rotation?
Our mesos master ended up with many gigabytes of logs once we started
running SPARK on it (approx 2GB of master INFO logs per day).
Re: best practices for log rotation
Posted by Charles Allen <ch...@metamarkets.com>.
Thanks, these are very helpful
On Tue, Jan 24, 2017 at 11:41 AM Cody Maloney <co...@mesosphere.io> wrote:
> For DC/OS we do two pieces. We simplify Mesos' log output via a module so
> that it doesn't have any of it's internal logrotate logic, and just writes
> to a single straight output file. We also include a mesos module so that
> mesos task output goes to systemd journald, making it so every piece of
> logging inside DC/OS ends up at a single source which can then be piped
> wherever people like. The modules:
> https://github.com/dcos/dcos-mesos-modules
>
> Systemd auto-rotates the journald logs and does cap sized logging with it.
> For the non-systemd logs we package logrotate (
> https://github.com/dcos/dcos/tree/master/packages/logrotate) + a simple
> helper script to catch accidentally orphaned files.
>
> On Tue, Jan 24, 2017 at 9:40 AM Tomek Janiszewski <ja...@gmail.com>
> wrote:
>
> We are using logrotate to rotate, compress and delete old data. To keep
> logs easier to search we put them into Elastic/Kibana.
>
> wt., 24.01.2017, 18:20 użytkownik Charles Allen <
> charles.allen@metamarkets.com> napisał:
>
> Anyone have good hints for best practices for log rotation?
>
> Our mesos master ended up with many gigabytes of logs once we started
> running SPARK on it (approx 2GB of master INFO logs per day).
>
>
Re: best practices for log rotation
Posted by Cody Maloney <co...@mesosphere.io>.
For DC/OS we do two pieces. We simplify Mesos' log output via a module so
that it doesn't have any of it's internal logrotate logic, and just writes
to a single straight output file. We also include a mesos module so that
mesos task output goes to systemd journald, making it so every piece of
logging inside DC/OS ends up at a single source which can then be piped
wherever people like. The modules:
https://github.com/dcos/dcos-mesos-modules
Systemd auto-rotates the journald logs and does cap sized logging with it.
For the non-systemd logs we package logrotate (
https://github.com/dcos/dcos/tree/master/packages/logrotate) + a simple
helper script to catch accidentally orphaned files.
On Tue, Jan 24, 2017 at 9:40 AM Tomek Janiszewski <ja...@gmail.com> wrote:
> We are using logrotate to rotate, compress and delete old data. To keep
> logs easier to search we put them into Elastic/Kibana.
>
> wt., 24.01.2017, 18:20 użytkownik Charles Allen <
> charles.allen@metamarkets.com> napisał:
>
> Anyone have good hints for best practices for log rotation?
>
> Our mesos master ended up with many gigabytes of logs once we started
> running SPARK on it (approx 2GB of master INFO logs per day).
>
>
Re: best practices for log rotation
Posted by Tomek Janiszewski <ja...@gmail.com>.
We are using logrotate to rotate, compress and delete old data. To keep
logs easier to search we put them into Elastic/Kibana.
wt., 24.01.2017, 18:20 użytkownik Charles Allen <
charles.allen@metamarkets.com> napisał:
> Anyone have good hints for best practices for log rotation?
>
> Our mesos master ended up with many gigabytes of logs once we started
> running SPARK on it (approx 2GB of master INFO logs per day).
>