You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by Bart Spaans <ba...@opencredo.com> on 2015/02/25 11:37:00 UTC
Log rotation
Hi all,
I was wondering if anyone knows a nice solution to rotating the
stdout/stderr logs, preferably without breaking the web ui log viewer?
We've seen some hard disks fill up, making whole nodes unusable, so I'd
like to put something around that.
Thanks,
Bart
--
Bart Spaans
Senior Consultant
OpenCredo Ltd -- Excellence in Enterprise Application Development
Mobile: +44(0) 7453 777 558
Registered Office: 5-11 Lavington St., London SE1 0NZ
Registered in UK. No 3943999
If you have received this e-mail in error please accept our apologies,
destroy it immediately and it would be greatly appreciated if you notified
the sender. It is your responsibility to protect your system from viruses
and any other harmful code or device. We try to eliminate them from
e-mails and attachments; but we accept no liability for any that remain. We
may monitor or access any or all e-mails sent to us.
Re: Log rotation
Posted by Benjamin Mahler <be...@gmail.com>.
Yes that is for master / slave logs only, sorry for misreading.
0.22.0 has optional disk quota enforcement you can enable to avoid jobs
going over their disk allocation.
On Wed, Feb 25, 2015 at 1:23 PM, Dick Davies <di...@hellooperator.net> wrote:
> I may be misreading that diff, but it seems that's specific to the
> master/slave logs,
> rather than the 'stdout/stderr' logs that you see in a task sandbox.
>
> I hope I'm wrong as we have similar issues - especially with apps
> deployed with tracing
> etc. on that merrily fill up /var/mesos on a slave, kill it, get
> deployed to another slave,
> kill that, etc...
>
> our plan is to log all tasks remotely to a fluentd or logstash agent
> on each slave.
>
> On 25 February 2015 at 18:05, Benjamin Mahler <be...@gmail.com>
> wrote:
> > +drob
> >
> > 0.22.0 introduces a flag on the master / slave called
> --external_log_file.
> > This allows you to point the master/slave to an external log file so
> that it
> > can be shown the webui. This means you can log to stderr, rotate the file
> > it's getting redirected to, and still see it in the webui log viewer:
> >
> > https://reviews.apache.org/r/30328/
> >
> > On Wed, Feb 25, 2015 at 2:37 AM, Bart Spaans <ba...@opencredo.com>
> > wrote:
> >>
> >> Hi all,
> >>
> >> I was wondering if anyone knows a nice solution to rotating the
> >> stdout/stderr logs, preferably without breaking the web ui log viewer?
> We've
> >> seen some hard disks fill up, making whole nodes unusable, so I'd like
> to
> >> put something around that.
> >>
> >> Thanks,
> >> Bart
> >>
> >> --
> >>
> >> Bart Spaans
> >> Senior Consultant
> >> OpenCredo Ltd -- Excellence in Enterprise Application Development
> >>
> >> Mobile: +44(0) 7453 777 558
> >>
> >> Registered Office: 5-11 Lavington St., London SE1 0NZ
> >> Registered in UK. No 3943999
> >>
> >> If you have received this e-mail in error please accept our apologies,
> >> destroy it immediately and it would be greatly appreciated if you
> notified
> >> the sender. It is your responsibility to protect your system from
> viruses
> >> and any other harmful code or device. We try to eliminate them from
> e-mails
> >> and attachments; but we accept no liability for any that remain. We may
> >> monitor or access any or all e-mails sent to us.
> >
> >
>
Re: Log rotation
Posted by Dick Davies <di...@hellooperator.net>.
I may be misreading that diff, but it seems that's specific to the
master/slave logs,
rather than the 'stdout/stderr' logs that you see in a task sandbox.
I hope I'm wrong as we have similar issues - especially with apps
deployed with tracing
etc. on that merrily fill up /var/mesos on a slave, kill it, get
deployed to another slave,
kill that, etc...
our plan is to log all tasks remotely to a fluentd or logstash agent
on each slave.
On 25 February 2015 at 18:05, Benjamin Mahler <be...@gmail.com> wrote:
> +drob
>
> 0.22.0 introduces a flag on the master / slave called --external_log_file.
> This allows you to point the master/slave to an external log file so that it
> can be shown the webui. This means you can log to stderr, rotate the file
> it's getting redirected to, and still see it in the webui log viewer:
>
> https://reviews.apache.org/r/30328/
>
> On Wed, Feb 25, 2015 at 2:37 AM, Bart Spaans <ba...@opencredo.com>
> wrote:
>>
>> Hi all,
>>
>> I was wondering if anyone knows a nice solution to rotating the
>> stdout/stderr logs, preferably without breaking the web ui log viewer? We've
>> seen some hard disks fill up, making whole nodes unusable, so I'd like to
>> put something around that.
>>
>> Thanks,
>> Bart
>>
>> --
>>
>> Bart Spaans
>> Senior Consultant
>> OpenCredo Ltd -- Excellence in Enterprise Application Development
>>
>> Mobile: +44(0) 7453 777 558
>>
>> Registered Office: 5-11 Lavington St., London SE1 0NZ
>> Registered in UK. No 3943999
>>
>> If you have received this e-mail in error please accept our apologies,
>> destroy it immediately and it would be greatly appreciated if you notified
>> the sender. It is your responsibility to protect your system from viruses
>> and any other harmful code or device. We try to eliminate them from e-mails
>> and attachments; but we accept no liability for any that remain. We may
>> monitor or access any or all e-mails sent to us.
>
>
Re: Log rotation
Posted by Benjamin Mahler <be...@gmail.com>.
+drob
0.22.0 introduces a flag on the master / slave called --external_log_file.
This allows you to point the master/slave to an external log file so that
it can be shown the webui. This means you can log to stderr, rotate the
file it's getting redirected to, and still see it in the webui log viewer:
https://reviews.apache.org/r/30328/
On Wed, Feb 25, 2015 at 2:37 AM, Bart Spaans <ba...@opencredo.com>
wrote:
> Hi all,
>
> I was wondering if anyone knows a nice solution to rotating the
> stdout/stderr logs, preferably without breaking the web ui log viewer?
> We've seen some hard disks fill up, making whole nodes unusable, so I'd
> like to put something around that.
>
> Thanks,
> Bart
>
> --
>
> Bart Spaans
> Senior Consultant
> OpenCredo Ltd -- Excellence in Enterprise Application Development
>
> Mobile: +44(0) 7453 777 558
>
> Registered Office: 5-11 Lavington St., London SE1 0NZ
> Registered in UK. No 3943999
>
> If you have received this e-mail in error please accept our apologies,
> destroy it immediately and it would be greatly appreciated if you notified
> the sender. It is your responsibility to protect your system from viruses
> and any other harmful code or device. We try to eliminate them from
> e-mails and attachments; but we accept no liability for any that remain. We
> may monitor or access any or all e-mails sent to us.
>