You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by Warren Kiser <wa...@hioscar.com> on 2015/02/28 01:12:25 UTC

Unlimited Log Retention

Does anyone know how to achieve unlimited log retention either globally or
on a per topic basis? I tried explicitly setting the log.retention.bytes to
-1 but the default time policy kicked in after 7 days and cleaned up the
messages.

Thanks!

Warren

Re: Unlimited Log Retention

Posted by Warren Kiser <wa...@hioscar.com>.

Thanks! 245,146 years is more than enough for me.

On Sat, Feb 28, 2015 at 2:58 PM, Jay Kreps <ja...@gmail.com> wrote:

> It is totally reasonable to have unlimited retention. We don't have an
> explicit setting for this but you can set the time based retention policy
> to something large
>   log.retention.hours=2147483647
> which will retain the log for 245,146 years. :-)
>
> -Jay
>
> On Fri, Feb 27, 2015 at 4:12 PM, Warren Kiser <wa...@hioscar.com> wrote:
>
> > Does anyone know how to achieve unlimited log retention either globally
> or
> > on a per topic basis? I tried explicitly setting the log.retention.bytes
> to
> > -1 but the default time policy kicked in after 7 days and cleaned up the
> > messages.
> >
> > Thanks!
> >
> > Warren
> >
>

Re: Unlimited Log Retention

Posted by Jay Kreps <ja...@gmail.com>.

It is totally reasonable to have unlimited retention. We don't have an
explicit setting for this but you can set the time based retention policy
to something large
  log.retention.hours=2147483647
which will retain the log for 245,146 years. :-)

-Jay

On Fri, Feb 27, 2015 at 4:12 PM, Warren Kiser <wa...@hioscar.com> wrote:

> Does anyone know how to achieve unlimited log retention either globally or
> on a per topic basis? I tried explicitly setting the log.retention.bytes to
> -1 but the default time policy kicked in after 7 days and cleaned up the
> messages.
>
> Thanks!
>
> Warren
>

Re: Unlimited Log Retention

Posted by Mark Roberts <wi...@gmail.com>.

Wouldn't it be a better choice to store the logs offline somewhere? HDFS and S3 are both good choices...

-Mark


> On Feb 27, 2015, at 16:12, Warren Kiser <wa...@hioscar.com> wrote:
> 
> Does anyone know how to achieve unlimited log retention either globally or
> on a per topic basis? I tried explicitly setting the log.retention.bytes to
> -1 but the default time policy kicked in after 7 days and cleaned up the
> messages.
> 
> Thanks!
> 
> Warren