You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by Madhukar Bharti <bh...@gmail.com> on 2015/12/30 13:30:15 UTC

__consumer_offsets Topic files are not getting deleted after log retention hrs

Dear Team,

We are using Kafka-0.8.2.1 and having log.retention.hours=168 but files of
__consumer_offsets are not getting deleted, due to this lots of disc spaces
are used.

Please help how to delete file of offset storage topic after specified time.


Thanks and Regards,
Madhukar

Re: __consumer_offsets Topic files are not getting deleted after log retention hrs

Posted by Madhukar Bharti <bh...@gmail.com>.

Thanks a lot Guozhnag, Maxim.

I haven't set "offsets.retention.minutes" and "offsets.topic.segment.bytes",
both are defaults only but we have "log.cleaner.enable" is set to false.
that might be the reason. We will set as mentioned and test.

Thanks again!!

On Fri, Jan 1, 2016 at 1:32 AM, Maxim Vladimirsky <ho...@gmail.com> wrote:

> Make sure that you have *log.cleaner.enable* set to *true* (it is false by
> default). Check this KAFKA-2988
> <https://issues.apache.org/jira/browse/KAFKA-2988> out for details.
>
>
> On Thu, Dec 31, 2015 at 11:13 AM, Guozhang Wang <wa...@gmail.com>
> wrote:
>
> > There is a special config for that topic named
> "offsets.retention.minutes"
> > besides the "log.retention.hours", and similarly a different
> > "offsets.topic.segment.bytes" for that topic as well. Could you check
> what
> > are the values?
> >
> > Note that only old log segments will be considered for deletion so if
> your
> > __consumer_offsets topic only have one segment file, it will not be
> > deleted.
> >
> > Guozhang
> >
> >
> > On Wed, Dec 30, 2015 at 4:30 AM, Madhukar Bharti <
> bhartimadhukar@gmail.com
> > >
> > wrote:
> >
> > > Dear Team,
> > >
> > > We are using Kafka-0.8.2.1 and having log.retention.hours=168 but files
> > of
> > > __consumer_offsets are not getting deleted, due to this lots of disc
> > spaces
> > > are used.
> > >
> > > Please help how to delete file of offset storage topic after specified
> > > time.
> > >
> > >
> > > Thanks and Regards,
> > > Madhukar
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>



-- 
Thanks and Regards,
Madhukar

Re: __consumer_offsets Topic files are not getting deleted after log retention hrs

Posted by Maxim Vladimirsky <ho...@gmail.com>.

Make sure that you have *log.cleaner.enable* set to *true* (it is false by
default). Check this KAFKA-2988
<https://issues.apache.org/jira/browse/KAFKA-2988> out for details.


On Thu, Dec 31, 2015 at 11:13 AM, Guozhang Wang <wa...@gmail.com> wrote:

> There is a special config for that topic named "offsets.retention.minutes"
> besides the "log.retention.hours", and similarly a different
> "offsets.topic.segment.bytes" for that topic as well. Could you check what
> are the values?
>
> Note that only old log segments will be considered for deletion so if your
> __consumer_offsets topic only have one segment file, it will not be
> deleted.
>
> Guozhang
>
>
> On Wed, Dec 30, 2015 at 4:30 AM, Madhukar Bharti <bhartimadhukar@gmail.com
> >
> wrote:
>
> > Dear Team,
> >
> > We are using Kafka-0.8.2.1 and having log.retention.hours=168 but files
> of
> > __consumer_offsets are not getting deleted, due to this lots of disc
> spaces
> > are used.
> >
> > Please help how to delete file of offset storage topic after specified
> > time.
> >
> >
> > Thanks and Regards,
> > Madhukar
> >
>
>
>
> --
> -- Guozhang
>

Re: __consumer_offsets Topic files are not getting deleted after log retention hrs

Posted by Guozhang Wang <wa...@gmail.com>.

There is a special config for that topic named "offsets.retention.minutes"
besides the "log.retention.hours", and similarly a different
"offsets.topic.segment.bytes" for that topic as well. Could you check what
are the values?

Note that only old log segments will be considered for deletion so if your
__consumer_offsets topic only have one segment file, it will not be deleted.

Guozhang

On Wed, Dec 30, 2015 at 4:30 AM, Madhukar Bharti <bh...@gmail.com>
wrote:

> Dear Team,
>
> We are using Kafka-0.8.2.1 and having log.retention.hours=168 but files of
> __consumer_offsets are not getting deleted, due to this lots of disc spaces
> are used.
>
> Please help how to delete file of offset storage topic after specified
> time.
>
>
> Thanks and Regards,
> Madhukar
>

-- 
-- Guozhang