You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by David Yu <gu...@gmail.com> on 2016/08/11 14:46:04 UTC
Failed to start kafka cleaner thread
Our Kafka cleaner has stopped running log compactions. From the log, we
found:
2016-08-03 20:57:19,240 ERROR kafka.log.LogCleaner:
[kafka-log-cleaner-thread-0], Error due to
java.lang.IllegalArgumentException: requirement failed: Last clean offset
is 104326 but segment base offset is 0 for log
session-store-2.0-tickets-changelog-62.
at scala.Predef$.require(Predef.scala:219)
at kafka.log.Cleaner.buildOffsetMap(LogCleaner.scala:584)
at kafka.log.Cleaner.clean(LogCleaner.scala:329)
at
kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:237)
at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:215)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
2016-08-03 20:57:19,310 INFO kafka.log.LogCleaner:
[kafka-log-cleaner-thread-0], Stopped
Can anyone let me know what this means?
Thanks,
David
Re: Failed to start kafka cleaner thread
Posted by David Yu <gu...@gmail.com>.
Ok, thanks for the info.
On Fri, Aug 12, 2016 at 8:29 AM Tom Crayford <tc...@heroku.com> wrote:
> Hey David,
>
> There have been numerous issues with the log cleaner thread.
> https://issues.apache.org/jira/browse/KAFKA-3894 talks about the history
> of
> "log cleaner thread crashing" issues. Some of them are only fixed in
> 0.10.0.0 (https://issues.apache.org/jira/browse/KAFKA-3587), and we're
> working on a patch for 0.10.1.0 or the next pertinent release.
>
> Thanks
>
> Tom Crayford
> Heroku Kafka
>
> On Thu, Aug 11, 2016 at 4:28 PM, David Yu <gu...@gmail.com> wrote:
>
> > BTW, we are using Kafka 0.9.0.0.
> >
> > And I found the following thread describing the same issue:
> > http://grokbase.com/t/kafka/users/159jbe18en/log-cleaner-thread-stops
> >
> > So is this still an issue in 0.9.0.0?
> >
> > Thanks,
> > David
> >
> > On Thu, Aug 11, 2016 at 9:46 AM David Yu <gu...@gmail.com> wrote:
> >
> > > Our Kafka cleaner has stopped running log compactions. From the log, we
> > > found:
> > >
> > > 2016-08-03 20:57:19,240 ERROR kafka.log.LogCleaner:
> > > [kafka-log-cleaner-thread-0], Error due to
> > >
> > > java.lang.IllegalArgumentException: requirement failed: Last clean
> > offset
> > > is 104326 but segment base offset is 0 for log
> > > session-store-2.0-tickets-changelog-62.
> > >
> > > at scala.Predef$.require(Predef.scala:219)
> > >
> > > at kafka.log.Cleaner.buildOffsetMap(LogCleaner.scala:584)
> > >
> > > at kafka.log.Cleaner.clean(LogCleaner.scala:329)
> > >
> > > at
> > > kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:237)
> > >
> > > at kafka.log.LogCleaner$CleanerThread.doWork(
> > LogCleaner.scala:215)
> > >
> > > at kafka.utils.ShutdownableThread.run(
> > ShutdownableThread.scala:63)
> > >
> > > 2016-08-03 20:57:19,310 INFO kafka.log.LogCleaner:
> > > [kafka-log-cleaner-thread-0], Stopped
> > >
> > >
> > > Can anyone let me know what this means?
> > >
> > > Thanks,
> > > David
> > >
> >
>
Re: Failed to start kafka cleaner thread
Posted by Tom Crayford <tc...@heroku.com>.
Hey David,
There have been numerous issues with the log cleaner thread.
https://issues.apache.org/jira/browse/KAFKA-3894 talks about the history of
"log cleaner thread crashing" issues. Some of them are only fixed in
0.10.0.0 (https://issues.apache.org/jira/browse/KAFKA-3587), and we're
working on a patch for 0.10.1.0 or the next pertinent release.
Thanks
Tom Crayford
Heroku Kafka
On Thu, Aug 11, 2016 at 4:28 PM, David Yu <gu...@gmail.com> wrote:
> BTW, we are using Kafka 0.9.0.0.
>
> And I found the following thread describing the same issue:
> http://grokbase.com/t/kafka/users/159jbe18en/log-cleaner-thread-stops
>
> So is this still an issue in 0.9.0.0?
>
> Thanks,
> David
>
> On Thu, Aug 11, 2016 at 9:46 AM David Yu <gu...@gmail.com> wrote:
>
> > Our Kafka cleaner has stopped running log compactions. From the log, we
> > found:
> >
> > 2016-08-03 20:57:19,240 ERROR kafka.log.LogCleaner:
> > [kafka-log-cleaner-thread-0], Error due to
> >
> > java.lang.IllegalArgumentException: requirement failed: Last clean
> offset
> > is 104326 but segment base offset is 0 for log
> > session-store-2.0-tickets-changelog-62.
> >
> > at scala.Predef$.require(Predef.scala:219)
> >
> > at kafka.log.Cleaner.buildOffsetMap(LogCleaner.scala:584)
> >
> > at kafka.log.Cleaner.clean(LogCleaner.scala:329)
> >
> > at
> > kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:237)
> >
> > at kafka.log.LogCleaner$CleanerThread.doWork(
> LogCleaner.scala:215)
> >
> > at kafka.utils.ShutdownableThread.run(
> ShutdownableThread.scala:63)
> >
> > 2016-08-03 20:57:19,310 INFO kafka.log.LogCleaner:
> > [kafka-log-cleaner-thread-0], Stopped
> >
> >
> > Can anyone let me know what this means?
> >
> > Thanks,
> > David
> >
>
Re: Failed to start kafka cleaner thread
Posted by David Yu <gu...@gmail.com>.
BTW, we are using Kafka 0.9.0.0.
And I found the following thread describing the same issue:
http://grokbase.com/t/kafka/users/159jbe18en/log-cleaner-thread-stops
So is this still an issue in 0.9.0.0?
Thanks,
David
On Thu, Aug 11, 2016 at 9:46 AM David Yu <gu...@gmail.com> wrote:
> Our Kafka cleaner has stopped running log compactions. From the log, we
> found:
>
> 2016-08-03 20:57:19,240 ERROR kafka.log.LogCleaner:
> [kafka-log-cleaner-thread-0], Error due to
>
> java.lang.IllegalArgumentException: requirement failed: Last clean offset
> is 104326 but segment base offset is 0 for log
> session-store-2.0-tickets-changelog-62.
>
> at scala.Predef$.require(Predef.scala:219)
>
> at kafka.log.Cleaner.buildOffsetMap(LogCleaner.scala:584)
>
> at kafka.log.Cleaner.clean(LogCleaner.scala:329)
>
> at
> kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:237)
>
> at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:215)
>
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
>
> 2016-08-03 20:57:19,310 INFO kafka.log.LogCleaner:
> [kafka-log-cleaner-thread-0], Stopped
>
>
> Can anyone let me know what this means?
>
> Thanks,
> David
>