You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Anatoly Deyneka <An...@ig.com> on 2016/03/08 10:07:20 UTC

kafka 0.9.0.1: FATAL exception on startup

Hi,

I need your advice how to start server in the next situation:
It fails on startup with FATAL error:
[2016-03-07 16:30:53,495] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
kafka.common.InvalidOffsetException: Attempt to append an offset (211046544) to position 40048 no larger than the last offset appended (211046546) to xyz/00000000000210467262.index.
        at kafka.log.OffsetIndex$$anonfun$append$1.apply$mcV$sp(OffsetIndex.scala:207)
        at kafka.log.OffsetIndex$$anonfun$append$1.apply(OffsetIndex.scala:197)
        at kafka.log.OffsetIndex$$anonfun$append$1.apply(OffsetIndex.scala:197)
        at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:262)
        at kafka.log.OffsetIndex.append(OffsetIndex.scala:197)
        at kafka.log.LogSegment.recover(LogSegment.scala:188)
        at kafka.log.Log$$anonfun$loadSegments$4.apply(Log.scala:188)
        at kafka.log.Log$$anonfun$loadSegments$4.apply(Log.scala:160)
        at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:778)
        at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
        at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
        at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:777)
        at kafka.log.Log.loadSegments(Log.scala:160)
        at kafka.log.Log.<init>(Log.scala:90)
        at kafka.log.LogManager$$anonfun$loadLogs$2$$anonfun$3$$anonfun$apply$10$$anonfun$apply$1.apply$mcV$sp(LogManager.scala:150)
        at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:60)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)

http://stackoverflow.com/questions/35849673/kafka-0-9-0-1-fails-with-fatal-exception-on-startup

Thanks,
Anatoly
________________________________

The information contained in this email is strictly confidential and for the use of the addressee only, unless otherwise indicated. If you are not the intended recipient, please do not read, copy, use or disclose to others this message or any attachment. Please also notify the sender by replying to this email or by telephone (+44 (0)20 7896 0011) and then delete the email and any copies of it. Opinions, conclusions (etc) that do not relate to the official business of this company shall be understood as neither given nor endorsed by it. IG Group Holdings plc is a company registered in England and Wales under number 04677092. VAT registration number 761 2978 07. Registered Office: Cannon Bridge House, 25 Dowgate Hill, London EC4R 2YA. Listed on the London Stock Exchange. Its subsidiaries IG Markets Limited and IG Index Limited are authorised and regulated by the Financial Conduct Authority (IG Markets Limited FCA registration number 195355 and IG Index Limited FCA registration number 114059).-

Re: kafka 0.9.0.1: FATAL exception on startup

Posted by Ismael Juma <is...@juma.me.uk>.
Hi Anatoly and James,

Can one of you please file a JIRA? Please describe the problem in detail
and please include logs, if available.

Ismael

On Wed, Mar 23, 2016 at 5:44 AM, James Cheng <jc...@tivo.com> wrote:

> Hi, we ran into this problem too. The only way we were able to bypass this
> was by stopping Kafka and deleting the log directory of the affected
> partition. Which means, we lost data for that partition on this broker.
>
> -James
>
> > On Mar 8, 2016, at 1:07 AM, Anatoly Deyneka <An...@ig.com>
> wrote:
> >
> > Hi,
> >
> > I need your advice how to start server in the next situation:
> > It fails on startup with FATAL error:
> > [2016-03-07 16:30:53,495] FATAL Fatal error during KafkaServerStartable
> startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
> > kafka.common.InvalidOffsetException: Attempt to append an offset
> (211046544) to position 40048 no larger than the last offset appended
> (211046546) to xyz/00000000000210467262.index.
> >        at
> kafka.log.OffsetIndex$$anonfun$append$1.apply$mcV$sp(OffsetIndex.scala:207)
> >        at
> kafka.log.OffsetIndex$$anonfun$append$1.apply(OffsetIndex.scala:197)
> >        at
> kafka.log.OffsetIndex$$anonfun$append$1.apply(OffsetIndex.scala:197)
> >        at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:262)
> >        at kafka.log.OffsetIndex.append(OffsetIndex.scala:197)
> >        at kafka.log.LogSegment.recover(LogSegment.scala:188)
> >        at kafka.log.Log$$anonfun$loadSegments$4.apply(Log.scala:188)
> >        at kafka.log.Log$$anonfun$loadSegments$4.apply(Log.scala:160)
> >        at
> scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:778)
> >        at
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
> >        at
> scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
> >        at
> scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:777)
> >        at kafka.log.Log.loadSegments(Log.scala:160)
> >        at kafka.log.Log.<init>(Log.scala:90)
> >        at
> kafka.log.LogManager$$anonfun$loadLogs$2$$anonfun$3$$anonfun$apply$10$$anonfun$apply$1.apply$mcV$sp(LogManager.scala:150)
> >        at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:60)
> >        at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> >        at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >        at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >        at java.lang.Thread.run(Thread.java:744)
> >
> >
> http://stackoverflow.com/questions/35849673/kafka-0-9-0-1-fails-with-fatal-exception-on-startup
> >
> > Thanks,
> > Anatoly
> > ________________________________
> >
> > The information contained in this email is strictly confidential and for
> the use of the addressee only, unless otherwise indicated. If you are not
> the intended recipient, please do not read, copy, use or disclose to others
> this message or any attachment. Please also notify the sender by replying
> to this email or by telephone (+44 (0)20 7896 0011) and then delete the
> email and any copies of it. Opinions, conclusions (etc) that do not relate
> to the official business of this company shall be understood as neither
> given nor endorsed by it. IG Group Holdings plc is a company registered in
> England and Wales under number 04677092. VAT registration number 761 2978
> 07. Registered Office: Cannon Bridge House, 25 Dowgate Hill, London EC4R
> 2YA. Listed on the London Stock Exchange. Its subsidiaries IG Markets
> Limited and IG Index Limited are authorised and regulated by the Financial
> Conduct Authority (IG Markets Limited FCA registration number 195355 and IG
> Index Limited FCA registration number 114059).-
>
>
> ________________________________
>
> This email and any attachments may contain confidential and privileged
> material for the sole use of the intended recipient. Any review, copying,
> or distribution of this email (or any attachments) by others is prohibited.
> If you are not the intended recipient, please contact the sender
> immediately and permanently delete this email and any attachments. No
> employee or agent of TiVo Inc. is authorized to conclude any binding
> agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo
> Inc. may only be made by a signed written agreement.
>

Re: kafka 0.9.0.1: FATAL exception on startup

Posted by James Cheng <jc...@tivo.com>.
Hi, we ran into this problem too. The only way we were able to bypass this was by stopping Kafka and deleting the log directory of the affected partition. Which means, we lost data for that partition on this broker.

-James

> On Mar 8, 2016, at 1:07 AM, Anatoly Deyneka <An...@ig.com> wrote:
>
> Hi,
>
> I need your advice how to start server in the next situation:
> It fails on startup with FATAL error:
> [2016-03-07 16:30:53,495] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
> kafka.common.InvalidOffsetException: Attempt to append an offset (211046544) to position 40048 no larger than the last offset appended (211046546) to xyz/00000000000210467262.index.
>        at kafka.log.OffsetIndex$$anonfun$append$1.apply$mcV$sp(OffsetIndex.scala:207)
>        at kafka.log.OffsetIndex$$anonfun$append$1.apply(OffsetIndex.scala:197)
>        at kafka.log.OffsetIndex$$anonfun$append$1.apply(OffsetIndex.scala:197)
>        at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:262)
>        at kafka.log.OffsetIndex.append(OffsetIndex.scala:197)
>        at kafka.log.LogSegment.recover(LogSegment.scala:188)
>        at kafka.log.Log$$anonfun$loadSegments$4.apply(Log.scala:188)
>        at kafka.log.Log$$anonfun$loadSegments$4.apply(Log.scala:160)
>        at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:778)
>        at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
>        at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
>        at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:777)
>        at kafka.log.Log.loadSegments(Log.scala:160)
>        at kafka.log.Log.<init>(Log.scala:90)
>        at kafka.log.LogManager$$anonfun$loadLogs$2$$anonfun$3$$anonfun$apply$10$$anonfun$apply$1.apply$mcV$sp(LogManager.scala:150)
>        at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:60)
>        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>        at java.lang.Thread.run(Thread.java:744)
>
> http://stackoverflow.com/questions/35849673/kafka-0-9-0-1-fails-with-fatal-exception-on-startup
>
> Thanks,
> Anatoly
> ________________________________
>
> The information contained in this email is strictly confidential and for the use of the addressee only, unless otherwise indicated. If you are not the intended recipient, please do not read, copy, use or disclose to others this message or any attachment. Please also notify the sender by replying to this email or by telephone (+44 (0)20 7896 0011) and then delete the email and any copies of it. Opinions, conclusions (etc) that do not relate to the official business of this company shall be understood as neither given nor endorsed by it. IG Group Holdings plc is a company registered in England and Wales under number 04677092. VAT registration number 761 2978 07. Registered Office: Cannon Bridge House, 25 Dowgate Hill, London EC4R 2YA. Listed on the London Stock Exchange. Its subsidiaries IG Markets Limited and IG Index Limited are authorised and regulated by the Financial Conduct Authority (IG Markets Limited FCA registration number 195355 and IG Index Limited FCA registration number 114059).-


________________________________

This email and any attachments may contain confidential and privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments) by others is prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete this email and any attachments. No employee or agent of TiVo Inc. is authorized to conclude any binding agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo Inc. may only be made by a signed written agreement.