[jira] [Comment Edited] (KAFKA-528) IndexOutOfBoundsException thrown by kafka.consumer.ConsumerFetcherThread

["John Fung (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466002#comment-13466002 ] 

John Fung edited comment on KAFKA-528 at 9/29/12 10:21 AM:
-----------------------------------------------------------

1. Tested kafka-528.patch with rev. 1389460 and there is no data loss now. The Exception specified in this JIRA is not showing.

2. However, there is still another exception showing in broker log (already filed under KAFKA-529) :

[2012-09-28 23:05:54,669] ERROR [ReplicaFetcherThread-2-0-on-broker-3], Error due to  (kafka.server.ReplicaFetcherThread)
java.lang.IndexOutOfBoundsException
        at java.nio.Buffer.checkIndex(Buffer.java:514)
        at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:121)
        at kafka.message.Message.magic(Message.scala:119)
        at kafka.message.Message.checksum(Message.scala:132)
        at kafka.message.Message.isValid(Message.scala:144)

                
      was (Author: jfung):
    Tested kafka-528.patch with rev. 1389460 and there is no data loss now. However, there is still another exception showing in broker log (already filed under KAFKA-529) :

[2012-09-28 23:05:54,669] ERROR [ReplicaFetcherThread-2-0-on-broker-3], Error due to  (kafka.server.ReplicaFetcherThread)
java.lang.IndexOutOfBoundsException
        at java.nio.Buffer.checkIndex(Buffer.java:514)
        at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:121)
        at kafka.message.Message.magic(Message.scala:119)
        at kafka.message.Message.checksum(Message.scala:132)
        at kafka.message.Message.isValid(Message.scala:144)

                  
> IndexOutOfBoundsException thrown by kafka.consumer.ConsumerFetcherThread
> ------------------------------------------------------------------------
>
>                 Key: KAFKA-528
>                 URL: https://issues.apache.org/jira/browse/KAFKA-528
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.8
>            Reporter: John Fung
>            Assignee: Jun Rao
>            Priority: Blocker
>              Labels: bugs
>             Fix For: 0.8
>
>         Attachments: kafka-528.patch, system_test_1348521165.tar.gz
>
>
> 1. Attached file system_test_1348521165.tar.gz contains all the associated log files for this test session.
> 2. The system test output log can be found at: system_test_1348521165/system_test_output.log
> 3. The following log message can be found at: system_test_1348521165/logs/console_consumer-5/console_consumer.log
> [2012-09-24 14:15:12,016] ERROR [ConsumerFetcherThread-console-consumer-16186_jfung-1348521311426-2c83ced7-0-1], Error due to  (kafka.consumer.ConsumerFetcherThread)
> java.lang.IndexOutOfBoundsException
>         at java.nio.Buffer.checkIndex(Buffer.java:512)
>         at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:121)
>         at kafka.message.Message.magic(Message.scala:119)
>         at kafka.message.Message.checksum(Message.scala:132)
>         at kafka.message.Message.isValid(Message.scala:144)
>         at kafka.message.ByteBufferMessageSet$$anon$1.makeNextOuter(ByteBufferMessageSet.scala:118)
>         at kafka.message.ByteBufferMessageSet$$anon$1.makeNext(ByteBufferMessageSet.scala:149)
>         at kafka.message.ByteBufferMessageSet$$anon$1.makeNext(ByteBufferMessageSet.scala:89)
>         at kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:61)
>         at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:53)
>         at kafka.message.ByteBufferMessageSet.shallowValidBytes(ByteBufferMessageSet.scala:54)
>         at kafka.message.ByteBufferMessageSet.validBytes(ByteBufferMessageSet.scala:49)
>         at kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:54)
>         at kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcherThread.scala:42)
>         at kafka.server.AbstractFetcherThread$$anonfun$doWork$5.apply(AbstractFetcherThread.scala:103)
>         at kafka.server.AbstractFetcherThread$$anonfun$doWork$5.apply(AbstractFetcherThread.scala:96)
>         at scala.collection.immutable.Map$Map2.foreach(Map.scala:127)
>         at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:96)
>         at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:50)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira