[jira] [Commented] (KAFKA-2236) offset request reply racing with segment rolling

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-2236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270380#comment-15270380 ] 

ASF GitHub Bot commented on KAFKA-2236:
---------------------------------------

GitHub user ijuma opened a pull request:

    https://github.com/apache/kafka/pull/1318

    KAFKA-2236; Offset request reply racing with segment rolling

    

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/ijuma/kafka KAFKA-2236-offset-request-reply-segment-rolling-race

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/1318.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1318
    
----
commit 7d6d50cd377de82215856e22fc09e137a724acfc
Author: William Thurston <wt...@linkedin.com>
Date:   2015-07-19T02:54:05Z

    Addresses Jira Kafka-2236 to eliminate an array index out of bounds exception when segments are appended between checks

----


> offset request reply racing with segment rolling
> ------------------------------------------------
>
>                 Key: KAFKA-2236
>                 URL: https://issues.apache.org/jira/browse/KAFKA-2236
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 0.8.2.0
>         Environment: Linux x86_64, java.1.7.0_72, discovered using librdkafka based client.
>            Reporter: Alfred Landrum
>            Assignee: Jason Gustafson
>            Priority: Critical
>              Labels: newbie
>
> My use case with kafka involves an aggressive retention policy that rolls segment files frequently. My librdkafka based client sees occasional errors to offset requests, showing up in the broker log like:
> [2015-06-02 02:33:38,047] INFO Rolled new log segment for 'receiver-93b40462-3850-47c1-bcda-8a3e221328ca-50' in 1 ms. (kafka.log.Log)
> [2015-06-02 02:33:38,049] WARN [KafkaApi-0] Error while responding to offset request (kafka.server.KafkaApis)
> java.lang.ArrayIndexOutOfBoundsException: 3
>         at kafka.server.KafkaApis.fetchOffsetsBefore(KafkaApis.scala:469)
>         at kafka.server.KafkaApis.fetchOffsets(KafkaApis.scala:449)
>         at kafka.server.KafkaApis$$anonfun$17.apply(KafkaApis.scala:411)
>         at kafka.server.KafkaApis$$anonfun$17.apply(KafkaApis.scala:402)
>         at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
>         at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
>         at scala.collection.immutable.Map$Map1.foreach(Map.scala:109)
>         at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
>         at scala.collection.AbstractTraversable.map(Traversable.scala:105)
>         at kafka.server.KafkaApis.handleOffsetRequest(KafkaApis.scala:402)
>         at kafka.server.KafkaApis.handle(KafkaApis.scala:61)
>         at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:59)
>         at java.lang.Thread.run(Thread.java:745)
> quoting Guozhang Wang's reply to my query on the users list:
> "I check the 0.8.2 code and may probably find a bug related to your issue.
> Basically, segsArray.last.size is called multiple times during handling
> offset requests, while segsArray.last could get concurrent appends. Hence
> it is possible that in line 461, if(segsArray.last.size > 0) returns false
> while later in line 468, if(segsArray.last.size > 0) could return true."
> http://mail-archives.apache.org/mod_mbox/kafka-users/201506.mbox/%3CCAHwHRrUK-3wdoEAaFbsD0E859Ea0gXixfxgDzF8E3%3D_8r7K%2Bpw%40mail.gmail.com%3E



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)