[jira] [Commented] (STORM-511) Storm-Kafka spout keeps sending fetch requests with invalid offset

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/STORM-511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14181937#comment-14181937 ] 

ASF GitHub Bot commented on STORM-511:
--------------------------------------

Github user ptgoetz commented on the pull request:

    https://github.com/apache/storm/pull/272#issuecomment-60306459
  
    @viktortnk Thanks for your work on this. I made some minor changes and submitted a pull request to your repo.
    
    I confirmed that the issue is reproducible in 0.9.3-rc1, and that it is fixed with (y)our changes.
    
    I'm +1 with the suggested changes.


> Storm-Kafka spout keeps sending fetch requests with invalid offset
> ------------------------------------------------------------------
>
>                 Key: STORM-511
>                 URL: https://issues.apache.org/jira/browse/STORM-511
>             Project: Apache Storm
>          Issue Type: Bug
>    Affects Versions: 0.9.3
>            Reporter: Viktor Taranenko
>              Labels: kafka
>             Fix For: 0.9.3
>
>
> With default behaviour (KafkaConfig.useStartOffsetTimeIfOffsetOutOfRange == true) when Kafka returns the error about offset being out of range, storm.kafka.KafkaUtils.fetchMessages tries to fix offset in local scope and retry fetch request. But if there are no more messages appeared under that specified partition it will never update the PartitionManager, but keep sending tons of requests with invalid offset to Kafka broker. On both sides Storm and Kafka logs grow extremely quick during that time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)