[jira] [Commented] (STORM-2340) KafkaSpout is blocked in AutoCommitMode

["Xu Mingmin (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/STORM-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852039#comment-15852039 ] 

Xu Mingmin commented on STORM-2340:
-----------------------------------

create pull request https://github.com/apache/storm/pull/1919 

> KafkaSpout is blocked in AutoCommitMode
> ---------------------------------------
>
>                 Key: STORM-2340
>                 URL: https://issues.apache.org/jira/browse/STORM-2340
>             Project: Apache Storm
>          Issue Type: Improvement
>          Components: storm-kafka-client
>            Reporter: Xu Mingmin
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> What's the issue?
> When Storm topology is run on 'At-Most-Once' mode, with ack executor=0, KafkaSpout cannot emit records after some time.
> What's the cause?
> Function poll() controls whether it need to pull more data from Kafka cluster. With the condition !waitingToEmit() && numUncommittedOffsets < maxUncommittedOffsets, it's always FALSE after reaching threshold, as numUncommittedOffsets is creasing and never reset on 'At-Most-Once' mode.
> What's the solution?
> I change the condition to !waitingToEmit() && (numUncommittedOffsets < maxUncommittedOffsets || consumerAutoCommitMode), that return TRUE with AutoCommitMode regarding of UnCommittedOffset, see the comments inline.
> Also, it's not required to track emitted(msgId) and numUncommittedOffsets with AutoCommitMode .



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)