You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apex.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/07/18 11:29:20 UTC
[jira] [Commented] (APEXMALHAR-2158) Duplication of data emitted
when the Kafka Input Operator(0.8 version) redeploys
[ https://issues.apache.org/jira/browse/APEXMALHAR-2158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382108#comment-15382108 ]
ASF GitHub Bot commented on APEXMALHAR-2158:
--------------------------------------------
GitHub user chaithu14 opened a pull request:
https://github.com/apache/apex-malhar/pull/346
APEXMALHAR-2158 Fixed the duplication of messages emitted issue when the Kafka Input operator redeployed
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/chaithu14/incubator-apex-malhar APEXMALHAR-2158-Dpdata-recovery
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/apex-malhar/pull/346.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #346
----
commit 96e0535e9c934e18ae10d71b9ee9fe9c5b9d6805
Author: Chaitanya <ch...@datatorrent.com>
Date: 2016-07-18T11:27:04Z
APEXMALHAR-2158 Fixed the duplication of messages emitted issue when the Kafka Input operator redeployed
----
> Duplication of data emitted when the Kafka Input Operator(0.8 version) redeploys
> --------------------------------------------------------------------------------
>
> Key: APEXMALHAR-2158
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2158
> Project: Apache Apex Malhar
> Issue Type: Bug
> Reporter: Chaitanya
> Assignee: Chaitanya
>
> Kafka Input Operator failure recovery depends on offsetStats map.
> "offsetStats" map is updated whenever the message is emitted from operator.
> When the operator goes down and comes back, operator sets this map to the consumer and start reading from these offsets. Consumer reads the last emitted message again and operator emits it.
> Solution: Increment the offsets before setting to the consumer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)