You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/06/08 03:37:18 UTC
[jira] [Commented] (KAFKA-5403) Transactions system test should
dedup consumed messages by offset
[ https://issues.apache.org/jira/browse/KAFKA-5403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16042147#comment-16042147 ]
ASF GitHub Bot commented on KAFKA-5403:
---------------------------------------
GitHub user apurvam opened a pull request:
https://github.com/apache/kafka/pull/3266
KAFKA-5403: Transaction system test consumer should dedup messages by offset
Since the consumer can consume duplicate offsets due to rebalances, we should dedup consumed messages by offset in order to ensure that the test doesn't fail spuriously.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apurvam/kafka KAFKA-5403-dedup-consumed-messages-transactions-system-test
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/3266.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3266
----
commit f4ba943358aad08950e0952f60eb98843efd42f2
Author: Apurva Mehta <ap...@confluent.io>
Date: 2017-06-08T01:40:40Z
WIP
commit 5a6ac9dcc83c5bb0f62a5c387923612c5a9f1212
Author: Apurva Mehta <ap...@confluent.io>
Date: 2017-06-08T02:51:48Z
WIP commit
----
> Transactions system test should dedup consumed messages by offset
> -----------------------------------------------------------------
>
> Key: KAFKA-5403
> URL: https://issues.apache.org/jira/browse/KAFKA-5403
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.11.0.0
> Reporter: Apurva Mehta
> Assignee: Apurva Mehta
> Fix For: 0.11.0.1
>
>
> In KAFKA-5396, we saw that the consumers which verify the data in multiple topics could read the same offsets multiple times, for instance when a rebalance happens.
> This would detect spurious duplicates, causing the test to fail. We should dedup the consumed messages by offset and only fail the test if we have duplicate values for a if for a unique set of offsets.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)