You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/11/08 23:40:00 UTC
[jira] [Commented] (KAFKA-6190) GlobalKTable never finishes
restoring when consuming transactional messages
[ https://issues.apache.org/jira/browse/KAFKA-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244941#comment-16244941 ]
ASF GitHub Bot commented on KAFKA-6190:
---------------------------------------
GitHub user alexjg opened a pull request:
https://github.com/apache/kafka/pull/4197
KAFKA-6190 GlobalKTable never finishes restoring when consuming transactional messages
Calculate offset using consumer.position() in GlobalStateManagerImp#restoreState
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/alexjg/kafka 0.11.0
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/4197.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4197
----
commit d96aac2de211ee4592e795b07d059dd4dea20f95
Author: Alex Good <al...@gmail.com>
Date: 2017-11-08T23:26:09Z
Calculate offset using consumer.position() in GlobalStateManagerImpl#restoreState
----
> GlobalKTable never finishes restoring when consuming transactional messages
> ---------------------------------------------------------------------------
>
> Key: KAFKA-6190
> URL: https://issues.apache.org/jira/browse/KAFKA-6190
> Project: Kafka
> Issue Type: Bug
> Components: clients
> Environment: Linux
> Reporter: Alex Good
>
> When creating a GlobalKTable from a topic that contains messages that were produced in a transaction the GlobalStreamThread never finishes restoring the table. This appears to be because the `GlobalStateManagerImpl#restoreState` method fails to take into account the transaction markers in it's calculation of it's offset when reading messages and so never reaches the high watermark for the topic it is restoring.
> To demonstrate the issue produce a few messages in a transaction to a topic, then attempt to restore a GlobalKTable from that topic, the store will never complete restoring.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)