You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2019/05/11 05:45:00 UTC
[jira] [Commented] (KAFKA-8347) Choose next record to process by
timestamp
[ https://issues.apache.org/jira/browse/KAFKA-8347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16837764#comment-16837764 ]
ASF GitHub Bot commented on KAFKA-8347:
---------------------------------------
ableegoldman commented on pull request #6719: KAFKA-8347: Choose next record to process by timestamp
URL: https://github.com/apache/kafka/pull/6719
When choosing the next record to process, we should look at the head record's timestamp of each partition and choose the lowest rather than choosing the lowest of the partition's streamtime.
This change effectively makes RecordQueue return the timestamp of the head record rather than its streamtime. Streamtime is removed (replaced) from RecordQueue as it was only being tracked in order to choose the next partition to poll from.
Will add some unit tests soon
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Choose next record to process by timestamp
> ------------------------------------------
>
> Key: KAFKA-8347
> URL: https://issues.apache.org/jira/browse/KAFKA-8347
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Sophie Blee-Goldman
> Priority: Major
>
> Currently PartitionGroup will determine the next record to process by choosing the partition with the lowest stream time. However if a partition contains out of order data its stream time may be significantly larger than the timestamp of the next record. The next record should instead be chosen as the record with the lowest timestamp across all partitions, regardless of which partition it comes from or what its partition time is.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)