You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/05/06 22:30:12 UTC
[jira] [Commented] (KAFKA-3658) Incorrect validation check on
maintenance period with join window size
[ https://issues.apache.org/jira/browse/KAFKA-3658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15274845#comment-15274845 ]
ASF GitHub Bot commented on KAFKA-3658:
---------------------------------------
GitHub user guozhangwang opened a pull request:
https://github.com/apache/kafka/pull/1337
KAFKA-3658: Validate retention period be longer than window size
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/guozhangwang/kafka K3658
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/1337.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1337
----
commit 0835c7e185d9477bd59f104cd69250ed856e1ed3
Author: Guozhang Wang <wa...@gmail.com>
Date: 2016-05-06T22:28:28Z
validate retention period be longer than window size
----
> Incorrect validation check on maintenance period with join window size
> ----------------------------------------------------------------------
>
> Key: KAFKA-3658
> URL: https://issues.apache.org/jira/browse/KAFKA-3658
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Reporter: Guozhang Wang
> Assignee: Guozhang Wang
> Labels: architecture
> Fix For: 0.10.0.1
>
>
> As [~hcai@pinterest.com] found out, the current validation check of {{KStreamJoinWindow}} requires the retention period to be at least twice than the join window size. This check was originally for making the segment interval to be larger than the join window size. But for windowed stream-stream join this is not necessary.
> More specifically, for example with a window size 6, and retention period 12, and num. segment 5, the segment size will be set to 3. This means after time 12, the first segment of [0, 3) will be dropped, then at time 13, a late record with timestamp (1) will not be accepted to the window store, and will not participate in the joining as well.
> The proposed change is to only require retention period to be > window size, not window size * 2.
> cc [~ymatsuda]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)