[jira] [Comment Edited] (KAFKA-6323) punctuate with WALL_CLOCK_TIME triggered immediately

["Matthias J. Sax (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16299140#comment-16299140 ] 

Matthias J. Sax edited comment on KAFKA-6323 at 12/20/17 9:40 PM:
------------------------------------------------------------------

[~frederica]

We cannot simply change the interface of {{schedule()}} method -- this is a public API change and requires a KIP (https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals). If you think it's a valuable addition, feel free to do a KIP on it (including a new JIRA to cover the change).

[~guozhang]

I see your point that aligning start wall-clock time punctuations might not be as valuable as aligning stream-time ones. However, I agree with [~frederica] that if we move from {{T2 (T2 >= T1)}} to {{T2 + T}} punctuation shift into the future and I think this would be undesired behavior. For long GC pauses etc, we would just skip the corresponding punctuation similarly to the skipping behavior for stream-time in case stream-time make a larger advance that 2x punctuation interval.


was (Author: mjsax):
[~frederica]

We cannot simply change the interface of `schedule` method -- this is a public API change and requires a KIP (https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals). If you think it's a valuable addition, feel free to do a KIP on it (including a new JIRA to cover the change).

[~guozhang]

I see your point that aligning start wall-clock time punctuations might not be as valuable as aligning stream-time ones. However, I agree with [~frederica] that if we move from `T2 (T2 >= T1)` to `T2 + T` punctuation shift into the future and I think this would be undesired behavior. For long GC pauses etc, we would just skip the corresponding punctuation similarly to the skipping behavior for stream-time in case stream-time make a larger advance that 2x punctuation interval.

> punctuate with WALL_CLOCK_TIME triggered immediately
> ----------------------------------------------------
>
>                 Key: KAFKA-6323
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6323
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 1.0.0
>            Reporter: Frederic Arno
>            Assignee: Frederic Arno
>             Fix For: 1.1.0, 1.0.1
>
>
> When working on a custom Processor from which I am scheduling a punctuation using WALL_CLOCK_TIME. I've noticed that whatever the punctuation interval I set, a call to my Punctuator is always triggered immediately.
> Having a quick look at kafka-streams' code, I could find that all PunctuationSchedule's timestamps are matched against the current time in order to decide whether or not to trigger the punctuator (org.apache.kafka.streams.processor.internals.PunctuationQueue#mayPunctuate). However, I've only seen code that initializes PunctuationSchedule's timestamp to 0, which I guess is what is causing an immediate punctuation.
> At least when using WALL_CLOCK_TIME, shouldn't the PunctuationSchedule's timestamp be initialized to current time + interval?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)