[jira] [Commented] (FLINK-5719) Let LatencyMarkers completely bypass operators / chains

["Tzu-Li (Gordon) Tai (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15853960#comment-15853960 ] 

Tzu-Li (Gordon) Tai commented on FLINK-5719:
--------------------------------------------

One note:
Currently, the {{LatencyMarker}} class is annotated as {{@PublicEvolving}}.
I think it actually should be {{@Internal}}. Is it ok to simply change this?

> Let LatencyMarkers completely bypass operators / chains
> -------------------------------------------------------
>
>                 Key: FLINK-5719
>                 URL: https://issues.apache.org/jira/browse/FLINK-5719
>             Project: Flink
>          Issue Type: Improvement
>          Components: Streaming
>            Reporter: Tzu-Li (Gordon) Tai
>
> Currently, {{LatencyMarker}} s are forwarded through operators via the operator interfaces and methods, i.e. {{AbstractStreamOperator#processLatencyMarker()}},  {{Output#emitLatencyMarker()}}, {{OneInputStreamOperator#processLatencyMarker()}} etc.
> The main issue with this is that {{LatencyMarker}} s are essentially internal elements, and the implementation on how to handle them should be final. Exposing them through operator interfaces will allow the user to override the implementation, and also makes the user interface for operators over-complicated.
> [~aljoscha] suggested to bypass such internal stream elements from the operator to keep the operator interfaces minimal, in FLINK-5017.
> We propose a similar approach here for {{LatencyMarker}} as well. Since the chaining output calls contribute very little to the measured latency and can be ignored, instead of passing it through operator chains, latency markers can simply be passed downstream once tasks receive them.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)