You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Jingsong Lee (Jira)" <ji...@apache.org> on 2020/05/22 14:32:00 UTC
[jira] [Comment Edited] (FLINK-17878) Transient watermark attribute
should be initial at runtime in streaming file operators
[ https://issues.apache.org/jira/browse/FLINK-17878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114082#comment-17114082 ]
Jingsong Lee edited comment on FLINK-17878 at 5/22/20, 2:31 PM:
----------------------------------------------------------------
IIUC, operators only be deserialized from {{JobGraph}} which comes from user initialization. So I don't mean removing transient help watermark persistence.
But it can work for initialization we want (Long.Min_value in runtime). If you take a look to {{SinkOperator}} and others, the watermark field works like this.
It is not a big problem. In this case, we don't want to make watermark field persistent.
was (Author: lzljs3620320):
IIUC, operators only be deserialized from {{JobGraph}} which comes from user initialization. So I don't mean removing transient help watermark persistence. But it can work. If you take a look to {{SinkOperator}} and others, the watermark field works like this.
It is not a big problem. In this case, we don't want to make watermark field persistent.
> Transient watermark attribute should be initial at runtime in streaming file operators
> --------------------------------------------------------------------------------------
>
> Key: FLINK-17878
> URL: https://issues.apache.org/jira/browse/FLINK-17878
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
> Affects Versions: 1.11.0
> Reporter: xiaogang zhou
> Assignee: xiaogang zhou
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.0
>
>
> StreamingFileWriter has a
> private transient long currentWatermark = Long.MIN_VALUE;
>
> in case developer wants to create a custom bucket assigner, it will receive a currentWatermark as 0, this might be conflict with the original flink approach to handle a min_long.
>
> should we remove the transient key word?
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)