[jira] [Commented] (FLINK-9962) allow users to specify TimeZone in DateTimeBucketer

["Kostas Kloudas (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-9962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565586#comment-16565586 ] 

Kostas Kloudas commented on FLINK-9962:
---------------------------------------

[~phoenixjiangnan] I agree that for *responsiveness* it would be nice to be able to roll also on event time. This is actually the reason why in the {{RollingPolicy}} interface of new {{StreamingFileSink}}, there is a method is called {{shouldRollOnProcessingTime}}. The idea was in the future to add a {{shouldRollOnEventTime}}, but this requires a bit more work also in operator level. In general, a motivating example for the new {{StreamingFileSink}} was to be able to do "windowing" but with files.
 

That said, being able to roll on event time is only a matter of *responsiveness*, not correctness. In the new {{StreamingFileSink}}, you can always use the {{Bucketer}} to assign an incoming element to the correct bucket (like the {{WindowAssigner}}), and use the inactivity interval of the default rolling policy to close the "window", i.e. the in-progress file, when no more data is arriving for that window.

Does this fit your need?

> allow users to specify TimeZone in DateTimeBucketer
> ---------------------------------------------------
>
>                 Key: FLINK-9962
>                 URL: https://issues.apache.org/jira/browse/FLINK-9962
>             Project: Flink
>          Issue Type: Improvement
>          Components: Streaming Connectors
>    Affects Versions: 1.5.1, 1.6.0
>            Reporter: Bowen Li
>            Assignee: Bowen Li
>            Priority: Major
>             Fix For: 1.7.0
>
>
> Currently {{DateTimeBucketer}} will return a bucket path by using local timezone. We should add a {{timezone}} constructor param to allow users to specify a timezone.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)