[jira] [Created] (BEAM-145) OutputTimeFn#assignOutputTime overrides WindowFn#getOutputTime in unfortunate ways

["Kenneth Knowles (JIRA)" <ji...@apache.org>]

Kenneth Knowles created BEAM-145:
------------------------------------

             Summary: OutputTimeFn#assignOutputTime overrides WindowFn#getOutputTime in unfortunate ways
                 Key: BEAM-145
                 URL: https://issues.apache.org/jira/browse/BEAM-145
             Project: Beam
          Issue Type: Bug
          Components: runner-core
            Reporter: Kenneth Knowles
            Assignee: Kenneth Knowles
            Priority: Minor


Today the {{OutputTimeFn}} includes {{#assignOutputTime}}, {{#combine}}, and {{#merge}}. Together these express the grouping of timestamps, analogous to the grouping of values in a GBK / Combine, in a canonical way.

The default {{OutputTimeFn}} is provided by the {{WindowFn}}. In particular, {{SlidingWindows}} provides an {{OutputTimeFn}} that shifts input timestamps later to avoid watermark stuckness and then takes the minimum to compute the output timestamp.

The SDK additionally provides instance for "min", "max" and "end of window" output timestamps.

Unfortunately,  if one overrides the {{OutputTimeFn}} to one of these, the shifting done by {{SlidingWindows}} is lost.

This is actually only a minor problem for now, since "min" is the default, "end of window" is unaffected, and "max" has only esoteric uses.The fix is easy:

This is interrelated with another suggested change:  Since there are only three common {{OutputTimeFn}} instances, and it is a high bandwidth API, it does not seem worthwhile to leave it in userland. So it is proposed to reduce it to an enum, which would leave only the {{WindowFn}} as a userland place for timestamp adjustments. (requiring special casing for end-of-window, since it cannot be implemented without owning {{#assignOutputTime}})



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)