You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/04/01 22:43:00 UTC

[jira] [Work logged] (BEAM-12011) WindowFn.getOutputTime break joins, adds needless complexity

     [ https://issues.apache.org/jira/browse/BEAM-12011?focusedWorklogId=575799&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-575799 ]

ASF GitHub Bot logged work on BEAM-12011:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 01/Apr/21 22:42
            Start Date: 01/Apr/21 22:42
    Worklog Time Spent: 10m 
      Work Description: boyuanzz commented on pull request #14268:
URL: https://github.com/apache/beam/pull/14268#issuecomment-812215546


   A high level question regarding to 
   
   > it was added as response to a real user problem: sliding windows + EARLIEST timestamp combiner delays downstream aggregations a lot
   but when that problem occurred, timestamp combiner EARLIEST was the default; with the change to how lateness is defined we changed the default to END_OF_WINDOW
   
   With this PR, what will happen to the real user problem mentioned above?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 575799)
    Time Spent: 2.5h  (was: 2h 20m)

> WindowFn.getOutputTime break joins, adds needless complexity
> ------------------------------------------------------------
>
>                 Key: BEAM-12011
>                 URL: https://issues.apache.org/jira/browse/BEAM-12011
>             Project: Beam
>          Issue Type: Bug
>          Components: beam-model
>            Reporter: Kenneth Knowles
>            Assignee: Kenneth Knowles
>            Priority: P2
>          Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> See https://lists.apache.org/thread.html/r99d0efa70f83815e17a1dbfb94effd16b38176046ac8ffbfe371c895%40%3Cdev.beam.apache.org%3E
> The consensus is that we really shouldn't be producing weird output, even if it causes a watermark problem we should solve it another way. It breaks CoGBK joins, and new runner authors have even missed the existence of the need to call into a UDF (hence implementing SlidingWindows the natural way instead of this strange way)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)