[jira] [Commented] (FLINK-8689) Add runtime support of distinct filter using MapView

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16372422#comment-16372422 ] 

ASF GitHub Bot commented on FLINK-8689:
---------------------------------------

GitHub user walterddr opened a pull request:

    https://github.com/apache/flink/pull/5555

    [FLINK-8689][table]Add runtime support of distinct filter using MapView for codegen

    
    ## What is the purpose of the change
    
    - Adding in runtime support using distinct aggregation delegate to support distinct filtering using MapView. This is only fixing the currently broken over-window aggregate with distinct filter, however this change is meant to be used by other distinct operations on datastream such as group window aggregate, group aggregate, etc.
    
    
    ## Brief change log
    
      - Adding a new DistinctAggDelegateFunction and DistinctAccumulator that encapsulates any real aggregate function.
      - change codegen to specifically generate distinct filter before invoking the actual aggregate function.
      - adding in more codegen specifically for using delegates on merge and reset.
    
    ## Verifying this change
    
    This change added tests and can be verified as follows:
    
      - Added over-window unit-test to verify generated plan before codegen.
      - Added integration tests for testing distinct over-window aggregate end-to-end.
    
    ## Does this pull request potentially affect one of the following parts:
    
    no
    
    ## Documentation
    
    this does not expose and additional external facing functionality, which will come in separated PR.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/walterddr/flink FLINK-8689

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/5555.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5555
    
----
commit cb853e6d1adfbb04819a234f8fbced32a8ffb21f
Author: Rong Rong <ro...@...>
Date:   2018-02-21T21:40:29Z

    support distinct agg function delegate using mapview accumulator in runtime environment

commit fff7fa48f07b350f6fd4565f58d6a35af88faa50
Author: Rong Rong <ro...@...>
Date:   2018-02-21T21:43:02Z

    add over window aggregate and unbounded aggregate ITCase

commit 03fc641ac01af6867d2516263bcea0ac96f1802c
Author: Rong Rong <wa...@...>
Date:   2018-02-22T05:15:26Z

    adding in overwindow logical test cases to verify distinct modifier

----


> Add runtime support of distinct filter using MapView 
> -----------------------------------------------------
>
>                 Key: FLINK-8689
>                 URL: https://issues.apache.org/jira/browse/FLINK-8689
>             Project: Flink
>          Issue Type: Sub-task
>            Reporter: Rong Rong
>            Assignee: Rong Rong
>            Priority: Major
>
> This ticket should cover distinct aggregate function support to codegen for *AggregateCall*, where *isDistinct* fields is set to true.
> This can be verified using the following SQL, which is not currently producing correct results.
> {code:java}
> SELECT
>   a,
>   SUM(b) OVER (PARTITION BY a ORDER BY proctime ROWS BETWEEN 5 PRECEDING AND CURRENT ROW)
> FROM
>   MyTable{code}
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)