[GitHub] flink issue #3765: [FLINK-6373] Add runtime support for distinct aggregation...

[sunjincheng121 <gi...@git.apache.org>]

Github user sunjincheng121 commented on the issue:

    https://github.com/apache/flink/pull/3765
  
    Hi @haohui @fhueske I am very interested in `DISTINCT`,  Let me share some ideas about this:
    First up, in standard database there are two situations can using `DISTINCT` keyword. 
    *  in `SELECT Clause`, e.g.: `SELECT DISTINCT name FROM table` 
    *  in `AGG Clause`, e.g.: `COUNT([ALL|DISTINCT] expression)`,`SUM([ALL|DISTINCT] expression)`, etc. 
    
    In this post,we talk about  `AGG Clause`. The `DISTINCT` keyword tells the database system to aggregate only the distinct, or unique, values within the scope of the aggregate function. i.e. database system will deal with the `DISTINCT` keyword, and put the unique value into `AGG`. Based on this understanding, I think FLINK FRAMEWORK(not the AGG) should deal with the `DISTINCT` keyword. we do not need `DistinctAccumulator.java`. About GROUP WINDOW, I think I like analyze whether the data is duplicated in `XXXWindowFunction` and `DataSetXXXAggFunction`, And add boolean variable `isFirstTimeProcess` identifies whether the data is duplicated as a parameter of `GeneratedAggregationsFunction`. `GeneratedAggregationsFunction` process data according to `aggCall.isDistinct` and `isFirstTimeProcess`. What do you think? @haohui @fhueske 
    
    Best,
    SunJincheng


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---