You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "lsyldliu (via GitHub)" <gi...@apache.org> on 2023/06/30 04:03:33 UTC

[GitHub] [flink] lsyldliu opened a new pull request, #22914: [FLINK-32281][table-planner] Enable two-phase HashAgg default when all aggregate functions in query support adaptive local HashAgg

lsyldliu opened a new pull request, #22914:
URL: https://github.com/apache/flink/pull/22914

   ## What is the purpose of the change
   
   *For the HashAgg operator, planner currently prefers a one-phase agg when the statistic cannot be accurately estimated. In some queries of production scenarios, it may be more reasonable to choose a two-phase agg. In the TPC-DS cases, we find that for some patterns actually choosing two-stage agg, the query runtime is significantly reduced. In https://issues.apache.org/jira/browse/FLINK-30542 , we have introduced the adaptive local hashagg, which can adaptively skip aggregation when the local phase aggregation degree is relatively low, which can greatly improve the performance of two-phase aggregation in some queries. Based on the above background, in this issue, we propose to turn on two-phase agg by default for functions that support adaptive local hashagg, such as sum/count/min/max, etc., so as to exploit the ability of adpative local hashgg to improve the performance of agg query. For OFCG, if we turn on two-phaseagg by default, we can also let the local agg operator be put in
 to the fused operator, so as to enjoy the benefit from OFCG.*
   
   
   ## Brief change log
   
     - *Enable two-phase HashAgg default when all aggregate functions in query support adaptive local HashAgg*
   
   
   ## Verifying this change
   
   This change is already covered by existing tests.
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): (no)
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
     - The serializers: (no)
     - The runtime per-record code paths (performance sensitive): (no)
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
     - The S3 file system connector: (no)
   
   ## Documentation
   
     - Does this pull request introduce a new feature? (no)
     - If yes, how is the feature documented? (not documented)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #22914: [FLINK-32281][table-planner] Enable two-phase HashAgg default when all aggregate functions in query support adaptive local HashAgg

Posted by "flinkbot (via GitHub)" <gi...@apache.org>.

flinkbot commented on PR #22914:
URL: https://github.com/apache/flink/pull/22914#issuecomment-1614094241

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "d25232d1bb3fc7a1e0e170b9bc40845db923bd1c",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d25232d1bb3fc7a1e0e170b9bc40845db923bd1c",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d25232d1bb3fc7a1e0e170b9bc40845db923bd1c UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-32281][table-planner] Enable two-phase HashAgg default when all aggregate functions in query support adaptive local HashAgg [flink]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.

github-actions[bot] closed pull request #22914: [FLINK-32281][table-planner] Enable two-phase HashAgg default when all aggregate functions in query support adaptive local HashAgg
URL: https://github.com/apache/flink/pull/22914


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-32281][table-planner] Enable two-phase HashAgg default when all aggregate functions in query support adaptive local HashAgg [flink]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.

github-actions[bot] commented on PR #22914:
URL: https://github.com/apache/flink/pull/22914#issuecomment-2595360809

   This PR is being marked as stale since it has not had any activity in the last 180 days. 
   If you would like to keep this PR alive, please leave a comment asking for a review. 
   If the PR has merge conflicts, update it with the latest from the base branch.
   
   If you are having difficulty finding a reviewer, please reach out to the 
   community, contact details can be found here: https://flink.apache.org/what-is-flink/community/
   
   If this PR is no longer valid or desired, please feel free to close it. 
   If no activity occurs in the next 90 days, it will be automatically closed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-32281][table-planner] Enable two-phase HashAgg default when all aggregate functions in query support adaptive local HashAgg [flink]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.

github-actions[bot] commented on PR #22914:
URL: https://github.com/apache/flink/pull/22914#issuecomment-2774619802

   This PR has been closed since it has not had any activity in 120 days. 
   If you feel like this was a mistake, or you would like to continue working on it, 
   please feel free to re-open the PR and ask for a review.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org