You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by "James Taylor (JIRA)" <ji...@apache.org> on 2016/06/12 07:12:21 UTC
[jira] [Updated] (PHOENIX-2988) Replace COUNT(DISTINCT...) with
COUNT(...) when possible
[ https://issues.apache.org/jira/browse/PHOENIX-2988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
James Taylor updated PHOENIX-2988:
----------------------------------
Description:
An optimization that would really benefit the SELECT COUNT(DISTINCT pkCol) case: if there's only a single COUNT(DISTINCT pkCol) and the GroupBy ends up being order preserving, you can replace the COUNT(DISTINCT pkCol) with a COUNT(pkCol) in the SELECT, HAVING, and ORDER BY clauses. That'll prevent the DistinctValueWithCountServerAggregator from being used which keeps a Map of all unique values and instead just keep a single overall count, which is all we need thanks to your DistinctPrefixFilter.
A few considerations in the implementation:
* Pass through select in the call to groupBy.compile() in QueryCompiler and change the return type to return a new select (as the SELECT, HAVING, and ORDER BY may have been rewritten). Probably easiest if the GroupBy object is just mutated in place.
* Within the groupBy.compile() call, use a visitor on the SELECT, HAVING and ORDER BY clauses to do the rewriting. You can do that by deriving a class from ParseNodeRewriter, overriding the {{visitLeave(final FunctionParseNode node, List<ParseNode> nodes)}} method to return a new COUNT parse node with the {{nodes}} passed in as children if {{node}} equals the DistinctCountParseNode that you replaced in the select statement.
* The compilation of the HAVING clause should be moved after the call to groupBy compile in QueryCompiler, like this since it may have been rewritten in the groupBy.compile call:
{code}
select = groupBy.compile(context, select, innerPlanTupleProjector);
Expression having = HavingCompiler.compile(context, select, groupBy);
{code}
was:
An optimization that would really benefit the SELECT COUNT(DISTINCT pkCol) case: if there's only a single COUNT(DISTINCT pkCol) and the GroupBy ends up being order preserving, you can replace the COUNT(DISTINCT pkCol) with a COUNT(pkCol) in the select expression nodes. That'll prevent the DistinctValueWithCountServerAggregator from being used which keeps a Map of all unique values and instead just keep a single overall count, which is all we need thanks to your DistinctPrefixFilter.
A few considerations in the implementation:
* Pass through select in the call to groupBy.compile() in QueryCompiler and you replacement the COUNT(DISTINCT ...) in place.
* The same replacements need to be done for the HAVING clause. We have a ParseNodeRewriter that'll help with that. You could do that by creating a derived class, overriding the {{visitLeave(final FunctionParseNode node, List<ParseNode> nodes)}} method to return a new COUNT parse node with the {{nodes}} passed in as children if {{node}} equals the DistinctCountParseNode that you replaced in the select statement.
* The compilation of the HAVING clause should be moved after the call to groupBy compile in QueryCompiler, like this:
{code}
groupBy = groupBy.compile(context, select, innerPlanTupleProjector);
Expression having = HavingCompiler.compile(context, select, groupBy);
{code}
> Replace COUNT(DISTINCT...) with COUNT(...) when possible
> --------------------------------------------------------
>
> Key: PHOENIX-2988
> URL: https://issues.apache.org/jira/browse/PHOENIX-2988
> Project: Phoenix
> Issue Type: Sub-task
> Reporter: James Taylor
> Fix For: 4.8.0
>
>
> An optimization that would really benefit the SELECT COUNT(DISTINCT pkCol) case: if there's only a single COUNT(DISTINCT pkCol) and the GroupBy ends up being order preserving, you can replace the COUNT(DISTINCT pkCol) with a COUNT(pkCol) in the SELECT, HAVING, and ORDER BY clauses. That'll prevent the DistinctValueWithCountServerAggregator from being used which keeps a Map of all unique values and instead just keep a single overall count, which is all we need thanks to your DistinctPrefixFilter.
> A few considerations in the implementation:
> * Pass through select in the call to groupBy.compile() in QueryCompiler and change the return type to return a new select (as the SELECT, HAVING, and ORDER BY may have been rewritten). Probably easiest if the GroupBy object is just mutated in place.
> * Within the groupBy.compile() call, use a visitor on the SELECT, HAVING and ORDER BY clauses to do the rewriting. You can do that by deriving a class from ParseNodeRewriter, overriding the {{visitLeave(final FunctionParseNode node, List<ParseNode> nodes)}} method to return a new COUNT parse node with the {{nodes}} passed in as children if {{node}} equals the DistinctCountParseNode that you replaced in the select statement.
> * The compilation of the HAVING clause should be moved after the call to groupBy compile in QueryCompiler, like this since it may have been rewritten in the groupBy.compile call:
> {code}
> select = groupBy.compile(context, select, innerPlanTupleProjector);
> Expression having = HavingCompiler.compile(context, select, groupBy);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)