You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2021/09/03 13:50:16 UTC
[GitHub] [hive] ujc714 opened a new pull request #2616: HIVE-25498: Query with more than 32 count distinct functions returns …
ujc714 opened a new pull request #2616:
URL: https://github.com/apache/hive/pull/2616
### What changes were proposed in this pull request?
Fix a bug in HiveExpandDistinctAggregatesRule.getGroupingIdValue() which causes "COUNT(DISTINCT COL)" function returns wrong result.
### Why are the changes needed?
If there are more than 32 COUNT(DISTINCT COL)" function in a query, the values returned from HiveExpandDistinctAggregatesRule.getGroupingIdValue() overflow so these COUNT functions return 0.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
mvn test -Dtest=TestMiniTezCliDriver -Dqfile=multi_count_distinct.q
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] kasakrisz commented on pull request #2616: HIVE-25498: Query with more than 32 count distinct functions returns …
Posted by GitBox <gi...@apache.org>.
kasakrisz commented on pull request #2616:
URL: https://github.com/apache/hive/pull/2616#issuecomment-915242945
@ujc714
With this patch the maximum number of `count(distinct)` expressions can be handled by the `HiveExpandDistinctAggregatesRule` is increased to 63 from 31. But this limitation still exists. Could you please add a check here:
https://github.com/apache/hive/blob/72d860ad7721e705c830ca5f141a79e899cc86f7/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveExpandDistinctAggregatesRule.java#L115
like
```
if (numCountDistinct == 0 || numCountDistinct > 63 || aggregate.getGroupType() != Group.SIMPLE) {
return;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] pgaref commented on pull request #2616: HIVE-25498: Query with more than 32 count distinct functions returns …
Posted by GitBox <gi...@apache.org>.
pgaref commented on pull request #2616:
URL: https://github.com/apache/hive/pull/2616#issuecomment-915356626
> @ujc714
> With this patch the maximum number of `count(distinct)` expressions can be handled by the `HiveExpandDistinctAggregatesRule` is increased to 63 from 31. But this limitation still exists. Could you please add a check here:
> https://github.com/apache/hive/blob/72d860ad7721e705c830ca5f141a79e899cc86f7/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveExpandDistinctAggregatesRule.java#L115
>
> like
>
> ```
> if (numCountDistinct == 0 || numCountDistinct > 63 || aggregate.getGroupType() != Group.SIMPLE) {
> return;
> }
> ```
Agree with @kasakrisz -- an alternative would be to change **getGroupingIdValue** logic but that could be tricky.
At the end of the day we should not be limited by the number of count distinct functions so having this extra check as part of the onMatch method makes sense to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] kasakrisz merged pull request #2616: HIVE-25498: Query with more than 31 count distinct functions returns …
Posted by GitBox <gi...@apache.org>.
kasakrisz merged pull request #2616:
URL: https://github.com/apache/hive/pull/2616
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org