You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2021/03/24 10:33:58 UTC
[GitHub] [hive] kasakrisz opened a new pull request #2112: HIVE-24929: Allow correlated exists subqueries with windowing clause
kasakrisz opened a new pull request #2112:
URL: https://github.com/apache/hive/pull/2112
### What changes were proposed in this pull request?
Loose correlated subquery restrictions: enable correlated exists/not exists subquery rewrite when subquery has windowing clause.
### Why are the changes needed?
In case of Exists/Not exists subqueries we are not interested in the result of the window function call but the existence of any record.
### Does this PR introduce _any_ user-facing change?
Yes. Currently such queries can not be executed with Hive and an error message is printed. With this patch queries will be executed and the result will be printed.
### How was this patch tested?
```
mvn test -Dtest.output.overwrite -DskipSparkTests -Dtest=TestMiniLlapLocalCliDriver -Dqfile=subquery_exists_windowfunc.q -pl itests/qtest -Pitests
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] kasakrisz merged pull request #2112: HIVE-24929: Allow correlated exists subqueries with windowing clause
Posted by GitBox <gi...@apache.org>.
kasakrisz merged pull request #2112:
URL: https://github.com/apache/hive/pull/2112
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] kasakrisz commented on a change in pull request #2112: HIVE-24929: Allow correlated exists subqueries with windowing clause
Posted by GitBox <gi...@apache.org>.
kasakrisz commented on a change in pull request #2112:
URL: https://github.com/apache/hive/pull/2112#discussion_r601070130
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/QBSubQuery.java
##########
@@ -606,7 +606,8 @@ void subqueryRestrictionsCheck(RowResolver parentQueryRR,
/*
* Restriction.14.h :: Correlated Sub Queries cannot contain Windowing clauses.
*/
- if ( hasWindowing && hasCorrelation) {
+ if (operator.getType() != SubQueryType.EXISTS && operator.getType() != SubQueryType.NOT_EXISTS &&
+ hasWindowing && hasCorrelation) {
throw new CalciteSubquerySemanticException(ASTErrorUtils.getMsg(
ErrorMsg.UNSUPPORTED_SUBQUERY_EXPRESSION.getMsg(),
subQueryAST, "Correlated Sub Queries cannot contain Windowing clauses."));
Review comment:
Yes. Thanks for pointing out. Created follow-up patch of #2039: #2118
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] jcamachor commented on a change in pull request #2112: HIVE-24929: Allow correlated exists subqueries with windowing clause
Posted by GitBox <gi...@apache.org>.
jcamachor commented on a change in pull request #2112:
URL: https://github.com/apache/hive/pull/2112#discussion_r600532604
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/QBSubQuery.java
##########
@@ -606,7 +606,8 @@ void subqueryRestrictionsCheck(RowResolver parentQueryRR,
/*
* Restriction.14.h :: Correlated Sub Queries cannot contain Windowing clauses.
*/
- if ( hasWindowing && hasCorrelation) {
+ if (operator.getType() != SubQueryType.EXISTS && operator.getType() != SubQueryType.NOT_EXISTS &&
+ hasWindowing && hasCorrelation) {
throw new CalciteSubquerySemanticException(ASTErrorUtils.getMsg(
ErrorMsg.UNSUPPORTED_SUBQUERY_EXPRESSION.getMsg(),
subQueryAST, "Correlated Sub Queries cannot contain Windowing clauses."));
Review comment:
Can we update this message as well as the comment below (I think it is outdated)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] kasakrisz commented on a change in pull request #2112: HIVE-24929: Allow correlated exists subqueries with windowing clause
Posted by GitBox <gi...@apache.org>.
kasakrisz commented on a change in pull request #2112:
URL: https://github.com/apache/hive/pull/2112#discussion_r600727171
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/QBSubQuery.java
##########
@@ -606,7 +606,8 @@ void subqueryRestrictionsCheck(RowResolver parentQueryRR,
/*
* Restriction.14.h :: Correlated Sub Queries cannot contain Windowing clauses.
*/
- if ( hasWindowing && hasCorrelation) {
+ if (operator.getType() != SubQueryType.EXISTS && operator.getType() != SubQueryType.NOT_EXISTS &&
+ hasWindowing && hasCorrelation) {
throw new CalciteSubquerySemanticException(ASTErrorUtils.getMsg(
ErrorMsg.UNSUPPORTED_SUBQUERY_EXPRESSION.getMsg(),
subQueryAST, "Correlated Sub Queries cannot contain Windowing clauses."));
Review comment:
Updated the error message.
Did you mean the comment above? `Restriction.14.h :: Correlated Sub Queries cannot contain Windowing clauses.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] jcamachor commented on a change in pull request #2112: HIVE-24929: Allow correlated exists subqueries with windowing clause
Posted by GitBox <gi...@apache.org>.
jcamachor commented on a change in pull request #2112:
URL: https://github.com/apache/hive/pull/2112#discussion_r600783271
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/QBSubQuery.java
##########
@@ -606,7 +606,8 @@ void subqueryRestrictionsCheck(RowResolver parentQueryRR,
/*
* Restriction.14.h :: Correlated Sub Queries cannot contain Windowing clauses.
*/
- if ( hasWindowing && hasCorrelation) {
+ if (operator.getType() != SubQueryType.EXISTS && operator.getType() != SubQueryType.NOT_EXISTS &&
+ hasWindowing && hasCorrelation) {
throw new CalciteSubquerySemanticException(ASTErrorUtils.getMsg(
ErrorMsg.UNSUPPORTED_SUBQUERY_EXPRESSION.getMsg(),
subQueryAST, "Correlated Sub Queries cannot contain Windowing clauses."));
Review comment:
I think `Restriction.13.m` is also out-of-date based on your latest work (not only related to this patch)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org