You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@calcite.apache.org by "NobiGo (via GitHub)" <gi...@apache.org> on 2024/06/08 03:12:32 UTC
[PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
NobiGo opened a new pull request, #3817:
URL: https://github.com/apache/calcite/pull/3817
…ct And SparkSQLDialect
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #3817:
URL: https://github.com/apache/calcite/pull/3817#issuecomment-2155790916
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=3817) **Quality Gate passed**
Issues
 [1 New issue](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=3817&resolved=false&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=3817&resolutions=WONTFIX)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=3817&resolved=false&sinceLeakPeriod=true)
 [100.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=3817&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=3817&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=3817)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on PR #3817:
URL: https://github.com/apache/calcite/pull/3817#issuecomment-2158989260
There are lots of changes in the quoting of identifiers.
Is this a bug that is being fixed?
If so, can it be a separate PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #3817:
URL: https://github.com/apache/calcite/pull/3817#issuecomment-2156227173
## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=3817) **Quality Gate passed**
Issues
 [1 New issue](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=3817&resolved=false&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=3817&resolutions=WONTFIX)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=3817&resolved=false&sinceLeakPeriod=true)
 [100.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=3817&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=3817&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=3817)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #3817:
URL: https://github.com/apache/calcite/pull/3817#discussion_r1644813475
##########
core/src/main/java/org/apache/calcite/util/RelToSqlConverterUtil.java:
##########
@@ -93,6 +101,41 @@ public static void unparseSparkArrayAndMap(SqlWriter writer,
writer.endList(frame);
}
+
+ /**
+ * Rewrite SINGLE_VALUE use SparkSQL Mode.
+ *
+ * <blockquote><pre>
+ * CASE COUNT(*)
+ * WHEN 0 THEN NULL
+ * WHEN 1 THEN MIN(<result>)
+ * ELSE (SELECT NULL UNION ALL SELECT NULL))
Review Comment:
Please add the explanations you gave here as comments in the code.
It will make maintaining the code much easier.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "YiwenWu (via GitHub)" <gi...@apache.org>.
YiwenWu commented on code in PR #3817:
URL: https://github.com/apache/calcite/pull/3817#discussion_r1642057291
##########
core/src/main/java/org/apache/calcite/util/RelToSqlConverterUtil.java:
##########
@@ -93,6 +101,41 @@ public static void unparseSparkArrayAndMap(SqlWriter writer,
writer.endList(frame);
}
+
+ /**
+ * Rewrite SINGLE_VALUE use SparkSQL Mode.
+ *
+ * <blockquote><pre>
+ * CASE COUNT(*)
+ * WHEN 0 THEN NULL
+ * WHEN 1 THEN MIN(<result>)
+ * ELSE (SELECT NULL UNION ALL SELECT NULL))
Review Comment:
> > 1. SELECT NULL UNION ALL SELECT NULL
>
> this is intentional to make sure when the subquery include more then one value ,This SQL will throw exception.
>
> 2: Suggestion: Use COUNT(field) instead of COUNT(*)
>
> Please check [CALCITE-6430](https://issues.apache.org/jira/browse/CALCITE-6430).
got it, thanks. Based on count(*) and union all, the result of the subquery is guaranteed to be only one row.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on PR #3817:
URL: https://github.com/apache/calcite/pull/3817#issuecomment-2176601325
why is this a draft?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "NobiGo (via GitHub)" <gi...@apache.org>.
NobiGo commented on code in PR #3817:
URL: https://github.com/apache/calcite/pull/3817#discussion_r1639121901
##########
core/src/main/java/org/apache/calcite/util/RelToSqlConverterUtil.java:
##########
@@ -93,6 +101,41 @@ public static void unparseSparkArrayAndMap(SqlWriter writer,
writer.endList(frame);
}
+
+ /**
+ * Rewrite SINGLE_VALUE use SparkSQL Mode.
+ *
+ * <blockquote><pre>
+ * CASE COUNT(*)
+ * WHEN 0 THEN NULL
+ * WHEN 1 THEN MIN(<result>)
+ * ELSE (SELECT NULL UNION ALL SELECT NULL))
Review Comment:
> 1. SELECT NULL UNION ALL SELECT NULL
this is intentional to make sure when the subquery include more then one value ,This SQL will throw exception.
2: Suggestion: Use COUNT(field) instead of COUNT(*)
Please check CALCITE-6430.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "YiwenWu (via GitHub)" <gi...@apache.org>.
YiwenWu commented on code in PR #3817:
URL: https://github.com/apache/calcite/pull/3817#discussion_r1639114996
##########
core/src/main/java/org/apache/calcite/util/RelToSqlConverterUtil.java:
##########
@@ -93,6 +101,41 @@ public static void unparseSparkArrayAndMap(SqlWriter writer,
writer.endList(frame);
}
+
+ /**
+ * Rewrite SINGLE_VALUE use SparkSQL Mode.
+ *
+ * <blockquote><pre>
+ * CASE COUNT(*)
+ * WHEN 0 THEN NULL
+ * WHEN 1 THEN MIN(<result>)
+ * ELSE (SELECT NULL UNION ALL SELECT NULL))
Review Comment:
1. A problem: `SELECT NULL UNION ALL SELECT NULL` will return two NULL values instead of one.
2. Suggestion: Use `COUNT(field)` instead of `COUNT(*)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "NobiGo (via GitHub)" <gi...@apache.org>.
NobiGo commented on PR #3817:
URL: https://github.com/apache/calcite/pull/3817#issuecomment-2177067073
> why is this a draft?
Hi @mihaibudiu . I have add comment in the JIRA. This SQL can't run as expected in Spark And Hive.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "NobiGo (via GitHub)" <gi...@apache.org>.
NobiGo commented on PR #3817:
URL: https://github.com/apache/calcite/pull/3817#issuecomment-2161690527
> Can you make a separate issue about quoting then? Too many unrelated changes in this PR.
Please check the CALCITE-6434.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "NobiGo (via GitHub)" <gi...@apache.org>.
NobiGo commented on PR #3817:
URL: https://github.com/apache/calcite/pull/3817#issuecomment-2159567167
> There are lots of changes in the quoting of identifiers. Is this a bug that is being fixed? If so, can it be a separate PR?
Yes,If we don't add the quating, Spark will parse failed the alias `$f0`, So we need add this. And Others database dialect almost have added this. Of course, We can separate this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "NobiGo (via GitHub)" <gi...@apache.org>.
NobiGo closed pull request #3817: [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale…
URL: https://github.com/apache/calcite/pull/3817
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on PR #3817:
URL: https://github.com/apache/calcite/pull/3817#issuecomment-2159568382
Can you make a separate issue about quoting then?
Too many unrelated changes in this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #3817:
URL: https://github.com/apache/calcite/pull/3817#discussion_r1640155876
##########
core/src/main/java/org/apache/calcite/util/RelToSqlConverterUtil.java:
##########
@@ -93,6 +101,41 @@ public static void unparseSparkArrayAndMap(SqlWriter writer,
writer.endList(frame);
}
+
+ /**
+ * Rewrite SINGLE_VALUE use SparkSQL Mode.
+ *
+ * <blockquote><pre>
+ * CASE COUNT(*)
+ * WHEN 0 THEN NULL
+ * WHEN 1 THEN MIN(<result>)
+ * ELSE (SELECT NULL UNION ALL SELECT NULL))
Review Comment:
You should add a comment in the code about the UNION ALL. Your intent is not obvious at all.
I am not sure how the runtime error will look; if the error will mention a UNION ALL operator, the users may be confused, since there is no UNION ALL in the original query, but I guess there isn't much you can do about this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org