You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@calcite.apache.org by "NobiGo (via GitHub)" <gi...@apache.org> on 2025/03/10 13:01:13 UTC
[PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
NobiGo opened a new pull request, #4235:
URL: https://github.com/apache/calcite/pull/4235
…ct And SparkSQLDialect
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "NobiGo (via GitHub)" <gi...@apache.org>.
NobiGo commented on code in PR #4235:
URL: https://github.com/apache/calcite/pull/4235#discussion_r2008458444
##########
core/src/main/java/org/apache/calcite/sql/dialect/HiveSqlDialect.java:
##########
@@ -166,4 +173,36 @@ public HiveSqlDialect(Context context) {
}
return super.getCastSpec(type);
}
+
+ /**
+ * Rewrite SINGLE_VALUE.
Review Comment:
Fixed.
##########
core/src/main/java/org/apache/calcite/sql/dialect/HiveSqlDialect.java:
##########
@@ -166,4 +173,36 @@ public HiveSqlDialect(Context context) {
}
return super.getCastSpec(type);
}
+
+ /**
+ * Rewrite SINGLE_VALUE.
+ *
+ * <pre>In Hive, ASSERT_TRUE(false) will throw assertion failed exception.</pre>
Review Comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "NobiGo (via GitHub)" <gi...@apache.org>.
NobiGo commented on PR #4235:
URL: https://github.com/apache/calcite/pull/4235#issuecomment-2710533389
Before this https://github.com/apache/calcite/pull/3817 Hive and Spark don't throw exceptions as we expected. It's okay now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "NobiGo (via GitHub)" <gi...@apache.org>.
NobiGo commented on PR #4235:
URL: https://github.com/apache/calcite/pull/4235#issuecomment-2743426655
@mihaibudiu Could you please review this PR? The previous PR was not executed as expected. I have tested it in our service.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "NobiGo (via GitHub)" <gi...@apache.org>.
NobiGo commented on PR #4235:
URL: https://github.com/apache/calcite/pull/4235#issuecomment-2785690741
> hi @NobiGo
>
> ```sql
> create table correlated_scalar_t1(c1 bigint, c2 bigint);
> create table correlated_scalar_t2(c1 bigint, c2 bigint);
> insert into correlated_scalar_t1 values (1,null),(null,1),(1,2), (null,2),(1,3), (2,4), (2,5), (3,3), (3,4), (20,2), (22,3), (24,4),(null,null);
> insert into correlated_scalar_t2 values (1,null),(null,1),(1,4), (1,2), (null,3), (2,4), (3,7), (3,9),(null,null),(5,1);
>
> select c1 from correlated_scalar_t1 where correlated_scalar_t1.c2 > (select c1 from correlated_scalar_t2 where correlated_scalar_t1.c1 = correlated_scalar_t2.c1 and correlated_scalar_t2.c2 < 4) order by c1;
> ```
>
> can you add this case into calicte and check correctness?
>
> ```sql
> with t1 (c1, c2) as (values (1,null),(null,1),(1,2), (null,2),(1,3), (2,4), (2,5), (3,3), (3,4), (20,2), (22,3), (24,4),(null,null)), t2 (c1, c2) as (values (1,null),(null,1),(1,4), (1,2), (null,3), (2,4), (3,7), (3,9),(null,null),(5,1)) select c1 from t1 where t1.c2 > (select c1 from t2 where t1.c1 = t2.c1 and t2.c2 < 4) order by c1;
> ```
This is an open-source project. You can freely verify it. If you encounter a new problem, you can register it in Jira.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "NobiGo (via GitHub)" <gi...@apache.org>.
NobiGo merged PR #4235:
URL: https://github.com/apache/calcite/pull/4235
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "iwanttobepowerful (via GitHub)" <gi...@apache.org>.
iwanttobepowerful commented on PR #4235:
URL: https://github.com/apache/calcite/pull/4235#issuecomment-2785312146
hi @NobiGo
```sql
create table correlated_scalar_t1(c1 bigint, c2 bigint);
create table correlated_scalar_t2(c1 bigint, c2 bigint);
insert into correlated_scalar_t1 values (1,null),(null,1),(1,2), (null,2),(1,3), (2,4), (2,5), (3,3), (3,4), (20,2), (22,3), (24,4),(null,null);
insert into correlated_scalar_t2 values (1,null),(null,1),(1,4), (1,2), (null,3), (2,4), (3,7), (3,9),(null,null),(5,1);
select c1 from correlated_scalar_t1 where correlated_scalar_t1.c2 > (select c1 from correlated_scalar_t2 where correlated_scalar_t1.c1 = correlated_scalar_t2.c1 and correlated_scalar_t2.c2 < 4) order by c1;
```
can you add this case into calicte and check correctness?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "NobiGo (via GitHub)" <gi...@apache.org>.
NobiGo commented on code in PR #4235:
URL: https://github.com/apache/calcite/pull/4235#discussion_r2008458303
##########
core/src/main/java/org/apache/calcite/sql/dialect/SparkSqlDialect.java:
##########
@@ -152,4 +159,39 @@ public SparkSqlDialect(SqlDialect.Context context) {
}
return super.getCastSpec(type);
}
+
+ /**
+ * Rewrite SINGLE_VALUE.
+ *
+ * <pre>In Spark, RAISE_ERROR("more than one value in agg SINGLE_VALUE")
+ * will throw exception.</pre>
+ *
+ * <blockquote><pre>
+ * CASE COUNT(*)
+ * WHEN 0 THEN NULL
+ * WHEN 1 THEN MIN(<result>)
+ * ELSE RAISE_ERROR("more than one value in agg SINGLE_VALUE")
+ * </pre></blockquote>
+ */
+ @Override public SqlNode rewriteSingleValueExpr(SqlNode aggCall, RelDataType relDataType) {
+ final SqlNode operand = ((SqlBasicCall) aggCall).operand(0);
Review Comment:
The current conversion here is consistent with other dialect conversions, so no adjustments have been made.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CALCITE-6431] Implement the SINGLE_VALUE aggregation in HiveSqlDiale… [calcite]
Posted by "mihaibudiu (via GitHub)" <gi...@apache.org>.
mihaibudiu commented on code in PR #4235:
URL: https://github.com/apache/calcite/pull/4235#discussion_r2008004135
##########
core/src/main/java/org/apache/calcite/sql/dialect/HiveSqlDialect.java:
##########
@@ -166,4 +173,36 @@ public HiveSqlDialect(Context context) {
}
return super.getCastSpec(type);
}
+
+ /**
+ * Rewrite SINGLE_VALUE.
Review Comment:
SINGLE_VALUE(result)
##########
core/src/main/java/org/apache/calcite/sql/dialect/HiveSqlDialect.java:
##########
@@ -166,4 +173,36 @@ public HiveSqlDialect(Context context) {
}
return super.getCastSpec(type);
}
+
+ /**
+ * Rewrite SINGLE_VALUE.
+ *
+ * <pre>In Hive, ASSERT_TRUE(false) will throw assertion failed exception.</pre>
Review Comment:
I would swap this <pre> with the next one.
It makes it look like you are solving a problem related to ASSERT_TRUE.
##########
core/src/main/java/org/apache/calcite/sql/dialect/SparkSqlDialect.java:
##########
@@ -152,4 +159,39 @@ public SparkSqlDialect(SqlDialect.Context context) {
}
return super.getCastSpec(type);
}
+
+ /**
+ * Rewrite SINGLE_VALUE.
+ *
+ * <pre>In Spark, RAISE_ERROR("more than one value in agg SINGLE_VALUE")
+ * will throw exception.</pre>
+ *
+ * <blockquote><pre>
+ * CASE COUNT(*)
+ * WHEN 0 THEN NULL
+ * WHEN 1 THEN MIN(<result>)
+ * ELSE RAISE_ERROR("more than one value in agg SINGLE_VALUE")
+ * </pre></blockquote>
+ */
+ @Override public SqlNode rewriteSingleValueExpr(SqlNode aggCall, RelDataType relDataType) {
+ final SqlNode operand = ((SqlBasicCall) aggCall).operand(0);
Review Comment:
I don't know if you care to preserve the position of the aggCall in the constructed tree.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org