You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Mark Lewis <ma...@outlook.com> on 2025/10/15 14:36:24 UTC
Regression in Calcite 1.41.0
Hi community,
I am making use of the Calcite SQLParser and SQLToRelSqlToRelConverter to convert SQL statements from the TPC-DS benchmark suite. In preparation to adopt the forthcoming Calcite 1.41.0, I tried using a version of Calcite I built locally from the current development codebase. I see failures that did not exist with Calcite 1.40.0 on TPC-DS queries 40, 67 and 80; all on CASE statements, with the underlying failure in SqlValidatorImpl.getValidatedNodeType():
Query 40 fails on:
CASE WHEN `CR_REFUNDED_CASH` IS NOT NULL THEN `CR_REFUNDED_CASH` ELSE 0 END
with the underlying error:
java.lang.UnsupportedOperationException: class org.apache.calcite.sql.SqlBasicCall: `CR_REFUNDED_CASH`
Query 67 fails on:
CASE WHEN `SS_SALES_PRICE` * `SS_QUANTITY` IS NOT NULL THEN `SS_SALES_PRICE` * `SS_QUANTITY` ELSE 0 END
with the underlying error:
java.lang.UnsupportedOperationException: class org.apache.calcite.sql.SqlBasicCall: `SS_SALES_PRICE` * `SS_QUANTITY`
Query 80 fails on:
CASE WHEN `SR_RETURN_AMT` IS NOT NULL THEN `SR_RETURN_AMT` ELSE 0 END
with the underlying error:
ava.lang.UnsupportedOperationException: class org.apache.calcite.sql.SqlBasicCall: `SR_RETURN_AMT`
This regression appears to be introduced in commit 12e7d621bbb19aa32cb45329cb84532115037b7a ([CALCITE-7044] Add internal operator CAST NOT NULL to enhance rewrite COALESCE operator). Since that commit, the SqlNode kind at the point of failure in each case has become SqlBasicCall with a CAST NOT NULL operator, and no node type is identified, which is a failure condition.
Regards,
Mark.
Re:Re: Regression in Calcite 1.41.0
Posted by 我 <cz...@163.com>.
I attempted to construct TPC-DS tables and data using WITH clauses in a test[1], but I'm uncertain if it can simulate the query scenario properly. I selected q80 for verification and didn't reproduce the error mentioned in the email. As Mihai mentioned, Calcite has many configurations. Could it be that certain configurations are causing the issue? Therefore, it would be best to provide a reproducible test case based on Calcite's current code to facilitate problem analysis.
[1] https://github.com/apache/calcite/pull/4582
在 2025-10-16 00:03:37，"Ruben Q L" <ru...@gmail.com> 写道：
>Definitely looks like a blocker.
>As others have said, we would need a Jira ticket and, ideally, a
>minimalistic unit test reproducing the issue.
>
>
>On Wed, Oct 15, 2025 at 4:36 PM Mihai Budiu <mb...@gmail.com> wrote:
>
>> Sounds like a blocking problem.
>>
>> Can you provide a reproduction? Calcite has many configuration settings.
>> In our tests we compile TPCDS without issues, but our configuration is not
>> the standard one.
>>
>> Mihai
>>
>>
>>
>> ________________________________
>> From: 我 <cz...@163.com>
>> Sent: Wednesday, October 15, 2025 8:34 AM
>> To: dev <de...@calcite.apache.org>
>> Subject: Re: Regression in Calcite 1.41.0
>>
>> Could you log this issue in Jira? It would be even better if you could
>> also provide a minimal test case!
>>
>> Best,
>> Zhen Chen
>>
>>
>>
>> ---- Replied Message ----
>> | From | Mark Lewis<ma...@outlook.com> |
>> | Date | 10/15/2025 22:36 |
>> | To | dev@calcite.apache.org |
>> | Cc | |
>> | Subject | Regression in Calcite 1.41.0 |
>> Hi community,
>>
>> I am making use of the Calcite SQLParser and SQLToRelSqlToRelConverter to
>> convert SQL statements from the TPC-DS benchmark suite. In preparation to
>> adopt the forthcoming Calcite 1.41.0, I tried using a version of Calcite I
>> built locally from the current development codebase. I see failures that
>> did not exist with Calcite 1.40.0 on TPC-DS queries 40, 67 and 80; all on
>> CASE statements, with the underlying failure in
>> SqlValidatorImpl.getValidatedNodeType():
>>
>> Query 40 fails on:
>>
>> CASE WHEN `CR_REFUNDED_CASH` IS NOT NULL THEN `CR_REFUNDED_CASH` ELSE 0 END
>>
>> with the underlying error:
>>
>> java.lang.UnsupportedOperationException: class
>> org.apache.calcite.sql.SqlBasicCall: `CR_REFUNDED_CASH`
>>
>> Query 67 fails on:
>>
>> CASE WHEN `SS_SALES_PRICE` * `SS_QUANTITY` IS NOT NULL THEN
>> `SS_SALES_PRICE` * `SS_QUANTITY` ELSE 0 END
>>
>> with the underlying error:
>>
>> java.lang.UnsupportedOperationException: class
>> org.apache.calcite.sql.SqlBasicCall: `SS_SALES_PRICE` * `SS_QUANTITY`
>>
>> Query 80 fails on:
>>
>> CASE WHEN `SR_RETURN_AMT` IS NOT NULL THEN `SR_RETURN_AMT` ELSE 0 END
>>
>> with the underlying error:
>>
>> ava.lang.UnsupportedOperationException: class
>> org.apache.calcite.sql.SqlBasicCall: `SR_RETURN_AMT`
>>
>> This regression appears to be introduced in commit
>> 12e7d621bbb19aa32cb45329cb84532115037b7a ([CALCITE-7044] Add internal
>> operator CAST NOT NULL to enhance rewrite COALESCE operator). Since that
>> commit, the SqlNode kind at the point of failure in each case has become
>> SqlBasicCall with a CAST NOT NULL operator, and no node type is identified,
>> which is a failure condition.
>>
>> Regards,
>>
>> Mark.
>>
Re: Regression in Calcite 1.41.0
Posted by Francis Chuang <fr...@apache.org>.
Mark,
I have not seen your request for a Jira account. Can you please try
applying again here: https://issues.apache.org/jira/issues/Dashboard.jspa
Francis
On 16/10/2025 9:33 pm, Mark Lewis wrote:
> Hi Ruben,
>
> I don't have an Apache Jira account so am not able to raise a Jira ticket. I did apply for an account yesterday using the self-serve form but still waiting.
>
> I actually hit the failures running tests within substrait-java, so this is not a completely standard configuration either:
>
> https://github.com/substrait-io/substrait-java/blob/e0f453059e5c8d07e0cf303bd76d19158f21bed2/isthmus/src/test/java/io/substrait/isthmus/TpcdsQueryTest.java#L33
>
> Here is the complete stacktrace for the failure on TPC-DS query 40:
>
> Caused by: java.lang.RuntimeException: while converting CASE WHEN CAST(`D_DATE` AS DATE) < CAST('2000-03-11' AS DATE) THEN `CS_SALES_PRICE` - CASE WHEN `CR_REFUNDED_CASH` IS NOT NULL THEN `CR_REFUNDED_CASH` ELSE 0 END ELSE 0 END
> at org.apache.calcite.sql2rel.ReflectiveConvertletTable.lambda$registerNodeTypeMethod$1(ReflectiveConvertletTable.java:99)
> at org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:62)
> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5930)
> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5093)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5744)
> at org.apache.calcite.sql2rel.AggConverter.translateAgg(AggConverter.java:471)
> at org.apache.calcite.sql2rel.AggConverter.translateAgg(AggConverter.java:338)
> at org.apache.calcite.sql2rel.AggConverter.visit(AggConverter.java:322)
> at org.apache.calcite.sql2rel.AggConverter$1.visit(AggConverter.java:177)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
> at org.apache.calcite.sql2rel.AggConverter.visit(AggConverter.java:329)
> at org.apache.calcite.sql2rel.AggConverter$1.visit(AggConverter.java:177)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
> at org.apache.calcite.sql2rel.SqlToRelConverter.createAggImpl(SqlToRelConverter.java:3587)
> at org.apache.calcite.sql2rel.SqlToRelConverter.convertAgg(SqlToRelConverter.java:3528)
> at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectList(SqlToRelConverter.java:4767)
> at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:811)
> at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:737)
> at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3943)
> at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:630)
> at io.substrait.isthmus.calcite.rel.DdlSqlToRelConverter.handleNonDdl(DdlSqlToRelConverter.java:50)
> at io.substrait.isthmus.calcite.rel.DdlSqlToRelConverter.visit(DdlSqlToRelConverter.java:46)
> at io.substrait.isthmus.calcite.rel.DdlSqlToRelConverter.visit(DdlSqlToRelConverter.java:15)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
> at io.substrait.isthmus.sql.SubstraitSqlToCalcite.lambda$convert$0(SubstraitSqlToCalcite.java:140)
> at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
> at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:720)
> at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
> at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
> at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
> at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
> at io.substrait.isthmus.sql.SubstraitSqlToCalcite.convert(SubstraitSqlToCalcite.java:147)
> at io.substrait.isthmus.sql.SubstraitSqlToCalcite.convertQueries(SubstraitSqlToCalcite.java:115)
> at io.substrait.isthmus.sql.SubstraitSqlToCalcite.convertQueries(SubstraitSqlToCalcite.java:89)
> at io.substrait.isthmus.SqlToSubstrait.convert(SqlToSubstrait.java:56)
> at io.substrait.isthmus.PlanTestBase.toSubstraitPlan(PlanTestBase.java:356)
> at io.substrait.isthmus.TpcdsQueryTest.toSubstraitPlan(TpcdsQueryTest.java:61)
> at io.substrait.isthmus.TpcdsQueryTest.lambda$testQuery$0(TpcdsQueryTest.java:34)
> at org.junit.jupiter.api.AssertDoesNotThrow.assertDoesNotThrow(AssertDoesNotThrow.java:71)
> ... 52 more
> Caused by: java.lang.reflect.InvocationTargetException
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at org.apache.calcite.sql2rel.ReflectiveConvertletTable.lambda$registerNodeTypeMethod$1(ReflectiveConvertletTable.java:95)
> ... 92 more
> Caused by: java.lang.RuntimeException: while converting CASE WHEN `CR_REFUNDED_CASH` IS NOT NULL THEN `CR_REFUNDED_CASH` ELSE 0 END
> at org.apache.calcite.sql2rel.ReflectiveConvertletTable.lambda$registerNodeTypeMethod$1(ReflectiveConvertletTable.java:99)
> at org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:62)
> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5930)
> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5093)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5744)
> at org.apache.calcite.sql2rel.StandardConvertletTable.convertOperands(StandardConvertletTable.java:1286)
> at org.apache.calcite.sql2rel.StandardConvertletTable.convertOperands(StandardConvertletTable.java:1278)
> at org.apache.calcite.sql2rel.StandardConvertletTable.convertCall(StandardConvertletTable.java:1184)
> at org.apache.calcite.sql2rel.StandardConvertletTable.lambda$new$2(StandardConvertletTable.java:163)
> at org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:62)
> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5930)
> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5093)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5744)
> at org.apache.calcite.sql2rel.StandardConvertletTable.convertCase(StandardConvertletTable.java:635)
> ... 97 more
> Caused by: java.lang.reflect.InvocationTargetException
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at org.apache.calcite.sql2rel.ReflectiveConvertletTable.lambda$registerNodeTypeMethod$1(ReflectiveConvertletTable.java:95)
> ... 112 more
> Caused by: java.lang.UnsupportedOperationException: class org.apache.calcite.sql.SqlBasicCall: `CR_REFUNDED_CASH`
> at org.apache.calcite.util.Util.needToImplement(Util.java:1112)
> at org.apache.calcite.sql.validate.SqlValidatorImpl.getValidatedNodeType(SqlValidatorImpl.java:1891)
> at org.apache.calcite.sql2rel.StandardConvertletTable.lambda$new$13(StandardConvertletTable.java:304)
> at org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:62)
> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5930)
> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5093)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5744)
> at org.apache.calcite.sql2rel.StandardConvertletTable.convertCase(StandardConvertletTable.java:635)
> ... 117 more
>
> It might be that something is wrong in the code paths that the stacktrace shows in the way the SqlNode objects are handled, but it seems that the problem stems from the creation of the SqlNode structure, which is much more standard Calcite usage:
>
> https://github.com/substrait-io/substrait-java/blob/e0f453059e5c8d07e0cf303bd76d19158f21bed2/isthmus/src/main/java/io/substrait/isthmus/sql/SubstraitSqlStatementParser.java#L32-L35
>
> Here we get the SqlBasicCall (with a cast operation) for the `CR_REFUNDED_CASH` SqlIdentifier.
>
> I will try to create a simpler reproduction to make diagnosis easier.
>
> Regards,
>
> Mark.
> ________________________________
> From: Ruben Q L <ru...@gmail.com>
> Sent: 16 October 2025 10:05
> To: dev@calcite.apache.org <de...@calcite.apache.org>; mark.s.lewis@outlook.com <ma...@outlook.com>
> Subject: Re: Regression in Calcite 1.41.0
>
> @Mark, could you please provide more details for reproducibility? (details about your system, Calcite configuration, full stack-trace, etc)
> Also, if you're not already subscribed to the dev list, I encourage you to do so (as explained in [1]) in order to be up-to-date with this (and other) discussions.
>
> Best,
> Ruben
>
> [1] https://calcite.apache.org/community/#mailing-lists
>
>
> On Wed, Oct 15, 2025 at 5:03 PM Ruben Q L <ru...@gmail.com>> wrote:
> Definitely looks like a blocker.
> As others have said, we would need a Jira ticket and, ideally, a minimalistic unit test reproducing the issue.
>
>
> On Wed, Oct 15, 2025 at 4:36 PM Mihai Budiu <mb...@gmail.com>> wrote:
> Sounds like a blocking problem.
>
> Can you provide a reproduction? Calcite has many configuration settings.
> In our tests we compile TPCDS without issues, but our configuration is not the standard one.
>
> Mihai
>
>
>
> ________________________________
> From: 我 <cz...@163.com>>
> Sent: Wednesday, October 15, 2025 8:34 AM
> To: dev <de...@calcite.apache.org>>
> Subject: Re: Regression in Calcite 1.41.0
>
> Could you log this issue in Jira? It would be even better if you could also provide a minimal test case!
>
> Best,
> Zhen Chen
>
>
>
> ---- Replied Message ----
> | From | Mark Lewis<ma...@outlook.com>> |
> | Date | 10/15/2025 22:36 |
> | To | dev@calcite.apache.org<ma...@calcite.apache.org> |
> | Cc | |
> | Subject | Regression in Calcite 1.41.0 |
> Hi community,
>
> I am making use of the Calcite SQLParser and SQLToRelSqlToRelConverter to convert SQL statements from the TPC-DS benchmark suite. In preparation to adopt the forthcoming Calcite 1.41.0, I tried using a version of Calcite I built locally from the current development codebase. I see failures that did not exist with Calcite 1.40.0 on TPC-DS queries 40, 67 and 80; all on CASE statements, with the underlying failure in SqlValidatorImpl.getValidatedNodeType():
>
> Query 40 fails on:
>
> CASE WHEN `CR_REFUNDED_CASH` IS NOT NULL THEN `CR_REFUNDED_CASH` ELSE 0 END
>
> with the underlying error:
>
> java.lang.UnsupportedOperationException: class org.apache.calcite.sql.SqlBasicCall: `CR_REFUNDED_CASH`
>
> Query 67 fails on:
>
> CASE WHEN `SS_SALES_PRICE` * `SS_QUANTITY` IS NOT NULL THEN `SS_SALES_PRICE` * `SS_QUANTITY` ELSE 0 END
>
> with the underlying error:
>
> java.lang.UnsupportedOperationException: class org.apache.calcite.sql.SqlBasicCall: `SS_SALES_PRICE` * `SS_QUANTITY`
>
> Query 80 fails on:
>
> CASE WHEN `SR_RETURN_AMT` IS NOT NULL THEN `SR_RETURN_AMT` ELSE 0 END
>
> with the underlying error:
>
> ava.lang.UnsupportedOperationException: class org.apache.calcite.sql.SqlBasicCall: `SR_RETURN_AMT`
>
> This regression appears to be introduced in commit 12e7d621bbb19aa32cb45329cb84532115037b7a ([CALCITE-7044] Add internal operator CAST NOT NULL to enhance rewrite COALESCE operator). Since that commit, the SqlNode kind at the point of failure in each case has become SqlBasicCall with a CAST NOT NULL operator, and no node type is identified, which is a failure condition.
>
> Regards,
>
> Mark.
Re: Regression in Calcite 1.41.0
Posted by Mark Lewis <Ma...@outlook.com>.
Hi Ruben,
I don't have an Apache Jira account so am not able to raise a Jira ticket. I did apply for an account yesterday using the self-serve form but still waiting.
I actually hit the failures running tests within substrait-java, so this is not a completely standard configuration either:
https://github.com/substrait-io/substrait-java/blob/e0f453059e5c8d07e0cf303bd76d19158f21bed2/isthmus/src/test/java/io/substrait/isthmus/TpcdsQueryTest.java#L33
Here is the complete stacktrace for the failure on TPC-DS query 40:
Caused by: java.lang.RuntimeException: while converting CASE WHEN CAST(`D_DATE` AS DATE) < CAST('2000-03-11' AS DATE) THEN `CS_SALES_PRICE` - CASE WHEN `CR_REFUNDED_CASH` IS NOT NULL THEN `CR_REFUNDED_CASH` ELSE 0 END ELSE 0 END
at org.apache.calcite.sql2rel.ReflectiveConvertletTable.lambda$registerNodeTypeMethod$1(ReflectiveConvertletTable.java:99)
at org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:62)
at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5930)
at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5093)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5744)
at org.apache.calcite.sql2rel.AggConverter.translateAgg(AggConverter.java:471)
at org.apache.calcite.sql2rel.AggConverter.translateAgg(AggConverter.java:338)
at org.apache.calcite.sql2rel.AggConverter.visit(AggConverter.java:322)
at org.apache.calcite.sql2rel.AggConverter$1.visit(AggConverter.java:177)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
at org.apache.calcite.sql2rel.AggConverter.visit(AggConverter.java:329)
at org.apache.calcite.sql2rel.AggConverter$1.visit(AggConverter.java:177)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
at org.apache.calcite.sql2rel.SqlToRelConverter.createAggImpl(SqlToRelConverter.java:3587)
at org.apache.calcite.sql2rel.SqlToRelConverter.convertAgg(SqlToRelConverter.java:3528)
at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectList(SqlToRelConverter.java:4767)
at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:811)
at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:737)
at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3943)
at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:630)
at io.substrait.isthmus.calcite.rel.DdlSqlToRelConverter.handleNonDdl(DdlSqlToRelConverter.java:50)
at io.substrait.isthmus.calcite.rel.DdlSqlToRelConverter.visit(DdlSqlToRelConverter.java:46)
at io.substrait.isthmus.calcite.rel.DdlSqlToRelConverter.visit(DdlSqlToRelConverter.java:15)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
at io.substrait.isthmus.sql.SubstraitSqlToCalcite.lambda$convert$0(SubstraitSqlToCalcite.java:140)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:720)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
at io.substrait.isthmus.sql.SubstraitSqlToCalcite.convert(SubstraitSqlToCalcite.java:147)
at io.substrait.isthmus.sql.SubstraitSqlToCalcite.convertQueries(SubstraitSqlToCalcite.java:115)
at io.substrait.isthmus.sql.SubstraitSqlToCalcite.convertQueries(SubstraitSqlToCalcite.java:89)
at io.substrait.isthmus.SqlToSubstrait.convert(SqlToSubstrait.java:56)
at io.substrait.isthmus.PlanTestBase.toSubstraitPlan(PlanTestBase.java:356)
at io.substrait.isthmus.TpcdsQueryTest.toSubstraitPlan(TpcdsQueryTest.java:61)
at io.substrait.isthmus.TpcdsQueryTest.lambda$testQuery$0(TpcdsQueryTest.java:34)
at org.junit.jupiter.api.AssertDoesNotThrow.assertDoesNotThrow(AssertDoesNotThrow.java:71)
... 52 more
Caused by: java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.calcite.sql2rel.ReflectiveConvertletTable.lambda$registerNodeTypeMethod$1(ReflectiveConvertletTable.java:95)
... 92 more
Caused by: java.lang.RuntimeException: while converting CASE WHEN `CR_REFUNDED_CASH` IS NOT NULL THEN `CR_REFUNDED_CASH` ELSE 0 END
at org.apache.calcite.sql2rel.ReflectiveConvertletTable.lambda$registerNodeTypeMethod$1(ReflectiveConvertletTable.java:99)
at org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:62)
at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5930)
at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5093)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5744)
at org.apache.calcite.sql2rel.StandardConvertletTable.convertOperands(StandardConvertletTable.java:1286)
at org.apache.calcite.sql2rel.StandardConvertletTable.convertOperands(StandardConvertletTable.java:1278)
at org.apache.calcite.sql2rel.StandardConvertletTable.convertCall(StandardConvertletTable.java:1184)
at org.apache.calcite.sql2rel.StandardConvertletTable.lambda$new$2(StandardConvertletTable.java:163)
at org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:62)
at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5930)
at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5093)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5744)
at org.apache.calcite.sql2rel.StandardConvertletTable.convertCase(StandardConvertletTable.java:635)
... 97 more
Caused by: java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.calcite.sql2rel.ReflectiveConvertletTable.lambda$registerNodeTypeMethod$1(ReflectiveConvertletTable.java:95)
... 112 more
Caused by: java.lang.UnsupportedOperationException: class org.apache.calcite.sql.SqlBasicCall: `CR_REFUNDED_CASH`
at org.apache.calcite.util.Util.needToImplement(Util.java:1112)
at org.apache.calcite.sql.validate.SqlValidatorImpl.getValidatedNodeType(SqlValidatorImpl.java:1891)
at org.apache.calcite.sql2rel.StandardConvertletTable.lambda$new$13(StandardConvertletTable.java:304)
at org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:62)
at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5930)
at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5093)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:175)
at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5744)
at org.apache.calcite.sql2rel.StandardConvertletTable.convertCase(StandardConvertletTable.java:635)
... 117 more
It might be that something is wrong in the code paths that the stacktrace shows in the way the SqlNode objects are handled, but it seems that the problem stems from the creation of the SqlNode structure, which is much more standard Calcite usage:
https://github.com/substrait-io/substrait-java/blob/e0f453059e5c8d07e0cf303bd76d19158f21bed2/isthmus/src/main/java/io/substrait/isthmus/sql/SubstraitSqlStatementParser.java#L32-L35
Here we get the SqlBasicCall (with a cast operation) for the `CR_REFUNDED_CASH` SqlIdentifier.
I will try to create a simpler reproduction to make diagnosis easier.
Regards,
Mark.
________________________________
From: Ruben Q L <ru...@gmail.com>
Sent: 16 October 2025 10:05
To: dev@calcite.apache.org <de...@calcite.apache.org>; mark.s.lewis@outlook.com <ma...@outlook.com>
Subject: Re: Regression in Calcite 1.41.0
@Mark, could you please provide more details for reproducibility? (details about your system, Calcite configuration, full stack-trace, etc)
Also, if you're not already subscribed to the dev list, I encourage you to do so (as explained in [1]) in order to be up-to-date with this (and other) discussions.
Best,
Ruben
[1] https://calcite.apache.org/community/#mailing-lists
On Wed, Oct 15, 2025 at 5:03 PM Ruben Q L <ru...@gmail.com>> wrote:
Definitely looks like a blocker.
As others have said, we would need a Jira ticket and, ideally, a minimalistic unit test reproducing the issue.
On Wed, Oct 15, 2025 at 4:36 PM Mihai Budiu <mb...@gmail.com>> wrote:
Sounds like a blocking problem.
Can you provide a reproduction? Calcite has many configuration settings.
In our tests we compile TPCDS without issues, but our configuration is not the standard one.
Mihai
________________________________
From: 我 <cz...@163.com>>
Sent: Wednesday, October 15, 2025 8:34 AM
To: dev <de...@calcite.apache.org>>
Subject: Re: Regression in Calcite 1.41.0
Could you log this issue in Jira? It would be even better if you could also provide a minimal test case!
Best,
Zhen Chen
---- Replied Message ----
| From | Mark Lewis<ma...@outlook.com>> |
| Date | 10/15/2025 22:36 |
| To | dev@calcite.apache.org<ma...@calcite.apache.org> |
| Cc | |
| Subject | Regression in Calcite 1.41.0 |
Hi community,
I am making use of the Calcite SQLParser and SQLToRelSqlToRelConverter to convert SQL statements from the TPC-DS benchmark suite. In preparation to adopt the forthcoming Calcite 1.41.0, I tried using a version of Calcite I built locally from the current development codebase. I see failures that did not exist with Calcite 1.40.0 on TPC-DS queries 40, 67 and 80; all on CASE statements, with the underlying failure in SqlValidatorImpl.getValidatedNodeType():
Query 40 fails on:
CASE WHEN `CR_REFUNDED_CASH` IS NOT NULL THEN `CR_REFUNDED_CASH` ELSE 0 END
with the underlying error:
java.lang.UnsupportedOperationException: class org.apache.calcite.sql.SqlBasicCall: `CR_REFUNDED_CASH`
Query 67 fails on:
CASE WHEN `SS_SALES_PRICE` * `SS_QUANTITY` IS NOT NULL THEN `SS_SALES_PRICE` * `SS_QUANTITY` ELSE 0 END
with the underlying error:
java.lang.UnsupportedOperationException: class org.apache.calcite.sql.SqlBasicCall: `SS_SALES_PRICE` * `SS_QUANTITY`
Query 80 fails on:
CASE WHEN `SR_RETURN_AMT` IS NOT NULL THEN `SR_RETURN_AMT` ELSE 0 END
with the underlying error:
ava.lang.UnsupportedOperationException: class org.apache.calcite.sql.SqlBasicCall: `SR_RETURN_AMT`
This regression appears to be introduced in commit 12e7d621bbb19aa32cb45329cb84532115037b7a ([CALCITE-7044] Add internal operator CAST NOT NULL to enhance rewrite COALESCE operator). Since that commit, the SqlNode kind at the point of failure in each case has become SqlBasicCall with a CAST NOT NULL operator, and no node type is identified, which is a failure condition.
Regards,
Mark.
Re: Regression in Calcite 1.41.0
Posted by Ruben Q L <ru...@gmail.com>.
@Mark, could you please provide more details for reproducibility? (details
about your system, Calcite configuration, full stack-trace, etc)
Also, if you're not already subscribed to the dev list, I encourage you to
do so (as explained in [1]) in order to be up-to-date with this (and other)
discussions.
Best,
Ruben
[1] https://calcite.apache.org/community/#mailing-lists
On Wed, Oct 15, 2025 at 5:03 PM Ruben Q L <ru...@gmail.com> wrote:
> Definitely looks like a blocker.
> As others have said, we would need a Jira ticket and, ideally, a
> minimalistic unit test reproducing the issue.
>
>
> On Wed, Oct 15, 2025 at 4:36 PM Mihai Budiu <mb...@gmail.com> wrote:
>
>> Sounds like a blocking problem.
>>
>> Can you provide a reproduction? Calcite has many configuration settings.
>> In our tests we compile TPCDS without issues, but our configuration is
>> not the standard one.
>>
>> Mihai
>>
>>
>>
>> ________________________________
>> From: 我 <cz...@163.com>
>> Sent: Wednesday, October 15, 2025 8:34 AM
>> To: dev <de...@calcite.apache.org>
>> Subject: Re: Regression in Calcite 1.41.0
>>
>> Could you log this issue in Jira? It would be even better if you could
>> also provide a minimal test case!
>>
>> Best,
>> Zhen Chen
>>
>>
>>
>> ---- Replied Message ----
>> | From | Mark Lewis<ma...@outlook.com> |
>> | Date | 10/15/2025 22:36 |
>> | To | dev@calcite.apache.org |
>> | Cc | |
>> | Subject | Regression in Calcite 1.41.0 |
>> Hi community,
>>
>> I am making use of the Calcite SQLParser and SQLToRelSqlToRelConverter to
>> convert SQL statements from the TPC-DS benchmark suite. In preparation to
>> adopt the forthcoming Calcite 1.41.0, I tried using a version of Calcite I
>> built locally from the current development codebase. I see failures that
>> did not exist with Calcite 1.40.0 on TPC-DS queries 40, 67 and 80; all on
>> CASE statements, with the underlying failure in
>> SqlValidatorImpl.getValidatedNodeType():
>>
>> Query 40 fails on:
>>
>> CASE WHEN `CR_REFUNDED_CASH` IS NOT NULL THEN `CR_REFUNDED_CASH` ELSE 0
>> END
>>
>> with the underlying error:
>>
>> java.lang.UnsupportedOperationException: class
>> org.apache.calcite.sql.SqlBasicCall: `CR_REFUNDED_CASH`
>>
>> Query 67 fails on:
>>
>> CASE WHEN `SS_SALES_PRICE` * `SS_QUANTITY` IS NOT NULL THEN
>> `SS_SALES_PRICE` * `SS_QUANTITY` ELSE 0 END
>>
>> with the underlying error:
>>
>> java.lang.UnsupportedOperationException: class
>> org.apache.calcite.sql.SqlBasicCall: `SS_SALES_PRICE` * `SS_QUANTITY`
>>
>> Query 80 fails on:
>>
>> CASE WHEN `SR_RETURN_AMT` IS NOT NULL THEN `SR_RETURN_AMT` ELSE 0 END
>>
>> with the underlying error:
>>
>> ava.lang.UnsupportedOperationException: class
>> org.apache.calcite.sql.SqlBasicCall: `SR_RETURN_AMT`
>>
>> This regression appears to be introduced in commit
>> 12e7d621bbb19aa32cb45329cb84532115037b7a ([CALCITE-7044] Add internal
>> operator CAST NOT NULL to enhance rewrite COALESCE operator). Since that
>> commit, the SqlNode kind at the point of failure in each case has become
>> SqlBasicCall with a CAST NOT NULL operator, and no node type is identified,
>> which is a failure condition.
>>
>> Regards,
>>
>> Mark.
>>
>
Re: Regression in Calcite 1.41.0
Posted by Ruben Q L <ru...@gmail.com>.
Definitely looks like a blocker.
As others have said, we would need a Jira ticket and, ideally, a
minimalistic unit test reproducing the issue.
On Wed, Oct 15, 2025 at 4:36 PM Mihai Budiu <mb...@gmail.com> wrote:
> Sounds like a blocking problem.
>
> Can you provide a reproduction? Calcite has many configuration settings.
> In our tests we compile TPCDS without issues, but our configuration is not
> the standard one.
>
> Mihai
>
>
>
> ________________________________
> From: 我 <cz...@163.com>
> Sent: Wednesday, October 15, 2025 8:34 AM
> To: dev <de...@calcite.apache.org>
> Subject: Re: Regression in Calcite 1.41.0
>
> Could you log this issue in Jira? It would be even better if you could
> also provide a minimal test case!
>
> Best,
> Zhen Chen
>
>
>
> ---- Replied Message ----
> | From | Mark Lewis<ma...@outlook.com> |
> | Date | 10/15/2025 22:36 |
> | To | dev@calcite.apache.org |
> | Cc | |
> | Subject | Regression in Calcite 1.41.0 |
> Hi community,
>
> I am making use of the Calcite SQLParser and SQLToRelSqlToRelConverter to
> convert SQL statements from the TPC-DS benchmark suite. In preparation to
> adopt the forthcoming Calcite 1.41.0, I tried using a version of Calcite I
> built locally from the current development codebase. I see failures that
> did not exist with Calcite 1.40.0 on TPC-DS queries 40, 67 and 80; all on
> CASE statements, with the underlying failure in
> SqlValidatorImpl.getValidatedNodeType():
>
> Query 40 fails on:
>
> CASE WHEN `CR_REFUNDED_CASH` IS NOT NULL THEN `CR_REFUNDED_CASH` ELSE 0 END
>
> with the underlying error:
>
> java.lang.UnsupportedOperationException: class
> org.apache.calcite.sql.SqlBasicCall: `CR_REFUNDED_CASH`
>
> Query 67 fails on:
>
> CASE WHEN `SS_SALES_PRICE` * `SS_QUANTITY` IS NOT NULL THEN
> `SS_SALES_PRICE` * `SS_QUANTITY` ELSE 0 END
>
> with the underlying error:
>
> java.lang.UnsupportedOperationException: class
> org.apache.calcite.sql.SqlBasicCall: `SS_SALES_PRICE` * `SS_QUANTITY`
>
> Query 80 fails on:
>
> CASE WHEN `SR_RETURN_AMT` IS NOT NULL THEN `SR_RETURN_AMT` ELSE 0 END
>
> with the underlying error:
>
> ava.lang.UnsupportedOperationException: class
> org.apache.calcite.sql.SqlBasicCall: `SR_RETURN_AMT`
>
> This regression appears to be introduced in commit
> 12e7d621bbb19aa32cb45329cb84532115037b7a ([CALCITE-7044] Add internal
> operator CAST NOT NULL to enhance rewrite COALESCE operator). Since that
> commit, the SqlNode kind at the point of failure in each case has become
> SqlBasicCall with a CAST NOT NULL operator, and no node type is identified,
> which is a failure condition.
>
> Regards,
>
> Mark.
>
Re: Regression in Calcite 1.41.0
Posted by Mihai Budiu <mb...@gmail.com>.
Sounds like a blocking problem.
Can you provide a reproduction? Calcite has many configuration settings.
In our tests we compile TPCDS without issues, but our configuration is not the standard one.
Mihai
________________________________
From: 我 <cz...@163.com>
Sent: Wednesday, October 15, 2025 8:34 AM
To: dev <de...@calcite.apache.org>
Subject: Re: Regression in Calcite 1.41.0
Could you log this issue in Jira? It would be even better if you could also provide a minimal test case!
Best,
Zhen Chen
---- Replied Message ----
| From | Mark Lewis<ma...@outlook.com> |
| Date | 10/15/2025 22:36 |
| To | dev@calcite.apache.org |
| Cc | |
| Subject | Regression in Calcite 1.41.0 |
Hi community,
I am making use of the Calcite SQLParser and SQLToRelSqlToRelConverter to convert SQL statements from the TPC-DS benchmark suite. In preparation to adopt the forthcoming Calcite 1.41.0, I tried using a version of Calcite I built locally from the current development codebase. I see failures that did not exist with Calcite 1.40.0 on TPC-DS queries 40, 67 and 80; all on CASE statements, with the underlying failure in SqlValidatorImpl.getValidatedNodeType():
Query 40 fails on:
CASE WHEN `CR_REFUNDED_CASH` IS NOT NULL THEN `CR_REFUNDED_CASH` ELSE 0 END
with the underlying error:
java.lang.UnsupportedOperationException: class org.apache.calcite.sql.SqlBasicCall: `CR_REFUNDED_CASH`
Query 67 fails on:
CASE WHEN `SS_SALES_PRICE` * `SS_QUANTITY` IS NOT NULL THEN `SS_SALES_PRICE` * `SS_QUANTITY` ELSE 0 END
with the underlying error:
java.lang.UnsupportedOperationException: class org.apache.calcite.sql.SqlBasicCall: `SS_SALES_PRICE` * `SS_QUANTITY`
Query 80 fails on:
CASE WHEN `SR_RETURN_AMT` IS NOT NULL THEN `SR_RETURN_AMT` ELSE 0 END
with the underlying error:
ava.lang.UnsupportedOperationException: class org.apache.calcite.sql.SqlBasicCall: `SR_RETURN_AMT`
This regression appears to be introduced in commit 12e7d621bbb19aa32cb45329cb84532115037b7a ([CALCITE-7044] Add internal operator CAST NOT NULL to enhance rewrite COALESCE operator). Since that commit, the SqlNode kind at the point of failure in each case has become SqlBasicCall with a CAST NOT NULL operator, and no node type is identified, which is a failure condition.
Regards,
Mark.
Re: Regression in Calcite 1.41.0
Posted by 我 <cz...@163.com>.
Could you log this issue in Jira? It would be even better if you could also provide a minimal test case!
Best,
Zhen Chen
---- Replied Message ----
| From | Mark Lewis<ma...@outlook.com> |
| Date | 10/15/2025 22:36 |
| To | dev@calcite.apache.org |
| Cc | |
| Subject | Regression in Calcite 1.41.0 |
Hi community,
I am making use of the Calcite SQLParser and SQLToRelSqlToRelConverter to convert SQL statements from the TPC-DS benchmark suite. In preparation to adopt the forthcoming Calcite 1.41.0, I tried using a version of Calcite I built locally from the current development codebase. I see failures that did not exist with Calcite 1.40.0 on TPC-DS queries 40, 67 and 80; all on CASE statements, with the underlying failure in SqlValidatorImpl.getValidatedNodeType():
Query 40 fails on:
CASE WHEN `CR_REFUNDED_CASH` IS NOT NULL THEN `CR_REFUNDED_CASH` ELSE 0 END
with the underlying error:
java.lang.UnsupportedOperationException: class org.apache.calcite.sql.SqlBasicCall: `CR_REFUNDED_CASH`
Query 67 fails on:
CASE WHEN `SS_SALES_PRICE` * `SS_QUANTITY` IS NOT NULL THEN `SS_SALES_PRICE` * `SS_QUANTITY` ELSE 0 END
with the underlying error:
java.lang.UnsupportedOperationException: class org.apache.calcite.sql.SqlBasicCall: `SS_SALES_PRICE` * `SS_QUANTITY`
Query 80 fails on:
CASE WHEN `SR_RETURN_AMT` IS NOT NULL THEN `SR_RETURN_AMT` ELSE 0 END
with the underlying error:
ava.lang.UnsupportedOperationException: class org.apache.calcite.sql.SqlBasicCall: `SR_RETURN_AMT`
This regression appears to be introduced in commit 12e7d621bbb19aa32cb45329cb84532115037b7a ([CALCITE-7044] Add internal operator CAST NOT NULL to enhance rewrite COALESCE operator). Since that commit, the SqlNode kind at the point of failure in each case has become SqlBasicCall with a CAST NOT NULL operator, and no node type is identified, which is a failure condition.
Regards,
Mark.