You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@calcite.apache.org by "Roman Churganov (Jira)" <ji...@apache.org> on 2022/04/19 18:42:00 UTC
[jira] [Updated] (CALCITE-5101) Query with LISTAGG, DISTINCT and ORDER BY
[ https://issues.apache.org/jira/browse/CALCITE-5101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roman Churganov updated CALCITE-5101:
-------------------------------------
Description:
Execute query like:
{code:sql}
SELECT C11, LISTAGG( DISTINCT C12) WITHIN GROUP ( ORDER BY C13 ) from FOO GROUP BY C11
{code}
fails with:
{noformat}
java.lang.IllegalStateException: Unable to implement EnumerableAggregate(group=[{0}], EXPR$1=[LISTAGG($1) WITHIN GROUP ([2])]): rowcount = 1.0, cumulative cost = {112.125 rows, 102.0 cpu, 0.0 io}, id = 1800
JdbcToEnumerableConverter: rowcount = 10.0, cumulative cost = {111.0 rows, 102.0 cpu, 0.0 io}, id = 1798
JdbcAggregate(group=[{1, 2}]): rowcount = 10.0, cumulative cost = {110.0 rows, 101.0 cpu, 0.0 io}, id = 1796
JdbcTableScan(table=[[TESTSCHEMA, FOO]]): rowcount = 100.0, cumulative cost = {100.0 rows, 101.0 cpu, 0.0 io}, id = 1705
at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:114)
at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:114)
at ru.roman.podd.test.UtilKt.runPlannerApiQuery(Util.kt:139)
at ru.roman.podd.test.UtilKt.runPlannerApiQuery$default(Util.kt:63)
at ru.roman.podd.test.JdbcSubQueryTest.should run LISTAGG(JdbcSubQueryTest.kt:144)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:686)
at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:205)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:201)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:137)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:71)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248)
at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211)
at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:132)
at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:71)
at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
Suppressed: java.lang.ArrayIndexOutOfBoundsException: 2
at com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:75)
at org.apache.calcite.adapter.enumerable.PhysTypeImpl.generateCollationKey(PhysTypeImpl.java:274)
at org.apache.calcite.adapter.enumerable.EnumerableAggregateBase.implementLambdaFactory(EnumerableAggregateBase.java:149)
at org.apache.calcite.adapter.enumerable.EnumerableAggregate.implement(EnumerableAggregate.java:233)
at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:111)
... 69 more
{noformat}
however queries without DISTINCT or without ORDER BY, or with ORDER BY aggregated colum works just fine
{code:sql}
SELECT C11, LISTAGG( DISTINCT C12 ) from FOO GROUP BY C11
{code}
{code:sql}
SELECT C11, LISTAGG( C12 ) WITHIN GROUP ( ORDER BY C13 ) from FOO GROUP BY C11
{code}
{code:sql}
SELECT C11, LISTAGG( DISTINCT C12 ) WITHIN GROUP ( ORDER BY C12 ) from FOO GROUP BY C11
{code}
was:
Execute query like:
{code:sql}
select C11, LISTAGG( distinct C12, ', ' )
from FOO
group by C11
{code}
{{In case of JDBC driver doesn't support LISTAGG function (Postresql), JDBC adapter creates subquery: }}
{code:sql}
SELECT "C11", "C12", ', ' AS "$f2"
FROM "FOO"
GROUP BY "C11", "C12", ', '
{code}
which cannot be executed on Postresql and gives error :
{noformat}
[42601] ERROR: non-integer constant in GROUP BY
{noformat}
> Query with LISTAGG, DISTINCT and ORDER BY
> -----------------------------------------
>
> Key: CALCITE-5101
> URL: https://issues.apache.org/jira/browse/CALCITE-5101
> Project: Calcite
> Issue Type: Bug
> Components: core
> Affects Versions: 1.29.0
> Reporter: Roman Churganov
> Priority: Blocker
>
> Execute query like:
> {code:sql}
> SELECT C11, LISTAGG( DISTINCT C12) WITHIN GROUP ( ORDER BY C13 ) from FOO GROUP BY C11
> {code}
> fails with:
> {noformat}
> java.lang.IllegalStateException: Unable to implement EnumerableAggregate(group=[{0}], EXPR$1=[LISTAGG($1) WITHIN GROUP ([2])]): rowcount = 1.0, cumulative cost = {112.125 rows, 102.0 cpu, 0.0 io}, id = 1800
> JdbcToEnumerableConverter: rowcount = 10.0, cumulative cost = {111.0 rows, 102.0 cpu, 0.0 io}, id = 1798
> JdbcAggregate(group=[{1, 2}]): rowcount = 10.0, cumulative cost = {110.0 rows, 101.0 cpu, 0.0 io}, id = 1796
> JdbcTableScan(table=[[TESTSCHEMA, FOO]]): rowcount = 100.0, cumulative cost = {100.0 rows, 101.0 cpu, 0.0 io}, id = 1705
> at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:114)
> at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:114)
> at ru.roman.podd.test.UtilKt.runPlannerApiQuery(Util.kt:139)
> at ru.roman.podd.test.UtilKt.runPlannerApiQuery$default(Util.kt:63)
> at ru.roman.podd.test.JdbcSubQueryTest.should run LISTAGG(JdbcSubQueryTest.kt:144)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:686)
> at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
> at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
> at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
> at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
> at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
> at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
> at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
> at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
> at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
> at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
> at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
> at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
> at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
> at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:205)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:201)
> at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:137)
> at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:71)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
> at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
> at java.util.ArrayList.forEach(ArrayList.java:1259)
> at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
> at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
> at java.util.ArrayList.forEach(ArrayList.java:1259)
> at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
> at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
> at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
> at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
> at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
> at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248)
> at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211)
> at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226)
> at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199)
> at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:132)
> at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:71)
> at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
> at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
> at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
> at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
> at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
> Suppressed: java.lang.ArrayIndexOutOfBoundsException: 2
> at com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:75)
> at org.apache.calcite.adapter.enumerable.PhysTypeImpl.generateCollationKey(PhysTypeImpl.java:274)
> at org.apache.calcite.adapter.enumerable.EnumerableAggregateBase.implementLambdaFactory(EnumerableAggregateBase.java:149)
> at org.apache.calcite.adapter.enumerable.EnumerableAggregate.implement(EnumerableAggregate.java:233)
> at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:111)
> ... 69 more
> {noformat}
> however queries without DISTINCT or without ORDER BY, or with ORDER BY aggregated colum works just fine
> {code:sql}
> SELECT C11, LISTAGG( DISTINCT C12 ) from FOO GROUP BY C11
> {code}
> {code:sql}
> SELECT C11, LISTAGG( C12 ) WITHIN GROUP ( ORDER BY C13 ) from FOO GROUP BY C11
> {code}
> {code:sql}
> SELECT C11, LISTAGG( DISTINCT C12 ) WITHIN GROUP ( ORDER BY C12 ) from FOO GROUP BY C11
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)