You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@calcite.apache.org by "Jia-Xuan Liu (Jira)" <ji...@apache.org> on 2023/04/12 08:21:00 UTC
[jira] [Created] (CALCITE-5643) Query with CROSS JOIN UNNEST(array) executed by BigQuery adaptor isn't supported
Jia-Xuan Liu created CALCITE-5643:
-------------------------------------
Summary: Query with CROSS JOIN UNNEST(array) executed by BigQuery adaptor isn't supported
Key: CALCITE-5643
URL: https://issues.apache.org/jira/browse/CALCITE-5643
Project: Calcite
Issue Type: Bug
Affects Versions: 1.34.0
Reporter: Jia-Xuan Liu
Given a query like
{code:sql}
WITH
tmp1 (custkey, name, orders) AS (
SELECT
c_custkey
, c_name
, array_agg(o_orderkey ORDER BY o_orderkey ASC) orders
FROM
(bq.customer
LEFT JOIN bq.orders ON (o_custkey = c_custkey))
GROUP BY c_custkey, c_name
)
SELECT
t.custkey
, t.name
, array_agg(o.o_orderstatus ORDER BY o.o_orderkey ASC)
FROM
((tmp1 t
CROSS JOIN UNNEST(t.orders) u (uc))
LEFT JOIN bq.orders o ON (u.uc = o.o_orderkey))
GROUP BY t.custkey, t.name
{code}
`bq` is a JdbcSchema connected with BigQuery.
It fails with exception:
{code:java}
Caused by: java.lang.IllegalStateException: Unable to implement EnumerableAggregate(group=[{0, 1}], EXPR$2=[ARRAY_AGG($5) WITHIN GROUP ([4])]): rowcount = 251.2225, cumulative cost = {7580.5628125 rows, 32182.2752199209 cpu, 0.0 io}, id = 394
EnumerableMergeJoin(condition=[=($3, $4)], joinType=[left]): rowcount = 2512.225, cumulative cost = {7297.9375 rows, 32182.2752199209 cpu, 0.0 io}, id = 392
EnumerableSort(sort0=[$3], dir0=[ASC]): rowcount = 158.5, cumulative cost = {4247.2125 rows, 23621.968885142334 cpu, 0.0 io}, id = 383
EnumerableCorrelate(correlation=[$cor0], joinType=[inner], requiredColumns=[{2}]): rowcount = 158.5, cumulative cost = {4088.7125 rows, 1140.15 cpu, 0.0 io}, id = 381
EnumerableAggregate(group=[{0, 1}], orders=[ARRAY_AGG($2) WITHIN GROUP ([2])]): rowcount = 158.5, cumulative cost = {2281.8125 rows, 680.5 cpu, 0.0 io}, id = 372
JdbcToEnumerableConverter: rowcount = 1585.0, cumulative cost = {2103.5 rows, 680.5 cpu, 0.0 io}, id = 370
JdbcJoin(condition=[=($3, $0)], joinType=[left]): rowcount = 1585.0, cumulative cost = {1945.0 rows, 522.0 cpu, 0.0 io}, id = 368
JdbcProject(c_custkey=[$0], c_name=[$1]): rowcount = 100.0, cumulative cost = {180.0 rows, 261.0 cpu, 0.0 io}, id = 363
JdbcTableScan(table=[[bq, customer]]): rowcount = 100.0, cumulative cost = {100.0 rows, 101.0 cpu, 0.0 io}, id = 1
JdbcProject(o_orderkey=[$0], o_custkey=[$1]): rowcount = 100.0, cumulative cost = {180.0 rows, 261.0 cpu, 0.0 io}, id = 366
JdbcTableScan(table=[[bq, orders]]): rowcount = 100.0, cumulative cost = {100.0 rows, 101.0 cpu, 0.0 io}, id = 3
EnumerableUncollect: rowcount = 1.0, cumulative cost = {2.9000000000000004 rows, 2.9000000000000004 cpu, 0.0 io}, id = 379
JdbcToEnumerableConverter: rowcount = 1.0, cumulative cost = {1.9000000000000001 rows, 1.9000000000000001 cpu, 0.0 io}, id = 377
JdbcProject(orders=[$cor0.orders]): rowcount = 1.0, cumulative cost = {1.8 rows, 1.8 cpu, 0.0 io}, id = 375
JdbcValues(tuples=[[{ 0 }]]): rowcount = 1.0, cumulative cost = {1.0 rows, 1.0 cpu, 0.0 io}, id = 263
JdbcToEnumerableConverter: rowcount = 100.0, cumulative cost = {280.0 rows, 8560.306334778565 cpu, 0.0 io}, id = 390
JdbcSort(sort0=[$0], dir0=[ASC]): rowcount = 100.0, cumulative cost = {270.0 rows, 8550.306334778565 cpu, 0.0 io}, id = 388
JdbcProject(o_orderkey=[$0], o_orderstatus=[$2]): rowcount = 100.0, cumulative cost = {180.0 rows, 261.0 cpu, 0.0 io}, id = 386
JdbcTableScan(table=[[bq, orders]]): rowcount = 100.0, cumulative cost = {100.0 rows, 101.0 cpu, 0.0 io}, id = 3
at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:117)
at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:112)
at org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1159)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:324)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)
at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:665)
at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:519)
at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:487)
at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:236)
at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:621)
at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:677)
at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:157)
... 2 more
Suppressed: java.lang.UnsupportedOperationException
at org.apache.calcite.rel.rel2sql.SqlImplementor$Context.getAliasContext(SqlImplementor.java:987)
at org.apache.calcite.rel.rel2sql.SqlImplementor$Context.toSql(SqlImplementor.java:657)
at org.apache.calcite.rel.rel2sql.RelToSqlConverter.visit(RelToSqlConverter.java:454)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
at java.base/java.lang.reflect.Method.invoke(Method.java:577)
at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:532)
at org.apache.calcite.rel.rel2sql.RelToSqlConverter.dispatch(RelToSqlConverter.java:139)
at org.apache.calcite.rel.rel2sql.RelToSqlConverter.visitInput(RelToSqlConverter.java:147)
at org.apache.calcite.rel.rel2sql.SqlImplementor.visitInput(SqlImplementor.java:216)
at org.apache.calcite.rel.rel2sql.SqlImplementor.visitInput(SqlImplementor.java:204)
at org.apache.calcite.rel.rel2sql.SqlImplementor.visitRoot(SqlImplementor.java:180)
at org.apache.calcite.adapter.jdbc.JdbcToEnumerableConverter.generateSql(JdbcToEnumerableConverter.java:364)
at org.apache.calcite.adapter.jdbc.JdbcToEnumerableConverter.implement(JdbcToEnumerableConverter.java:108)
at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.visitChild(EnumerableRelImplementor.java:107)
at org.apache.calcite.adapter.enumerable.EnumerableUncollect.implement(EnumerableUncollect.java:84)
at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.visitChild(EnumerableRelImplementor.java:107)
at org.apache.calcite.adapter.enumerable.EnumerableCorrelate.implement(EnumerableCorrelate.java:140)
at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.visitChild(EnumerableRelImplementor.java:107)
at org.apache.calcite.adapter.enumerable.EnumerableSort.implement(EnumerableSort.java:74)
at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.visitChild(EnumerableRelImplementor.java:107)
at org.apache.calcite.adapter.enumerable.EnumerableMergeJoin.implement(EnumerableMergeJoin.java:412)
at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.visitChild(EnumerableRelImplementor.java:107)
at org.apache.calcite.adapter.enumerable.EnumerableAggregate.implement(EnumerableAggregate.java:105)
at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:114)
... 13 more
Suppressed: java.lang.Throwable: Error while converting RelNode to SqlNode:
JdbcProject(orders=[$cor0.orders])
JdbcValues(tuples=[[{ 0 }]])
at org.apache.calcite.util.Util.throwAsRuntime(Util.java:973)
at org.apache.calcite.rel.rel2sql.SqlImplementor.visitRoot(SqlImplementor.java:182)
... 26 more
{code}
I used JDBC adaptor to connect with BigQuery (Simba BigQuery JDBC). Created the Calcite connection with `jdbc:calcite:fun=bigquery`.
I also tried a similar SQL without BigQuery table, as followed
{code:sql}
with Book as (
SELECT *
FROM
(
VALUES
ROW (1, 'book1', 1)
, ROW (2, 'book2', 2)
, ROW (3, 'book3', 1)
) Book (bookId, name, authorId)
),
People as (
SELECT *
FROM (
VALUES
ROW (1, 'user1', array[1,3]),
ROW (2, 'user2', array[2])
) People (userId, name, books)
)
select b.bookId, b.name from People p
cross join unnest(p.books) u(uc)
left join Book b on u.uc = b.bookId
{code}
It works fine.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)