[jira] [Comment Edited] (CALCITE-3055) TableScan type mismatch in VolcanoPlanner

["Danny Chan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CALCITE-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16846378#comment-16846378 ] 

Danny Chan edited comment on CALCITE-3055 at 5/23/19 2:31 AM:
--------------------------------------------------------------

+1 to add the RelDataType pair back, we can not remove it if we can not make sure that all the node's digest are implemented correctly. It would be a vital function regression if the planner produce wrong plan just because of the type difference of the nodes with same digest.

AFAIK, Flink has also encounter this problem after upgrade Calcite to version 1.19.0.


was (Author: danny0405):
+1 to add the RelDataType pair back, we can not remove it if we can not make sure that all the node's digest are implemented correctly. I would be a vital function regression if the planner produce wrong plan because of the type difference of the nodes. AFAIK, Flink has also encounter this problem after upgrade Calcite to version 1.19.0.

> TableScan type mismatch in VolcanoPlanner
> -----------------------------------------
>
>                 Key: CALCITE-3055
>                 URL: https://issues.apache.org/jira/browse/CALCITE-3055
>             Project: Calcite
>          Issue Type: Bug
>    Affects Versions: 1.19.0
>            Reporter: Bohdan Kazydub
>            Assignee: Danny Chan
>            Priority: Blocker
>             Fix For: 1.20.0
>
>
> In Drill with Calcite updated to 1.19.0 we observe following problem:
> the following query
> {code}
> CREATE VIEW nation_view_testunionall_expandable_star AS select n_name, n_nationkey FROM cp.`tpch/nation.parquet`;
> (SELECT n_comment, n_regionkey FROM cp.`tpch/nation.parquet` LIMIT 5) UNION ALL (SELECT * FROM nation_view_testunionall_expandable_star  LIMIT 5)
> {code}
> fails with
> {code}
> 15:46:06.005 [2333c7f1-89f3-0b80-8369-68e24895729e:foreman] ERROR o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR: AssertionError: Type mismatch:
> left:
> RecordType(DYNAMIC_STAR **, ANY n_comment, ANY n_regionkey) NOT NULL
> right:
> RecordType(DYNAMIC_STAR **, ANY n_comment, ANY n_regionkey, ANY n_name, ANY n_nationkey) NOT NULL
> {code}
> when Volcano planner is registering rel nodes. The initial plan is the following:
> {code}
> LogicalUnion(all=[true]): rowcount = 10.0, cumulative cost = \{520.0 rows, 940.755032994728 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 74
>   LogicalSort(fetch=[5]): rowcount = 5.0, cumulative cost = \{205.0 rows, 365.377516497364 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 68
>     LogicalProject(n_comment=[$1], n_regionkey=[$2]): rowcount = 100.0, cumulative cost = \{200.0 rows, 301.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 67
>       EnumerableTableScan(table=[[cp, tpch/nation.parquet]]): rowcount = 100.0, cumulative cost = \{100.0 rows, 101.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 29
>   LogicalSort(fetch=[5]): rowcount = 5.0, cumulative cost = \{305.0 rows, 565.377516497364 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 73
>     LogicalProject(n_name=[$0], n_nationkey=[$1]): rowcount = 100.0, cumulative cost = \{300.0 rows, 501.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 72
>       LogicalProject(n_name=[$3], n_nationkey=[$4]): rowcount = 100.0, cumulative cost = \{200.0 rows, 301.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 71
>         EnumerableTableScan(table=[[cp, tpch/nation.parquet]]): rowcount = 100.0, cumulative cost = \{100.0 rows, 101.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 34
> {code}
> It contains two EnumerableTableScan operators with different row types specified in the error message above. In the scope of CALCITE-2454 was removed rel data type from the key in the \{{VolcanoPlanner.mapDigestToRel}} map.
> To fix this, `{{TableScan` should include rowType}} item in {{AbstractRelNode#explainTerms(RelWriter)}} method because in case of dynamic tables there is the same digest for different operators. The same thing was done for Values operator and literals in conditions.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)