You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by xiaobo <gu...@qq.com.INVALID> on 2022/03/14 01:03:54 UTC
re：applying the wrong rule for calculated columns
our runnable test is at https://github.com/guxiaobo/calcite-json-adapter/blob/main/src/test/java/org/apache/calcite/adapter/json/test/JsonSchemaTest.java
------------------ original mail ------------------
@Test
public void test1_2()throws SQLException{
Map<string, list<jsonobject="">> map = makeJsonMap();
Schema schema = new JsonSchema<>("js", map);
CalciteConnection conn = this.openConn1(schema, "js");
System.out.println("c1 + c11 = " + exeGetLong(conn, "select c1 + c11 from js.t1"));
conn.close();
}
For the above simple calculated sql, calcite throws the following error, it seems calcite refers to our Table class to return data type information for c1 + c11, which we think calcite should infer itself. calcite calls deriveRowType() of our JsonTableScan class with field [EXPR$0] which is not a field of ourtable.test1_2(org.apache.calcite.adapter.json.test.JsonSchemaTest)
Error while preparing plan [LogicalProject(EXPR$0=[+($1, $0)])
JsonTableScan(table=[[vars, vars]])
]
at com.xsmartware.pbcm.policy.CalciteBasedVarGenerator.generateVariable(CalciteBasedVarGenerator.java:65) ~[classes/:?]
at com.xsmartware.pbcm.thread.VarGenThread.doCompute(VarGenThread.java:24) ~[classes/:?]
at com.xsmartware.pbcm.thread.VarGenThread.doCompute(VarGenThread.java:10) ~[classes/:?]
at com.xsmartware.pbcm.thread.PBCMBaseThread.run(PBCMBaseThread.java:38) [classes/:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.sql.SQLException: Error while preparing plan [LogicalProject(EXPR$0=[+($1, $0)])
JsonTableScan(table=[[vars, vars]])
]
at org.apache.calcite.avatica.Helper.createException(Helper.java:56) ~[avatica-core-1.20.0.jar:1.20.0]
at org.apache.calcite.avatica.Helper.createException(Helper.java:41) ~[avatica-core-1.20.0.jar:1.20.0]
at org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement_(CalciteConnectionImpl.java:239) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.jdbc.CalciteConnectionImpl.access$100(CalciteConnectionImpl.java:101) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.jdbc.CalciteConnectionImpl$2.prepareStatement(CalciteConnectionImpl.java:188) ~[calcite-core-1.29.0.jar:1.29.0]
at com.xsmartware.common.calcite.CalciteDatabase.executeQuery(CalciteDatabase.java:109) ~[classes/:?]
at com.xsmartware.common.calcite.CalciteDatabase.executeQuery(CalciteDatabase.java:103) ~[classes/:?]
at com.xsmartware.common.calcite.CalciteDatabase.exeGetLong(CalciteDatabase.java:138) ~[classes/:?]
at com.xsmartware.pbcm.policy.CalciteBasedVarGenerator.generateVariable(CalciteBasedVarGenerator.java:43) ~[classes/:?]
... 8 more
Caused by: java.lang.RuntimeException: Error while applying rule JsonProjectTableScanRule, args [rel#500:LogicalProject.NONE.[](input=RelSubset#499,exprs=[+($1, $0)]), rel#494:JsonTableScan.ENUMERABLE.[](table=[vars, vars])]
at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:250) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.plan.volcano.IterativeRuleDriver.drive(IterativeRuleDriver.java:59) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:523) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:276) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:336) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:177) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.prepare_(CalcitePrepareImpl.java:1026) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.prepareRel(CalcitePrepareImpl.java:988) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:668) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:513) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:483) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:249) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement_(CalciteConnectionImpl.java:229) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.jdbc.CalciteConnectionImpl.access$100(CalciteConnectionImpl.java:101) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.jdbc.CalciteConnectionImpl$2.prepareStatement(CalciteConnectionImpl.java:188) ~[calcite-core-1.29.0.jar:1.29.0]
at com.xsmartware.common.calcite.CalciteDatabase.executeQuery(CalciteDatabase.java:109) ~[classes/:?]
at com.xsmartware.common.calcite.CalciteDatabase.executeQuery(CalciteDatabase.java:103) ~[classes/:?]
at com.xsmartware.common.calcite.CalciteDatabase.exeGetLong(CalciteDatabase.java:138) ~[classes/:?]
at com.xsmartware.pbcm.policy.CalciteBasedVarGenerator.generateVariable(CalciteBasedVarGenerator.java:43) ~[classes/:?]
... 8 more
Caused by: java.lang.RuntimeException: Error occurred while applying rule JsonProjectTableScanRule
at org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:157) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:269) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:284) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.adapter.json.JsonProjectTableScanRule.onMatch(JsonProjectTableScanRule.java:80) ~[classes/:?]
at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:223) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.plan.volcano.IterativeRuleDriver.drive(IterativeRuleDriver.java:59) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:523) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:276) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:336) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:177) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.prepare_(CalcitePrepareImpl.java:1026) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.prepareRel(CalcitePrepareImpl.java:988) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:668) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:513) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:483) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:249) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement_(CalciteConnectionImpl.java:229) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.jdbc.CalciteConnectionImpl.access$100(CalciteConnectionImpl.java:101) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.jdbc.CalciteConnectionImpl$2.prepareStatement(CalciteConnectionImpl.java:188) ~[calcite-core-1.29.0.jar:1.29.0]
at com.xsmartware.common.calcite.CalciteDatabase.executeQuery(CalciteDatabase.java:109) ~[classes/:?]
at com.xsmartware.common.calcite.CalciteDatabase.executeQuery(CalciteDatabase.java:103) ~[classes/:?]
at com.xsmartware.common.calcite.CalciteDatabase.exeGetLong(CalciteDatabase.java:138) ~[classes/:?]
at com.xsmartware.pbcm.policy.CalciteBasedVarGenerator.generateVariable(CalciteBasedVarGenerator.java:43) ~[classes/:?]
... 8 more
Caused by: java.lang.NullPointerException
at org.apache.calcite.rel.type.RelDataTypeFactory$Builder.add(RelDataTypeFactory.java:533) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.rel.type.RelDataTypeFactory$FieldInfoBuilder.add(RelDataTypeFactory.java:403) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.rel.type.RelDataTypeFactory$FieldInfoBuilder.add(RelDataTypeFactory.java:367) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.adapter.json.JsonTableScan.lambda$0(JsonTableScan.java:42) ~[classes/:?]
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) ~[?:?]
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658) ~[?:?]
at org.apache.calcite.adapter.json.JsonTableScan.deriveRowType(JsonTableScan.java:42) ~[classes/:?]
at org.apache.calcite.rel.AbstractRelNode.getRowType(AbstractRelNode.java:173) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:587) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:613) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:144) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:269) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:284) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.adapter.json.JsonProjectTableScanRule.onMatch(JsonProjectTableScanRule.java:80) ~[classes/:?]
at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:223) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.plan.volcano.IterativeRuleDriver.drive(IterativeRuleDriver.java:59) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:523) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:276) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:336) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:177) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.prepare_(CalcitePrepareImpl.java:1026) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.prepareRel(CalcitePrepareImpl.java:988) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:668) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:513) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:483) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:249) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement_(CalciteConnectionImpl.java:229) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.jdbc.CalciteConnectionImpl.access$100(CalciteConnectionImpl.java:101) ~[calcite-core-1.29.0.jar:1.29.0]
at org.apache.calcite.jdbc.CalciteConnectionImpl$2.prepareStatement(CalciteConnectionImpl.java:188) ~[calcite-core-1.29.0.jar:1.29.0]
at com.xsmartware.common.calcite.CalciteDatabase.executeQuery(CalciteDatabase.java:109) ~[classes/:?]
at com.xsmartware.common.calcite.CalciteDatabase.executeQuery(CalciteDatabase.java:103) ~[classes/:?]
at com.xsmartware.common.calcite.CalciteDatabase.exeGetLong(CalciteDatabase.java:138) ~[classes/:?]
at com.xsmartware.pbcm.policy.CalciteBasedVarGenerator.generateVariable(CalciteBasedVarGenerator.java:43) ~[classes/:?]
... 8 more</string,>