You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2020/03/11 13:01:31 UTC
[GitHub] [incubator-iotdb] Alima777 opened a new pull request #902:
[IOTDB-28] Calcite Integration for IoTDB
Alima777 opened a new pull request #902: [IOTDB-28] Calcite Integration for IoTDB
URL: https://github.com/apache/incubator-iotdb/pull/902
IoTDB-Calcite Adapter.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [iotdb] yuqi1129 commented on pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
yuqi1129 commented on pull request #902:
URL: https://github.com/apache/iotdb/pull/902#issuecomment-771303075
@wangchao316 , Hi, about your problem, we may do the following as far as i known
1. Skip the optimization stage, that is, we only use the parser to convert sql to ast then validate and transfer the ast to logical RelNode, avatica builtin logical indeed is heavy for simple query, especially for time scales sql, this may need to change the calcite source code and maintain our own version if we want to solve this.
2. this problem seems the problem the SQL itself , we do not recommend too many values in in clause, due to the limit about the field in java language, we can't solve it well.
The above is a personal point of view, discussion is wanted
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yuqi1129 commented on pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
yuqi1129 commented on pull request #902:
URL: https://github.com/apache/iotdb/pull/902#issuecomment-771303075
@wangchao316 , Hi, about your problem, we may do the following as far as i known
1. Skip the optimization stage, that is, we only use the parser to convert sql to ast then validate and transfer the ast to logical RelNode, avatica builtin logical indeed is heavy for simple query, especially for time scales sql, this may need to change the calcite source code and maintain our own version if we want to solve this.
2. this problem seems the problem the SQL itself , we do not recommend too many values in in clause, due to the limit about the field in java language, we can't solve it well.
The above is a personal point of view, discussion is wanted
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls commented on pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
coveralls commented on pull request #902:
URL: https://github.com/apache/iotdb/pull/902#issuecomment-908873726
[](https://coveralls.io/builds/42528628)
Coverage increased (+0.005%) to 67.367% when pulling **77eede7d4914017a27d33652d49ec66cc7a4fffe on Alima777:dev_calcite1** into **fb18357edee14e60893e968aeddc3177fba868bc on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] wangchao316 commented on pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
wangchao316 commented on pull request #902:
URL: https://github.com/apache/iotdb/pull/902#issuecomment-771285262
hi all, The project was developed based on calcite and avatica. calcite has some problems:
1. For simple query optimization, using calcite is too heavy.
2. When calcite generates the execution plan tree, some class files are generated, which may cause memory overflow. This is because there are too many fields in the search criteria.
For example, slelect * from a in (1, 2, 3, 4.....500),
This will cause oom.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] Alima777 commented on a change in pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
Alima777 commented on a change in pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#discussion_r487761087
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBFilter.java
##########
@@ -0,0 +1,321 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.calcite.plan.RelOptCluster;
+import org.apache.calcite.plan.RelOptCost;
+import org.apache.calcite.plan.RelOptPlanner;
+import org.apache.calcite.plan.RelTraitSet;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Filter;
+import org.apache.calcite.rel.metadata.RelMetadataQuery;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexLiteral;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.util.Util;
+import org.apache.iotdb.db.exception.query.LogicalOptimizeException;
+import org.apache.iotdb.db.qp.constant.SQLConstant;
+import org.apache.iotdb.db.qp.logical.crud.BasicFunctionOperator;
+import org.apache.iotdb.db.qp.logical.crud.FilterOperator;
+import org.apache.iotdb.db.qp.strategy.optimizer.DnfFilterOptimizer;
+import org.apache.iotdb.tsfile.read.common.Path;
+
+public class IoTDBFilter extends Filter implements IoTDBRel {
+
+ private final FilterOperator filterOperator;
+ private final List<String> fieldNames;
+ private List<String> predicates; // for global predicate
+ Map<String, String> deviceToFilterMap = new LinkedHashMap<>(); // for device predicate
+
+ protected IoTDBFilter(RelOptCluster cluster, RelTraitSet traits, RelNode child, RexNode condition)
+ throws LogicalOptimizeException {
+ super(cluster, traits, child, condition);
+ this.fieldNames = IoTDBRules.IoTDBFieldNames(getRowType());
+ this.filterOperator = getIoTDBOperator(condition);
+ this.predicates = translateWhere(filterOperator);
+
+ // add global predicate to each device if both global and device predicate exist
+ if (!this.predicates.isEmpty() && !this.deviceToFilterMap.isEmpty()) {
+ for (String device : deviceToFilterMap.keySet()) {
+ StringBuilder builder = new StringBuilder(this.deviceToFilterMap.get(device));
+ builder.append(Util.toString(predicates, " OR ", " OR ", ""));
+ this.deviceToFilterMap.put(device, builder.toString());
+ }
+ }
+ assert getConvention() == IoTDBRel.CONVENTION;
+ assert getConvention() == child.getConvention();
+ }
+
+ @Override
+ public RelOptCost computeSelfCost(RelOptPlanner planner,
+ RelMetadataQuery mq) {
+ return super.computeSelfCost(planner, mq).multiplyBy(0.1);
+ }
+
+ @Override
+ public Filter copy(RelTraitSet relTraitSet, RelNode input, RexNode condition) {
+ try {
+ return new IoTDBFilter(getCluster(), traitSet, input, condition);
+ } catch (LogicalOptimizeException e) {
+ throw new AssertionError(e.getMessage());
+ }
+ }
+
+ @Override
+ public void implement(Implementor implementor) {
+ implementor.visitChild(0, getInput());
+ implementor.add(deviceToFilterMap, predicates);
+ }
+
+ /**
+ * Translates {@link RexNode} expressions into IoTDB filter operator.
+ */
+ private FilterOperator getIoTDBOperator(RexNode filter) {
+ switch (filter.getKind()) {
+ case EQUALS:
+ return getBasicOperator(SQLConstant.EQUAL, (RexCall) filter);
+ case NOT_EQUALS:
+ return getBasicOperator(SQLConstant.NOTEQUAL, (RexCall) filter);
+ case GREATER_THAN:
+ return getBasicOperator(SQLConstant.GREATERTHAN, (RexCall) filter);
+ case GREATER_THAN_OR_EQUAL:
+ return getBasicOperator(SQLConstant.GREATERTHANOREQUALTO, (RexCall) filter);
+ case LESS_THAN:
+ return getBasicOperator(SQLConstant.LESSTHAN, (RexCall) filter);
+ case LESS_THAN_OR_EQUAL:
+ return getBasicOperator(SQLConstant.LESSTHANOREQUALTO, (RexCall) filter);
+ case AND:
+ return getBinaryOperator(SQLConstant.KW_AND, ((RexCall) filter).getOperands());
+ case OR:
+ return getBinaryOperator(SQLConstant.KW_OR, ((RexCall) filter).getOperands());
+ default:
+ throw new AssertionError("cannot get IoTDBOperator from " + filter);
+ }
+ }
+
+ private FilterOperator getBasicOperator(int tokenIntType, RexCall call) {
+ final RexNode left = call.operands.get(0);
+ final RexNode right = call.operands.get(1);
+
+ FilterOperator operator = getBasicOperator2(tokenIntType, left, right);
+ if (operator != null) {
+ return operator;
+ }
+ operator = getBasicOperator2(tokenIntType, right, left);
+ if (operator != null) {
+ return operator;
+ }
+ throw new AssertionError("cannot translate basic operator: " + call);
+ }
+
+ private FilterOperator getBasicOperator2(int tokenIntType, RexNode left, RexNode right) {
+ switch (right.getKind()) {
+ case LITERAL:
+ break;
+ default:
+ return null;
+ }
+ final RexLiteral rightLiteral = (RexLiteral) right;
+ switch (left.getKind()) {
+ case INPUT_REF:
+ String name = fieldNames.get(((RexInputRef) left).getIndex());
+ return new BasicFunctionOperator(tokenIntType, new Path(name), literalValue(rightLiteral));
+ case CAST:
+ return getBasicOperator2(tokenIntType, ((RexCall) left).getOperands().get(0), right);
+ default:
+ return null;
+ }
+
+ }
+
+ private FilterOperator getBinaryOperator(int tokenIntType, List<RexNode> operands) {
+ FilterOperator filterBinaryTree = new FilterOperator(tokenIntType);
+ FilterOperator currentNode = filterBinaryTree;
+ for (int i = 0; i < operands.size(); i++) {
Review comment:
Fixed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] wangchao316 commented on pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
wangchao316 commented on pull request #902:
URL: https://github.com/apache/iotdb/pull/902#issuecomment-771285262
hi all, The project was developed based on calcite and avatica. calcite has some problems:
1. For simple query optimization, using calcite is too heavy.
2. When calcite generates the execution plan tree, some class files are generated, which may cause memory overflow. This is because there are too many fields in the search criteria.
For example, slelect * from a in (1, 2, 3, 4.....500),
This will cause oom.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] Alima777 commented on a change in pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
Alima777 commented on a change in pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#discussion_r487775869
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBTable.java
##########
@@ -0,0 +1,306 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import com.google.common.collect.ImmutableList;
+import java.sql.Connection;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import org.apache.calcite.adapter.java.AbstractQueryableTable;
+import org.apache.calcite.linq4j.AbstractEnumerable;
+import org.apache.calcite.linq4j.Enumerable;
+import org.apache.calcite.linq4j.Enumerator;
+import org.apache.calcite.linq4j.QueryProvider;
+import org.apache.calcite.linq4j.Queryable;
+import org.apache.calcite.linq4j.function.Function1;
+import org.apache.calcite.plan.RelOptCluster;
+import org.apache.calcite.plan.RelOptTable;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rel.type.RelDataTypeFactory;
+import org.apache.calcite.rel.type.RelDataTypeImpl;
+import org.apache.calcite.rel.type.RelDataTypeSystem;
+import org.apache.calcite.rel.type.RelProtoDataType;
+import org.apache.calcite.schema.SchemaPlus;
+import org.apache.calcite.schema.TranslatableTable;
+import org.apache.calcite.schema.impl.AbstractTableQueryable;
+import org.apache.calcite.sql.type.SqlTypeFactoryImpl;
+import org.apache.calcite.sql.type.SqlTypeName;
+import org.apache.calcite.util.Util;
+import org.apache.iotdb.db.exception.query.QueryProcessException;
+
+public class IoTDBTable extends AbstractQueryableTable
+ implements TranslatableTable {
+
+ RelProtoDataType protoRowType;
+ private final IoTDBSchema schema;
+ private final String storageGroup;
+
+ public IoTDBTable(IoTDBSchema schema, String storageGroup) {
+ super(Object[].class);
+ this.schema = schema;
+ this.storageGroup = storageGroup;
+ }
+
+ public String toString() {
+ return "IoTDBTable {" + storageGroup + "}";
+ }
+
+ @Override
+ public RelDataType getRowType(RelDataTypeFactory typeFactory) {
+ try {
+ if (protoRowType == null) {
+ protoRowType = schema.getRelDataType(storageGroup);
+ }
+ } catch (SQLException | QueryProcessException e) {
+ // print exception error here
+ e.printStackTrace();
+ }
+ return protoRowType.apply(typeFactory);
+ }
+
+ @Override
+ public <T> Queryable<T> asQueryable(QueryProvider queryProvider, SchemaPlus schema,
+ String tableName) {
+ return new IoTDBQueryable<>(queryProvider, schema, this, storageGroup);
+ }
+
+ @Override
+ public RelNode toRel(RelOptTable.ToRelContext context, RelOptTable relOptTable) {
+ final RelOptCluster cluster = context.getCluster();
+ return new IoTDBTableScan(cluster, cluster.traitSetOf(IoTDBRel.CONVENTION),
+ relOptTable, this, null);
+ }
+
+ public Enumerable<Object> query(final Connection connection) {
+ return query(connection, ImmutableList.of(), ImmutableList.of(), ImmutableList.of(),
+ ImmutableList.of(), 0, 0);
Review comment:
It's a default empty value of query() method.
Actually, I don't know what's the function of this method exactly... I just referred to the Cassandra Adapter. I will appreciate that if you can explain it to me :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] yuqi1129 commented on a change in pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
yuqi1129 commented on a change in pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#discussion_r488604981
##########
File path: calcite/src/test/java/org/apache/iotdb/calcite/IoTDBAdapterTest.java
##########
@@ -0,0 +1,339 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import com.google.common.collect.ImmutableMap;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.Statement;
+import org.apache.calcite.test.CalciteAssert;
+import org.apache.calcite.util.Sources;
+import org.apache.iotdb.db.utils.EnvironmentUtils;
+import org.apache.iotdb.jdbc.Config;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+public class IoTDBAdapterTest {
+
+ public static final ImmutableMap<String, String> MODEL =
+ ImmutableMap.of("model",
+ Sources.of(IoTDBAdapterTest.class.getResource("/model.json"))
+ .file().getAbsolutePath());
+ private static String[] sqls = new String[]{
+
+ "SET STORAGE GROUP TO root.vehicle",
+ "SET STORAGE GROUP TO root.other",
+
+ "CREATE timeSERIES root.vehicle.d0.s0 WITH DATATYPE=INT32, ENCODING=RLE",
+ "CREATE timeSERIES root.vehicle.d0.s1 WITH DATATYPE=INT64, ENCODING=RLE",
+ "CREATE timeSERIES root.vehicle.d0.s2 WITH DATATYPE=FLOAT, ENCODING=RLE",
+ "CREATE timeSERIES root.vehicle.d0.s3 WITH DATATYPE=TEXT, ENCODING=PLAIN",
+ "CREATE timeSERIES root.vehicle.d0.s4 WITH DATATYPE=BOOLEAN, ENCODING=PLAIN",
+
+ "CREATE timeSERIES root.vehicle.d1.s0 WITH DATATYPE=INT32, ENCODING=RLE",
+ "CREATE timeSERIES root.vehicle.d1.s1 WITH DATATYPE=INT64, ENCODING=RLE",
+ "CREATE timeSERIES root.vehicle.d1.s2 WITH DATATYPE=FLOAT, ENCODING=RLE",
+ "CREATE timeSERIES root.vehicle.d1.s3 WITH DATATYPE=TEXT, ENCODING=PLAIN",
+ "CREATE timeSERIES root.vehicle.d1.s4 WITH DATATYPE=BOOLEAN, ENCODING=PLAIN",
+
+ "CREATE timeSERIES root.other.d1.s0 WITH DATATYPE=FLOAT, ENCODING=RLE",
+
+ "insert into root.vehicle.d0(timestamp,s0) values(1,101)",
+ "insert into root.vehicle.d0(timestamp,s0) values(2,198)",
+ "insert into root.vehicle.d0(timestamp,s0) values(100,99)",
+ "insert into root.vehicle.d0(timestamp,s0) values(101,99)",
+ "insert into root.vehicle.d0(timestamp,s0) values(102,80)",
+ "insert into root.vehicle.d0(timestamp,s0) values(103,99)",
+ "insert into root.vehicle.d0(timestamp,s0) values(104,90)",
+ "insert into root.vehicle.d0(timestamp,s0) values(105,99)",
+ "insert into root.vehicle.d0(timestamp,s0) values(106,99)",
+ "insert into root.vehicle.d0(timestamp,s0) values(2,10000)",
+ "insert into root.vehicle.d0(timestamp,s0) values(50,10000)",
+ "insert into root.vehicle.d0(timestamp,s0) values(1000,22222)",
+
+ "insert into root.vehicle.d0(timestamp,s1) values(1,1101)",
+ "insert into root.vehicle.d0(timestamp,s1) values(2,198)",
+ "insert into root.vehicle.d0(timestamp,s1) values(100,199)",
+ "insert into root.vehicle.d0(timestamp,s1) values(101,199)",
+ "insert into root.vehicle.d0(timestamp,s1) values(102,180)",
+ "insert into root.vehicle.d0(timestamp,s1) values(103,199)",
+ "insert into root.vehicle.d0(timestamp,s1) values(104,190)",
+ "insert into root.vehicle.d0(timestamp,s1) values(105,199)",
+ "insert into root.vehicle.d0(timestamp,s1) values(2,40000)",
+ "insert into root.vehicle.d0(timestamp,s1) values(50,50000)",
+ "insert into root.vehicle.d0(timestamp,s1) values(1000,55555)",
+
+ "insert into root.vehicle.d0(timestamp,s2) values(1000,55555)",
+ "insert into root.vehicle.d0(timestamp,s2) values(2,2.22)",
+ "insert into root.vehicle.d0(timestamp,s2) values(3,3.33)",
+ "insert into root.vehicle.d0(timestamp,s2) values(4,4.44)",
+ "insert into root.vehicle.d0(timestamp,s2) values(102,10.00)",
+ "insert into root.vehicle.d0(timestamp,s2) values(105,11.11)",
+ "insert into root.vehicle.d0(timestamp,s2) values(1000,1000.11)",
+
+ "insert into root.vehicle.d0(timestamp,s3) values(60,'aaaaa')",
+ "insert into root.vehicle.d0(timestamp,s3) values(70,'bbbbb')",
+ "insert into root.vehicle.d0(timestamp,s3) values(80,'ccccc')",
+ "insert into root.vehicle.d0(timestamp,s3) values(101,'ddddd')",
+ "insert into root.vehicle.d0(timestamp,s3) values(102,'fffff')",
+
+ "insert into root.vehicle.d0(timestamp,s4) values(23, false)",
+ "insert into root.vehicle.d0(timestamp,s4) values(100, true)",
+
+ "insert into root.vehicle.d1(timestamp,s0) values(1,999)",
+ "insert into root.vehicle.d1(timestamp,s0) values(1000,10)",
+
+ "insert into root.vehicle.d1(timestamp,s1) values(2,9999)",
+ "insert into root.vehicle.d1(timestamp,s1) values(1000,5)",
+
+ "insert into root.vehicle.d1(timestamp,s2) values(2,12345.6)",
+ "insert into root.vehicle.d1(timestamp,s2) values(2222,2.22)",
+
+ "insert into root.vehicle.d1(timestamp,s3) values(10,'ten')",
+ "insert into root.vehicle.d1(timestamp,s3) values(1000,'thousand')",
+
+ "insert into root.vehicle.d1(timestamp,s4) values(100, false)",
+ "insert into root.vehicle.d1(timestamp,s4) values(10000, true)",
+
+ "insert into root.vehicle.d0(timestamp,s1) values(2000-01-01T08:00:00+08:00, 100)",
+ "insert into root.vehicle.d0(timestamp,s3) values(2000-01-01T08:00:00+08:00, 'good')",
+
+ "insert into root.other.d1(timestamp,s0) values(2, 3.14)",};
+
+ @BeforeClass
+ public static void setUp() throws Exception {
+ EnvironmentUtils.closeStatMonitor();
+ EnvironmentUtils.envSetUp();
+
+ insertData();
+ }
+
+ @AfterClass
+ public static void tearDown() throws Exception {
+ EnvironmentUtils.cleanEnv();
+ }
+
+ private static void insertData() throws ClassNotFoundException {
+ Class.forName(Config.JDBC_DRIVER_NAME);
+ try (Connection connection = DriverManager
+ .getConnection(Config.IOTDB_URL_PREFIX + "127.0.0.1:6667/", "root", "root");
+ Statement statement = connection.createStatement()) {
+
+ for (String sql : sqls) {
+ statement.execute(sql);
+ }
+ } catch (Exception e) {
+ e.printStackTrace();
+ }
+ }
+
+ @Test
+ public void testSelect() {
+ CalciteAssert.that()
+ .with(MODEL)
+ .with("UnQuotedCasing", IoTDBConstant.UNQUOTED_CASING)
+ .query("select * from \"root.vehicle\"")
Review comment:
What's the sql dialect here ? oracle , mysql or else ?
##########
File path: calcite/IoTDB-Calcite-Adapter.md
##########
@@ -0,0 +1,232 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+# IoTDB - Calcite Adapter 功能文档
+
+## 关系表结构
+
+IoTDB - Calcite Adapter 中使用的关系表结构为：
+
+| time | device | sensor1 | sensor2 | sensor3 | ... |
+| ---- | ------ | ------- | ------- | ------- | --- |
+| | | | | | |
+
+其中，IoTDB 中每个存储组作为一张表，表中的列包括 time , device 列以及该存储组中所有设备中传感器的最大并集，其中不同设备的同名传感器应该具有相同的数据类型。
+
+例如对于 IoTDB 中存储组 `root.sg`，其中设备及其对应的传感器为：
+- d1 -> s1, s2
+- d2 -> s2, s3
+- d3 -> s1, s4
+
+则在 IoTDB - Calcite Adapter 中的表名为 `root.sg`，其表结构为
+
+| time | device | s1 | s2 | s3 | s4 |
+| ---- | ------ | --- | --- | --- | --- |
+| | | | | | |
+
+## 工作原理
+
+接下来简单介绍 IoTDB - Calcite Adapter 的工作原理。
+
+输入的 SQL 语句在经过 Calicte 的解析验证后，对 `IoTDBRules` 中定义的优化（下推）规则进行匹配，对于能够下推的节点做相应转化后，得到能够在 IoTDB 端执行的 SQL 语句，然后在 IoTDB 端执行查询语句获取源数据；对于不能下推的节点则调用 Calcite 默认的物理计划进行执行，最后通过 `IoTDBEnumerator` 遍历结果集获取结果。
+
+
+## 查询介绍
+
+当前在 `IoTDBRules` 中定义的下推规则有：`IoTDBProjectRule`, `IoTDBFilterRule`, `IoTDBLimitRule`。
+
+### IoTDBProjectRule
+
+IoTDBProjectRule 实现了将查询语句中出现的投影列下推到 IoTDB 端进行执行。
+
+例如：（以下 sql 均为测试中的语句）
+
+1. 对于通配符
+
+```sql
+select * from "root.vehicle"
+```
+
+对于通配符 `*`，将在转化中保持原样，而不转化为列名，得到 IoTDB 中的查询语句为：
+
+```sql
+select * from root.vehicle.* align by device
+```
+
+2. 对于非通配符的传感器列
+
+```sql
+select s0 from "root.vehicle"
+```
+
+将转化为：
+
+```sql
+select s0 from root.vehicle.* align by device
+```
+
+3. 对于非通配符的非传感器列
+
+```sql
+select "time", device, s2 from "root.vehicle"
+```
+
+该语句中的 time 及 device 列是 IoTDB 的查询语句中不需要包括的，因此转化将去掉这两列，得到 IoTDB 中的查询语句为：
+
+```sql
+select s2 from root.vehicle.* align by device
+```
+
+特别地，如果查询语句中仅包含 time 及 device 列，则投影部分将转化为通配符 `*`。
+
+4. 重命名 Alias
+
+当前 IoTDB - Calcite Adapter 仅支持在 SELECT 语句中对投影列进行重命名，不支持在后续语句中使用重命名后的名称。
+
+```sql
+select "time" AS t, device AS d, s2 from "root.vehicle"
+```
+
+将得到结果中 time 列的名字为 t，device 列的名字为 d。
+
+### IoTDBFilterRule
+
+IoTDBFilterRule 实现了将查询语句中的 WHERE 子句下推到 IoTDB 端进行执行。
+
+1. WHERE 子句中不限制 device 列
+
+```sql
+select * from "root.vehicle" where "time" < 10 AND s0 >= 150
+```
+
+对于 time 列将不作改变，由于未限制具体的设备，因此传感器列不会与具体的设备名进行拼接，得到 IoTDB 中的查询语句为：
+
+```sql
+select * from root.vehicle.* where time < 10 AND s0 >= 150
+```
+
+2. WHERE 子句中限制 device 列
+
+- 仅限制单个设备
+
+```sql
+select * from "root.vehicle" where device = 'root.vehicle.d0' AND "time" > 10 AND s0 <= 100
+```
+
+如果 WHERE 中只限制了单个设备且其它限制条件均是对该设备的限制，则在 IoTDB 中将转化为对该设备的查询，上述查询将转化为：
+
+```sql
+select * from root.vehicle.d0 where time > 10 AND s0 <= 100
+```
+
+- 限制多个设备
+
+```sql
+select * from "root.vehicle" where (device = 'root.vehicle.d0' AND "time" <= 1) OR (device = 'root.vehicle.d1' AND s0 < 100)
+```
+
+如果 WHERE 中限制了多个设备，将转化为多条查询语句，根据对每个设备的限制条件分别进行查询。
+
+如上述查询语句将转化为两条 SQL 在 IoTDB 中执行：
+
+```sql
+select * from root.vehicle.d0 where time <= 1
+select * from root.vehicle.d1 where s0 < 100
+```
+
+- 既有限制设备的条件，又有全局条件
+
+```sql
+select * from "root.vehicle" where (device = 'root.vehicle.d0' AND "time" <= 1) OR s0 = 999
+```
+
+在上述 SQL 语句中，除了有对设备 `root.vehicle.d0` 的单独限制外，还有一个限制条件 `s0 = 999`，该限制条件被认为是一个全局条件，任何设备只要满足该条件都被认为是正确结果。
+
+因此上述查询将转化为对存储组中所有设备的查询，对于有单独限制条件的设备将单独处理，其它剩余设备将使用全局条件统一查询。
+
+```sql
+select * from root.vehicle.d0 where time <= 1 OR s0 = 999
+select * from root.vehicle.d1 where s0 = 999
+```
+
+注：由于测试中恰好只有两个设备，如果再有一个设备 d2，则将在 FROM 子句加上 `root.vehicle.d2` 而非为设备 d2 单独再次查询。
+
+### IoTDBLimitRule
+
+IoTDBLimitRule 实现了将查询语句中的 LIMIT 及 OFFSET 子句下推到 IoTDB 端进行执行。
Review comment:
In fact, IoTDBLimitRule can't push limit and offset down to IoTDBTable, the limit and offset logic in done in code generated by calcite see `IoTDBToEnumerableConverter#implement` which will call `IoTDBTable#query` by reflection
you can debug the `query` method and see it
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBTable.java
##########
@@ -0,0 +1,306 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import com.google.common.collect.ImmutableList;
+import java.sql.Connection;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import org.apache.calcite.adapter.java.AbstractQueryableTable;
+import org.apache.calcite.linq4j.AbstractEnumerable;
+import org.apache.calcite.linq4j.Enumerable;
+import org.apache.calcite.linq4j.Enumerator;
+import org.apache.calcite.linq4j.QueryProvider;
+import org.apache.calcite.linq4j.Queryable;
+import org.apache.calcite.linq4j.function.Function1;
+import org.apache.calcite.plan.RelOptCluster;
+import org.apache.calcite.plan.RelOptTable;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rel.type.RelDataTypeFactory;
+import org.apache.calcite.rel.type.RelDataTypeImpl;
+import org.apache.calcite.rel.type.RelDataTypeSystem;
+import org.apache.calcite.rel.type.RelProtoDataType;
+import org.apache.calcite.schema.SchemaPlus;
+import org.apache.calcite.schema.TranslatableTable;
+import org.apache.calcite.schema.impl.AbstractTableQueryable;
+import org.apache.calcite.sql.type.SqlTypeFactoryImpl;
+import org.apache.calcite.sql.type.SqlTypeName;
+import org.apache.calcite.util.Util;
+import org.apache.iotdb.db.exception.query.QueryProcessException;
+
+public class IoTDBTable extends AbstractQueryableTable
+ implements TranslatableTable {
+
+ RelProtoDataType protoRowType;
+ private final IoTDBSchema schema;
+ private final String storageGroup;
+
+ public IoTDBTable(IoTDBSchema schema, String storageGroup) {
+ super(Object[].class);
+ this.schema = schema;
+ this.storageGroup = storageGroup;
+ }
+
+ public String toString() {
+ return "IoTDBTable {" + storageGroup + "}";
+ }
+
+ @Override
+ public RelDataType getRowType(RelDataTypeFactory typeFactory) {
+ try {
+ if (protoRowType == null) {
+ protoRowType = schema.getRelDataType(storageGroup);
+ }
+ } catch (SQLException | QueryProcessException e) {
+ // print exception error here
+ e.printStackTrace();
+ }
+ return protoRowType.apply(typeFactory);
+ }
+
+ @Override
+ public <T> Queryable<T> asQueryable(QueryProvider queryProvider, SchemaPlus schema,
+ String tableName) {
+ return new IoTDBQueryable<>(queryProvider, schema, this, storageGroup);
+ }
+
+ @Override
+ public RelNode toRel(RelOptTable.ToRelContext context, RelOptTable relOptTable) {
+ final RelOptCluster cluster = context.getCluster();
+ return new IoTDBTableScan(cluster, cluster.traitSetOf(IoTDBRel.CONVENTION),
+ relOptTable, this, null);
+ }
+
+ public Enumerable<Object> query(final Connection connection) {
+ return query(connection, ImmutableList.of(), ImmutableList.of(), ImmutableList.of(),
+ ImmutableList.of(), 0, 0);
Review comment:
Cassandra do not support to push limit and offset down to table scan, so just set 0 here and do the logic in upper relnode
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] vesense commented on pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
vesense commented on pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#issuecomment-626624352
@Alima777 Thanks for your contribution, I will have a review this month.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls commented on pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
coveralls commented on pull request #902:
URL: https://github.com/apache/iotdb/pull/902#issuecomment-908873726
[](https://coveralls.io/builds/42528628)
Coverage increased (+0.005%) to 67.367% when pulling **77eede7d4914017a27d33652d49ec66cc7a4fffe on Alima777:dev_calcite1** into **fb18357edee14e60893e968aeddc3177fba868bc on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] yuqi1129 commented on pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
yuqi1129 commented on pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#issuecomment-691814857
@Test
public void testFilter7() {
CalciteAssert.that()
.with(MODEL)
.with("UnQuotedCasing", IoTDBConstant.UnQuotedCasing)
.query("select * from \"root.vehicle\" " +
"where (device = 'root.vehicle.d0' AND \"time\" <= 1) OR s2 = 2.22 and 1 = 1")
.returns("time=1; device=root.vehicle.d0; s0=101; s1=1101; s2=null; s3=null; s4=null\n" +
"time=2; device=root.vehicle.d0; s0=10000; s1=40000; s2=2.22; s3=null; s4=null\n" +
"time=2222; device=root.vehicle.d1; s0=null; s1=null; s2=2.22; s3=null; s4=null\n")
.explainContains("PLAN=IoTDBToEnumerableConverter\n" +
" IoTDBFilter(condition=[OR(AND(=($1, 'root.vehicle.d0'), <=($0, 1)), =(CAST($4):DOUBLE NOT NULL, 2.22))])\n" +
" IoTDBTableScan(table=[[IoTDBSchema, root.vehicle]])");
}
Should add rules to reduce constant value
like where 1 = 1 and a = 1 should be folded as where a = 1, see above
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] Alima777 edited a comment on pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
Alima777 edited a comment on pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#issuecomment-691922941
@yuqi1129 Hi, thank you very much for your patient review!
Since it's the first large module I implemented, ignoring lots of code standards, like: exception processing, error log, standard code style and something else... And It has not been maintained for a long time.
So Thanks again! I've fixed it and please have a check.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] Alima777 commented on a change in pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
Alima777 commented on a change in pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#discussion_r488656800
##########
File path: calcite/src/test/java/org/apache/iotdb/calcite/IoTDBAdapterTest.java
##########
@@ -0,0 +1,339 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import com.google.common.collect.ImmutableMap;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.Statement;
+import org.apache.calcite.test.CalciteAssert;
+import org.apache.calcite.util.Sources;
+import org.apache.iotdb.db.utils.EnvironmentUtils;
+import org.apache.iotdb.jdbc.Config;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+public class IoTDBAdapterTest {
+
+ public static final ImmutableMap<String, String> MODEL =
+ ImmutableMap.of("model",
+ Sources.of(IoTDBAdapterTest.class.getResource("/model.json"))
+ .file().getAbsolutePath());
+ private static String[] sqls = new String[]{
+
+ "SET STORAGE GROUP TO root.vehicle",
+ "SET STORAGE GROUP TO root.other",
+
+ "CREATE timeSERIES root.vehicle.d0.s0 WITH DATATYPE=INT32, ENCODING=RLE",
+ "CREATE timeSERIES root.vehicle.d0.s1 WITH DATATYPE=INT64, ENCODING=RLE",
+ "CREATE timeSERIES root.vehicle.d0.s2 WITH DATATYPE=FLOAT, ENCODING=RLE",
+ "CREATE timeSERIES root.vehicle.d0.s3 WITH DATATYPE=TEXT, ENCODING=PLAIN",
+ "CREATE timeSERIES root.vehicle.d0.s4 WITH DATATYPE=BOOLEAN, ENCODING=PLAIN",
+
+ "CREATE timeSERIES root.vehicle.d1.s0 WITH DATATYPE=INT32, ENCODING=RLE",
+ "CREATE timeSERIES root.vehicle.d1.s1 WITH DATATYPE=INT64, ENCODING=RLE",
+ "CREATE timeSERIES root.vehicle.d1.s2 WITH DATATYPE=FLOAT, ENCODING=RLE",
+ "CREATE timeSERIES root.vehicle.d1.s3 WITH DATATYPE=TEXT, ENCODING=PLAIN",
+ "CREATE timeSERIES root.vehicle.d1.s4 WITH DATATYPE=BOOLEAN, ENCODING=PLAIN",
+
+ "CREATE timeSERIES root.other.d1.s0 WITH DATATYPE=FLOAT, ENCODING=RLE",
+
+ "insert into root.vehicle.d0(timestamp,s0) values(1,101)",
+ "insert into root.vehicle.d0(timestamp,s0) values(2,198)",
+ "insert into root.vehicle.d0(timestamp,s0) values(100,99)",
+ "insert into root.vehicle.d0(timestamp,s0) values(101,99)",
+ "insert into root.vehicle.d0(timestamp,s0) values(102,80)",
+ "insert into root.vehicle.d0(timestamp,s0) values(103,99)",
+ "insert into root.vehicle.d0(timestamp,s0) values(104,90)",
+ "insert into root.vehicle.d0(timestamp,s0) values(105,99)",
+ "insert into root.vehicle.d0(timestamp,s0) values(106,99)",
+ "insert into root.vehicle.d0(timestamp,s0) values(2,10000)",
+ "insert into root.vehicle.d0(timestamp,s0) values(50,10000)",
+ "insert into root.vehicle.d0(timestamp,s0) values(1000,22222)",
+
+ "insert into root.vehicle.d0(timestamp,s1) values(1,1101)",
+ "insert into root.vehicle.d0(timestamp,s1) values(2,198)",
+ "insert into root.vehicle.d0(timestamp,s1) values(100,199)",
+ "insert into root.vehicle.d0(timestamp,s1) values(101,199)",
+ "insert into root.vehicle.d0(timestamp,s1) values(102,180)",
+ "insert into root.vehicle.d0(timestamp,s1) values(103,199)",
+ "insert into root.vehicle.d0(timestamp,s1) values(104,190)",
+ "insert into root.vehicle.d0(timestamp,s1) values(105,199)",
+ "insert into root.vehicle.d0(timestamp,s1) values(2,40000)",
+ "insert into root.vehicle.d0(timestamp,s1) values(50,50000)",
+ "insert into root.vehicle.d0(timestamp,s1) values(1000,55555)",
+
+ "insert into root.vehicle.d0(timestamp,s2) values(1000,55555)",
+ "insert into root.vehicle.d0(timestamp,s2) values(2,2.22)",
+ "insert into root.vehicle.d0(timestamp,s2) values(3,3.33)",
+ "insert into root.vehicle.d0(timestamp,s2) values(4,4.44)",
+ "insert into root.vehicle.d0(timestamp,s2) values(102,10.00)",
+ "insert into root.vehicle.d0(timestamp,s2) values(105,11.11)",
+ "insert into root.vehicle.d0(timestamp,s2) values(1000,1000.11)",
+
+ "insert into root.vehicle.d0(timestamp,s3) values(60,'aaaaa')",
+ "insert into root.vehicle.d0(timestamp,s3) values(70,'bbbbb')",
+ "insert into root.vehicle.d0(timestamp,s3) values(80,'ccccc')",
+ "insert into root.vehicle.d0(timestamp,s3) values(101,'ddddd')",
+ "insert into root.vehicle.d0(timestamp,s3) values(102,'fffff')",
+
+ "insert into root.vehicle.d0(timestamp,s4) values(23, false)",
+ "insert into root.vehicle.d0(timestamp,s4) values(100, true)",
+
+ "insert into root.vehicle.d1(timestamp,s0) values(1,999)",
+ "insert into root.vehicle.d1(timestamp,s0) values(1000,10)",
+
+ "insert into root.vehicle.d1(timestamp,s1) values(2,9999)",
+ "insert into root.vehicle.d1(timestamp,s1) values(1000,5)",
+
+ "insert into root.vehicle.d1(timestamp,s2) values(2,12345.6)",
+ "insert into root.vehicle.d1(timestamp,s2) values(2222,2.22)",
+
+ "insert into root.vehicle.d1(timestamp,s3) values(10,'ten')",
+ "insert into root.vehicle.d1(timestamp,s3) values(1000,'thousand')",
+
+ "insert into root.vehicle.d1(timestamp,s4) values(100, false)",
+ "insert into root.vehicle.d1(timestamp,s4) values(10000, true)",
+
+ "insert into root.vehicle.d0(timestamp,s1) values(2000-01-01T08:00:00+08:00, 100)",
+ "insert into root.vehicle.d0(timestamp,s3) values(2000-01-01T08:00:00+08:00, 'good')",
+
+ "insert into root.other.d1(timestamp,s0) values(2, 3.14)",};
+
+ @BeforeClass
+ public static void setUp() throws Exception {
+ EnvironmentUtils.closeStatMonitor();
+ EnvironmentUtils.envSetUp();
+
+ insertData();
+ }
+
+ @AfterClass
+ public static void tearDown() throws Exception {
+ EnvironmentUtils.cleanEnv();
+ }
+
+ private static void insertData() throws ClassNotFoundException {
+ Class.forName(Config.JDBC_DRIVER_NAME);
+ try (Connection connection = DriverManager
+ .getConnection(Config.IOTDB_URL_PREFIX + "127.0.0.1:6667/", "root", "root");
+ Statement statement = connection.createStatement()) {
+
+ for (String sql : sqls) {
+ statement.execute(sql);
+ }
+ } catch (Exception e) {
+ e.printStackTrace();
+ }
+ }
+
+ @Test
+ public void testSelect() {
+ CalciteAssert.that()
+ .with(MODEL)
+ .with("UnQuotedCasing", IoTDBConstant.UNQUOTED_CASING)
+ .query("select * from \"root.vehicle\"")
Review comment:
According to the calcite doc:
> SQL conformance level. Values: DEFAULT (the default, similar to PRAGMATIC_2003)
It could be customed by properties.
##########
File path: calcite/src/test/java/org/apache/iotdb/calcite/IoTDBAdapterTest.java
##########
@@ -0,0 +1,339 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import com.google.common.collect.ImmutableMap;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.Statement;
+import org.apache.calcite.test.CalciteAssert;
+import org.apache.calcite.util.Sources;
+import org.apache.iotdb.db.utils.EnvironmentUtils;
+import org.apache.iotdb.jdbc.Config;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+public class IoTDBAdapterTest {
+
+ public static final ImmutableMap<String, String> MODEL =
+ ImmutableMap.of("model",
+ Sources.of(IoTDBAdapterTest.class.getResource("/model.json"))
+ .file().getAbsolutePath());
+ private static String[] sqls = new String[]{
+
+ "SET STORAGE GROUP TO root.vehicle",
+ "SET STORAGE GROUP TO root.other",
+
+ "CREATE timeSERIES root.vehicle.d0.s0 WITH DATATYPE=INT32, ENCODING=RLE",
+ "CREATE timeSERIES root.vehicle.d0.s1 WITH DATATYPE=INT64, ENCODING=RLE",
+ "CREATE timeSERIES root.vehicle.d0.s2 WITH DATATYPE=FLOAT, ENCODING=RLE",
+ "CREATE timeSERIES root.vehicle.d0.s3 WITH DATATYPE=TEXT, ENCODING=PLAIN",
+ "CREATE timeSERIES root.vehicle.d0.s4 WITH DATATYPE=BOOLEAN, ENCODING=PLAIN",
+
+ "CREATE timeSERIES root.vehicle.d1.s0 WITH DATATYPE=INT32, ENCODING=RLE",
+ "CREATE timeSERIES root.vehicle.d1.s1 WITH DATATYPE=INT64, ENCODING=RLE",
+ "CREATE timeSERIES root.vehicle.d1.s2 WITH DATATYPE=FLOAT, ENCODING=RLE",
+ "CREATE timeSERIES root.vehicle.d1.s3 WITH DATATYPE=TEXT, ENCODING=PLAIN",
+ "CREATE timeSERIES root.vehicle.d1.s4 WITH DATATYPE=BOOLEAN, ENCODING=PLAIN",
+
+ "CREATE timeSERIES root.other.d1.s0 WITH DATATYPE=FLOAT, ENCODING=RLE",
+
+ "insert into root.vehicle.d0(timestamp,s0) values(1,101)",
+ "insert into root.vehicle.d0(timestamp,s0) values(2,198)",
+ "insert into root.vehicle.d0(timestamp,s0) values(100,99)",
+ "insert into root.vehicle.d0(timestamp,s0) values(101,99)",
+ "insert into root.vehicle.d0(timestamp,s0) values(102,80)",
+ "insert into root.vehicle.d0(timestamp,s0) values(103,99)",
+ "insert into root.vehicle.d0(timestamp,s0) values(104,90)",
+ "insert into root.vehicle.d0(timestamp,s0) values(105,99)",
+ "insert into root.vehicle.d0(timestamp,s0) values(106,99)",
+ "insert into root.vehicle.d0(timestamp,s0) values(2,10000)",
+ "insert into root.vehicle.d0(timestamp,s0) values(50,10000)",
+ "insert into root.vehicle.d0(timestamp,s0) values(1000,22222)",
+
+ "insert into root.vehicle.d0(timestamp,s1) values(1,1101)",
+ "insert into root.vehicle.d0(timestamp,s1) values(2,198)",
+ "insert into root.vehicle.d0(timestamp,s1) values(100,199)",
+ "insert into root.vehicle.d0(timestamp,s1) values(101,199)",
+ "insert into root.vehicle.d0(timestamp,s1) values(102,180)",
+ "insert into root.vehicle.d0(timestamp,s1) values(103,199)",
+ "insert into root.vehicle.d0(timestamp,s1) values(104,190)",
+ "insert into root.vehicle.d0(timestamp,s1) values(105,199)",
+ "insert into root.vehicle.d0(timestamp,s1) values(2,40000)",
+ "insert into root.vehicle.d0(timestamp,s1) values(50,50000)",
+ "insert into root.vehicle.d0(timestamp,s1) values(1000,55555)",
+
+ "insert into root.vehicle.d0(timestamp,s2) values(1000,55555)",
+ "insert into root.vehicle.d0(timestamp,s2) values(2,2.22)",
+ "insert into root.vehicle.d0(timestamp,s2) values(3,3.33)",
+ "insert into root.vehicle.d0(timestamp,s2) values(4,4.44)",
+ "insert into root.vehicle.d0(timestamp,s2) values(102,10.00)",
+ "insert into root.vehicle.d0(timestamp,s2) values(105,11.11)",
+ "insert into root.vehicle.d0(timestamp,s2) values(1000,1000.11)",
+
+ "insert into root.vehicle.d0(timestamp,s3) values(60,'aaaaa')",
+ "insert into root.vehicle.d0(timestamp,s3) values(70,'bbbbb')",
+ "insert into root.vehicle.d0(timestamp,s3) values(80,'ccccc')",
+ "insert into root.vehicle.d0(timestamp,s3) values(101,'ddddd')",
+ "insert into root.vehicle.d0(timestamp,s3) values(102,'fffff')",
+
+ "insert into root.vehicle.d0(timestamp,s4) values(23, false)",
+ "insert into root.vehicle.d0(timestamp,s4) values(100, true)",
+
+ "insert into root.vehicle.d1(timestamp,s0) values(1,999)",
+ "insert into root.vehicle.d1(timestamp,s0) values(1000,10)",
+
+ "insert into root.vehicle.d1(timestamp,s1) values(2,9999)",
+ "insert into root.vehicle.d1(timestamp,s1) values(1000,5)",
+
+ "insert into root.vehicle.d1(timestamp,s2) values(2,12345.6)",
+ "insert into root.vehicle.d1(timestamp,s2) values(2222,2.22)",
+
+ "insert into root.vehicle.d1(timestamp,s3) values(10,'ten')",
+ "insert into root.vehicle.d1(timestamp,s3) values(1000,'thousand')",
+
+ "insert into root.vehicle.d1(timestamp,s4) values(100, false)",
+ "insert into root.vehicle.d1(timestamp,s4) values(10000, true)",
+
+ "insert into root.vehicle.d0(timestamp,s1) values(2000-01-01T08:00:00+08:00, 100)",
+ "insert into root.vehicle.d0(timestamp,s3) values(2000-01-01T08:00:00+08:00, 'good')",
+
+ "insert into root.other.d1(timestamp,s0) values(2, 3.14)",};
+
+ @BeforeClass
+ public static void setUp() throws Exception {
+ EnvironmentUtils.closeStatMonitor();
+ EnvironmentUtils.envSetUp();
+
+ insertData();
+ }
+
+ @AfterClass
+ public static void tearDown() throws Exception {
+ EnvironmentUtils.cleanEnv();
+ }
+
+ private static void insertData() throws ClassNotFoundException {
+ Class.forName(Config.JDBC_DRIVER_NAME);
+ try (Connection connection = DriverManager
+ .getConnection(Config.IOTDB_URL_PREFIX + "127.0.0.1:6667/", "root", "root");
+ Statement statement = connection.createStatement()) {
+
+ for (String sql : sqls) {
+ statement.execute(sql);
+ }
+ } catch (Exception e) {
+ e.printStackTrace();
+ }
+ }
+
+ @Test
+ public void testSelect() {
+ CalciteAssert.that()
+ .with(MODEL)
+ .with("UnQuotedCasing", IoTDBConstant.UNQUOTED_CASING)
+ .query("select * from \"root.vehicle\"")
Review comment:
According to the calcite doc:
> SQL conformance level. Values: DEFAULT (the default, similar to PRAGMATIC_2003)
It could be customed by properties.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] Alima777 commented on a change in pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
Alima777 commented on a change in pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#discussion_r487762769
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBEnumerator.java
##########
@@ -0,0 +1,169 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import java.sql.ResultSet;
+import java.sql.ResultSetMetaData;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.List;
+import org.apache.calcite.linq4j.Enumerator;
+import org.apache.calcite.rel.type.RelDataTypeFactory;
+import org.apache.calcite.rel.type.RelDataTypeField;
+import org.apache.calcite.rel.type.RelDataTypeSystem;
+import org.apache.calcite.rel.type.RelProtoDataType;
+import org.apache.calcite.sql.type.SqlTypeFactoryImpl;
+import org.apache.calcite.sql.type.SqlTypeName;
+
+public class IoTDBEnumerator implements Enumerator<Object> {
+
+ private ResultSet currentResultSet;
+ private Iterator<ResultSet> iterator;
+ private List<Integer> indexInResultSet = new ArrayList<>();
+ private List<RelDataTypeField> fieldTypes;
+
+ /**
+ * Creates a IoTDBEnumerator.
+ *
+ * @param resultList IoTDB result set
+ * @param protoRowType The type of protecting rows
+ */
+ IoTDBEnumerator(List<ResultSet> resultList, RelProtoDataType protoRowType) throws SQLException {
+ this.iterator = resultList.iterator();
+ if (iterator.hasNext()) {
+ this.currentResultSet = iterator.next();
+ }
+
+ final RelDataTypeFactory typeFactory =
+ new SqlTypeFactoryImpl(RelDataTypeSystem.DEFAULT);
+ this.fieldTypes = protoRowType.apply(typeFactory).getFieldList();
+
+ // get the corresponding index of project columns in result set
+ // as we have 'time' and 'device' columns in result that the project columns may don't include
+ // e.g. If resultSet includes [time, device, s0, s1], project columns only include [device, s0, s1],
+ // then the indexInResultSet is [2, 3, 4].
+ ResultSetMetaData metaData = currentResultSet.getMetaData();
+ int indexInFieldTypes = 0;
+ for (int i = 1; i <= metaData.getColumnCount(); i++) {
+ if (i <= 2 && !metaData.getColumnName(i).toLowerCase()
+ .equals(fieldTypes.get(indexInFieldTypes).getName())) {
+ continue;
+ } else {
+ indexInFieldTypes++;
+ indexInResultSet.add(i);
+ }
+ }
+ }
+
+ /**
+ * Produce and get the next row from the results
+ *
+ * @return A new row from the results
+ */
+ @Override
+ public Object current() {
+ if (fieldTypes.size() == 1) {
+ // If we just have one field, produce it directly
+ return currentRowField(indexInResultSet.get(0), fieldTypes.get(0).getType().getSqlTypeName());
+ } else {
+ // Build an array with all fields in this row
+ Object[] row = new Object[fieldTypes.size()];
+ for (int i = 0; i < fieldTypes.size(); i++) {
+ row[i] = currentRowField(indexInResultSet.get(i),
+ fieldTypes.get(i).getType().getSqlTypeName());
+ }
+
+ return row;
+ }
+ }
+
+ /**
+ * Get a field for the current row from the underlying object.
+ *
+ * @param index Index of the field within the Row object
+ * @param type Type of the field in this row
+ */
+ private Object currentRowField(int index, SqlTypeName type) {
+ try {
+ switch (type) {
+ case VARCHAR:
+ return currentResultSet.getString(index);
+ case INTEGER:
+ return currentResultSet.getInt(index);
+ case BIGINT:
+ return currentResultSet.getLong(index);
+ case DOUBLE:
+ return currentResultSet.getDouble(index);
+ case REAL:
+ return currentResultSet.getFloat(index);
+ case BOOLEAN:
+ return currentResultSet.getBoolean(index);
+ default:
+ return null;
+ }
+ } catch (SQLException e) {
+ if (e.getMessage().endsWith("NULL.")) {
+ return null;
+ } else {
+ e.printStackTrace();
Review comment:
Yes! I will fix it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] Alima777 commented on a change in pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
Alima777 commented on a change in pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#discussion_r495533642
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBSchema.java
##########
@@ -0,0 +1,167 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import com.google.common.collect.ImmutableMap;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.calcite.rel.type.RelDataTypeFactory;
+import org.apache.calcite.rel.type.RelDataTypeImpl;
+import org.apache.calcite.rel.type.RelDataTypeSystem;
+import org.apache.calcite.rel.type.RelProtoDataType;
+import org.apache.calcite.schema.SchemaPlus;
+import org.apache.calcite.schema.Table;
+import org.apache.calcite.schema.impl.AbstractSchema;
+import org.apache.calcite.sql.type.SqlTypeFactoryImpl;
+import org.apache.iotdb.db.exception.query.QueryProcessException;
+import org.apache.iotdb.jdbc.Config;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class IoTDBSchema extends AbstractSchema {
+
+ private static final Logger logger = LoggerFactory.getLogger(IoTDBSchema.class);
+ final Connection connection;
+ private Map<String, Table> tableMap;
+ private final SchemaPlus parentSchema;
+ final String name;
+ public static Map<String, List<String>> sgToDeviceMap = new HashMap<>();
+
+ /**
+ * Creates a IoTDB schema.
+ *
+ * @param host IoTDB host, e.g. "localhost"
+ * @param port IoTDB port, e.g. 6667
+ * @param username IoTDB username
+ * @param password IoTDB password
+ */
+ public IoTDBSchema(String host, int port, String username, String password,
+ SchemaPlus parentSchema, String name) {
+ super();
+ try {
+ Class.forName(Config.JDBC_DRIVER_NAME);
+ this.connection = DriverManager
+ .getConnection(Config.IOTDB_URL_PREFIX + host + ":" + port + "/", username, password);
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+
+ this.parentSchema = parentSchema;
+ this.name = name;
+ }
+
+ /**
+ * Generate the columns' names and data types in the given table.
+ * @param storageGroup the table name
+ * @return the columns' names and data types
+ */
+ RelProtoDataType getRelDataType(String storageGroup) throws SQLException, QueryProcessException {
Review comment:
Hi, it's explained as follows in Calcite:
> // Temporary type factory, just for the duration of this method. Allowable
> // because we're creating a proto-type, not a type; before being used, the
> // proto-type will be copied into a real type factory.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] Alima777 commented on a change in pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
Alima777 commented on a change in pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#discussion_r495535581
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBSchema.java
##########
@@ -0,0 +1,167 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import com.google.common.collect.ImmutableMap;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.calcite.rel.type.RelDataTypeFactory;
+import org.apache.calcite.rel.type.RelDataTypeImpl;
+import org.apache.calcite.rel.type.RelDataTypeSystem;
+import org.apache.calcite.rel.type.RelProtoDataType;
+import org.apache.calcite.schema.SchemaPlus;
+import org.apache.calcite.schema.Table;
+import org.apache.calcite.schema.impl.AbstractSchema;
+import org.apache.calcite.sql.type.SqlTypeFactoryImpl;
+import org.apache.iotdb.db.exception.query.QueryProcessException;
+import org.apache.iotdb.jdbc.Config;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class IoTDBSchema extends AbstractSchema {
+
+ private static final Logger logger = LoggerFactory.getLogger(IoTDBSchema.class);
+ final Connection connection;
+ private Map<String, Table> tableMap;
+ private final SchemaPlus parentSchema;
+ final String name;
+ public static Map<String, List<String>> sgToDeviceMap = new HashMap<>();
+
+ /**
+ * Creates a IoTDB schema.
+ *
+ * @param host IoTDB host, e.g. "localhost"
+ * @param port IoTDB port, e.g. 6667
+ * @param username IoTDB username
+ * @param password IoTDB password
+ */
+ public IoTDBSchema(String host, int port, String username, String password,
+ SchemaPlus parentSchema, String name) {
+ super();
+ try {
+ Class.forName(Config.JDBC_DRIVER_NAME);
+ this.connection = DriverManager
+ .getConnection(Config.IOTDB_URL_PREFIX + host + ":" + port + "/", username, password);
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+
+ this.parentSchema = parentSchema;
+ this.name = name;
+ }
+
+ /**
+ * Generate the columns' names and data types in the given table.
+ * @param storageGroup the table name
+ * @return the columns' names and data types
+ */
+ RelProtoDataType getRelDataType(String storageGroup) throws SQLException, QueryProcessException {
Review comment:
> Generally, if i understand it correctly we map the whole storage group to a single table where we have one column for the device and then unroll all possible sensors of each device as a column, is that right?
Yes, that's right.
> This could be avoided in situations where the query is something like
SELECT * FROM 'root.vehicles' where device = 'mycar' as we only have this single device there.
As you mentioned, it does could be avoided in this situation. But before querying, we have to build a relational table here whose column data type has to be just one. The table will be validated by Calcite, in this situation which type of sensors should we adopt?
So I think the same sensor with different types should be avoided. If some queries are really needed, we can build a view in model file.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] Alima777 commented on a change in pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
Alima777 commented on a change in pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#discussion_r495533642
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBSchema.java
##########
@@ -0,0 +1,167 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import com.google.common.collect.ImmutableMap;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.calcite.rel.type.RelDataTypeFactory;
+import org.apache.calcite.rel.type.RelDataTypeImpl;
+import org.apache.calcite.rel.type.RelDataTypeSystem;
+import org.apache.calcite.rel.type.RelProtoDataType;
+import org.apache.calcite.schema.SchemaPlus;
+import org.apache.calcite.schema.Table;
+import org.apache.calcite.schema.impl.AbstractSchema;
+import org.apache.calcite.sql.type.SqlTypeFactoryImpl;
+import org.apache.iotdb.db.exception.query.QueryProcessException;
+import org.apache.iotdb.jdbc.Config;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class IoTDBSchema extends AbstractSchema {
+
+ private static final Logger logger = LoggerFactory.getLogger(IoTDBSchema.class);
+ final Connection connection;
+ private Map<String, Table> tableMap;
+ private final SchemaPlus parentSchema;
+ final String name;
+ public static Map<String, List<String>> sgToDeviceMap = new HashMap<>();
+
+ /**
+ * Creates a IoTDB schema.
+ *
+ * @param host IoTDB host, e.g. "localhost"
+ * @param port IoTDB port, e.g. 6667
+ * @param username IoTDB username
+ * @param password IoTDB password
+ */
+ public IoTDBSchema(String host, int port, String username, String password,
+ SchemaPlus parentSchema, String name) {
+ super();
+ try {
+ Class.forName(Config.JDBC_DRIVER_NAME);
+ this.connection = DriverManager
+ .getConnection(Config.IOTDB_URL_PREFIX + host + ":" + port + "/", username, password);
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+
+ this.parentSchema = parentSchema;
+ this.name = name;
+ }
+
+ /**
+ * Generate the columns' names and data types in the given table.
+ * @param storageGroup the table name
+ * @return the columns' names and data types
+ */
+ RelProtoDataType getRelDataType(String storageGroup) throws SQLException, QueryProcessException {
Review comment:
Hi, it's explained as follows in Calcite:
```
// Temporary type factory, just for the duration of this method. Allowable
// because we're creating a proto-type, not a type; before being used, the
// proto-type will be copied into a real type factory.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] yuqi1129 edited a comment on pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
yuqi1129 edited a comment on pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#issuecomment-691814857
@Test
```
public void testFilter7() {
CalciteAssert.that()
.with(MODEL)
.with("UnQuotedCasing", IoTDBConstant.UnQuotedCasing)
.query("select * from \"root.vehicle\" " +
"where (device = 'root.vehicle.d0' AND \"time\" <= 1) OR s2 = 2.22 and 1 = 1")
.returns("time=1; device=root.vehicle.d0; s0=101; s1=1101; s2=null; s3=null; s4=null\n" +
"time=2; device=root.vehicle.d0; s0=10000; s1=40000; s2=2.22; s3=null; s4=null\n" +
"time=2222; device=root.vehicle.d1; s0=null; s1=null; s2=2.22; s3=null; s4=null\n")
.explainContains("PLAN=IoTDBToEnumerableConverter\n" +
" IoTDBFilter(condition=[OR(AND(=($1, 'root.vehicle.d0'), <=($0, 1)), =(CAST($4):DOUBLE NOT NULL, 2.22))])\n" +
" IoTDBTableScan(table=[[IoTDBSchema, root.vehicle]])");
}
```
Should add rules to reduce constant value
like where 1 = 1 and a = 1 should be folded as where a = 1, see above
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] Alima777 commented on a change in pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
Alima777 commented on a change in pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#discussion_r487763220
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBConstant.java
##########
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+public class IoTDBConstant {
+
+ public static final String AlignByDevice = "align by device";
Review comment:
Agree with you. I've fixed it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] Alima777 commented on a change in pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
Alima777 commented on a change in pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#discussion_r495534402
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBSchema.java
##########
@@ -0,0 +1,167 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import com.google.common.collect.ImmutableMap;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.calcite.rel.type.RelDataTypeFactory;
+import org.apache.calcite.rel.type.RelDataTypeImpl;
+import org.apache.calcite.rel.type.RelDataTypeSystem;
+import org.apache.calcite.rel.type.RelProtoDataType;
+import org.apache.calcite.schema.SchemaPlus;
+import org.apache.calcite.schema.Table;
+import org.apache.calcite.schema.impl.AbstractSchema;
+import org.apache.calcite.sql.type.SqlTypeFactoryImpl;
+import org.apache.iotdb.db.exception.query.QueryProcessException;
+import org.apache.iotdb.jdbc.Config;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class IoTDBSchema extends AbstractSchema {
+
+ private static final Logger logger = LoggerFactory.getLogger(IoTDBSchema.class);
+ final Connection connection;
+ private Map<String, Table> tableMap;
+ private final SchemaPlus parentSchema;
+ final String name;
+ public static Map<String, List<String>> sgToDeviceMap = new HashMap<>();
+
+ /**
+ * Creates a IoTDB schema.
+ *
+ * @param host IoTDB host, e.g. "localhost"
+ * @param port IoTDB port, e.g. 6667
+ * @param username IoTDB username
+ * @param password IoTDB password
+ */
+ public IoTDBSchema(String host, int port, String username, String password,
+ SchemaPlus parentSchema, String name) {
+ super();
+ try {
+ Class.forName(Config.JDBC_DRIVER_NAME);
+ this.connection = DriverManager
+ .getConnection(Config.IOTDB_URL_PREFIX + host + ":" + port + "/", username, password);
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+
+ this.parentSchema = parentSchema;
+ this.name = name;
+ }
+
+ /**
+ * Generate the columns' names and data types in the given table.
+ * @param storageGroup the table name
+ * @return the columns' names and data types
+ */
+ RelProtoDataType getRelDataType(String storageGroup) throws SQLException, QueryProcessException {
Review comment:
But actually I don't get the point from this... So I just copied the implementation way.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] Alima777 commented on pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
Alima777 commented on pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#issuecomment-691922941
@yuqi1129 Hi, thank you very much for your patient review!
Since it's my first large implemented module, I ignored lots of code standards, like: exception processing, standard code style and something else... And It has not been maintained for a long time.
So Thanks again! I've fixed it and please have a check.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] Alima777 edited a comment on pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
Alima777 edited a comment on pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#issuecomment-691922941
@yuqi1129 Hi, thank you very much for your patient review!
Since it's the first large module I implemented, ignoring lots of code standards, like: exception processing, standard code style and something else... And It has not been maintained for a long time.
So Thanks again! I've fixed it and please have a check.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] Alima777 commented on a change in pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
Alima777 commented on a change in pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#discussion_r487762459
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBRel.java
##########
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.calcite.plan.Convention;
+import org.apache.calcite.plan.RelOptTable;
+import org.apache.calcite.rel.RelNode;
+
+public interface IoTDBRel extends RelNode {
+
+ void implement(Implementor implementor);
+
+ /**
+ * Calling convention for relational operations that occur in IoTDB.
+ */
+ Convention CONVENTION = new Convention.Impl("IOTDB", IoTDBRel.class);
+
+ /**
+ * Callback for the implementation process that converts a tree of {@link IoTDBRel} nodes into a
+ * IoTDB SQL query.
+ */
+ class Implementor {
+
+ final List<String> selectFields = new ArrayList<>();
+ final Map<String, String> deviceToFilterMap = new LinkedHashMap<>();
+ final List<String> globalPredicate = new ArrayList<>();
+ int limit = 0;
+ int offset = 0;
+
+ RelOptTable table;
+ IoTDBTable ioTDBTable;
+
+ /**
+ * Adds newly projected fields.
+ *
+ * @param fields New fields to be projected from a query
+ */
+ public void addFields(List<String> fields) {
+ if (selectFields != null) {
+ selectFields.addAll(fields);
+ }
+ }
+
+ /**
+ * Adds newly restricted devices and predicates.
+ *
+ * @param deviceToFilterMap predicate of given device
+ * @param predicates global predicates to be applied to the query
+ */
+ public void add(Map<String, String> deviceToFilterMap, List<String> predicates) {
+ if (this.deviceToFilterMap != null) {
Review comment:
You are right! I've removed it.
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBTable.java
##########
@@ -0,0 +1,306 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import com.google.common.collect.ImmutableList;
+import java.sql.Connection;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import org.apache.calcite.adapter.java.AbstractQueryableTable;
+import org.apache.calcite.linq4j.AbstractEnumerable;
+import org.apache.calcite.linq4j.Enumerable;
+import org.apache.calcite.linq4j.Enumerator;
+import org.apache.calcite.linq4j.QueryProvider;
+import org.apache.calcite.linq4j.Queryable;
+import org.apache.calcite.linq4j.function.Function1;
+import org.apache.calcite.plan.RelOptCluster;
+import org.apache.calcite.plan.RelOptTable;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rel.type.RelDataTypeFactory;
+import org.apache.calcite.rel.type.RelDataTypeImpl;
+import org.apache.calcite.rel.type.RelDataTypeSystem;
+import org.apache.calcite.rel.type.RelProtoDataType;
+import org.apache.calcite.schema.SchemaPlus;
+import org.apache.calcite.schema.TranslatableTable;
+import org.apache.calcite.schema.impl.AbstractTableQueryable;
+import org.apache.calcite.sql.type.SqlTypeFactoryImpl;
+import org.apache.calcite.sql.type.SqlTypeName;
+import org.apache.calcite.util.Util;
+import org.apache.iotdb.db.exception.query.QueryProcessException;
+
+public class IoTDBTable extends AbstractQueryableTable
+ implements TranslatableTable {
+
+ RelProtoDataType protoRowType;
+ private final IoTDBSchema schema;
+ private final String storageGroup;
+
+ public IoTDBTable(IoTDBSchema schema, String storageGroup) {
+ super(Object[].class);
+ this.schema = schema;
+ this.storageGroup = storageGroup;
+ }
+
+ public String toString() {
+ return "IoTDBTable {" + storageGroup + "}";
+ }
+
+ @Override
+ public RelDataType getRowType(RelDataTypeFactory typeFactory) {
+ try {
+ if (protoRowType == null) {
+ protoRowType = schema.getRelDataType(storageGroup);
+ }
+ } catch (SQLException | QueryProcessException e) {
+ // print exception error here
+ e.printStackTrace();
+ }
+ return protoRowType.apply(typeFactory);
+ }
+
+ @Override
+ public <T> Queryable<T> asQueryable(QueryProvider queryProvider, SchemaPlus schema,
+ String tableName) {
+ return new IoTDBQueryable<>(queryProvider, schema, this, storageGroup);
+ }
+
+ @Override
+ public RelNode toRel(RelOptTable.ToRelContext context, RelOptTable relOptTable) {
+ final RelOptCluster cluster = context.getCluster();
+ return new IoTDBTableScan(cluster, cluster.traitSetOf(IoTDBRel.CONVENTION),
+ relOptTable, this, null);
+ }
+
+ public Enumerable<Object> query(final Connection connection) {
+ return query(connection, ImmutableList.of(), ImmutableList.of(), ImmutableList.of(),
+ ImmutableList.of(), 0, 0);
+ }
+
+ /**
+ * Executes a IoTDB SQL query.
+ *
+ * @param connection IoTDB connection
+ * @param fields List of fields to project
+ * @return Enumerator of results
+ */
+ public Enumerable<Object> query(final Connection connection,
+ List<Map.Entry<String, Class>> fields,
+ final List<String> selectFields, final List<Map.Entry<String, String>> deviceToFilterList,
+ List<String> globalPredicates, final Integer limit, final Integer offset) {
+ // Build the type of the resulting row based on the provided fields
+ final RelDataTypeFactory typeFactory =
+ new SqlTypeFactoryImpl(RelDataTypeSystem.DEFAULT);
+ final RelDataTypeFactory.Builder fieldInfo = typeFactory.builder();
+ final RelDataType rowType = getRowType(typeFactory);
+
+ Function1<String, Void> addField = fieldName -> {
+ SqlTypeName typeName =
+ rowType.getField(fieldName, true, false).getType().getSqlTypeName();
+ fieldInfo.add(fieldName, typeFactory.createSqlType(typeName))
+ .nullable(true);
+ return null;
+ };
+
+ if (selectFields.isEmpty()) {
+ for (Map.Entry<String, Class> field : fields) {
+ addField.apply(field.getKey());
+ }
+ } else {
+ for (String field : selectFields) {
+ if (field.startsWith("'")) {
+ fieldInfo.add(field, SqlTypeName.VARCHAR);
+ } else {
+ addField.apply(field);
+ }
+ }
+ }
+
+ final RelProtoDataType resultRowType = RelDataTypeImpl.proto(fieldInfo.build());
+
+ // Construct the list of fields to project
+ String selectString = "";
+ if (selectFields.isEmpty()) {
+ selectString = "*";
+ } else {
+ // delete the 'time', 'device' string in select list
+ // this has to be here rather than init "selectFields" otherwise the resultRowType will be wrong
+ StringBuilder selectBuilder = new StringBuilder();
+ for (int i = 0; i < selectFields.size(); i++) {
+ String selectField = selectFields.get(i);
+ if (selectField.equals(IoTDBConstant.DeviceColumn) || selectField
+ .equals(IoTDBConstant.TimeColumn)) {
+ continue;
+ } else {
+ selectBuilder.append(selectField);
+ if (i < selectFields.size() - 1) {
+ selectBuilder.append(", ");
+ }
+ }
+ }
+ selectString = selectBuilder.toString();
+ if (selectString.equals("")) {
+ selectString = "*";
+ }
+ }
+
+ List<String> queryList = new ArrayList<>();
+ Set<String> tmpDevices = new HashSet<>(); // to deduplicate in global query
+ // construct query by device
+ if (!deviceToFilterList.isEmpty()) {
+ for (Entry<String, String> deviceToFilter : deviceToFilterList) {
+ String fromClause = " FROM ";
+ fromClause += deviceToFilter.getKey();
+ tmpDevices.add(deviceToFilter.getKey());
+
+ String whereClause = "";
+ if (deviceToFilter.getValue() != null) {
+ whereClause = " WHERE ";
+ whereClause += deviceToFilter.getValue();
+ }
+
+ // Build and issue the query and return an Enumerator over the results
+ StringBuilder queryBuilder = new StringBuilder("SELECT ");
+ queryBuilder.append(selectString);
+ queryBuilder.append(fromClause);
+ queryBuilder.append(whereClause);
+
+ if (limit > 0) {
+ queryBuilder.append(" LIMIT " + limit);
+ }
+ if (offset > 0) {
+ queryBuilder.append(" OFFSET " + offset);
+ }
+
+ // append align by device
+ queryBuilder.append(" " + IoTDBConstant.AlignByDevice);
+ queryList.add(queryBuilder.toString());
+ }
+ }
+
+ // construct global query
+ if (deviceToFilterList.isEmpty() || !globalPredicates.isEmpty()) {
+ String fromClause = " FROM ";
+ // deduplicate redundant device
+ if (!deviceToFilterList.isEmpty() && !globalPredicates.isEmpty()) {
Review comment:
Removed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] Alima777 commented on a change in pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
Alima777 commented on a change in pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#discussion_r495533642
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBSchema.java
##########
@@ -0,0 +1,167 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import com.google.common.collect.ImmutableMap;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.calcite.rel.type.RelDataTypeFactory;
+import org.apache.calcite.rel.type.RelDataTypeImpl;
+import org.apache.calcite.rel.type.RelDataTypeSystem;
+import org.apache.calcite.rel.type.RelProtoDataType;
+import org.apache.calcite.schema.SchemaPlus;
+import org.apache.calcite.schema.Table;
+import org.apache.calcite.schema.impl.AbstractSchema;
+import org.apache.calcite.sql.type.SqlTypeFactoryImpl;
+import org.apache.iotdb.db.exception.query.QueryProcessException;
+import org.apache.iotdb.jdbc.Config;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class IoTDBSchema extends AbstractSchema {
+
+ private static final Logger logger = LoggerFactory.getLogger(IoTDBSchema.class);
+ final Connection connection;
+ private Map<String, Table> tableMap;
+ private final SchemaPlus parentSchema;
+ final String name;
+ public static Map<String, List<String>> sgToDeviceMap = new HashMap<>();
+
+ /**
+ * Creates a IoTDB schema.
+ *
+ * @param host IoTDB host, e.g. "localhost"
+ * @param port IoTDB port, e.g. 6667
+ * @param username IoTDB username
+ * @param password IoTDB password
+ */
+ public IoTDBSchema(String host, int port, String username, String password,
+ SchemaPlus parentSchema, String name) {
+ super();
+ try {
+ Class.forName(Config.JDBC_DRIVER_NAME);
+ this.connection = DriverManager
+ .getConnection(Config.IOTDB_URL_PREFIX + host + ":" + port + "/", username, password);
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+
+ this.parentSchema = parentSchema;
+ this.name = name;
+ }
+
+ /**
+ * Generate the columns' names and data types in the given table.
+ * @param storageGroup the table name
+ * @return the columns' names and data types
+ */
+ RelProtoDataType getRelDataType(String storageGroup) throws SQLException, QueryProcessException {
Review comment:
Hi, it's explained as follows in Calcite:
> // Temporary type factory, just for the duration of this method. Allowable
// because we're creating a proto-type, not a type; before being used, the
// proto-type will be copied into a real type factory.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] JulianFeinauer commented on a change in pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
JulianFeinauer commented on a change in pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#discussion_r495433577
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBSchema.java
##########
@@ -0,0 +1,167 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import com.google.common.collect.ImmutableMap;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.calcite.rel.type.RelDataTypeFactory;
+import org.apache.calcite.rel.type.RelDataTypeImpl;
+import org.apache.calcite.rel.type.RelDataTypeSystem;
+import org.apache.calcite.rel.type.RelProtoDataType;
+import org.apache.calcite.schema.SchemaPlus;
+import org.apache.calcite.schema.Table;
+import org.apache.calcite.schema.impl.AbstractSchema;
+import org.apache.calcite.sql.type.SqlTypeFactoryImpl;
+import org.apache.iotdb.db.exception.query.QueryProcessException;
+import org.apache.iotdb.jdbc.Config;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class IoTDBSchema extends AbstractSchema {
+
+ private static final Logger logger = LoggerFactory.getLogger(IoTDBSchema.class);
+ final Connection connection;
+ private Map<String, Table> tableMap;
+ private final SchemaPlus parentSchema;
+ final String name;
+ public static Map<String, List<String>> sgToDeviceMap = new HashMap<>();
+
+ /**
+ * Creates a IoTDB schema.
+ *
+ * @param host IoTDB host, e.g. "localhost"
+ * @param port IoTDB port, e.g. 6667
+ * @param username IoTDB username
+ * @param password IoTDB password
+ */
+ public IoTDBSchema(String host, int port, String username, String password,
+ SchemaPlus parentSchema, String name) {
+ super();
+ try {
+ Class.forName(Config.JDBC_DRIVER_NAME);
+ this.connection = DriverManager
+ .getConnection(Config.IOTDB_URL_PREFIX + host + ":" + port + "/", username, password);
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+
+ this.parentSchema = parentSchema;
+ this.name = name;
+ }
+
+ /**
+ * Generate the columns' names and data types in the given table.
+ * @param storageGroup the table name
+ * @return the columns' names and data types
+ */
+ RelProtoDataType getRelDataType(String storageGroup) throws SQLException, QueryProcessException {
Review comment:
I would pass the RelDataTypeFactory here via Parameter (from calling IoTDBTable and not create a new one). WDYT?
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBSchema.java
##########
@@ -0,0 +1,167 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import com.google.common.collect.ImmutableMap;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.calcite.rel.type.RelDataTypeFactory;
+import org.apache.calcite.rel.type.RelDataTypeImpl;
+import org.apache.calcite.rel.type.RelDataTypeSystem;
+import org.apache.calcite.rel.type.RelProtoDataType;
+import org.apache.calcite.schema.SchemaPlus;
+import org.apache.calcite.schema.Table;
+import org.apache.calcite.schema.impl.AbstractSchema;
+import org.apache.calcite.sql.type.SqlTypeFactoryImpl;
+import org.apache.iotdb.db.exception.query.QueryProcessException;
+import org.apache.iotdb.jdbc.Config;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class IoTDBSchema extends AbstractSchema {
+
+ private static final Logger logger = LoggerFactory.getLogger(IoTDBSchema.class);
+ final Connection connection;
+ private Map<String, Table> tableMap;
+ private final SchemaPlus parentSchema;
+ final String name;
+ public static Map<String, List<String>> sgToDeviceMap = new HashMap<>();
+
+ /**
+ * Creates a IoTDB schema.
+ *
+ * @param host IoTDB host, e.g. "localhost"
+ * @param port IoTDB port, e.g. 6667
+ * @param username IoTDB username
+ * @param password IoTDB password
+ */
+ public IoTDBSchema(String host, int port, String username, String password,
+ SchemaPlus parentSchema, String name) {
+ super();
+ try {
+ Class.forName(Config.JDBC_DRIVER_NAME);
+ this.connection = DriverManager
+ .getConnection(Config.IOTDB_URL_PREFIX + host + ":" + port + "/", username, password);
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+
+ this.parentSchema = parentSchema;
+ this.name = name;
+ }
+
+ /**
+ * Generate the columns' names and data types in the given table.
+ * @param storageGroup the table name
+ * @return the columns' names and data types
+ */
+ RelProtoDataType getRelDataType(String storageGroup) throws SQLException, QueryProcessException {
Review comment:
A rather general question about the data model itself. Generally, if i understand it correctly we map the whole storage group to a single table where we have one column for the device and then unroll all possible sensors of each device as a column, is that right?
And if so, this could lead to issues in cases where I have two devices which have the same sensors but with different types (see Line 119 here). This could be avoided in situations where the query is something like
`SELECT * FROM 'root.vehicles' where device = 'mycar'` as we only have this single device there. What do you think, should we, perhaps in a later PR try to push the query further down to to evaluate the `RelDataType` with further information to avoid the situation descibed above?
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBRules.java
##########
@@ -0,0 +1,192 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import java.util.List;
+import org.apache.calcite.adapter.enumerable.EnumerableLimit;
+import org.apache.calcite.adapter.java.JavaTypeFactory;
+import org.apache.calcite.plan.Convention;
+import org.apache.calcite.plan.RelOptRule;
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelTraitSet;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.convert.ConverterRule;
+import org.apache.calcite.rel.logical.LogicalFilter;
+import org.apache.calcite.rel.logical.LogicalProject;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.rex.RexVisitorImpl;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.validate.SqlValidatorUtil;
+import org.apache.iotdb.db.exception.query.LogicalOptimizeException;
+
+/**
+ * Rules and relational operators for {@link IoTDBRel#CONVENTION} calling convention.
+ */
+public class IoTDBRules {
+
+ private IoTDBRules() {
+ }
+
+ public static final RelOptRule[] RULES = {
+ IoTDBFilterRule.INSTANCE,
+ IoTDBProjectRule.INSTANCE,
Review comment:
I am very unshure about this Rule or, the use of a custom Project Rule in general. By default Calcite will transform a LogicalProject into EnumerableCalc Rule which especially implements all Rex-Operations (like addition, substraction, ...).
In the current Implementation the `IoTDBProject` implements the default `RexVisitorImpl` which translates any relational expression to its first operand. So, e.g. `a + b` which is the REX `+(a, b)` will simply be transformed to `a`.
If we remove this rule by commenting out L51 Calcite will use the default `EnumerableCalc` and Rex Expressions will work as expected. You can easily see that by adding a test case like the following:
```
@Test
public void testProject() {
CalciteAssert.that()
.with(MODEL)
.with("UnQuotedCasing", IoTDBConstant.UNQUOTED_CASING)
.query("select *, s0 + s1 as test from \"root.vehicle\" " +
"where s0 <= 10")
.limit(1)
.returns("time=1000; device=root.vehicle.d1; s0=10; s1=5; s2=null; s3=thousand; s4=null; test=15\n")
.explainContains("PLAN=EnumerableCalc(expr#0..6=[{inputs}], expr#7=[+($t2, $t3)], proj#0..7=[{exprs}])\n" +
" IoTDBToEnumerableConverter\n" +
" IoTDBFilter(condition=[<=($2, 10)])\n" +
" IoTDBTableScan(table=[[IoTDBSchema, root.vehicle]])");
}
```
If the IotDBProjectRule is active, then this will give the wrong result 5 (only s1), without this RULE (commenting out L51) this will give the correct result 15 and the test will pass.
Is there a specific reason why you decided to implement the Project Rel explicitly? We could do this partial at least to push down aggregate queries of course but I would do this in a second step as I personally consider rex expressions "more important" so to say.
WDYT?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] yuqi1129 commented on a change in pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
yuqi1129 commented on a change in pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#discussion_r487632569
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBEnumerator.java
##########
@@ -0,0 +1,169 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import java.sql.ResultSet;
+import java.sql.ResultSetMetaData;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.List;
+import org.apache.calcite.linq4j.Enumerator;
+import org.apache.calcite.rel.type.RelDataTypeFactory;
+import org.apache.calcite.rel.type.RelDataTypeField;
+import org.apache.calcite.rel.type.RelDataTypeSystem;
+import org.apache.calcite.rel.type.RelProtoDataType;
+import org.apache.calcite.sql.type.SqlTypeFactoryImpl;
+import org.apache.calcite.sql.type.SqlTypeName;
+
+public class IoTDBEnumerator implements Enumerator<Object> {
+
+ private ResultSet currentResultSet;
+ private Iterator<ResultSet> iterator;
+ private List<Integer> indexInResultSet = new ArrayList<>();
+ private List<RelDataTypeField> fieldTypes;
+
+ /**
+ * Creates a IoTDBEnumerator.
+ *
+ * @param resultList IoTDB result set
+ * @param protoRowType The type of protecting rows
+ */
+ IoTDBEnumerator(List<ResultSet> resultList, RelProtoDataType protoRowType) throws SQLException {
+ this.iterator = resultList.iterator();
+ if (iterator.hasNext()) {
+ this.currentResultSet = iterator.next();
+ }
+
+ final RelDataTypeFactory typeFactory =
+ new SqlTypeFactoryImpl(RelDataTypeSystem.DEFAULT);
+ this.fieldTypes = protoRowType.apply(typeFactory).getFieldList();
+
+ // get the corresponding index of project columns in result set
+ // as we have 'time' and 'device' columns in result that the project columns may don't include
+ // e.g. If resultSet includes [time, device, s0, s1], project columns only include [device, s0, s1],
+ // then the indexInResultSet is [2, 3, 4].
+ ResultSetMetaData metaData = currentResultSet.getMetaData();
+ int indexInFieldTypes = 0;
+ for (int i = 1; i <= metaData.getColumnCount(); i++) {
+ if (i <= 2 && !metaData.getColumnName(i).toLowerCase()
+ .equals(fieldTypes.get(indexInFieldTypes).getName())) {
+ continue;
+ } else {
+ indexInFieldTypes++;
+ indexInResultSet.add(i);
+ }
+ }
+ }
+
+ /**
+ * Produce and get the next row from the results
+ *
+ * @return A new row from the results
+ */
+ @Override
+ public Object current() {
+ if (fieldTypes.size() == 1) {
+ // If we just have one field, produce it directly
+ return currentRowField(indexInResultSet.get(0), fieldTypes.get(0).getType().getSqlTypeName());
+ } else {
+ // Build an array with all fields in this row
+ Object[] row = new Object[fieldTypes.size()];
+ for (int i = 0; i < fieldTypes.size(); i++) {
+ row[i] = currentRowField(indexInResultSet.get(i),
+ fieldTypes.get(i).getType().getSqlTypeName());
+ }
+
+ return row;
+ }
+ }
+
+ /**
+ * Get a field for the current row from the underlying object.
+ *
+ * @param index Index of the field within the Row object
+ * @param type Type of the field in this row
+ */
+ private Object currentRowField(int index, SqlTypeName type) {
+ try {
+ switch (type) {
+ case VARCHAR:
+ return currentResultSet.getString(index);
+ case INTEGER:
+ return currentResultSet.getInt(index);
+ case BIGINT:
+ return currentResultSet.getLong(index);
+ case DOUBLE:
+ return currentResultSet.getDouble(index);
+ case REAL:
+ return currentResultSet.getFloat(index);
+ case BOOLEAN:
+ return currentResultSet.getBoolean(index);
+ default:
+ return null;
+ }
+ } catch (SQLException e) {
+ if (e.getMessage().endsWith("NULL.")) {
+ return null;
+ } else {
+ e.printStackTrace();
Review comment:
use log ?
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBTable.java
##########
@@ -0,0 +1,306 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import com.google.common.collect.ImmutableList;
+import java.sql.Connection;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import org.apache.calcite.adapter.java.AbstractQueryableTable;
+import org.apache.calcite.linq4j.AbstractEnumerable;
+import org.apache.calcite.linq4j.Enumerable;
+import org.apache.calcite.linq4j.Enumerator;
+import org.apache.calcite.linq4j.QueryProvider;
+import org.apache.calcite.linq4j.Queryable;
+import org.apache.calcite.linq4j.function.Function1;
+import org.apache.calcite.plan.RelOptCluster;
+import org.apache.calcite.plan.RelOptTable;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rel.type.RelDataTypeFactory;
+import org.apache.calcite.rel.type.RelDataTypeImpl;
+import org.apache.calcite.rel.type.RelDataTypeSystem;
+import org.apache.calcite.rel.type.RelProtoDataType;
+import org.apache.calcite.schema.SchemaPlus;
+import org.apache.calcite.schema.TranslatableTable;
+import org.apache.calcite.schema.impl.AbstractTableQueryable;
+import org.apache.calcite.sql.type.SqlTypeFactoryImpl;
+import org.apache.calcite.sql.type.SqlTypeName;
+import org.apache.calcite.util.Util;
+import org.apache.iotdb.db.exception.query.QueryProcessException;
+
+public class IoTDBTable extends AbstractQueryableTable
+ implements TranslatableTable {
+
+ RelProtoDataType protoRowType;
+ private final IoTDBSchema schema;
+ private final String storageGroup;
+
+ public IoTDBTable(IoTDBSchema schema, String storageGroup) {
+ super(Object[].class);
+ this.schema = schema;
+ this.storageGroup = storageGroup;
+ }
+
+ public String toString() {
+ return "IoTDBTable {" + storageGroup + "}";
+ }
+
+ @Override
+ public RelDataType getRowType(RelDataTypeFactory typeFactory) {
+ try {
+ if (protoRowType == null) {
+ protoRowType = schema.getRelDataType(storageGroup);
+ }
+ } catch (SQLException | QueryProcessException e) {
+ // print exception error here
+ e.printStackTrace();
+ }
+ return protoRowType.apply(typeFactory);
+ }
+
+ @Override
+ public <T> Queryable<T> asQueryable(QueryProvider queryProvider, SchemaPlus schema,
+ String tableName) {
+ return new IoTDBQueryable<>(queryProvider, schema, this, storageGroup);
+ }
+
+ @Override
+ public RelNode toRel(RelOptTable.ToRelContext context, RelOptTable relOptTable) {
+ final RelOptCluster cluster = context.getCluster();
+ return new IoTDBTableScan(cluster, cluster.traitSetOf(IoTDBRel.CONVENTION),
+ relOptTable, this, null);
+ }
+
+ public Enumerable<Object> query(final Connection connection) {
+ return query(connection, ImmutableList.of(), ImmutableList.of(), ImmutableList.of(),
+ ImmutableList.of(), 0, 0);
+ }
+
+ /**
+ * Executes a IoTDB SQL query.
+ *
+ * @param connection IoTDB connection
+ * @param fields List of fields to project
+ * @return Enumerator of results
+ */
+ public Enumerable<Object> query(final Connection connection,
+ List<Map.Entry<String, Class>> fields,
+ final List<String> selectFields, final List<Map.Entry<String, String>> deviceToFilterList,
+ List<String> globalPredicates, final Integer limit, final Integer offset) {
+ // Build the type of the resulting row based on the provided fields
+ final RelDataTypeFactory typeFactory =
+ new SqlTypeFactoryImpl(RelDataTypeSystem.DEFAULT);
+ final RelDataTypeFactory.Builder fieldInfo = typeFactory.builder();
+ final RelDataType rowType = getRowType(typeFactory);
+
+ Function1<String, Void> addField = fieldName -> {
+ SqlTypeName typeName =
+ rowType.getField(fieldName, true, false).getType().getSqlTypeName();
+ fieldInfo.add(fieldName, typeFactory.createSqlType(typeName))
+ .nullable(true);
+ return null;
+ };
+
+ if (selectFields.isEmpty()) {
+ for (Map.Entry<String, Class> field : fields) {
+ addField.apply(field.getKey());
+ }
+ } else {
+ for (String field : selectFields) {
+ if (field.startsWith("'")) {
+ fieldInfo.add(field, SqlTypeName.VARCHAR);
+ } else {
+ addField.apply(field);
+ }
+ }
+ }
+
+ final RelProtoDataType resultRowType = RelDataTypeImpl.proto(fieldInfo.build());
+
+ // Construct the list of fields to project
+ String selectString = "";
+ if (selectFields.isEmpty()) {
+ selectString = "*";
+ } else {
+ // delete the 'time', 'device' string in select list
+ // this has to be here rather than init "selectFields" otherwise the resultRowType will be wrong
+ StringBuilder selectBuilder = new StringBuilder();
+ for (int i = 0; i < selectFields.size(); i++) {
+ String selectField = selectFields.get(i);
+ if (selectField.equals(IoTDBConstant.DeviceColumn) || selectField
+ .equals(IoTDBConstant.TimeColumn)) {
+ continue;
+ } else {
+ selectBuilder.append(selectField);
+ if (i < selectFields.size() - 1) {
+ selectBuilder.append(", ");
+ }
+ }
+ }
+ selectString = selectBuilder.toString();
+ if (selectString.equals("")) {
+ selectString = "*";
+ }
+ }
+
+ List<String> queryList = new ArrayList<>();
+ Set<String> tmpDevices = new HashSet<>(); // to deduplicate in global query
+ // construct query by device
+ if (!deviceToFilterList.isEmpty()) {
+ for (Entry<String, String> deviceToFilter : deviceToFilterList) {
+ String fromClause = " FROM ";
+ fromClause += deviceToFilter.getKey();
+ tmpDevices.add(deviceToFilter.getKey());
+
+ String whereClause = "";
+ if (deviceToFilter.getValue() != null) {
+ whereClause = " WHERE ";
+ whereClause += deviceToFilter.getValue();
+ }
+
+ // Build and issue the query and return an Enumerator over the results
+ StringBuilder queryBuilder = new StringBuilder("SELECT ");
+ queryBuilder.append(selectString);
+ queryBuilder.append(fromClause);
+ queryBuilder.append(whereClause);
+
+ if (limit > 0) {
+ queryBuilder.append(" LIMIT " + limit);
+ }
+ if (offset > 0) {
+ queryBuilder.append(" OFFSET " + offset);
+ }
+
+ // append align by device
+ queryBuilder.append(" " + IoTDBConstant.AlignByDevice);
+ queryList.add(queryBuilder.toString());
+ }
+ }
+
+ // construct global query
+ if (deviceToFilterList.isEmpty() || !globalPredicates.isEmpty()) {
+ String fromClause = " FROM ";
+ // deduplicate redundant device
+ if (!deviceToFilterList.isEmpty() && !globalPredicates.isEmpty()) {
Review comment:
`!globalPredicates.isEmpty()` always true when comes here
##########
File path: calcite/pom.xml
##########
@@ -0,0 +1,99 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <artifactId>iotdb-parent</artifactId>
+ <groupId>org.apache.iotdb</groupId>
+ <version>0.10.0-SNAPSHOT</version>
+ <relativePath>../pom.xml</relativePath>
+ </parent>
+ <artifactId>iotdb-calcite</artifactId>
+ <version>0.10.0-SNAPSHOT</version>
+ <packaging>jar</packaging>
+ <description>IoTDB adapter for Calcite</description>
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.calcite</groupId>
+ <artifactId>calcite-core</artifactId>
+ <version>1.21.0</version>
Review comment:
better to use place holder like ${xxx.versionn} than direct constant value
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBConstant.java
##########
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+public class IoTDBConstant {
+
+ public static final String AlignByDevice = "align by device";
Review comment:
i advice to use upper name to represent `static final` value
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBTable.java
##########
@@ -0,0 +1,306 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import com.google.common.collect.ImmutableList;
+import java.sql.Connection;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import org.apache.calcite.adapter.java.AbstractQueryableTable;
+import org.apache.calcite.linq4j.AbstractEnumerable;
+import org.apache.calcite.linq4j.Enumerable;
+import org.apache.calcite.linq4j.Enumerator;
+import org.apache.calcite.linq4j.QueryProvider;
+import org.apache.calcite.linq4j.Queryable;
+import org.apache.calcite.linq4j.function.Function1;
+import org.apache.calcite.plan.RelOptCluster;
+import org.apache.calcite.plan.RelOptTable;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rel.type.RelDataTypeFactory;
+import org.apache.calcite.rel.type.RelDataTypeImpl;
+import org.apache.calcite.rel.type.RelDataTypeSystem;
+import org.apache.calcite.rel.type.RelProtoDataType;
+import org.apache.calcite.schema.SchemaPlus;
+import org.apache.calcite.schema.TranslatableTable;
+import org.apache.calcite.schema.impl.AbstractTableQueryable;
+import org.apache.calcite.sql.type.SqlTypeFactoryImpl;
+import org.apache.calcite.sql.type.SqlTypeName;
+import org.apache.calcite.util.Util;
+import org.apache.iotdb.db.exception.query.QueryProcessException;
+
+public class IoTDBTable extends AbstractQueryableTable
+ implements TranslatableTable {
+
+ RelProtoDataType protoRowType;
+ private final IoTDBSchema schema;
+ private final String storageGroup;
+
+ public IoTDBTable(IoTDBSchema schema, String storageGroup) {
+ super(Object[].class);
+ this.schema = schema;
+ this.storageGroup = storageGroup;
+ }
+
+ public String toString() {
+ return "IoTDBTable {" + storageGroup + "}";
+ }
+
+ @Override
+ public RelDataType getRowType(RelDataTypeFactory typeFactory) {
+ try {
+ if (protoRowType == null) {
+ protoRowType = schema.getRelDataType(storageGroup);
+ }
+ } catch (SQLException | QueryProcessException e) {
+ // print exception error here
+ e.printStackTrace();
+ }
+ return protoRowType.apply(typeFactory);
+ }
+
+ @Override
+ public <T> Queryable<T> asQueryable(QueryProvider queryProvider, SchemaPlus schema,
+ String tableName) {
+ return new IoTDBQueryable<>(queryProvider, schema, this, storageGroup);
+ }
+
+ @Override
+ public RelNode toRel(RelOptTable.ToRelContext context, RelOptTable relOptTable) {
+ final RelOptCluster cluster = context.getCluster();
+ return new IoTDBTableScan(cluster, cluster.traitSetOf(IoTDBRel.CONVENTION),
+ relOptTable, this, null);
+ }
+
+ public Enumerable<Object> query(final Connection connection) {
+ return query(connection, ImmutableList.of(), ImmutableList.of(), ImmutableList.of(),
+ ImmutableList.of(), 0, 0);
+ }
+
+ /**
+ * Executes a IoTDB SQL query.
+ *
+ * @param connection IoTDB connection
+ * @param fields List of fields to project
+ * @return Enumerator of results
+ */
+ public Enumerable<Object> query(final Connection connection,
+ List<Map.Entry<String, Class>> fields,
+ final List<String> selectFields, final List<Map.Entry<String, String>> deviceToFilterList,
+ List<String> globalPredicates, final Integer limit, final Integer offset) {
+ // Build the type of the resulting row based on the provided fields
+ final RelDataTypeFactory typeFactory =
+ new SqlTypeFactoryImpl(RelDataTypeSystem.DEFAULT);
+ final RelDataTypeFactory.Builder fieldInfo = typeFactory.builder();
+ final RelDataType rowType = getRowType(typeFactory);
+
+ Function1<String, Void> addField = fieldName -> {
+ SqlTypeName typeName =
+ rowType.getField(fieldName, true, false).getType().getSqlTypeName();
+ fieldInfo.add(fieldName, typeFactory.createSqlType(typeName))
+ .nullable(true);
+ return null;
+ };
+
+ if (selectFields.isEmpty()) {
+ for (Map.Entry<String, Class> field : fields) {
+ addField.apply(field.getKey());
+ }
+ } else {
+ for (String field : selectFields) {
+ if (field.startsWith("'")) {
+ fieldInfo.add(field, SqlTypeName.VARCHAR);
+ } else {
+ addField.apply(field);
+ }
+ }
+ }
+
+ final RelProtoDataType resultRowType = RelDataTypeImpl.proto(fieldInfo.build());
+
+ // Construct the list of fields to project
+ String selectString = "";
+ if (selectFields.isEmpty()) {
+ selectString = "*";
+ } else {
+ // delete the 'time', 'device' string in select list
+ // this has to be here rather than init "selectFields" otherwise the resultRowType will be wrong
+ StringBuilder selectBuilder = new StringBuilder();
+ for (int i = 0; i < selectFields.size(); i++) {
+ String selectField = selectFields.get(i);
+ if (selectField.equals(IoTDBConstant.DeviceColumn) || selectField
+ .equals(IoTDBConstant.TimeColumn)) {
+ continue;
Review comment:
Unnecessary
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBEnumerator.java
##########
@@ -0,0 +1,169 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import java.sql.ResultSet;
+import java.sql.ResultSetMetaData;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.List;
+import org.apache.calcite.linq4j.Enumerator;
+import org.apache.calcite.rel.type.RelDataTypeFactory;
+import org.apache.calcite.rel.type.RelDataTypeField;
+import org.apache.calcite.rel.type.RelDataTypeSystem;
+import org.apache.calcite.rel.type.RelProtoDataType;
+import org.apache.calcite.sql.type.SqlTypeFactoryImpl;
+import org.apache.calcite.sql.type.SqlTypeName;
+
+public class IoTDBEnumerator implements Enumerator<Object> {
+
+ private ResultSet currentResultSet;
+ private Iterator<ResultSet> iterator;
+ private List<Integer> indexInResultSet = new ArrayList<>();
+ private List<RelDataTypeField> fieldTypes;
+
+ /**
+ * Creates a IoTDBEnumerator.
+ *
+ * @param resultList IoTDB result set
+ * @param protoRowType The type of protecting rows
+ */
+ IoTDBEnumerator(List<ResultSet> resultList, RelProtoDataType protoRowType) throws SQLException {
+ this.iterator = resultList.iterator();
+ if (iterator.hasNext()) {
+ this.currentResultSet = iterator.next();
+ }
+
+ final RelDataTypeFactory typeFactory =
+ new SqlTypeFactoryImpl(RelDataTypeSystem.DEFAULT);
+ this.fieldTypes = protoRowType.apply(typeFactory).getFieldList();
+
+ // get the corresponding index of project columns in result set
+ // as we have 'time' and 'device' columns in result that the project columns may don't include
+ // e.g. If resultSet includes [time, device, s0, s1], project columns only include [device, s0, s1],
+ // then the indexInResultSet is [2, 3, 4].
+ ResultSetMetaData metaData = currentResultSet.getMetaData();
+ int indexInFieldTypes = 0;
+ for (int i = 1; i <= metaData.getColumnCount(); i++) {
+ if (i <= 2 && !metaData.getColumnName(i).toLowerCase()
+ .equals(fieldTypes.get(indexInFieldTypes).getName())) {
+ continue;
Review comment:
Unnecessary `continue`
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBTable.java
##########
@@ -0,0 +1,306 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import com.google.common.collect.ImmutableList;
+import java.sql.Connection;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import org.apache.calcite.adapter.java.AbstractQueryableTable;
+import org.apache.calcite.linq4j.AbstractEnumerable;
+import org.apache.calcite.linq4j.Enumerable;
+import org.apache.calcite.linq4j.Enumerator;
+import org.apache.calcite.linq4j.QueryProvider;
+import org.apache.calcite.linq4j.Queryable;
+import org.apache.calcite.linq4j.function.Function1;
+import org.apache.calcite.plan.RelOptCluster;
+import org.apache.calcite.plan.RelOptTable;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rel.type.RelDataTypeFactory;
+import org.apache.calcite.rel.type.RelDataTypeImpl;
+import org.apache.calcite.rel.type.RelDataTypeSystem;
+import org.apache.calcite.rel.type.RelProtoDataType;
+import org.apache.calcite.schema.SchemaPlus;
+import org.apache.calcite.schema.TranslatableTable;
+import org.apache.calcite.schema.impl.AbstractTableQueryable;
+import org.apache.calcite.sql.type.SqlTypeFactoryImpl;
+import org.apache.calcite.sql.type.SqlTypeName;
+import org.apache.calcite.util.Util;
+import org.apache.iotdb.db.exception.query.QueryProcessException;
+
+public class IoTDBTable extends AbstractQueryableTable
+ implements TranslatableTable {
+
+ RelProtoDataType protoRowType;
+ private final IoTDBSchema schema;
+ private final String storageGroup;
+
+ public IoTDBTable(IoTDBSchema schema, String storageGroup) {
+ super(Object[].class);
+ this.schema = schema;
+ this.storageGroup = storageGroup;
+ }
+
+ public String toString() {
+ return "IoTDBTable {" + storageGroup + "}";
+ }
+
+ @Override
+ public RelDataType getRowType(RelDataTypeFactory typeFactory) {
+ try {
+ if (protoRowType == null) {
+ protoRowType = schema.getRelDataType(storageGroup);
+ }
+ } catch (SQLException | QueryProcessException e) {
+ // print exception error here
+ e.printStackTrace();
+ }
+ return protoRowType.apply(typeFactory);
+ }
+
+ @Override
+ public <T> Queryable<T> asQueryable(QueryProvider queryProvider, SchemaPlus schema,
+ String tableName) {
+ return new IoTDBQueryable<>(queryProvider, schema, this, storageGroup);
+ }
+
+ @Override
+ public RelNode toRel(RelOptTable.ToRelContext context, RelOptTable relOptTable) {
+ final RelOptCluster cluster = context.getCluster();
+ return new IoTDBTableScan(cluster, cluster.traitSetOf(IoTDBRel.CONVENTION),
+ relOptTable, this, null);
+ }
+
+ public Enumerable<Object> query(final Connection connection) {
+ return query(connection, ImmutableList.of(), ImmutableList.of(), ImmutableList.of(),
+ ImmutableList.of(), 0, 0);
Review comment:
Why limit and offset are 0?
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBRel.java
##########
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.calcite.plan.Convention;
+import org.apache.calcite.plan.RelOptTable;
+import org.apache.calcite.rel.RelNode;
+
+public interface IoTDBRel extends RelNode {
+
+ void implement(Implementor implementor);
+
+ /**
+ * Calling convention for relational operations that occur in IoTDB.
+ */
+ Convention CONVENTION = new Convention.Impl("IOTDB", IoTDBRel.class);
+
+ /**
+ * Callback for the implementation process that converts a tree of {@link IoTDBRel} nodes into a
+ * IoTDB SQL query.
+ */
+ class Implementor {
+
+ final List<String> selectFields = new ArrayList<>();
+ final Map<String, String> deviceToFilterMap = new LinkedHashMap<>();
+ final List<String> globalPredicate = new ArrayList<>();
+ int limit = 0;
+ int offset = 0;
+
+ RelOptTable table;
+ IoTDBTable ioTDBTable;
+
+ /**
+ * Adds newly projected fields.
+ *
+ * @param fields New fields to be projected from a query
+ */
+ public void addFields(List<String> fields) {
+ if (selectFields != null) {
+ selectFields.addAll(fields);
+ }
+ }
+
+ /**
+ * Adds newly restricted devices and predicates.
+ *
+ * @param deviceToFilterMap predicate of given device
+ * @param predicates global predicates to be applied to the query
+ */
+ public void add(Map<String, String> deviceToFilterMap, List<String> predicates) {
+ if (this.deviceToFilterMap != null) {
Review comment:
As `deviceToFilterMap ` and `globalPredicate` are marked as `final`, there is not need to make null check
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBFilter.java
##########
@@ -0,0 +1,321 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.calcite.plan.RelOptCluster;
+import org.apache.calcite.plan.RelOptCost;
+import org.apache.calcite.plan.RelOptPlanner;
+import org.apache.calcite.plan.RelTraitSet;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Filter;
+import org.apache.calcite.rel.metadata.RelMetadataQuery;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexLiteral;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.util.Util;
+import org.apache.iotdb.db.exception.query.LogicalOptimizeException;
+import org.apache.iotdb.db.qp.constant.SQLConstant;
+import org.apache.iotdb.db.qp.logical.crud.BasicFunctionOperator;
+import org.apache.iotdb.db.qp.logical.crud.FilterOperator;
+import org.apache.iotdb.db.qp.strategy.optimizer.DnfFilterOptimizer;
+import org.apache.iotdb.tsfile.read.common.Path;
+
+public class IoTDBFilter extends Filter implements IoTDBRel {
+
+ private final FilterOperator filterOperator;
+ private final List<String> fieldNames;
+ private List<String> predicates; // for global predicate
+ Map<String, String> deviceToFilterMap = new LinkedHashMap<>(); // for device predicate
+
+ protected IoTDBFilter(RelOptCluster cluster, RelTraitSet traits, RelNode child, RexNode condition)
+ throws LogicalOptimizeException {
+ super(cluster, traits, child, condition);
+ this.fieldNames = IoTDBRules.IoTDBFieldNames(getRowType());
+ this.filterOperator = getIoTDBOperator(condition);
+ this.predicates = translateWhere(filterOperator);
+
+ // add global predicate to each device if both global and device predicate exist
+ if (!this.predicates.isEmpty() && !this.deviceToFilterMap.isEmpty()) {
+ for (String device : deviceToFilterMap.keySet()) {
+ StringBuilder builder = new StringBuilder(this.deviceToFilterMap.get(device));
+ builder.append(Util.toString(predicates, " OR ", " OR ", ""));
+ this.deviceToFilterMap.put(device, builder.toString());
+ }
+ }
+ assert getConvention() == IoTDBRel.CONVENTION;
+ assert getConvention() == child.getConvention();
+ }
+
+ @Override
+ public RelOptCost computeSelfCost(RelOptPlanner planner,
+ RelMetadataQuery mq) {
+ return super.computeSelfCost(planner, mq).multiplyBy(0.1);
+ }
+
+ @Override
+ public Filter copy(RelTraitSet relTraitSet, RelNode input, RexNode condition) {
+ try {
+ return new IoTDBFilter(getCluster(), traitSet, input, condition);
+ } catch (LogicalOptimizeException e) {
+ throw new AssertionError(e.getMessage());
+ }
+ }
+
+ @Override
+ public void implement(Implementor implementor) {
+ implementor.visitChild(0, getInput());
+ implementor.add(deviceToFilterMap, predicates);
+ }
+
+ /**
+ * Translates {@link RexNode} expressions into IoTDB filter operator.
+ */
+ private FilterOperator getIoTDBOperator(RexNode filter) {
+ switch (filter.getKind()) {
+ case EQUALS:
+ return getBasicOperator(SQLConstant.EQUAL, (RexCall) filter);
+ case NOT_EQUALS:
+ return getBasicOperator(SQLConstant.NOTEQUAL, (RexCall) filter);
+ case GREATER_THAN:
+ return getBasicOperator(SQLConstant.GREATERTHAN, (RexCall) filter);
+ case GREATER_THAN_OR_EQUAL:
+ return getBasicOperator(SQLConstant.GREATERTHANOREQUALTO, (RexCall) filter);
+ case LESS_THAN:
+ return getBasicOperator(SQLConstant.LESSTHAN, (RexCall) filter);
+ case LESS_THAN_OR_EQUAL:
+ return getBasicOperator(SQLConstant.LESSTHANOREQUALTO, (RexCall) filter);
+ case AND:
+ return getBinaryOperator(SQLConstant.KW_AND, ((RexCall) filter).getOperands());
+ case OR:
+ return getBinaryOperator(SQLConstant.KW_OR, ((RexCall) filter).getOperands());
+ default:
+ throw new AssertionError("cannot get IoTDBOperator from " + filter);
Review comment:
why just throws error not exception ?
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBSchema.java
##########
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import com.google.common.collect.ImmutableMap;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.calcite.rel.type.RelDataTypeFactory;
+import org.apache.calcite.rel.type.RelDataTypeImpl;
+import org.apache.calcite.rel.type.RelDataTypeSystem;
+import org.apache.calcite.rel.type.RelProtoDataType;
+import org.apache.calcite.schema.SchemaPlus;
+import org.apache.calcite.schema.Table;
+import org.apache.calcite.schema.impl.AbstractSchema;
+import org.apache.calcite.sql.type.SqlTypeFactoryImpl;
+import org.apache.iotdb.db.exception.query.QueryProcessException;
+import org.apache.iotdb.jdbc.Config;
+
+public class IoTDBSchema extends AbstractSchema {
+
+ final Connection connection;
+ private Map<String, Table> tableMap;
+ private final SchemaPlus parentSchema;
+ final String name;
+ public static Map<String, List<String>> sgToDeviceMap = new HashMap<>();
+
+ /**
+ * Creates a IoTDB schema.
+ *
+ * @param host IoTDB host, e.g. "localhost"
+ * @param port IoTDB port, e.g. 6667
+ * @param username IoTDB username
+ * @param password IoTDB password
+ */
+ public IoTDBSchema(String host, int port, String username, String password,
+ SchemaPlus parentSchema, String name) {
+ super();
+ try {
+ Class.forName(Config.JDBC_DRIVER_NAME);
+ this.connection = DriverManager
+ .getConnection(Config.IOTDB_URL_PREFIX + host + ":" + port + "/", username, password);
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+
+ this.parentSchema = parentSchema;
+ this.name = name;
+ }
+
+ /**
+ * Generate the columns' names and data types in the given table.
+ * @param storageGroup the table name
+ * @return the columns' names and data types
+ */
+ RelProtoDataType getRelDataType(String storageGroup) throws SQLException, QueryProcessException {
+ final RelDataTypeFactory typeFactory =
+ new SqlTypeFactoryImpl(RelDataTypeSystem.DEFAULT);
+ final RelDataTypeFactory.Builder fieldInfo = typeFactory.builder();
+
+ // add time, device columns in relational table
+ fieldInfo.add(IoTDBConstant.TimeColumn,
+ typeFactory.createSqlType(IoTDBFieldType.INT64.getSqlType()));
+ fieldInfo.add(IoTDBConstant.DeviceColumn,
+ typeFactory.createSqlType(IoTDBFieldType.TEXT.getSqlType()));
+
+ // get devices in this storage group
+ Statement statement = connection.createStatement();
+ boolean hasDevices = statement.execute("show devices " + storageGroup);
+ if (hasDevices) {
+ ResultSet devices = statement.getResultSet();
+ List<String> deviceList = new ArrayList<>();
+ while (devices.next()) {
+ deviceList.add(devices.getString(1));
+ }
+ this.sgToDeviceMap.put(storageGroup, deviceList);
+ }
+
+ // get deduplicated measurements in table
+ boolean hasTS = statement.execute("show timeseries " + storageGroup);
+ if (hasTS) {
+ ResultSet timeseries = statement.getResultSet();
+ Map<String, IoTDBFieldType> tmpMeasurementMap = new HashMap<>();
+ while (timeseries.next()) {
+ String sensorName = timeseries.getString(1);
+ sensorName = sensorName.substring(sensorName.lastIndexOf('.') + 1);
+ IoTDBFieldType sensorType = IoTDBFieldType.of(timeseries.getString(4));
+
+ if (!tmpMeasurementMap.containsKey(sensorName)) {
+ tmpMeasurementMap.put(sensorName, sensorType);
+ fieldInfo.add(sensorName, typeFactory.createSqlType(sensorType.getSqlType()));
+ } else {
+ if (!tmpMeasurementMap.get(sensorName).equals(sensorType)) {
+ throw new QueryProcessException(
+ "The data types of the same measurement column should be the same across "
+ + "devices in ALIGN_BY_DEVICE sql. For more details please refer to the "
+ + "SQL document.");
+ }
+ }
+ }
+ }
+
+ return RelDataTypeImpl.proto(fieldInfo.build());
+ }
+
+ /**
+ * Generate a map whose key is table name and value is table instance. The implementations of
+ * AbstractSchema.getTableNames() and AbstractSchema.getTable(String) depend on this map.
+ */
+ @Override
+ protected Map<String, Table> getTableMap() {
+ try {
+ if (tableMap == null) {
+ tableMap = createTableMap();
+ }
+ } catch (SQLException e) {
+ e.printStackTrace();
Review comment:
log
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBFilter.java
##########
@@ -0,0 +1,321 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.calcite.plan.RelOptCluster;
+import org.apache.calcite.plan.RelOptCost;
+import org.apache.calcite.plan.RelOptPlanner;
+import org.apache.calcite.plan.RelTraitSet;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Filter;
+import org.apache.calcite.rel.metadata.RelMetadataQuery;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexLiteral;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.util.Util;
+import org.apache.iotdb.db.exception.query.LogicalOptimizeException;
+import org.apache.iotdb.db.qp.constant.SQLConstant;
+import org.apache.iotdb.db.qp.logical.crud.BasicFunctionOperator;
+import org.apache.iotdb.db.qp.logical.crud.FilterOperator;
+import org.apache.iotdb.db.qp.strategy.optimizer.DnfFilterOptimizer;
+import org.apache.iotdb.tsfile.read.common.Path;
+
+public class IoTDBFilter extends Filter implements IoTDBRel {
+
+ private final FilterOperator filterOperator;
+ private final List<String> fieldNames;
+ private List<String> predicates; // for global predicate
+ Map<String, String> deviceToFilterMap = new LinkedHashMap<>(); // for device predicate
+
+ protected IoTDBFilter(RelOptCluster cluster, RelTraitSet traits, RelNode child, RexNode condition)
+ throws LogicalOptimizeException {
+ super(cluster, traits, child, condition);
+ this.fieldNames = IoTDBRules.IoTDBFieldNames(getRowType());
+ this.filterOperator = getIoTDBOperator(condition);
+ this.predicates = translateWhere(filterOperator);
+
+ // add global predicate to each device if both global and device predicate exist
+ if (!this.predicates.isEmpty() && !this.deviceToFilterMap.isEmpty()) {
+ for (String device : deviceToFilterMap.keySet()) {
+ StringBuilder builder = new StringBuilder(this.deviceToFilterMap.get(device));
+ builder.append(Util.toString(predicates, " OR ", " OR ", ""));
+ this.deviceToFilterMap.put(device, builder.toString());
+ }
+ }
+ assert getConvention() == IoTDBRel.CONVENTION;
+ assert getConvention() == child.getConvention();
+ }
+
+ @Override
+ public RelOptCost computeSelfCost(RelOptPlanner planner,
+ RelMetadataQuery mq) {
+ return super.computeSelfCost(planner, mq).multiplyBy(0.1);
+ }
+
+ @Override
+ public Filter copy(RelTraitSet relTraitSet, RelNode input, RexNode condition) {
+ try {
+ return new IoTDBFilter(getCluster(), traitSet, input, condition);
+ } catch (LogicalOptimizeException e) {
+ throw new AssertionError(e.getMessage());
+ }
+ }
+
+ @Override
+ public void implement(Implementor implementor) {
+ implementor.visitChild(0, getInput());
+ implementor.add(deviceToFilterMap, predicates);
+ }
+
+ /**
+ * Translates {@link RexNode} expressions into IoTDB filter operator.
+ */
+ private FilterOperator getIoTDBOperator(RexNode filter) {
+ switch (filter.getKind()) {
+ case EQUALS:
+ return getBasicOperator(SQLConstant.EQUAL, (RexCall) filter);
+ case NOT_EQUALS:
+ return getBasicOperator(SQLConstant.NOTEQUAL, (RexCall) filter);
+ case GREATER_THAN:
+ return getBasicOperator(SQLConstant.GREATERTHAN, (RexCall) filter);
+ case GREATER_THAN_OR_EQUAL:
+ return getBasicOperator(SQLConstant.GREATERTHANOREQUALTO, (RexCall) filter);
+ case LESS_THAN:
+ return getBasicOperator(SQLConstant.LESSTHAN, (RexCall) filter);
+ case LESS_THAN_OR_EQUAL:
+ return getBasicOperator(SQLConstant.LESSTHANOREQUALTO, (RexCall) filter);
+ case AND:
+ return getBinaryOperator(SQLConstant.KW_AND, ((RexCall) filter).getOperands());
+ case OR:
+ return getBinaryOperator(SQLConstant.KW_OR, ((RexCall) filter).getOperands());
+ default:
+ throw new AssertionError("cannot get IoTDBOperator from " + filter);
+ }
+ }
+
+ private FilterOperator getBasicOperator(int tokenIntType, RexCall call) {
+ final RexNode left = call.operands.get(0);
+ final RexNode right = call.operands.get(1);
+
+ FilterOperator operator = getBasicOperator2(tokenIntType, left, right);
+ if (operator != null) {
+ return operator;
+ }
+ operator = getBasicOperator2(tokenIntType, right, left);
+ if (operator != null) {
+ return operator;
+ }
+ throw new AssertionError("cannot translate basic operator: " + call);
+ }
+
+ private FilterOperator getBasicOperator2(int tokenIntType, RexNode left, RexNode right) {
+ switch (right.getKind()) {
+ case LITERAL:
+ break;
+ default:
+ return null;
+ }
+ final RexLiteral rightLiteral = (RexLiteral) right;
+ switch (left.getKind()) {
+ case INPUT_REF:
+ String name = fieldNames.get(((RexInputRef) left).getIndex());
+ return new BasicFunctionOperator(tokenIntType, new Path(name), literalValue(rightLiteral));
+ case CAST:
+ return getBasicOperator2(tokenIntType, ((RexCall) left).getOperands().get(0), right);
+ default:
+ return null;
+ }
+
+ }
+
+ private FilterOperator getBinaryOperator(int tokenIntType, List<RexNode> operands) {
+ FilterOperator filterBinaryTree = new FilterOperator(tokenIntType);
+ FilterOperator currentNode = filterBinaryTree;
+ for (int i = 0; i < operands.size(); i++) {
Review comment:
better to add a local variance `size = operands.size()` to avoid frequently called in loop
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] Alima777 commented on pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
Alima777 commented on pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#issuecomment-691788390
Hi, @yuqi1129 would you like to have a review of this PR? :D
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] Alima777 commented on a change in pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
Alima777 commented on a change in pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#discussion_r487762031
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBFilter.java
##########
@@ -0,0 +1,321 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.calcite.plan.RelOptCluster;
+import org.apache.calcite.plan.RelOptCost;
+import org.apache.calcite.plan.RelOptPlanner;
+import org.apache.calcite.plan.RelTraitSet;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Filter;
+import org.apache.calcite.rel.metadata.RelMetadataQuery;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexLiteral;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.util.Util;
+import org.apache.iotdb.db.exception.query.LogicalOptimizeException;
+import org.apache.iotdb.db.qp.constant.SQLConstant;
+import org.apache.iotdb.db.qp.logical.crud.BasicFunctionOperator;
+import org.apache.iotdb.db.qp.logical.crud.FilterOperator;
+import org.apache.iotdb.db.qp.strategy.optimizer.DnfFilterOptimizer;
+import org.apache.iotdb.tsfile.read.common.Path;
+
+public class IoTDBFilter extends Filter implements IoTDBRel {
+
+ private final FilterOperator filterOperator;
+ private final List<String> fieldNames;
+ private List<String> predicates; // for global predicate
+ Map<String, String> deviceToFilterMap = new LinkedHashMap<>(); // for device predicate
+
+ protected IoTDBFilter(RelOptCluster cluster, RelTraitSet traits, RelNode child, RexNode condition)
+ throws LogicalOptimizeException {
+ super(cluster, traits, child, condition);
+ this.fieldNames = IoTDBRules.IoTDBFieldNames(getRowType());
+ this.filterOperator = getIoTDBOperator(condition);
+ this.predicates = translateWhere(filterOperator);
+
+ // add global predicate to each device if both global and device predicate exist
+ if (!this.predicates.isEmpty() && !this.deviceToFilterMap.isEmpty()) {
+ for (String device : deviceToFilterMap.keySet()) {
+ StringBuilder builder = new StringBuilder(this.deviceToFilterMap.get(device));
+ builder.append(Util.toString(predicates, " OR ", " OR ", ""));
+ this.deviceToFilterMap.put(device, builder.toString());
+ }
+ }
+ assert getConvention() == IoTDBRel.CONVENTION;
+ assert getConvention() == child.getConvention();
+ }
+
+ @Override
+ public RelOptCost computeSelfCost(RelOptPlanner planner,
+ RelMetadataQuery mq) {
+ return super.computeSelfCost(planner, mq).multiplyBy(0.1);
+ }
+
+ @Override
+ public Filter copy(RelTraitSet relTraitSet, RelNode input, RexNode condition) {
+ try {
+ return new IoTDBFilter(getCluster(), traitSet, input, condition);
+ } catch (LogicalOptimizeException e) {
+ throw new AssertionError(e.getMessage());
+ }
+ }
+
+ @Override
+ public void implement(Implementor implementor) {
+ implementor.visitChild(0, getInput());
+ implementor.add(deviceToFilterMap, predicates);
+ }
+
+ /**
+ * Translates {@link RexNode} expressions into IoTDB filter operator.
+ */
+ private FilterOperator getIoTDBOperator(RexNode filter) {
+ switch (filter.getKind()) {
+ case EQUALS:
+ return getBasicOperator(SQLConstant.EQUAL, (RexCall) filter);
+ case NOT_EQUALS:
+ return getBasicOperator(SQLConstant.NOTEQUAL, (RexCall) filter);
+ case GREATER_THAN:
+ return getBasicOperator(SQLConstant.GREATERTHAN, (RexCall) filter);
+ case GREATER_THAN_OR_EQUAL:
+ return getBasicOperator(SQLConstant.GREATERTHANOREQUALTO, (RexCall) filter);
+ case LESS_THAN:
+ return getBasicOperator(SQLConstant.LESSTHAN, (RexCall) filter);
+ case LESS_THAN_OR_EQUAL:
+ return getBasicOperator(SQLConstant.LESSTHANOREQUALTO, (RexCall) filter);
+ case AND:
+ return getBinaryOperator(SQLConstant.KW_AND, ((RexCall) filter).getOperands());
+ case OR:
+ return getBinaryOperator(SQLConstant.KW_OR, ((RexCall) filter).getOperands());
+ default:
+ throw new AssertionError("cannot get IoTDBOperator from " + filter);
Review comment:
I just used it for convenience before...Sorry, I've added a new exception for it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] Alima777 commented on a change in pull request #902: [IOTDB-28] Calcite Integration for IoTDB
Posted by GitBox <gi...@apache.org>.
Alima777 commented on a change in pull request #902:
URL: https://github.com/apache/incubator-iotdb/pull/902#discussion_r495537954
##########
File path: calcite/src/main/java/org/apache/iotdb/calcite/IoTDBRules.java
##########
@@ -0,0 +1,192 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.calcite;
+
+import java.util.List;
+import org.apache.calcite.adapter.enumerable.EnumerableLimit;
+import org.apache.calcite.adapter.java.JavaTypeFactory;
+import org.apache.calcite.plan.Convention;
+import org.apache.calcite.plan.RelOptRule;
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelTraitSet;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.convert.ConverterRule;
+import org.apache.calcite.rel.logical.LogicalFilter;
+import org.apache.calcite.rel.logical.LogicalProject;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.rex.RexVisitorImpl;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.validate.SqlValidatorUtil;
+import org.apache.iotdb.db.exception.query.LogicalOptimizeException;
+
+/**
+ * Rules and relational operators for {@link IoTDBRel#CONVENTION} calling convention.
+ */
+public class IoTDBRules {
+
+ private IoTDBRules() {
+ }
+
+ public static final RelOptRule[] RULES = {
+ IoTDBFilterRule.INSTANCE,
+ IoTDBProjectRule.INSTANCE,
Review comment:
Hi,
Good point. I impletented the Project rel to push down more operations to IoTDB to avoid querying all data in each query.
For example, I want to see the column `s1` in table, if I removed the ProjectRule, the query ` select s1 from "root.vehicle"` will be transformed to ` select * from root.vehicle align by device`. I have to query ALL DATA in storage group `root.vehicle`. The time cost will be unacceptable.
But as you mentioned, `a + b` will give the wrong result if it's active, so we can try to modify the rule of Project. Try to push down the single column, and leave `a + b` column for Calcite if possible.
And by the way, the aggregation is not push down in this PR as the aggregation method of IoTDB is a bit different from those relations'. We can try to push down partiallly in later PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org