You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2023/01/13 05:14:30 UTC

[GitHub] [iotdb] Wei-hao-Li opened a new pull request, #8870: [IOTDB-5382]Support DIFF as built-in scalar function in IoTDB

Wei-hao-Li opened a new pull request, #8870:
URL: https://github.com/apache/iotdb/pull/8870

   user doc: https://apache-iotdb.feishu.cn/docx/W88rdWTZqo3yaLxvisUc3czFnCe
   design doc: https://apache-iotdb.feishu.cn/docx/CPbIddG8joqt0QxmPrrcMMw4nae


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] Wei-hao-Li commented on a diff in pull request #8870: [IOTDB-5382]Support DIFF as built-in scalar function in IoTDB

Posted by GitBox <gi...@apache.org>.
Wei-hao-Li commented on code in PR #8870:
URL: https://github.com/apache/iotdb/pull/8870#discussion_r1071708871


##########
docs/zh/UserGuide/Operators-Functions/Overview.md:
##########
@@ -216,6 +216,11 @@ OR, |, ||
 | DERIVATIVE              | INT32 / INT64 / FLOAT / DOUBLE                  | DOUBLE                   | 统计序列中某数据点相对于前一数据点的变化率,数量上等同于 DIFFERENCE /  TIME_DIFFERENCE。范围内第一个数据点没有对应的结果输出。 |
 | NON_NEGATIVE_DERIVATIVE | INT32 / INT64 / FLOAT / DOUBLE                  | DOUBLE                   | 统计序列中某数据点相对于前一数据点的变化率的绝对值,数量上等同于 NON_NEGATIVE_DIFFERENCE /  TIME_DIFFERENCE。范围内第一个数据点没有对应的结果输出。 |
 
+
+| 函数名  | 输入序列类型                         | 参数                                                                                                                     | 输出序列类型 | 功能描述                                           |

Review Comment:
   > Does english doc have overview section?
   
   No.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] JackieTien97 merged pull request #8870: [IOTDB-5382]Support DIFF as built-in scalar function in IoTDB

Posted by GitBox <gi...@apache.org>.
JackieTien97 merged PR #8870:
URL: https://github.com/apache/iotdb/pull/8870


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] Wei-hao-Li closed pull request #8870: [IOTDB-5382]Support DIFF as built-in scalar function in IoTDB

Posted by GitBox <gi...@apache.org>.
Wei-hao-Li closed pull request #8870: [IOTDB-5382]Support DIFF as built-in scalar function in IoTDB
URL: https://github.com/apache/iotdb/pull/8870


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] JackieTien97 commented on a diff in pull request #8870: [IOTDB-5382]Support DIFF as built-in scalar function in IoTDB

Posted by GitBox <gi...@apache.org>.
JackieTien97 commented on code in PR #8870:
URL: https://github.com/apache/iotdb/pull/8870#discussion_r1071694767


##########
integration-test/src/test/java/org/apache/iotdb/db/it/builtinfunction/scalar/IoTDBDiffFunctionIT.java:
##########
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.it.builtinfunction.scalar;
+
+import org.apache.iotdb.it.env.EnvFactory;
+import org.apache.iotdb.it.framework.IoTDBTestRunner;
+import org.apache.iotdb.itbase.category.ClusterIT;
+import org.apache.iotdb.itbase.category.LocalStandaloneIT;
+
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+
+import static org.apache.iotdb.db.it.utils.TestUtils.prepareData;
+import static org.apache.iotdb.db.it.utils.TestUtils.resultSetEqualTest;
+import static org.apache.iotdb.itbase.constant.TestConstant.TIMESTAMP_STR;
+
+@RunWith(IoTDBTestRunner.class)
+@Category({LocalStandaloneIT.class, ClusterIT.class})
+public class IoTDBDiffFunctionIT {

Review Comment:
   add more test case as we discussed [here](https://apache-iotdb.feishu.cn/docx/CPbIddG8joqt0QxmPrrcMMw4nae)



##########
server/src/main/java/org/apache/iotdb/db/mpp/plan/analyze/ExpressionAnalyzer.java:
##########
@@ -1267,4 +1268,36 @@ public static Expression bindTypeForTimeSeriesOperand(
           "unsupported expression type: " + predicate.getExpressionType());
     }
   }
+
+  public static boolean isDeviceViewNeedSpecialProcess(Expression predicate) {

Review Comment:
   ```suggestion
     public static boolean isDeviceViewNeedSpecialProcess(Expression expression) {
   ```



##########
node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/BuiltinScalarFunction.java:
##########
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.commons.udf.builtin;
+
+import com.google.common.collect.ImmutableSet;
+
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+public enum BuiltinScalarFunction {
+  DIFF("diff"),
+  ;
+
+  private final String functionName;
+
+  BuiltinScalarFunction(String functionName) {
+    this.functionName = functionName;
+  }
+
+  public String getFunctionName() {
+    return functionName;
+  }
+
+  private static final Set<String> NATIVE_FUNCTION_NAMES =
+      new HashSet<>(
+          Arrays.stream(BuiltinScalarFunction.values())
+              .map(BuiltinScalarFunction::getFunctionName)
+              .collect(Collectors.toList()));
+
+  public static final Set<String> DEVICE_VIEW_SPECIAL_PROCESS_FUNCTIONS = ImmutableSet.of("diff");

Review Comment:
   add some comments to explain this field, like
   we shouldn't apply these functions to each DataRegion for one device, because these functions need all data to calculate correct result. So we need to collect all data for one device in one DataRegion, and then apply these functions to only that one DataRegion.



##########
docs/zh/UserGuide/Operators-Functions/Overview.md:
##########
@@ -216,6 +216,11 @@ OR, |, ||
 | DERIVATIVE              | INT32 / INT64 / FLOAT / DOUBLE                  | DOUBLE                   | 统计序列中某数据点相对于前一数据点的变化率,数量上等同于 DIFFERENCE /  TIME_DIFFERENCE。范围内第一个数据点没有对应的结果输出。 |
 | NON_NEGATIVE_DERIVATIVE | INT32 / INT64 / FLOAT / DOUBLE                  | DOUBLE                   | 统计序列中某数据点相对于前一数据点的变化率的绝对值,数量上等同于 NON_NEGATIVE_DIFFERENCE /  TIME_DIFFERENCE。范围内第一个数据点没有对应的结果输出。 |
 
+
+| 函数名  | 输入序列类型                         | 参数                                                                                                                     | 输出序列类型 | 功能描述                                           |

Review Comment:
   Does english doc have overview section?



##########
server/src/main/java/org/apache/iotdb/db/utils/TypeInferenceUtils.java:
##########
@@ -158,6 +158,33 @@ private static boolean verifyIsAggregationDataTypeMatched(
     }
   }
 
+  public static TSDataType getScalarFunctionDataType(String funcName, TSDataType dataType) {
+    if (funcName == null) {
+      throw new IllegalArgumentException("ScalarFunction Name must not be null");
+    }
+    if (!verifyIsScalarFunctionDataTypeMatched(funcName, dataType)) {
+      throw new SemanticException(
+          "Scalar functions [DIFF] only support numeric data types [INT32, INT64, FLOAT, DOUBLE]");

Review Comment:
   This error messgae should be more generic. `verifyIsScalarFunctionDataTypeMatched` is used to check whether data type is matched, not only numeric data types and [DIFF] shuold be replaced with `funcName`



##########
server/src/main/java/org/apache/iotdb/db/mpp/plan/analyze/Analysis.java:
##########
@@ -137,6 +137,8 @@ public class Analysis {
 
   private Set<Expression> deviceViewOutputExpressions;
 
+  private boolean deviceViewSpecialProcess;

Review Comment:
   add java doc to describe it.



##########
server/src/main/java/org/apache/iotdb/db/mpp/plan/analyze/ExpressionAnalyzer.java:
##########
@@ -1267,4 +1268,36 @@ public static Expression bindTypeForTimeSeriesOperand(
           "unsupported expression type: " + predicate.getExpressionType());
     }
   }
+
+  public static boolean isDeviceViewNeedSpecialProcess(Expression predicate) {

Review Comment:
   not only predicate expression in where clause will use this function



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org