You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2021/11/02 08:05:44 UTC
[GitHub] [iotdb] xuanronaldo opened a new pull request #4292: [IOTDB-1775] CAST transition function
xuanronaldo opened a new pull request #4292:
URL: https://github.com/apache/iotdb/pull/4292
Add cast function to tranform the data type in query.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] SteveYurongSu merged pull request #4292: [IOTDB-1775] CAST: data type conversion function
Posted by GitBox <gi...@apache.org>.
SteveYurongSu merged pull request #4292:
URL: https://github.com/apache/iotdb/pull/4292
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] SteveYurongSu commented on pull request #4292: [IOTDB-1775] CAST conversion function
Posted by GitBox <gi...@apache.org>.
SteveYurongSu commented on pull request #4292:
URL: https://github.com/apache/iotdb/pull/4292#issuecomment-957301445
@xuanronaldo Please refer to [1] for better performance.
[1] https://github.com/apache/iotdb/compare/cast?expand=1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] SteveYurongSu commented on a change in pull request #4292: [IOTDB-1775] CAST conversion function
Posted by GitBox <gi...@apache.org>.
SteveYurongSu commented on a change in pull request #4292:
URL: https://github.com/apache/iotdb/pull/4292#discussion_r741597078
##########
File path: docs/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -459,7 +459,41 @@ select s1, s2, const(s1, 'value'='1024', 'type'='INT64'), pi(s2), e(s1, s2) from
Total line number = 5
It costs 0.005s
```
-
+#### Data Type Conversion Function
+The IoTDB currently supports 6 data types, including INT32, INT64 ,FLOAT, DOUBLE, BOOLEAN, TEXT. When we query or evaluate data, we may need to convert data types, such as TEXT to INT32, or improve the accuracy of the data, such as FLOAT to DOUBLE. Therefore, IoTDB supports the use of cast functions to convert data types.
+##### Notes
+1.The value of type BOOLEAN is `true`, when data is converted to BOOLEAN if INT32 and INT64 are not 0, FLOAT and DOUBLE are not 0.0, TEXT is not empty string or "false", otherwise `false`.
+2.The value of type INT32, INT64, FLOAT, DOUBLE are 1 or 1.0 and TEXT is "true", when BOOLEAN data is true, otherwise 0, 0.0 or "false".
+##### Syntax
+Example data:
+```
+IoTDB> select text from root.test;
++-----------------------------+--------------+
+| Time|root.test.text|
++-----------------------------+--------------+
+|1970-01-01T08:00:00.001+08:00| 1.1|
+|1970-01-01T08:00:00.002+08:00| 1|
+|1970-01-01T08:00:00.003+08:00| hello world|
+|1970-01-01T08:00:00.004+08:00| false|
++-----------------------------+--------------+
+```
+SQL:
+```
+select cast(text,'type'='BOOLEAN') from root.test;
Review comment:
Please introduce 'type'
##########
File path: server/src/test/java/org/apache/iotdb/db/integration/IoTDBUDTFBuiltinFunctionIT.java
##########
@@ -364,4 +364,38 @@ public void testConstantTimeSeriesGeneratingFunctions() {
fail(throwable.getMessage());
}
}
+
+ @Test
+ public void testConversionFunction() {
Review comment:
Add some tests in Session
##########
File path: docs/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -459,7 +459,41 @@ select s1, s2, const(s1, 'value'='1024', 'type'='INT64'), pi(s2), e(s1, s2) from
Total line number = 5
It costs 0.005s
```
-
+#### Data Type Conversion Function
+The IoTDB currently supports 6 data types, including INT32, INT64 ,FLOAT, DOUBLE, BOOLEAN, TEXT. When we query or evaluate data, we may need to convert data types, such as TEXT to INT32, or improve the accuracy of the data, such as FLOAT to DOUBLE. Therefore, IoTDB supports the use of cast functions to convert data types.
+##### Notes
+1.The value of type BOOLEAN is `true`, when data is converted to BOOLEAN if INT32 and INT64 are not 0, FLOAT and DOUBLE are not 0.0, TEXT is not empty string or "false", otherwise `false`.
+2.The value of type INT32, INT64, FLOAT, DOUBLE are 1 or 1.0 and TEXT is "true", when BOOLEAN data is true, otherwise 0, 0.0 or "false".
Review comment:
Introduce more about the binary cast process.
##########
File path: session/src/test/java/org/apache/iotdb/session/IoTDBSessionSimpleIT.java
##########
@@ -1060,6 +1055,80 @@ public void testInsertIlligalPath() throws IoTDBConnectionException {
session.close();
}
+ @Test
+ public void testConversionFunction()
+ throws IoTDBConnectionException, StatementExecutionException {
+ // connect
+ session = new Session("127.0.0.1", 6667, "root", "root");
+ session.open();
Review comment:
Don't forget to close the session.
##########
File path: docs/zh/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -469,6 +469,52 @@ Total line number = 5
It costs 0.005s
```
+#### 数据类型转换函数
+
+当前IoTDB支持6种数据类型,其中包括INT32、INT64、FLOAT、DOUBLE、BOOLEAN以及TEXT。当我们对数据进行查询或者计算时可能需要进行数据类型的转换, 比如说将TEXT转换为INT32,或者提高数据精度,比如说将FLOAT转换为DOUBLE。所以,IoTDB支持使用cast函数对数据类型进行转换。
+
+| 函数名 | 必要的属性参数 | 输出序列类型 | 功能类型 |
+| ------ | ------------------------------------------------------------ | ------------------------ | ---------------------------------- |
+| CAST | `type`:输出的数据点的类型,只能是 INT32 / INT64 / FLOAT / DOUBLE / BOOLEAN / TEXT | 由输入属性参数`type`决定 | 将数据转换为`type`参数指定的类型。 |
+
+##### 类型转换说明
+
+1.当INT32、INT64类型的值不为0时,FLOAT与DOUBLE类型的值不为0.0时,TEXT类型不为空字符串或者"false"时,转换为BOOLEAN类型时值为true,否则为false。
+
+2.当BOOLEAN类型的值为true时,转换为INT32与INT64类型的值为1,转换为FLOAT或者DOUBLE类型时值为1.0,转换为TEXT类型时值为"true"。当BOOLEAN类型的值为false时,转换为INT32与INT64类型的值为0,转换为FLOAT或者DOUBLE类型时值为0.0,转换为TEXT类型时值为"false"。
+
+3.当TEXT类型转换为INT32、INT64、FLOAT类型时,会先将TEXT类型的数据转换为DOUBLE类型,然后再转换为对应的类型,此时可能会存在损失精度的问题。
Review comment:
如果无法转换(比如不是数字),……
##########
File path: server/src/main/java/org/apache/iotdb/db/query/udf/builtin/UDTFCast.java
##########
@@ -0,0 +1,261 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.query.udf.builtin;
+
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.exception.query.QueryProcessException;
+import org.apache.iotdb.db.query.udf.api.UDTF;
+import org.apache.iotdb.db.query.udf.api.access.Row;
+import org.apache.iotdb.db.query.udf.api.collector.PointCollector;
+import org.apache.iotdb.db.query.udf.api.customizer.config.UDTFConfigurations;
+import org.apache.iotdb.db.query.udf.api.customizer.parameter.UDFParameterValidator;
+import org.apache.iotdb.db.query.udf.api.customizer.parameter.UDFParameters;
+import org.apache.iotdb.db.query.udf.api.customizer.strategy.RowByRowAccessStrategy;
+import org.apache.iotdb.db.query.udf.api.exception.UDFAttributeNotProvidedException;
+import org.apache.iotdb.db.query.udf.api.exception.UDFInputSeriesNumberNotValidException;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.utils.Binary;
+
+import java.io.IOException;
+
+public class UDTFCast implements UDTF {
+
+ private TSDataType sourceDataType;
+ private TSDataType targetDataType;
+
+ @Override
+ public void validate(UDFParameterValidator validator)
+ throws UDFInputSeriesNumberNotValidException, UDFAttributeNotProvidedException {
+ validator.validateInputSeriesNumber(1).validateRequiredAttribute("type");
+ }
+
+ @Override
+ public void beforeStart(UDFParameters parameters, UDTFConfigurations configurations)
+ throws MetadataException {
+ sourceDataType = parameters.getDataType(0);
+ targetDataType = TSDataType.valueOf(parameters.getString("type"));
+
+ configurations
+ .setAccessStrategy(new RowByRowAccessStrategy())
+ .setOutputDataType(targetDataType);
+ }
+
+ @Override
+ public void transform(Row row, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (sourceDataType) {
+ case INT32:
+ cast(row.getTime(), row.getInt(0), collector);
+ return;
+ case INT64:
+ cast(row.getTime(), row.getLong(0), collector);
+ return;
+ case FLOAT:
+ cast(row.getTime(), row.getFloat(0), collector);
+ return;
+ case DOUBLE:
+ cast(row.getTime(), row.getDouble(0), collector);
+ return;
+ case BOOLEAN:
+ cast(row.getTime(), row.getBoolean(0), collector);
+ return;
+ case TEXT:
+ cast(row.getTime(), row.getBinary(0), collector);
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, int value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (targetDataType) {
+ case INT32:
+ collector.putInt(time, value);
+ return;
+ case INT64:
+ collector.putLong(time, value);
+ return;
+ case FLOAT:
+ collector.putFloat(time, value);
+ return;
+ case DOUBLE:
+ collector.putDouble(time, value);
+ return;
+ case BOOLEAN:
+ collector.putBoolean(time, value != 0);
+ return;
+ case TEXT:
+ collector.putString(time, String.valueOf(value));
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, long value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (targetDataType) {
+ case INT32:
+ collector.putInt(time, (int) value);
+ return;
+ case INT64:
+ collector.putLong(time, value);
+ return;
+ case FLOAT:
+ collector.putFloat(time, value);
+ return;
+ case DOUBLE:
+ collector.putDouble(time, value);
+ return;
+ case BOOLEAN:
+ collector.putBoolean(time, value != 0l);
+ return;
+ case TEXT:
+ collector.putString(time, String.valueOf(value));
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, float value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (targetDataType) {
+ case INT32:
+ collector.putInt(time, (int) value);
+ return;
+ case INT64:
+ collector.putLong(time, (long) value);
+ return;
+ case FLOAT:
+ collector.putFloat(time, value);
+ return;
+ case DOUBLE:
+ collector.putDouble(time, value);
+ return;
+ case BOOLEAN:
+ collector.putBoolean(time, value != 0f);
+ return;
+ case TEXT:
+ collector.putString(time, String.valueOf(value));
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, double value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (targetDataType) {
+ case INT32:
+ collector.putInt(time, (int) value);
+ return;
+ case INT64:
+ collector.putLong(time, (long) value);
+ return;
+ case FLOAT:
+ collector.putFloat(time, (float) value);
+ return;
+ case DOUBLE:
+ collector.putDouble(time, value);
+ return;
+ case BOOLEAN:
+ collector.putBoolean(time, value != 0.0);
+ return;
+ case TEXT:
+ collector.putString(time, String.valueOf(value));
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, boolean value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (targetDataType) {
+ case INT32:
+ collector.putInt(time, value ? 1 : 0);
+ return;
+ case INT64:
+ collector.putLong(time, value ? 1L : 0L);
+ return;
+ case FLOAT:
+ collector.putFloat(time, value ? 1.0f : 0.0f);
+ return;
+ case DOUBLE:
+ collector.putDouble(time, value ? 1.0 : 0.0);
+ return;
+ case BOOLEAN:
+ collector.putBoolean(time, value);
+ return;
+ case TEXT:
+ collector.putString(time, String.valueOf(value));
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, Binary value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ String stringValue = value.getStringValue();
+ boolean numeric = isNumeric(stringValue);
+ switch (targetDataType) {
+ case INT32:
+ if (numeric) {
+ collector.putInt(time, (int) Double.parseDouble(stringValue));
+ }
Review comment:
```suggestion
try {
collector.putInt(time, (int) Double.parseDouble(stringValue));
} catch (Exception ignored) {
// skip
}
```
##########
File path: docs/zh/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -469,6 +469,52 @@ Total line number = 5
It costs 0.005s
```
+#### 数据类型转换函数
+
+当前IoTDB支持6种数据类型,其中包括INT32、INT64、FLOAT、DOUBLE、BOOLEAN以及TEXT。当我们对数据进行查询或者计算时可能需要进行数据类型的转换, 比如说将TEXT转换为INT32,或者提高数据精度,比如说将FLOAT转换为DOUBLE。所以,IoTDB支持使用cast函数对数据类型进行转换。
+
+| 函数名 | 必要的属性参数 | 输出序列类型 | 功能类型 |
+| ------ | ------------------------------------------------------------ | ------------------------ | ---------------------------------- |
+| CAST | `type`:输出的数据点的类型,只能是 INT32 / INT64 / FLOAT / DOUBLE / BOOLEAN / TEXT | 由输入属性参数`type`决定 | 将数据转换为`type`参数指定的类型。 |
+
+##### 类型转换说明
+
+1.当INT32、INT64类型的值不为0时,FLOAT与DOUBLE类型的值不为0.0时,TEXT类型不为空字符串或者"false"时,转换为BOOLEAN类型时值为true,否则为false。
+
+2.当BOOLEAN类型的值为true时,转换为INT32与INT64类型的值为1,转换为FLOAT或者DOUBLE类型时值为1.0,转换为TEXT类型时值为"true"。当BOOLEAN类型的值为false时,转换为INT32与INT64类型的值为0,转换为FLOAT或者DOUBLE类型时值为0.0,转换为TEXT类型时值为"false"。
+
+3.当TEXT类型转换为INT32、INT64、FLOAT类型时,会先将TEXT类型的数据转换为DOUBLE类型,然后再转换为对应的类型,此时可能会存在损失精度的问题。
+
+##### 演示
+测试数据:
+```
+IoTDB> select text from root.test;
++-----------------------------+--------------+
+| Time|root.test.text|
++-----------------------------+--------------+
+|1970-01-01T08:00:00.001+08:00| 1.1|
+|1970-01-01T08:00:00.002+08:00| 1|
+|1970-01-01T08:00:00.003+08:00| hello world|
+|1970-01-01T08:00:00.004+08:00| false|
++-----------------------------+--------------+
+```
+SQL语句:
+```sql
+select cast(text,'type'='BOOLEAN') from root.test;
Review comment:
```suggestion
select cast(text, 'type'='BOOLEAN') from root.test;
```
为每一种数据类型增加一列
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] SteveYurongSu commented on a change in pull request #4292: [IOTDB-1775] CAST conversion function
Posted by GitBox <gi...@apache.org>.
SteveYurongSu commented on a change in pull request #4292:
URL: https://github.com/apache/iotdb/pull/4292#discussion_r741778145
##########
File path: session/src/test/java/org/apache/iotdb/session/IoTDBSessionSimpleIT.java
##########
@@ -1060,6 +1055,80 @@ public void testInsertIlligalPath() throws IoTDBConnectionException {
session.close();
}
+ @Test
+ public void testConversionFunction()
+ throws IoTDBConnectionException, StatementExecutionException {
+ // connect
+ session = new Session("127.0.0.1", 6667, "root", "root");
+ session.open();
Review comment:
Don't forget to close the session.
##########
File path: docs/zh/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -469,6 +469,52 @@ Total line number = 5
It costs 0.005s
```
+#### 数据类型转换函数
+
+当前IoTDB支持6种数据类型,其中包括INT32、INT64、FLOAT、DOUBLE、BOOLEAN以及TEXT。当我们对数据进行查询或者计算时可能需要进行数据类型的转换, 比如说将TEXT转换为INT32,或者提高数据精度,比如说将FLOAT转换为DOUBLE。所以,IoTDB支持使用cast函数对数据类型进行转换。
+
+| 函数名 | 必要的属性参数 | 输出序列类型 | 功能类型 |
+| ------ | ------------------------------------------------------------ | ------------------------ | ---------------------------------- |
+| CAST | `type`:输出的数据点的类型,只能是 INT32 / INT64 / FLOAT / DOUBLE / BOOLEAN / TEXT | 由输入属性参数`type`决定 | 将数据转换为`type`参数指定的类型。 |
+
+##### 类型转换说明
+
+1.当INT32、INT64类型的值不为0时,FLOAT与DOUBLE类型的值不为0.0时,TEXT类型不为空字符串或者"false"时,转换为BOOLEAN类型时值为true,否则为false。
+
+2.当BOOLEAN类型的值为true时,转换为INT32与INT64类型的值为1,转换为FLOAT或者DOUBLE类型时值为1.0,转换为TEXT类型时值为"true"。当BOOLEAN类型的值为false时,转换为INT32与INT64类型的值为0,转换为FLOAT或者DOUBLE类型时值为0.0,转换为TEXT类型时值为"false"。
+
+3.当TEXT类型转换为INT32、INT64、FLOAT类型时,会先将TEXT类型的数据转换为DOUBLE类型,然后再转换为对应的类型,此时可能会存在损失精度的问题。
Review comment:
如果无法转换(比如不是数字),……
##########
File path: server/src/main/java/org/apache/iotdb/db/query/udf/builtin/UDTFCast.java
##########
@@ -0,0 +1,261 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.query.udf.builtin;
+
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.exception.query.QueryProcessException;
+import org.apache.iotdb.db.query.udf.api.UDTF;
+import org.apache.iotdb.db.query.udf.api.access.Row;
+import org.apache.iotdb.db.query.udf.api.collector.PointCollector;
+import org.apache.iotdb.db.query.udf.api.customizer.config.UDTFConfigurations;
+import org.apache.iotdb.db.query.udf.api.customizer.parameter.UDFParameterValidator;
+import org.apache.iotdb.db.query.udf.api.customizer.parameter.UDFParameters;
+import org.apache.iotdb.db.query.udf.api.customizer.strategy.RowByRowAccessStrategy;
+import org.apache.iotdb.db.query.udf.api.exception.UDFAttributeNotProvidedException;
+import org.apache.iotdb.db.query.udf.api.exception.UDFInputSeriesNumberNotValidException;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.utils.Binary;
+
+import java.io.IOException;
+
+public class UDTFCast implements UDTF {
+
+ private TSDataType sourceDataType;
+ private TSDataType targetDataType;
+
+ @Override
+ public void validate(UDFParameterValidator validator)
+ throws UDFInputSeriesNumberNotValidException, UDFAttributeNotProvidedException {
+ validator.validateInputSeriesNumber(1).validateRequiredAttribute("type");
+ }
+
+ @Override
+ public void beforeStart(UDFParameters parameters, UDTFConfigurations configurations)
+ throws MetadataException {
+ sourceDataType = parameters.getDataType(0);
+ targetDataType = TSDataType.valueOf(parameters.getString("type"));
+
+ configurations
+ .setAccessStrategy(new RowByRowAccessStrategy())
+ .setOutputDataType(targetDataType);
+ }
+
+ @Override
+ public void transform(Row row, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (sourceDataType) {
+ case INT32:
+ cast(row.getTime(), row.getInt(0), collector);
+ return;
+ case INT64:
+ cast(row.getTime(), row.getLong(0), collector);
+ return;
+ case FLOAT:
+ cast(row.getTime(), row.getFloat(0), collector);
+ return;
+ case DOUBLE:
+ cast(row.getTime(), row.getDouble(0), collector);
+ return;
+ case BOOLEAN:
+ cast(row.getTime(), row.getBoolean(0), collector);
+ return;
+ case TEXT:
+ cast(row.getTime(), row.getBinary(0), collector);
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, int value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (targetDataType) {
+ case INT32:
+ collector.putInt(time, value);
+ return;
+ case INT64:
+ collector.putLong(time, value);
+ return;
+ case FLOAT:
+ collector.putFloat(time, value);
+ return;
+ case DOUBLE:
+ collector.putDouble(time, value);
+ return;
+ case BOOLEAN:
+ collector.putBoolean(time, value != 0);
+ return;
+ case TEXT:
+ collector.putString(time, String.valueOf(value));
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, long value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (targetDataType) {
+ case INT32:
+ collector.putInt(time, (int) value);
+ return;
+ case INT64:
+ collector.putLong(time, value);
+ return;
+ case FLOAT:
+ collector.putFloat(time, value);
+ return;
+ case DOUBLE:
+ collector.putDouble(time, value);
+ return;
+ case BOOLEAN:
+ collector.putBoolean(time, value != 0l);
+ return;
+ case TEXT:
+ collector.putString(time, String.valueOf(value));
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, float value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (targetDataType) {
+ case INT32:
+ collector.putInt(time, (int) value);
+ return;
+ case INT64:
+ collector.putLong(time, (long) value);
+ return;
+ case FLOAT:
+ collector.putFloat(time, value);
+ return;
+ case DOUBLE:
+ collector.putDouble(time, value);
+ return;
+ case BOOLEAN:
+ collector.putBoolean(time, value != 0f);
+ return;
+ case TEXT:
+ collector.putString(time, String.valueOf(value));
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, double value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (targetDataType) {
+ case INT32:
+ collector.putInt(time, (int) value);
+ return;
+ case INT64:
+ collector.putLong(time, (long) value);
+ return;
+ case FLOAT:
+ collector.putFloat(time, (float) value);
+ return;
+ case DOUBLE:
+ collector.putDouble(time, value);
+ return;
+ case BOOLEAN:
+ collector.putBoolean(time, value != 0.0);
+ return;
+ case TEXT:
+ collector.putString(time, String.valueOf(value));
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, boolean value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (targetDataType) {
+ case INT32:
+ collector.putInt(time, value ? 1 : 0);
+ return;
+ case INT64:
+ collector.putLong(time, value ? 1L : 0L);
+ return;
+ case FLOAT:
+ collector.putFloat(time, value ? 1.0f : 0.0f);
+ return;
+ case DOUBLE:
+ collector.putDouble(time, value ? 1.0 : 0.0);
+ return;
+ case BOOLEAN:
+ collector.putBoolean(time, value);
+ return;
+ case TEXT:
+ collector.putString(time, String.valueOf(value));
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, Binary value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ String stringValue = value.getStringValue();
+ boolean numeric = isNumeric(stringValue);
+ switch (targetDataType) {
+ case INT32:
+ if (numeric) {
+ collector.putInt(time, (int) Double.parseDouble(stringValue));
+ }
Review comment:
```suggestion
try {
collector.putInt(time, (int) Double.parseDouble(stringValue));
} catch (Exception ignored) {
// skip
}
```
##########
File path: docs/zh/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -469,6 +469,52 @@ Total line number = 5
It costs 0.005s
```
+#### 数据类型转换函数
+
+当前IoTDB支持6种数据类型,其中包括INT32、INT64、FLOAT、DOUBLE、BOOLEAN以及TEXT。当我们对数据进行查询或者计算时可能需要进行数据类型的转换, 比如说将TEXT转换为INT32,或者提高数据精度,比如说将FLOAT转换为DOUBLE。所以,IoTDB支持使用cast函数对数据类型进行转换。
+
+| 函数名 | 必要的属性参数 | 输出序列类型 | 功能类型 |
+| ------ | ------------------------------------------------------------ | ------------------------ | ---------------------------------- |
+| CAST | `type`:输出的数据点的类型,只能是 INT32 / INT64 / FLOAT / DOUBLE / BOOLEAN / TEXT | 由输入属性参数`type`决定 | 将数据转换为`type`参数指定的类型。 |
+
+##### 类型转换说明
+
+1.当INT32、INT64类型的值不为0时,FLOAT与DOUBLE类型的值不为0.0时,TEXT类型不为空字符串或者"false"时,转换为BOOLEAN类型时值为true,否则为false。
+
+2.当BOOLEAN类型的值为true时,转换为INT32与INT64类型的值为1,转换为FLOAT或者DOUBLE类型时值为1.0,转换为TEXT类型时值为"true"。当BOOLEAN类型的值为false时,转换为INT32与INT64类型的值为0,转换为FLOAT或者DOUBLE类型时值为0.0,转换为TEXT类型时值为"false"。
+
+3.当TEXT类型转换为INT32、INT64、FLOAT类型时,会先将TEXT类型的数据转换为DOUBLE类型,然后再转换为对应的类型,此时可能会存在损失精度的问题。
+
+##### 演示
+测试数据:
+```
+IoTDB> select text from root.test;
++-----------------------------+--------------+
+| Time|root.test.text|
++-----------------------------+--------------+
+|1970-01-01T08:00:00.001+08:00| 1.1|
+|1970-01-01T08:00:00.002+08:00| 1|
+|1970-01-01T08:00:00.003+08:00| hello world|
+|1970-01-01T08:00:00.004+08:00| false|
++-----------------------------+--------------+
+```
+SQL语句:
+```sql
+select cast(text,'type'='BOOLEAN') from root.test;
Review comment:
```suggestion
select cast(text, 'type'='BOOLEAN') from root.test;
```
为每一种数据类型增加一列
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] SteveYurongSu commented on pull request #4292: [IOTDB-1775] CAST conversion function
Posted by GitBox <gi...@apache.org>.
SteveYurongSu commented on pull request #4292:
URL: https://github.com/apache/iotdb/pull/4292#issuecomment-957301445
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] SteveYurongSu commented on pull request #4292: [IOTDB-1775] CAST conversion function
Posted by GitBox <gi...@apache.org>.
SteveYurongSu commented on pull request #4292:
URL: https://github.com/apache/iotdb/pull/4292#issuecomment-957440918
Please add some UTs/ITs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] SteveYurongSu commented on a change in pull request #4292: [IOTDB-1775] CAST conversion function
Posted by GitBox <gi...@apache.org>.
SteveYurongSu commented on a change in pull request #4292:
URL: https://github.com/apache/iotdb/pull/4292#discussion_r741597078
##########
File path: docs/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -459,7 +459,41 @@ select s1, s2, const(s1, 'value'='1024', 'type'='INT64'), pi(s2), e(s1, s2) from
Total line number = 5
It costs 0.005s
```
-
+#### Data Type Conversion Function
+The IoTDB currently supports 6 data types, including INT32, INT64 ,FLOAT, DOUBLE, BOOLEAN, TEXT. When we query or evaluate data, we may need to convert data types, such as TEXT to INT32, or improve the accuracy of the data, such as FLOAT to DOUBLE. Therefore, IoTDB supports the use of cast functions to convert data types.
+##### Notes
+1.The value of type BOOLEAN is `true`, when data is converted to BOOLEAN if INT32 and INT64 are not 0, FLOAT and DOUBLE are not 0.0, TEXT is not empty string or "false", otherwise `false`.
+2.The value of type INT32, INT64, FLOAT, DOUBLE are 1 or 1.0 and TEXT is "true", when BOOLEAN data is true, otherwise 0, 0.0 or "false".
+##### Syntax
+Example data:
+```
+IoTDB> select text from root.test;
++-----------------------------+--------------+
+| Time|root.test.text|
++-----------------------------+--------------+
+|1970-01-01T08:00:00.001+08:00| 1.1|
+|1970-01-01T08:00:00.002+08:00| 1|
+|1970-01-01T08:00:00.003+08:00| hello world|
+|1970-01-01T08:00:00.004+08:00| false|
++-----------------------------+--------------+
+```
+SQL:
+```
+select cast(text,'type'='BOOLEAN') from root.test;
Review comment:
Please introduce 'type'
##########
File path: server/src/test/java/org/apache/iotdb/db/integration/IoTDBUDTFBuiltinFunctionIT.java
##########
@@ -364,4 +364,38 @@ public void testConstantTimeSeriesGeneratingFunctions() {
fail(throwable.getMessage());
}
}
+
+ @Test
+ public void testConversionFunction() {
Review comment:
Add some tests in Session
##########
File path: docs/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -459,7 +459,41 @@ select s1, s2, const(s1, 'value'='1024', 'type'='INT64'), pi(s2), e(s1, s2) from
Total line number = 5
It costs 0.005s
```
-
+#### Data Type Conversion Function
+The IoTDB currently supports 6 data types, including INT32, INT64 ,FLOAT, DOUBLE, BOOLEAN, TEXT. When we query or evaluate data, we may need to convert data types, such as TEXT to INT32, or improve the accuracy of the data, such as FLOAT to DOUBLE. Therefore, IoTDB supports the use of cast functions to convert data types.
+##### Notes
+1.The value of type BOOLEAN is `true`, when data is converted to BOOLEAN if INT32 and INT64 are not 0, FLOAT and DOUBLE are not 0.0, TEXT is not empty string or "false", otherwise `false`.
+2.The value of type INT32, INT64, FLOAT, DOUBLE are 1 or 1.0 and TEXT is "true", when BOOLEAN data is true, otherwise 0, 0.0 or "false".
Review comment:
Introduce more about the binary cast process.
##########
File path: session/src/test/java/org/apache/iotdb/session/IoTDBSessionSimpleIT.java
##########
@@ -1060,6 +1055,80 @@ public void testInsertIlligalPath() throws IoTDBConnectionException {
session.close();
}
+ @Test
+ public void testConversionFunction()
+ throws IoTDBConnectionException, StatementExecutionException {
+ // connect
+ session = new Session("127.0.0.1", 6667, "root", "root");
+ session.open();
Review comment:
Don't forget to close the session.
##########
File path: docs/zh/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -469,6 +469,52 @@ Total line number = 5
It costs 0.005s
```
+#### 数据类型转换函数
+
+当前IoTDB支持6种数据类型,其中包括INT32、INT64、FLOAT、DOUBLE、BOOLEAN以及TEXT。当我们对数据进行查询或者计算时可能需要进行数据类型的转换, 比如说将TEXT转换为INT32,或者提高数据精度,比如说将FLOAT转换为DOUBLE。所以,IoTDB支持使用cast函数对数据类型进行转换。
+
+| 函数名 | 必要的属性参数 | 输出序列类型 | 功能类型 |
+| ------ | ------------------------------------------------------------ | ------------------------ | ---------------------------------- |
+| CAST | `type`:输出的数据点的类型,只能是 INT32 / INT64 / FLOAT / DOUBLE / BOOLEAN / TEXT | 由输入属性参数`type`决定 | 将数据转换为`type`参数指定的类型。 |
+
+##### 类型转换说明
+
+1.当INT32、INT64类型的值不为0时,FLOAT与DOUBLE类型的值不为0.0时,TEXT类型不为空字符串或者"false"时,转换为BOOLEAN类型时值为true,否则为false。
+
+2.当BOOLEAN类型的值为true时,转换为INT32与INT64类型的值为1,转换为FLOAT或者DOUBLE类型时值为1.0,转换为TEXT类型时值为"true"。当BOOLEAN类型的值为false时,转换为INT32与INT64类型的值为0,转换为FLOAT或者DOUBLE类型时值为0.0,转换为TEXT类型时值为"false"。
+
+3.当TEXT类型转换为INT32、INT64、FLOAT类型时,会先将TEXT类型的数据转换为DOUBLE类型,然后再转换为对应的类型,此时可能会存在损失精度的问题。
Review comment:
如果无法转换(比如不是数字),……
##########
File path: server/src/main/java/org/apache/iotdb/db/query/udf/builtin/UDTFCast.java
##########
@@ -0,0 +1,261 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.query.udf.builtin;
+
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.exception.query.QueryProcessException;
+import org.apache.iotdb.db.query.udf.api.UDTF;
+import org.apache.iotdb.db.query.udf.api.access.Row;
+import org.apache.iotdb.db.query.udf.api.collector.PointCollector;
+import org.apache.iotdb.db.query.udf.api.customizer.config.UDTFConfigurations;
+import org.apache.iotdb.db.query.udf.api.customizer.parameter.UDFParameterValidator;
+import org.apache.iotdb.db.query.udf.api.customizer.parameter.UDFParameters;
+import org.apache.iotdb.db.query.udf.api.customizer.strategy.RowByRowAccessStrategy;
+import org.apache.iotdb.db.query.udf.api.exception.UDFAttributeNotProvidedException;
+import org.apache.iotdb.db.query.udf.api.exception.UDFInputSeriesNumberNotValidException;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.utils.Binary;
+
+import java.io.IOException;
+
+public class UDTFCast implements UDTF {
+
+ private TSDataType sourceDataType;
+ private TSDataType targetDataType;
+
+ @Override
+ public void validate(UDFParameterValidator validator)
+ throws UDFInputSeriesNumberNotValidException, UDFAttributeNotProvidedException {
+ validator.validateInputSeriesNumber(1).validateRequiredAttribute("type");
+ }
+
+ @Override
+ public void beforeStart(UDFParameters parameters, UDTFConfigurations configurations)
+ throws MetadataException {
+ sourceDataType = parameters.getDataType(0);
+ targetDataType = TSDataType.valueOf(parameters.getString("type"));
+
+ configurations
+ .setAccessStrategy(new RowByRowAccessStrategy())
+ .setOutputDataType(targetDataType);
+ }
+
+ @Override
+ public void transform(Row row, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (sourceDataType) {
+ case INT32:
+ cast(row.getTime(), row.getInt(0), collector);
+ return;
+ case INT64:
+ cast(row.getTime(), row.getLong(0), collector);
+ return;
+ case FLOAT:
+ cast(row.getTime(), row.getFloat(0), collector);
+ return;
+ case DOUBLE:
+ cast(row.getTime(), row.getDouble(0), collector);
+ return;
+ case BOOLEAN:
+ cast(row.getTime(), row.getBoolean(0), collector);
+ return;
+ case TEXT:
+ cast(row.getTime(), row.getBinary(0), collector);
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, int value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (targetDataType) {
+ case INT32:
+ collector.putInt(time, value);
+ return;
+ case INT64:
+ collector.putLong(time, value);
+ return;
+ case FLOAT:
+ collector.putFloat(time, value);
+ return;
+ case DOUBLE:
+ collector.putDouble(time, value);
+ return;
+ case BOOLEAN:
+ collector.putBoolean(time, value != 0);
+ return;
+ case TEXT:
+ collector.putString(time, String.valueOf(value));
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, long value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (targetDataType) {
+ case INT32:
+ collector.putInt(time, (int) value);
+ return;
+ case INT64:
+ collector.putLong(time, value);
+ return;
+ case FLOAT:
+ collector.putFloat(time, value);
+ return;
+ case DOUBLE:
+ collector.putDouble(time, value);
+ return;
+ case BOOLEAN:
+ collector.putBoolean(time, value != 0l);
+ return;
+ case TEXT:
+ collector.putString(time, String.valueOf(value));
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, float value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (targetDataType) {
+ case INT32:
+ collector.putInt(time, (int) value);
+ return;
+ case INT64:
+ collector.putLong(time, (long) value);
+ return;
+ case FLOAT:
+ collector.putFloat(time, value);
+ return;
+ case DOUBLE:
+ collector.putDouble(time, value);
+ return;
+ case BOOLEAN:
+ collector.putBoolean(time, value != 0f);
+ return;
+ case TEXT:
+ collector.putString(time, String.valueOf(value));
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, double value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (targetDataType) {
+ case INT32:
+ collector.putInt(time, (int) value);
+ return;
+ case INT64:
+ collector.putLong(time, (long) value);
+ return;
+ case FLOAT:
+ collector.putFloat(time, (float) value);
+ return;
+ case DOUBLE:
+ collector.putDouble(time, value);
+ return;
+ case BOOLEAN:
+ collector.putBoolean(time, value != 0.0);
+ return;
+ case TEXT:
+ collector.putString(time, String.valueOf(value));
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, boolean value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (targetDataType) {
+ case INT32:
+ collector.putInt(time, value ? 1 : 0);
+ return;
+ case INT64:
+ collector.putLong(time, value ? 1L : 0L);
+ return;
+ case FLOAT:
+ collector.putFloat(time, value ? 1.0f : 0.0f);
+ return;
+ case DOUBLE:
+ collector.putDouble(time, value ? 1.0 : 0.0);
+ return;
+ case BOOLEAN:
+ collector.putBoolean(time, value);
+ return;
+ case TEXT:
+ collector.putString(time, String.valueOf(value));
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, Binary value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ String stringValue = value.getStringValue();
+ boolean numeric = isNumeric(stringValue);
+ switch (targetDataType) {
+ case INT32:
+ if (numeric) {
+ collector.putInt(time, (int) Double.parseDouble(stringValue));
+ }
Review comment:
```suggestion
try {
collector.putInt(time, (int) Double.parseDouble(stringValue));
} catch (Exception ignored) {
// skip
}
```
##########
File path: docs/zh/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -469,6 +469,52 @@ Total line number = 5
It costs 0.005s
```
+#### 数据类型转换函数
+
+当前IoTDB支持6种数据类型,其中包括INT32、INT64、FLOAT、DOUBLE、BOOLEAN以及TEXT。当我们对数据进行查询或者计算时可能需要进行数据类型的转换, 比如说将TEXT转换为INT32,或者提高数据精度,比如说将FLOAT转换为DOUBLE。所以,IoTDB支持使用cast函数对数据类型进行转换。
+
+| 函数名 | 必要的属性参数 | 输出序列类型 | 功能类型 |
+| ------ | ------------------------------------------------------------ | ------------------------ | ---------------------------------- |
+| CAST | `type`:输出的数据点的类型,只能是 INT32 / INT64 / FLOAT / DOUBLE / BOOLEAN / TEXT | 由输入属性参数`type`决定 | 将数据转换为`type`参数指定的类型。 |
+
+##### 类型转换说明
+
+1.当INT32、INT64类型的值不为0时,FLOAT与DOUBLE类型的值不为0.0时,TEXT类型不为空字符串或者"false"时,转换为BOOLEAN类型时值为true,否则为false。
+
+2.当BOOLEAN类型的值为true时,转换为INT32与INT64类型的值为1,转换为FLOAT或者DOUBLE类型时值为1.0,转换为TEXT类型时值为"true"。当BOOLEAN类型的值为false时,转换为INT32与INT64类型的值为0,转换为FLOAT或者DOUBLE类型时值为0.0,转换为TEXT类型时值为"false"。
+
+3.当TEXT类型转换为INT32、INT64、FLOAT类型时,会先将TEXT类型的数据转换为DOUBLE类型,然后再转换为对应的类型,此时可能会存在损失精度的问题。
+
+##### 演示
+测试数据:
+```
+IoTDB> select text from root.test;
++-----------------------------+--------------+
+| Time|root.test.text|
++-----------------------------+--------------+
+|1970-01-01T08:00:00.001+08:00| 1.1|
+|1970-01-01T08:00:00.002+08:00| 1|
+|1970-01-01T08:00:00.003+08:00| hello world|
+|1970-01-01T08:00:00.004+08:00| false|
++-----------------------------+--------------+
+```
+SQL语句:
+```sql
+select cast(text,'type'='BOOLEAN') from root.test;
Review comment:
```suggestion
select cast(text, 'type'='BOOLEAN') from root.test;
```
为每一种数据类型增加一列
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] SteveYurongSu commented on pull request #4292: [IOTDB-1775] CAST conversion function
Posted by GitBox <gi...@apache.org>.
SteveYurongSu commented on pull request #4292:
URL: https://github.com/apache/iotdb/pull/4292#issuecomment-957301445
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] SteveYurongSu commented on a change in pull request #4292: [IOTDB-1775] CAST conversion function
Posted by GitBox <gi...@apache.org>.
SteveYurongSu commented on a change in pull request #4292:
URL: https://github.com/apache/iotdb/pull/4292#discussion_r741597078
##########
File path: docs/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -459,7 +459,41 @@ select s1, s2, const(s1, 'value'='1024', 'type'='INT64'), pi(s2), e(s1, s2) from
Total line number = 5
It costs 0.005s
```
-
+#### Data Type Conversion Function
+The IoTDB currently supports 6 data types, including INT32, INT64 ,FLOAT, DOUBLE, BOOLEAN, TEXT. When we query or evaluate data, we may need to convert data types, such as TEXT to INT32, or improve the accuracy of the data, such as FLOAT to DOUBLE. Therefore, IoTDB supports the use of cast functions to convert data types.
+##### Notes
+1.The value of type BOOLEAN is `true`, when data is converted to BOOLEAN if INT32 and INT64 are not 0, FLOAT and DOUBLE are not 0.0, TEXT is not empty string or "false", otherwise `false`.
+2.The value of type INT32, INT64, FLOAT, DOUBLE are 1 or 1.0 and TEXT is "true", when BOOLEAN data is true, otherwise 0, 0.0 or "false".
+##### Syntax
+Example data:
+```
+IoTDB> select text from root.test;
++-----------------------------+--------------+
+| Time|root.test.text|
++-----------------------------+--------------+
+|1970-01-01T08:00:00.001+08:00| 1.1|
+|1970-01-01T08:00:00.002+08:00| 1|
+|1970-01-01T08:00:00.003+08:00| hello world|
+|1970-01-01T08:00:00.004+08:00| false|
++-----------------------------+--------------+
+```
+SQL:
+```
+select cast(text,'type'='BOOLEAN') from root.test;
Review comment:
Please introduce 'type'
##########
File path: server/src/test/java/org/apache/iotdb/db/integration/IoTDBUDTFBuiltinFunctionIT.java
##########
@@ -364,4 +364,38 @@ public void testConstantTimeSeriesGeneratingFunctions() {
fail(throwable.getMessage());
}
}
+
+ @Test
+ public void testConversionFunction() {
Review comment:
Add some tests in Session
##########
File path: docs/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -459,7 +459,41 @@ select s1, s2, const(s1, 'value'='1024', 'type'='INT64'), pi(s2), e(s1, s2) from
Total line number = 5
It costs 0.005s
```
-
+#### Data Type Conversion Function
+The IoTDB currently supports 6 data types, including INT32, INT64 ,FLOAT, DOUBLE, BOOLEAN, TEXT. When we query or evaluate data, we may need to convert data types, such as TEXT to INT32, or improve the accuracy of the data, such as FLOAT to DOUBLE. Therefore, IoTDB supports the use of cast functions to convert data types.
+##### Notes
+1.The value of type BOOLEAN is `true`, when data is converted to BOOLEAN if INT32 and INT64 are not 0, FLOAT and DOUBLE are not 0.0, TEXT is not empty string or "false", otherwise `false`.
+2.The value of type INT32, INT64, FLOAT, DOUBLE are 1 or 1.0 and TEXT is "true", when BOOLEAN data is true, otherwise 0, 0.0 or "false".
Review comment:
Introduce more about the binary cast process.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] SteveYurongSu commented on a change in pull request #4292: [IOTDB-1775] CAST conversion function
Posted by GitBox <gi...@apache.org>.
SteveYurongSu commented on a change in pull request #4292:
URL: https://github.com/apache/iotdb/pull/4292#discussion_r741597078
##########
File path: docs/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -459,7 +459,41 @@ select s1, s2, const(s1, 'value'='1024', 'type'='INT64'), pi(s2), e(s1, s2) from
Total line number = 5
It costs 0.005s
```
-
+#### Data Type Conversion Function
+The IoTDB currently supports 6 data types, including INT32, INT64 ,FLOAT, DOUBLE, BOOLEAN, TEXT. When we query or evaluate data, we may need to convert data types, such as TEXT to INT32, or improve the accuracy of the data, such as FLOAT to DOUBLE. Therefore, IoTDB supports the use of cast functions to convert data types.
+##### Notes
+1.The value of type BOOLEAN is `true`, when data is converted to BOOLEAN if INT32 and INT64 are not 0, FLOAT and DOUBLE are not 0.0, TEXT is not empty string or "false", otherwise `false`.
+2.The value of type INT32, INT64, FLOAT, DOUBLE are 1 or 1.0 and TEXT is "true", when BOOLEAN data is true, otherwise 0, 0.0 or "false".
+##### Syntax
+Example data:
+```
+IoTDB> select text from root.test;
++-----------------------------+--------------+
+| Time|root.test.text|
++-----------------------------+--------------+
+|1970-01-01T08:00:00.001+08:00| 1.1|
+|1970-01-01T08:00:00.002+08:00| 1|
+|1970-01-01T08:00:00.003+08:00| hello world|
+|1970-01-01T08:00:00.004+08:00| false|
++-----------------------------+--------------+
+```
+SQL:
+```
+select cast(text,'type'='BOOLEAN') from root.test;
Review comment:
Please introduce 'type'
##########
File path: server/src/test/java/org/apache/iotdb/db/integration/IoTDBUDTFBuiltinFunctionIT.java
##########
@@ -364,4 +364,38 @@ public void testConstantTimeSeriesGeneratingFunctions() {
fail(throwable.getMessage());
}
}
+
+ @Test
+ public void testConversionFunction() {
Review comment:
Add some tests in Session
##########
File path: docs/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -459,7 +459,41 @@ select s1, s2, const(s1, 'value'='1024', 'type'='INT64'), pi(s2), e(s1, s2) from
Total line number = 5
It costs 0.005s
```
-
+#### Data Type Conversion Function
+The IoTDB currently supports 6 data types, including INT32, INT64 ,FLOAT, DOUBLE, BOOLEAN, TEXT. When we query or evaluate data, we may need to convert data types, such as TEXT to INT32, or improve the accuracy of the data, such as FLOAT to DOUBLE. Therefore, IoTDB supports the use of cast functions to convert data types.
+##### Notes
+1.The value of type BOOLEAN is `true`, when data is converted to BOOLEAN if INT32 and INT64 are not 0, FLOAT and DOUBLE are not 0.0, TEXT is not empty string or "false", otherwise `false`.
+2.The value of type INT32, INT64, FLOAT, DOUBLE are 1 or 1.0 and TEXT is "true", when BOOLEAN data is true, otherwise 0, 0.0 or "false".
Review comment:
Introduce more about the binary cast process.
##########
File path: session/src/test/java/org/apache/iotdb/session/IoTDBSessionSimpleIT.java
##########
@@ -1060,6 +1055,80 @@ public void testInsertIlligalPath() throws IoTDBConnectionException {
session.close();
}
+ @Test
+ public void testConversionFunction()
+ throws IoTDBConnectionException, StatementExecutionException {
+ // connect
+ session = new Session("127.0.0.1", 6667, "root", "root");
+ session.open();
Review comment:
Don't forget to close the session.
##########
File path: docs/zh/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -469,6 +469,52 @@ Total line number = 5
It costs 0.005s
```
+#### 数据类型转换函数
+
+当前IoTDB支持6种数据类型,其中包括INT32、INT64、FLOAT、DOUBLE、BOOLEAN以及TEXT。当我们对数据进行查询或者计算时可能需要进行数据类型的转换, 比如说将TEXT转换为INT32,或者提高数据精度,比如说将FLOAT转换为DOUBLE。所以,IoTDB支持使用cast函数对数据类型进行转换。
+
+| 函数名 | 必要的属性参数 | 输出序列类型 | 功能类型 |
+| ------ | ------------------------------------------------------------ | ------------------------ | ---------------------------------- |
+| CAST | `type`:输出的数据点的类型,只能是 INT32 / INT64 / FLOAT / DOUBLE / BOOLEAN / TEXT | 由输入属性参数`type`决定 | 将数据转换为`type`参数指定的类型。 |
+
+##### 类型转换说明
+
+1.当INT32、INT64类型的值不为0时,FLOAT与DOUBLE类型的值不为0.0时,TEXT类型不为空字符串或者"false"时,转换为BOOLEAN类型时值为true,否则为false。
+
+2.当BOOLEAN类型的值为true时,转换为INT32与INT64类型的值为1,转换为FLOAT或者DOUBLE类型时值为1.0,转换为TEXT类型时值为"true"。当BOOLEAN类型的值为false时,转换为INT32与INT64类型的值为0,转换为FLOAT或者DOUBLE类型时值为0.0,转换为TEXT类型时值为"false"。
+
+3.当TEXT类型转换为INT32、INT64、FLOAT类型时,会先将TEXT类型的数据转换为DOUBLE类型,然后再转换为对应的类型,此时可能会存在损失精度的问题。
Review comment:
如果无法转换(比如不是数字),……
##########
File path: server/src/main/java/org/apache/iotdb/db/query/udf/builtin/UDTFCast.java
##########
@@ -0,0 +1,261 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.query.udf.builtin;
+
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.exception.query.QueryProcessException;
+import org.apache.iotdb.db.query.udf.api.UDTF;
+import org.apache.iotdb.db.query.udf.api.access.Row;
+import org.apache.iotdb.db.query.udf.api.collector.PointCollector;
+import org.apache.iotdb.db.query.udf.api.customizer.config.UDTFConfigurations;
+import org.apache.iotdb.db.query.udf.api.customizer.parameter.UDFParameterValidator;
+import org.apache.iotdb.db.query.udf.api.customizer.parameter.UDFParameters;
+import org.apache.iotdb.db.query.udf.api.customizer.strategy.RowByRowAccessStrategy;
+import org.apache.iotdb.db.query.udf.api.exception.UDFAttributeNotProvidedException;
+import org.apache.iotdb.db.query.udf.api.exception.UDFInputSeriesNumberNotValidException;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.utils.Binary;
+
+import java.io.IOException;
+
+public class UDTFCast implements UDTF {
+
+ private TSDataType sourceDataType;
+ private TSDataType targetDataType;
+
+ @Override
+ public void validate(UDFParameterValidator validator)
+ throws UDFInputSeriesNumberNotValidException, UDFAttributeNotProvidedException {
+ validator.validateInputSeriesNumber(1).validateRequiredAttribute("type");
+ }
+
+ @Override
+ public void beforeStart(UDFParameters parameters, UDTFConfigurations configurations)
+ throws MetadataException {
+ sourceDataType = parameters.getDataType(0);
+ targetDataType = TSDataType.valueOf(parameters.getString("type"));
+
+ configurations
+ .setAccessStrategy(new RowByRowAccessStrategy())
+ .setOutputDataType(targetDataType);
+ }
+
+ @Override
+ public void transform(Row row, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (sourceDataType) {
+ case INT32:
+ cast(row.getTime(), row.getInt(0), collector);
+ return;
+ case INT64:
+ cast(row.getTime(), row.getLong(0), collector);
+ return;
+ case FLOAT:
+ cast(row.getTime(), row.getFloat(0), collector);
+ return;
+ case DOUBLE:
+ cast(row.getTime(), row.getDouble(0), collector);
+ return;
+ case BOOLEAN:
+ cast(row.getTime(), row.getBoolean(0), collector);
+ return;
+ case TEXT:
+ cast(row.getTime(), row.getBinary(0), collector);
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, int value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (targetDataType) {
+ case INT32:
+ collector.putInt(time, value);
+ return;
+ case INT64:
+ collector.putLong(time, value);
+ return;
+ case FLOAT:
+ collector.putFloat(time, value);
+ return;
+ case DOUBLE:
+ collector.putDouble(time, value);
+ return;
+ case BOOLEAN:
+ collector.putBoolean(time, value != 0);
+ return;
+ case TEXT:
+ collector.putString(time, String.valueOf(value));
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, long value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (targetDataType) {
+ case INT32:
+ collector.putInt(time, (int) value);
+ return;
+ case INT64:
+ collector.putLong(time, value);
+ return;
+ case FLOAT:
+ collector.putFloat(time, value);
+ return;
+ case DOUBLE:
+ collector.putDouble(time, value);
+ return;
+ case BOOLEAN:
+ collector.putBoolean(time, value != 0l);
+ return;
+ case TEXT:
+ collector.putString(time, String.valueOf(value));
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, float value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (targetDataType) {
+ case INT32:
+ collector.putInt(time, (int) value);
+ return;
+ case INT64:
+ collector.putLong(time, (long) value);
+ return;
+ case FLOAT:
+ collector.putFloat(time, value);
+ return;
+ case DOUBLE:
+ collector.putDouble(time, value);
+ return;
+ case BOOLEAN:
+ collector.putBoolean(time, value != 0f);
+ return;
+ case TEXT:
+ collector.putString(time, String.valueOf(value));
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, double value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (targetDataType) {
+ case INT32:
+ collector.putInt(time, (int) value);
+ return;
+ case INT64:
+ collector.putLong(time, (long) value);
+ return;
+ case FLOAT:
+ collector.putFloat(time, (float) value);
+ return;
+ case DOUBLE:
+ collector.putDouble(time, value);
+ return;
+ case BOOLEAN:
+ collector.putBoolean(time, value != 0.0);
+ return;
+ case TEXT:
+ collector.putString(time, String.valueOf(value));
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, boolean value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ switch (targetDataType) {
+ case INT32:
+ collector.putInt(time, value ? 1 : 0);
+ return;
+ case INT64:
+ collector.putLong(time, value ? 1L : 0L);
+ return;
+ case FLOAT:
+ collector.putFloat(time, value ? 1.0f : 0.0f);
+ return;
+ case DOUBLE:
+ collector.putDouble(time, value ? 1.0 : 0.0);
+ return;
+ case BOOLEAN:
+ collector.putBoolean(time, value);
+ return;
+ case TEXT:
+ collector.putString(time, String.valueOf(value));
+ return;
+ default:
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ private void cast(long time, Binary value, PointCollector collector)
+ throws IOException, QueryProcessException {
+ String stringValue = value.getStringValue();
+ boolean numeric = isNumeric(stringValue);
+ switch (targetDataType) {
+ case INT32:
+ if (numeric) {
+ collector.putInt(time, (int) Double.parseDouble(stringValue));
+ }
Review comment:
```suggestion
try {
collector.putInt(time, (int) Double.parseDouble(stringValue));
} catch (Exception ignored) {
// skip
}
```
##########
File path: docs/zh/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -469,6 +469,52 @@ Total line number = 5
It costs 0.005s
```
+#### 数据类型转换函数
+
+当前IoTDB支持6种数据类型,其中包括INT32、INT64、FLOAT、DOUBLE、BOOLEAN以及TEXT。当我们对数据进行查询或者计算时可能需要进行数据类型的转换, 比如说将TEXT转换为INT32,或者提高数据精度,比如说将FLOAT转换为DOUBLE。所以,IoTDB支持使用cast函数对数据类型进行转换。
+
+| 函数名 | 必要的属性参数 | 输出序列类型 | 功能类型 |
+| ------ | ------------------------------------------------------------ | ------------------------ | ---------------------------------- |
+| CAST | `type`:输出的数据点的类型,只能是 INT32 / INT64 / FLOAT / DOUBLE / BOOLEAN / TEXT | 由输入属性参数`type`决定 | 将数据转换为`type`参数指定的类型。 |
+
+##### 类型转换说明
+
+1.当INT32、INT64类型的值不为0时,FLOAT与DOUBLE类型的值不为0.0时,TEXT类型不为空字符串或者"false"时,转换为BOOLEAN类型时值为true,否则为false。
+
+2.当BOOLEAN类型的值为true时,转换为INT32与INT64类型的值为1,转换为FLOAT或者DOUBLE类型时值为1.0,转换为TEXT类型时值为"true"。当BOOLEAN类型的值为false时,转换为INT32与INT64类型的值为0,转换为FLOAT或者DOUBLE类型时值为0.0,转换为TEXT类型时值为"false"。
+
+3.当TEXT类型转换为INT32、INT64、FLOAT类型时,会先将TEXT类型的数据转换为DOUBLE类型,然后再转换为对应的类型,此时可能会存在损失精度的问题。
+
+##### 演示
+测试数据:
+```
+IoTDB> select text from root.test;
++-----------------------------+--------------+
+| Time|root.test.text|
++-----------------------------+--------------+
+|1970-01-01T08:00:00.001+08:00| 1.1|
+|1970-01-01T08:00:00.002+08:00| 1|
+|1970-01-01T08:00:00.003+08:00| hello world|
+|1970-01-01T08:00:00.004+08:00| false|
++-----------------------------+--------------+
+```
+SQL语句:
+```sql
+select cast(text,'type'='BOOLEAN') from root.test;
Review comment:
```suggestion
select cast(text, 'type'='BOOLEAN') from root.test;
```
为每一种数据类型增加一列
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org