You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iotdb.apache.org by qi...@apache.org on 2020/06/06 11:29:05 UTC
[incubator-iotdb] 02/02: [IOTDB-622] split group by level to two
parts
This is an automated email from the ASF dual-hosted git repository.
qiaojialin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-iotdb.git
commit 2e5d6820fa5930dd19ac8feba3e3831462d443f1
Author: mychaow <94...@qq.com>
AuthorDate: Sun May 24 17:19:07 2020 +0800
[IOTDB-622] split group by level to two parts
---
docs/SystemDesign/DataQuery/AggregationQuery.md | 22 +++
docs/SystemDesign/DataQuery/GroupByQuery.md | 2 +-
.../DML Data Manipulation Language.md | 65 ++++---
docs/UserGuide/Operation Manual/SQL Reference.md | 5 +-
docs/zh/SystemDesign/DataQuery/AggregationQuery.md | 20 ++
docs/zh/SystemDesign/DataQuery/GroupByQuery.md | 2 +-
.../DML Data Manipulation Language.md | 73 +++++---
.../zh/UserGuide/Operation Manual/SQL Reference.md | 5 +-
.../org/apache/iotdb/db/qp/strategy/SqlBase.g4 | 6 +-
.../org/apache/iotdb/db/metadata/MetaUtils.java | 2 +-
.../apache/iotdb/db/qp/executor/PlanExecutor.java | 4 +-
.../iotdb/db/qp/physical/crud/AggregationPlan.java | 10 +
...yFillTimePlan.java => GroupByTimeFillPlan.java} | 4 +-
.../iotdb/db/qp/physical/crud/GroupByTimePlan.java | 9 -
.../iotdb/db/qp/physical/crud/QueryPlan.java | 10 -
.../iotdb/db/qp/strategy/LogicalGenerator.java | 45 +++--
.../iotdb/db/qp/strategy/PhysicalGenerator.java | 31 +--
.../iotdb/db/query/dataset/SingleDataSet.java | 4 +-
.../query/dataset/groupby/GroupByFillDataSet.java | 10 +-
.../query/dataset/groupby/GroupByLevelDataSet.java | 208 ---------------------
.../query/dataset/groupby/GroupByTimeDataSet.java | 96 ++++++++++
.../db/query/executor/AggregationExecutor.java | 30 ++-
.../iotdb/db/query/executor/IQueryRouter.java | 2 +-
.../iotdb/db/query/executor/QueryRouter.java | 85 +++++----
.../org/apache/iotdb/db/service/TSServiceImpl.java | 14 +-
.../org/apache/iotdb/db/utils/FilePathUtils.java | 45 ++++-
.../apache/iotdb/db/qp/plan/PhysicalPlanTest.java | 14 +-
.../db/query/dataset/GroupByLevelDataSetTest.java | 64 +++----
...ataSetTest.java => GroupByTimeDataSetTest.java} | 57 +++---
29 files changed, 486 insertions(+), 458 deletions(-)
diff --git a/docs/SystemDesign/DataQuery/AggregationQuery.md b/docs/SystemDesign/DataQuery/AggregationQuery.md
index 486f8de..4d096d1 100644
--- a/docs/SystemDesign/DataQuery/AggregationQuery.md
+++ b/docs/SystemDesign/DataQuery/AggregationQuery.md
@@ -112,3 +112,25 @@ while (timestampGenerator.hasNext()) {
}
}
```
+
+## Aggregated query with level
+
+After aggregated query, we could also to count the total number of points of
+
+each node at the given level in current Metadata Tree.
+
+The logic is in the `AggregationExecutor` class.
+
+1. In the beginning, get the final paths group by level and the origin path index to final path.
+ > For example, we could get final path `root.sg1` by `root.sg1.d1.s0,root.sg1.d2.s1` and `level=1`.
+
+2. Then, get the aggregated query result: RowRecord.
+
+3. Finally, merge each RowRecord to NewRecord, which has fields like <final path, count>.
+
+ > For example, we will get new RowRecord `<root.sg1,7>` by `<root.sg1.d1.s0, 3>, <root.sg1.d2.s1, 4>` and level=1.
+
+
+> Attention:
+> 1. only support count aggregation
+> 2. root's level == 0
\ No newline at end of file
diff --git a/docs/SystemDesign/DataQuery/GroupByQuery.md b/docs/SystemDesign/DataQuery/GroupByQuery.md
index b11eaae..7ec7173 100644
--- a/docs/SystemDesign/DataQuery/GroupByQuery.md
+++ b/docs/SystemDesign/DataQuery/GroupByQuery.md
@@ -266,7 +266,7 @@ After down-frequency query, we could also to count the total number of points of
each node at the given level in current Metadata Tree.
-The logic is in the `GroupByLevelDataSet` class.
+The logic is in the `GroupByTimeDataSet` class.
1. In the beginning, get the final paths group by level and the origin path index to final path.
> For example, we could get final path `root.sg1` by `root.sg1.d1.s0,root.sg1.d2.s1` and `level=1`.
diff --git a/docs/UserGuide/Operation Manual/DML Data Manipulation Language.md b/docs/UserGuide/Operation Manual/DML Data Manipulation Language.md
index 113a6ca..27afcb7 100644
--- a/docs/UserGuide/Operation Manual/DML Data Manipulation Language.md
+++ b/docs/UserGuide/Operation Manual/DML Data Manipulation Language.md
@@ -146,6 +146,46 @@ The selected timeseries are "the power supply status of ln group wf01 plant wt01
The execution result of this SQL statement is as follows:
<center><img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" src="https://user-images.githubusercontent.com/13203019/51577450-dcfe0800-1ef4-11e9-9399-4ba2b2b7fb73.jpg"></center>
+### Aggregate Query
+This section mainly introduces the related examples of aggregate query.
+
+#### Count Points
+
+```
+select count(status) from root.ln.wf01.wt01;
+```
+
+| count(root.ln.wf01.wt01.status) |
+| -------------- |
+| 4 |
+
+
+##### Count Points By Level
+
+Level could be defined to show count the number of points of each node at the given level in current Metadata Tree.
+
+This could be used to query the number of points under each device.
+
+The SQL statement is:
+
+```
+select count(status) from root.ln.wf01.wt01 group by level=1;
+```
+
+
+| Time | count(root.ln) |
+| ------ | -------------- |
+| 0 | 7 |
+
+
+```
+select count(status) from root.ln.wf01.wt01 group by level=2;
+```
+
+| Time | count(root.ln.wf01) | count(root.ln.wf02) |
+| ------ | ------------------- | ------------------- |
+| 0 | 4 | 3 |
+
### Down-Frequency Aggregate Query
This section mainly introduces the related examples of down-frequency aggregation query,
@@ -263,7 +303,7 @@ We will get the result like following:
| 40 | 5 |
-### Down-Frequency Aggregate Query with Level Clause
+#### Down-Frequency Aggregate Query with Level Clause
Level could be defined to show count the number of points of each node at the given level in current Metadata Tree.
@@ -271,28 +311,7 @@ This could be used to query the number of points under each device.
The SQL statement is:
-This means aggregate query only with path by level.
-
-```
-select count(status) from root.ln.wf01.wt01 group by level=1;
-```
-
-
-| Time | count(root.ln) |
-| ------ | -------------- |
-| 0 | 7 |
-
-
-```
-select count(status) from root.ln.wf01.wt01 group by level=2;
-```
-
-| Time | count(root.ln.wf01) | count(root.ln.wf02) |
-| ------ | ------------------- | ------------------- |
-| 0 | 4 | 3 |
-
-
-We can also get down-frequency agggate query by level.
+Get down-frequency aggregate query by level.
```
select count(status) from root.ln.wf01.wt01 group by ([0,20),3ms), level=1;
diff --git a/docs/UserGuide/Operation Manual/SQL Reference.md b/docs/UserGuide/Operation Manual/SQL Reference.md
index 53eadd7..3e15816 100644
--- a/docs/UserGuide/Operation Manual/SQL Reference.md
+++ b/docs/UserGuide/Operation Manual/SQL Reference.md
@@ -338,6 +338,7 @@ Eg. IoTDB > SELECT MIN_TIME(temperature) FROM root.ln.wf01.wt01 WHERE root.ln.wf
Eg. IoTDB > SELECT MAX_TIME(temperature) FROM root.ln.wf01.wt01 WHERE root.ln.wf01.wt01.temperature > 24
Eg. IoTDB > SELECT MIN_VALUE(temperature) FROM root.ln.wf01.wt01 WHERE root.ln.wf01.wt01.temperature > 23
Eg. IoTDB > SELECT MAX_VALUE(temperature) FROM root.ln.wf01.wt01 WHERE root.ln.wf01.wt01.temperature < 25
+Eg. IoTDB > SELECT COUNT(temperature) FROM root.ln.wf01.wt01 WHERE root.ln.wf01.wt01.temperature < 25 GROUP BY LEVEL=1
Note: the statement needs to satisfy this constraint: <Path>(SelectClause) + <PrefixPath>(FromClause) = <Timeseries>
Note: If the <SensorExpr>(WhereClause) is started with <Path> and not with ROOT, the statement needs to satisfy this constraint: <PrefixPath>(FromClause) + <Path>(SensorExpr) = <Timeseries>
Note: In Version 0.7.0, if <WhereClause> includes `OR`, time filter can not be used.
@@ -347,7 +348,7 @@ Note: There must be a space on both sides of the plus and minus operator appeari
* Group By Statement
```
-SELECT <SelectClause> FROM <FromClause> WHERE <WhereClause> GROUP BY <GroupByClause>
+SELECT <SelectClause> FROM <FromClause> WHERE <WhereClause> GROUP BY <GroupByTimeClause>
SelectClause : <Function> [COMMA < Function >]*
Function : <AggregationFunction> LPAREN <Path> RPAREN
FromClause : <PrefixPath>
@@ -358,7 +359,7 @@ TimeExpr : TIME PrecedenceEqualOperator (<TimeValue> | <RelativeTime>)
RelativeTimeDurationUnit = Integer ('Y'|'MO'|'W'|'D'|'H'|'M'|'S'|'MS'|'US'|'NS')
RelativeTime : (now() | <TimeValue>) [(+|-) RelativeTimeDurationUnit]+
SensorExpr : (<Timeseries> | <Path>) PrecedenceEqualOperator <PointValue>
-GroupByClause : LPAREN <TimeInterval> COMMA <TimeUnit> (COMMA <TimeUnit>)? RPAREN
+GroupByTimeClause : LPAREN <TimeInterval> COMMA <TimeUnit> (COMMA <TimeUnit>)? RPAREN
TimeInterval: LSBRACKET <TimeValue> COMMA <TimeValue> RRBRACKET | LRBRACKET <TimeValue> COMMA <TimeValue> RSBRACKET
TimeUnit : Integer <DurationUnit>
DurationUnit : "ms" | "s" | "m" | "h" | "d" | "w"
diff --git a/docs/zh/SystemDesign/DataQuery/AggregationQuery.md b/docs/zh/SystemDesign/DataQuery/AggregationQuery.md
index 56b2589..519d147 100644
--- a/docs/zh/SystemDesign/DataQuery/AggregationQuery.md
+++ b/docs/zh/SystemDesign/DataQuery/AggregationQuery.md
@@ -112,3 +112,23 @@ while (timestampGenerator.hasNext()) {
}
}
```
+
+## 使用Level来统计点数
+
+对于count聚合查询，我们也可以使用level关键字来进一步汇总点数。
+
+这个逻辑在 `AggregationExecutor`类里。
+
+1. 首先，把所有涉及到的时序按level来进行汇集，最后的路径。
+ > 例如把root.sg1.d1.s0,root.sg1.d2.s1按level=1汇集成root.sg1。
+
+2. 然后调用上述的聚合逻辑求出所有时序的总点数信息，这个会返回RowRecord数据结构。
+
+3. 最后，把聚合查询返回的RowRecord按上述的final paths，进行累加，组合成新的RowRecord。
+
+ > 例如，把《root.sg1.d1.s0,3》，《root.sg1.d2.s1，4》聚合成《root.sg1，7》
+
+
+> 注意:
+> 1. 这里只支持count操作
+> 2. root的层级level=0
\ No newline at end of file
diff --git a/docs/zh/SystemDesign/DataQuery/GroupByQuery.md b/docs/zh/SystemDesign/DataQuery/GroupByQuery.md
index 0c12398..2b32473 100644
--- a/docs/zh/SystemDesign/DataQuery/GroupByQuery.md
+++ b/docs/zh/SystemDesign/DataQuery/GroupByQuery.md
@@ -271,7 +271,7 @@ for (int cnt = 1; cnt < timeStampFetchSize && timestampGenerator.hasNext(); cnt+
降采样后，我们也可以使用level关键字来进一步汇总点数。
-这个逻辑在 `GroupByLevelDataSet`类里。
+这个逻辑在 `GroupByTimeDataSet`类里。
1. 首先，把所有涉及到的时序按level来进行汇集，最后的路径。
> 例如把root.sg1.d1.s0,root.sg1.d2.s1按level=1汇集成root.sg1。
diff --git a/docs/zh/UserGuide/Operation Manual/DML Data Manipulation Language.md b/docs/zh/UserGuide/Operation Manual/DML Data Manipulation Language.md
index 87cf119..21a9828 100644
--- a/docs/zh/UserGuide/Operation Manual/DML Data Manipulation Language.md
+++ b/docs/zh/UserGuide/Operation Manual/DML Data Manipulation Language.md
@@ -170,6 +170,48 @@ select s1,s2 from root.sg1.* GROUP BY DEVICE
'disable align' 意味着每条时序就有3列存在。更多语法请参照 SQL REFERENCE.
+### 聚合查询
+本章节主要介绍聚合查询的相关示例，
+主要使用的是IoTDB SELECT语句的聚合查询函数。
+
+#### 统计总点数
+
+
+```
+select count(status) from root.ln.wf01.wt01;
+```
+
+| count(root.ln.wf01.wt01.status) |
+| -------------- |
+| 4 |
+
+
+##### 按层级统计
+通过定义LEVEL来统计指定层级下的数据点个数。
+
+这可以用来查询不同层级下的数据点总个数
+
+语法是：
+
+这个可以用来查询某个路径下的总数据点数
+
+```
+select count(status) from root.ln.wf01.wt01 group by level=1;
+```
+
+
+| Time | count(root.ln) |
+| ------ | -------------- |
+| 0 | 7 |
+
+
+```
+select count(status) from root.ln.wf01.wt01 group by level=2;
+```
+
+| Time | count(root.ln.wf01) | count(root.ln.wf02) |
+| ------ | ------------------- | ------------------- |
+| 0 | 4 | 3 |
### 降频聚合查询
@@ -295,36 +337,13 @@ SQL执行后的结果集如下所示：
| 35 | 3 |
| 40 | 5 |
-### 降采样后按Level聚合查询
+#### 降采样后按Level聚合查询
除此之外，还可以通过定义LEVEL来统计指定层级下的数据点个数。
-这可以用来查询不同层级下的数据点总个数
-
-语法是：
-
-这个可以用来查询某个路径下的总数据点数
-
-```
-select count(status) from root.ln.wf01.wt01 group by level=1;
-```
-
-
-| Time | count(root.ln) |
-| ------ | -------------- |
-| 0 | 7 |
-
-
-```
-select count(status) from root.ln.wf01.wt01 group by level=2;
-```
-
-| Time | count(root.ln.wf01) | count(root.ln.wf02) |
-| ------ | ------------------- | ------------------- |
-| 0 | 4 | 3 |
-
+例如：
-也可以用来统计降采样后的数据点个数
+统计降采样后的数据点个数
```
select count(status) from root.ln.wf01.wt01 group by ([0,20),3ms), level=1;
@@ -717,7 +736,7 @@ SQL语句将不会执行，并且相应的错误提示如下：
<center><img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" src="https://user-images.githubusercontent.com/13203019/51577867-577b5780-1ef6-11e9-978c-e02c1294bcc5.jpg"></center>
-#### Row and Column Control over Query Results
+#### 控制查询结果的行和列
除了对查询结果进行行或列控制之外，IoTDB还允许用户控制查询结果的行和列。 这是同时包含LIMIT子句和SLIMIT子句的完整示例。
diff --git a/docs/zh/UserGuide/Operation Manual/SQL Reference.md b/docs/zh/UserGuide/Operation Manual/SQL Reference.md
index add2f04..6a6fb5a 100644
--- a/docs/zh/UserGuide/Operation Manual/SQL Reference.md
+++ b/docs/zh/UserGuide/Operation Manual/SQL Reference.md
@@ -329,6 +329,7 @@ Eg. IoTDB > SELECT MIN_TIME(temperature) FROM root.ln.wf01.wt01 WHERE root.ln.wf
Eg. IoTDB > SELECT MAX_TIME(temperature) FROM root.ln.wf01.wt01 WHERE root.ln.wf01.wt01.temperature > 24
Eg. IoTDB > SELECT MIN_VALUE(temperature) FROM root.ln.wf01.wt01 WHERE root.ln.wf01.wt01.temperature > 23
Eg. IoTDB > SELECT MAX_VALUE(temperature) FROM root.ln.wf01.wt01 WHERE root.ln.wf01.wt01.temperature < 25
+Eg. IoTDB > SELECT COUNT(temperature) FROM root.ln.wf01.wt01 WHERE root.ln.wf01.wt01.temperature < 25 GROUP BY LEVEL=1
Note: the statement needs to satisfy this constraint: <Path>(SelectClause) + <PrefixPath>(FromClause) = <Timeseries>
Note: If the <SensorExpr>(WhereClause) is started with <Path> and not with ROOT, the statement needs to satisfy this constraint: <PrefixPath>(FromClause) + <Path>(SensorExpr) = <Timeseries>
Note: In Version 0.7.0, if <WhereClause> includes `OR`, time filter can not be used.
@@ -338,7 +339,7 @@ Note: There must be a space on both sides of the plus and minus operator appeari
* Group By 语句
```
-SELECT <SelectClause> FROM <FromClause> WHERE <WhereClause> GROUP BY <GroupByClause>
+SELECT <SelectClause> FROM <FromClause> WHERE <WhereClause> GROUP BY <GroupByTimeClause>
SelectClause : <Function> [COMMA < Function >]*
Function : <AggregationFunction> LPAREN <Path> RPAREN
FromClause : <PrefixPath>
@@ -349,7 +350,7 @@ TimeExpr : TIME PrecedenceEqualOperator (<TimeValue> | <RelativeTime>)
RelativeTimeDurationUnit = Integer ('Y'|'MO'|'W'|'D'|'H'|'M'|'S'|'MS'|'US'|'NS')
RelativeTime : (now() | <TimeValue>) [(+|-) RelativeTimeDurationUnit]+
SensorExpr : (<Timeseries> | <Path>) PrecedenceEqualOperator <PointValue>
-GroupByClause : LPAREN <TimeInterval> COMMA <TimeUnit> (COMMA <TimeUnit>)? RPAREN
+GroupByTimeClause : LPAREN <TimeInterval> COMMA <TimeUnit> (COMMA <TimeUnit>)? RPAREN
TimeInterval: LSBRACKET <TimeValue> COMMA <TimeValue> RRBRACKET | LRBRACKET <TimeValue> COMMA <TimeValue> RSBRACKET
TimeUnit : Integer <DurationUnit>
DurationUnit : "ms" | "s" | "m" | "h" | "d" | "w"
diff --git a/server/src/main/antlr4/org/apache/iotdb/db/qp/strategy/SqlBase.g4 b/server/src/main/antlr4/org/apache/iotdb/db/qp/strategy/SqlBase.g4
index b9c30ea..a7b8904 100644
--- a/server/src/main/antlr4/org/apache/iotdb/db/qp/strategy/SqlBase.g4
+++ b/server/src/main/antlr4/org/apache/iotdb/db/qp/strategy/SqlBase.g4
@@ -196,6 +196,7 @@ specialClause
| groupByFillClause
| fillClause slimitClause? alignByDeviceClauseOrDisableAlign?
| alignByDeviceClauseOrDisableAlign
+ | groupByLevelClause specialLimit?
;
specialLimit
@@ -246,7 +247,6 @@ groupByTimeClause
COMMA DURATION
(COMMA DURATION)?
RR_BRACKET
- | GROUP BY LEVEL OPERATOR_EQ INT
| GROUP BY LR_BRACKET
timeInterval
COMMA DURATION
@@ -263,6 +263,10 @@ groupByFillClause
FILL LR_BRACKET typeClause (COMMA typeClause)* RR_BRACKET
;
+groupByLevelClause
+ : GROUP BY LEVEL OPERATOR_EQ INT
+ ;
+
typeClause
: dataType LS_BRACKET linearClause RS_BRACKET
| dataType LS_BRACKET previousClause RS_BRACKET
diff --git a/server/src/main/java/org/apache/iotdb/db/metadata/MetaUtils.java b/server/src/main/java/org/apache/iotdb/db/metadata/MetaUtils.java
index de3e9c1..0e1d953 100644
--- a/server/src/main/java/org/apache/iotdb/db/metadata/MetaUtils.java
+++ b/server/src/main/java/org/apache/iotdb/db/metadata/MetaUtils.java
@@ -24,7 +24,7 @@ import org.apache.iotdb.db.exception.metadata.MetadataException;
import static org.apache.iotdb.db.conf.IoTDBConstant.PATH_WILDCARD;
-class MetaUtils {
+public class MetaUtils {
public static final String PATH_SEPARATOR = "\\.";
diff --git a/server/src/main/java/org/apache/iotdb/db/qp/executor/PlanExecutor.java b/server/src/main/java/org/apache/iotdb/db/qp/executor/PlanExecutor.java
index dd84071..8c6b228 100644
--- a/server/src/main/java/org/apache/iotdb/db/qp/executor/PlanExecutor.java
+++ b/server/src/main/java/org/apache/iotdb/db/qp/executor/PlanExecutor.java
@@ -292,8 +292,8 @@ public class PlanExecutor implements IPlanExecutor {
if (queryPlan.getPaths() == null || queryPlan.getPaths().isEmpty()) {
// no time series are selected, return EmptyDataSet
return new EmptyDataSet();
- } else if (queryPlan instanceof GroupByFillTimePlan) {
- GroupByFillTimePlan groupByFillPlan = (GroupByFillTimePlan) queryPlan;
+ } else if (queryPlan instanceof GroupByTimeFillPlan) {
+ GroupByTimeFillPlan groupByFillPlan = (GroupByTimeFillPlan) queryPlan;
return queryRouter.groupByFill(groupByFillPlan, context);
} else if (queryPlan instanceof GroupByTimePlan) {
GroupByTimePlan groupByTimePlan = (GroupByTimePlan) queryPlan;
diff --git a/server/src/main/java/org/apache/iotdb/db/qp/physical/crud/AggregationPlan.java b/server/src/main/java/org/apache/iotdb/db/qp/physical/crud/AggregationPlan.java
index 1f06dbf..972be84 100644
--- a/server/src/main/java/org/apache/iotdb/db/qp/physical/crud/AggregationPlan.java
+++ b/server/src/main/java/org/apache/iotdb/db/qp/physical/crud/AggregationPlan.java
@@ -31,6 +31,8 @@ public class AggregationPlan extends RawDataQueryPlan {
private List<String> aggregations = new ArrayList<>();
private List<String> deduplicatedAggregations = new ArrayList<>();
+ private int level = -1;
+
public AggregationPlan() {
super();
setOperatorType(Operator.OperatorType.AGGREGATION);
@@ -56,4 +58,12 @@ public class AggregationPlan extends RawDataQueryPlan {
public void setDeduplicatedAggregations(List<String> deduplicatedAggregations) {
this.deduplicatedAggregations = deduplicatedAggregations;
}
+
+ public int getLevel() {
+ return level;
+ }
+
+ public void setLevel(int level) {
+ this.level = level;
+ }
}
diff --git a/server/src/main/java/org/apache/iotdb/db/qp/physical/crud/GroupByFillTimePlan.java b/server/src/main/java/org/apache/iotdb/db/qp/physical/crud/GroupByTimeFillPlan.java
similarity index 93%
rename from server/src/main/java/org/apache/iotdb/db/qp/physical/crud/GroupByFillTimePlan.java
rename to server/src/main/java/org/apache/iotdb/db/qp/physical/crud/GroupByTimeFillPlan.java
index 55400f7..c5d741b 100644
--- a/server/src/main/java/org/apache/iotdb/db/qp/physical/crud/GroupByFillTimePlan.java
+++ b/server/src/main/java/org/apache/iotdb/db/qp/physical/crud/GroupByTimeFillPlan.java
@@ -24,11 +24,11 @@ import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
import java.util.Map;
-public class GroupByFillTimePlan extends GroupByTimePlan {
+public class GroupByTimeFillPlan extends GroupByTimePlan {
private Map<TSDataType, IFill> fillTypes;
- public GroupByFillTimePlan() {
+ public GroupByTimeFillPlan() {
super();
setOperatorType(Operator.OperatorType.GROUP_BY_FILL);
}
diff --git a/server/src/main/java/org/apache/iotdb/db/qp/physical/crud/GroupByTimePlan.java b/server/src/main/java/org/apache/iotdb/db/qp/physical/crud/GroupByTimePlan.java
index cc76465..7d00bdd 100644
--- a/server/src/main/java/org/apache/iotdb/db/qp/physical/crud/GroupByTimePlan.java
+++ b/server/src/main/java/org/apache/iotdb/db/qp/physical/crud/GroupByTimePlan.java
@@ -33,8 +33,6 @@ public class GroupByTimePlan extends AggregationPlan {
// if it is left close and right open interval
private boolean leftCRightO = true;
- private boolean byTime = false;
-
public GroupByTimePlan() {
super();
setOperatorType(Operator.OperatorType.GROUPBYTIME);
@@ -80,11 +78,4 @@ public class GroupByTimePlan extends AggregationPlan {
this.leftCRightO = leftCRightO;
}
- public boolean isByTime() {
- return byTime;
- }
-
- public void setByTime(boolean isByTime) {
- this.byTime = isByTime;
- }
}
diff --git a/server/src/main/java/org/apache/iotdb/db/qp/physical/crud/QueryPlan.java b/server/src/main/java/org/apache/iotdb/db/qp/physical/crud/QueryPlan.java
index 7f3b9bd..58b4c13 100644
--- a/server/src/main/java/org/apache/iotdb/db/qp/physical/crud/QueryPlan.java
+++ b/server/src/main/java/org/apache/iotdb/db/qp/physical/crud/QueryPlan.java
@@ -36,8 +36,6 @@ public abstract class QueryPlan extends PhysicalPlan {
private int rowLimit = 0;
private int rowOffset = 0;
- private int level = -1;
-
private Map<String, Integer> pathToIndex = new HashMap<>();
public QueryPlan() {
@@ -101,12 +99,4 @@ public abstract class QueryPlan extends PhysicalPlan {
public Map<String, Integer> getPathToIndex() {
return pathToIndex;
}
-
- public int getLevel() {
- return level;
- }
-
- public void setLevel(int level) {
- this.level = level;
- }
}
diff --git a/server/src/main/java/org/apache/iotdb/db/qp/strategy/LogicalGenerator.java b/server/src/main/java/org/apache/iotdb/db/qp/strategy/LogicalGenerator.java
index 41dfd6a..dda397d 100644
--- a/server/src/main/java/org/apache/iotdb/db/qp/strategy/LogicalGenerator.java
+++ b/server/src/main/java/org/apache/iotdb/db/qp/strategy/LogicalGenerator.java
@@ -96,7 +96,7 @@ import org.apache.iotdb.db.qp.strategy.SqlBaseParser.GrantRoleContext;
import org.apache.iotdb.db.qp.strategy.SqlBaseParser.GrantRoleToUserContext;
import org.apache.iotdb.db.qp.strategy.SqlBaseParser.GrantUserContext;
import org.apache.iotdb.db.qp.strategy.SqlBaseParser.GrantWatermarkEmbeddingContext;
-import org.apache.iotdb.db.qp.strategy.SqlBaseParser.GroupByClauseContext;
+import org.apache.iotdb.db.qp.strategy.SqlBaseParser.GroupByTimeClauseContext;
import org.apache.iotdb.db.qp.strategy.SqlBaseParser.InClauseContext;
import org.apache.iotdb.db.qp.strategy.SqlBaseParser.InsertColumnSpecContext;
import org.apache.iotdb.db.qp.strategy.SqlBaseParser.InsertStatementContext;
@@ -164,6 +164,8 @@ import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
import org.apache.iotdb.tsfile.file.metadata.enums.TSEncoding;
import org.apache.iotdb.tsfile.read.common.Path;
import org.apache.iotdb.tsfile.utils.StringContainer;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
/**
* This class is a listener and you can get an operator which is a logical plan.
@@ -788,9 +790,8 @@ public class LogicalGenerator extends SqlBaseBaseListener {
@Override
public void enterGroupByFillClause(SqlBaseParser.GroupByFillClauseContext ctx) {
super.enterGroupByFillClause(ctx);
- queryOp.setGroupByLevel(true);
- queryOp.setFill(true);
queryOp.setGroupByTime(true);
+ queryOp.setFill(true);
queryOp.setLeftCRightO(ctx.timeInterval().LS_BRACKET() != null);
@@ -864,33 +865,37 @@ public class LogicalGenerator extends SqlBaseBaseListener {
@Override
public void enterGroupByTimeClause(GroupByTimeClauseContext ctx) {
super.enterGroupByTimeClause(ctx);
- queryOp.setGroupByLevel(true);
- if (ctx.timeInterval() != null) {
- queryOp.setGroupByTime(true);
- queryOp.setLeftCRightO(ctx.timeInterval().LS_BRACKET() != null);
- // parse timeUnit
- queryOp.setUnit(parseDuration(ctx.DURATION(0).getText()));
- queryOp.setSlidingStep(queryOp.getUnit());
- // parse sliding step
- if (ctx.DURATION().size() == 2) {
- queryOp.setSlidingStep(parseDuration(ctx.DURATION(1).getText()));
- if (queryOp.getSlidingStep() < queryOp.getUnit()) {
- throw new SQLParserException(
- "The third parameter sliding step shouldn't be smaller than the second parameter time interval.");
- }
+ queryOp.setGroupByTime(true);
+ queryOp.setLeftCRightO(ctx.timeInterval().LS_BRACKET() != null);
+ // parse timeUnit
+ queryOp.setUnit(parseDuration(ctx.DURATION(0).getText()));
+ queryOp.setSlidingStep(queryOp.getUnit());
+ // parse sliding step
+ if (ctx.DURATION().size() == 2) {
+ queryOp.setSlidingStep(parseDuration(ctx.DURATION(1).getText()));
+ if (queryOp.getSlidingStep() < queryOp.getUnit()) {
+ throw new SQLParserException(
+ "The third parameter sliding step shouldn't be smaller than the second parameter time interval.");
}
-
- parseTimeInterval(ctx.timeInterval());
}
+ parseTimeInterval(ctx.timeInterval());
+
if (ctx.INT() != null) {
- logger.debug("group by level:" + ctx.INT().getText() );
queryOp.setLevel(Integer.parseInt(ctx.INT().getText()));
}
}
@Override
+ public void enterGroupByLevelClause(SqlBaseParser.GroupByLevelClauseContext ctx) {
+ super.enterGroupByLevelClause(ctx);
+ queryOp.setGroupByLevel(true);
+
+ queryOp.setLevel(Integer.parseInt(ctx.INT().getText()));
+ }
+
+ @Override
public void enterFillClause(FillClauseContext ctx) {
super.enterFillClause(ctx);
FilterOperator filterOperator = queryOp.getFilterOperator();
diff --git a/server/src/main/java/org/apache/iotdb/db/qp/strategy/PhysicalGenerator.java b/server/src/main/java/org/apache/iotdb/db/qp/strategy/PhysicalGenerator.java
index 33d7e33..f0024fd 100644
--- a/server/src/main/java/org/apache/iotdb/db/qp/strategy/PhysicalGenerator.java
+++ b/server/src/main/java/org/apache/iotdb/db/qp/strategy/PhysicalGenerator.java
@@ -283,11 +283,11 @@ public class PhysicalGenerator {
private PhysicalPlan transformQuery(QueryOperator queryOperator) throws QueryProcessException {
QueryPlan queryPlan;
- if (queryOperator.isGroupByLevel() && queryOperator.isFill()) {
- queryPlan = new GroupByFillTimePlan();
- ((GroupByFillTimePlan) queryPlan).setInterval(queryOperator.getUnit());
- ((GroupByFillTimePlan) queryPlan).setSlidingStep(queryOperator.getSlidingStep());
- ((GroupByFillTimePlan) queryPlan).setLeftCRightO(queryOperator.isLeftCRightO());
+ if (queryOperator.isGroupByTime() && queryOperator.isFill()) {
+ queryPlan = new GroupByTimeFillPlan();
+ ((GroupByTimeFillPlan) queryPlan).setInterval(queryOperator.getUnit());
+ ((GroupByTimeFillPlan) queryPlan).setSlidingStep(queryOperator.getSlidingStep());
+ ((GroupByTimeFillPlan) queryPlan).setLeftCRightO(queryOperator.isLeftCRightO());
if (!queryOperator.isLeftCRightO()) {
((GroupByTimePlan) queryPlan).setStartTime(queryOperator.getStartTime() + 1);
((GroupByTimePlan) queryPlan).setEndTime(queryOperator.getEndTime() + 1);
@@ -295,16 +295,15 @@ public class PhysicalGenerator {
((GroupByTimePlan) queryPlan).setStartTime(queryOperator.getStartTime());
((GroupByTimePlan) queryPlan).setEndTime(queryOperator.getEndTime());
}
- ((GroupByFillTimePlan) queryPlan)
+ ((GroupByTimeFillPlan) queryPlan)
.setAggregations(queryOperator.getSelectOperator().getAggregations());
for (String aggregation : queryPlan.getAggregations()) {
if (!SQLConstant.LAST_VALUE.equals(aggregation)) {
throw new QueryProcessException("Group By Fill only support last_value function");
}
}
- ((GroupByFillTimePlan) queryPlan).setFillType(queryOperator.getFillTypes());
- ((GroupByFillTimePlan) queryPlan).setByTime(queryOperator.isGroupByTime());
- } else if (queryOperator.isGroupByLevel()) {
+ ((GroupByTimeFillPlan) queryPlan).setFillType(queryOperator.getFillTypes());
+ } else if (queryOperator.isGroupByTime()) {
queryPlan = new GroupByTimePlan();
((GroupByTimePlan) queryPlan).setInterval(queryOperator.getUnit());
((GroupByTimePlan) queryPlan).setSlidingStep(queryOperator.getSlidingStep());
@@ -319,12 +318,11 @@ public class PhysicalGenerator {
((GroupByTimePlan) queryPlan)
.setAggregations(queryOperator.getSelectOperator().getAggregations());
((GroupByTimePlan) queryPlan).setLevel(queryOperator.getLevel());
- ((GroupByTimePlan) queryPlan).setByTime(queryOperator.isGroupByTime());
if (queryOperator.getLevel() >= 0) {
for (int i = 0; i < queryOperator.getSelectOperator().getAggregations().size(); i++) {
if (!SQLConstant.COUNT.equals(queryOperator.getSelectOperator().getAggregations().get(i))) {
- throw new QueryProcessException("group by level only support count");
+ throw new QueryProcessException("group by level only support count now.");
}
}
}
@@ -339,8 +337,16 @@ public class PhysicalGenerator {
((FillQueryPlan) queryPlan).setFillType(queryOperator.getFillTypes());
} else if (queryOperator.hasAggregation()) {
queryPlan = new AggregationPlan();
+ ((AggregationPlan)queryPlan).setLevel(queryOperator.getLevel());
((AggregationPlan) queryPlan)
.setAggregations(queryOperator.getSelectOperator().getAggregations());
+ if (queryOperator.getLevel() >= 0) {
+ for (int i = 0; i < queryOperator.getSelectOperator().getAggregations().size(); i++) {
+ if (!SQLConstant.COUNT.equals(queryOperator.getSelectOperator().getAggregations().get(i))) {
+ throw new QueryProcessException("group by level only support count now.");
+ }
+ }
+ }
} else if (queryOperator.isLastQuery()) {
queryPlan = new LastQueryPlan();
} else {
@@ -361,6 +367,9 @@ public class PhysicalGenerator {
} else if (queryPlan instanceof FillQueryPlan) {
alignByDevicePlan.setFillQueryPlan((FillQueryPlan) queryPlan);
} else if (queryPlan instanceof AggregationPlan) {
+ if (((AggregationPlan)queryPlan).getLevel() >= 0) {
+ throw new QueryProcessException("group by level does not support align by device now.");
+ }
alignByDevicePlan.setAggregationPlan((AggregationPlan) queryPlan);
}
diff --git a/server/src/main/java/org/apache/iotdb/db/query/dataset/SingleDataSet.java b/server/src/main/java/org/apache/iotdb/db/query/dataset/SingleDataSet.java
index 668b28f..9580b63 100644
--- a/server/src/main/java/org/apache/iotdb/db/query/dataset/SingleDataSet.java
+++ b/server/src/main/java/org/apache/iotdb/db/query/dataset/SingleDataSet.java
@@ -39,12 +39,12 @@ public class SingleDataSet extends QueryDataSet {
}
@Override
- protected boolean hasNextWithoutConstraint() throws IOException {
+ public boolean hasNextWithoutConstraint() throws IOException {
return i == 0;
}
@Override
- protected RowRecord nextWithoutConstraint() throws IOException {
+ public RowRecord nextWithoutConstraint() throws IOException {
i++;
return record;
}
diff --git a/server/src/main/java/org/apache/iotdb/db/query/dataset/groupby/GroupByFillDataSet.java b/server/src/main/java/org/apache/iotdb/db/query/dataset/groupby/GroupByFillDataSet.java
index d826f23..697cd3b 100644
--- a/server/src/main/java/org/apache/iotdb/db/query/dataset/groupby/GroupByFillDataSet.java
+++ b/server/src/main/java/org/apache/iotdb/db/query/dataset/groupby/GroupByFillDataSet.java
@@ -21,7 +21,7 @@ package org.apache.iotdb.db.query.dataset.groupby;
import org.apache.iotdb.db.conf.IoTDBDescriptor;
import org.apache.iotdb.db.exception.StorageEngineException;
import org.apache.iotdb.db.exception.query.QueryProcessException;
-import org.apache.iotdb.db.qp.physical.crud.GroupByFillTimePlan;
+import org.apache.iotdb.db.qp.physical.crud.GroupByTimeFillPlan;
import org.apache.iotdb.db.query.context.QueryContext;
import org.apache.iotdb.db.query.executor.LastQueryExecutor;
import org.apache.iotdb.db.query.executor.fill.IFill;
@@ -49,7 +49,7 @@ public class GroupByFillDataSet extends QueryDataSet {
public GroupByFillDataSet(List<Path> paths, List<TSDataType> dataTypes,
GroupByEngineDataSet groupByEngineDataSet,
- Map<TSDataType, IFill> fillTypes, QueryContext context, GroupByFillTimePlan groupByFillPlan)
+ Map<TSDataType, IFill> fillTypes, QueryContext context, GroupByTimeFillPlan groupByFillPlan)
throws StorageEngineException, IOException, QueryProcessException {
super(paths, dataTypes);
this.groupByEngineDataSet = groupByEngineDataSet;
@@ -57,8 +57,8 @@ public class GroupByFillDataSet extends QueryDataSet {
initPreviousParis(context, groupByFillPlan);
initLastTimeArray(context, groupByFillPlan);
}
-
- private void initPreviousParis(QueryContext context, GroupByFillTimePlan groupByFillPlan)
+
+ private void initPreviousParis(QueryContext context, GroupByTimeFillPlan groupByFillPlan)
throws StorageEngineException, IOException, QueryProcessException {
previousValue = new Object[paths.size()];
for (int i = 0; i < paths.size(); i++) {
@@ -85,7 +85,7 @@ public class GroupByFillDataSet extends QueryDataSet {
}
}
- private void initLastTimeArray(QueryContext context, GroupByFillTimePlan groupByFillPlan)
+ private void initLastTimeArray(QueryContext context, GroupByTimeFillPlan groupByFillPlan)
throws IOException, StorageEngineException, QueryProcessException {
lastTimeArray = new long[paths.size()];
Arrays.fill(lastTimeArray, Long.MAX_VALUE);
diff --git a/server/src/main/java/org/apache/iotdb/db/query/dataset/groupby/GroupByLevelDataSet.java b/server/src/main/java/org/apache/iotdb/db/query/dataset/groupby/GroupByLevelDataSet.java
deleted file mode 100644
index b24c610..0000000
--- a/server/src/main/java/org/apache/iotdb/db/query/dataset/groupby/GroupByLevelDataSet.java
+++ /dev/null
@@ -1,208 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.iotdb.db.query.dataset.groupby;
-
-import org.apache.iotdb.db.exception.StorageEngineException;
-import org.apache.iotdb.db.exception.query.QueryProcessException;
-import org.apache.iotdb.db.qp.physical.crud.GroupByTimePlan;
-import org.apache.iotdb.db.query.aggregation.AggregateResult;
-import org.apache.iotdb.db.query.context.QueryContext;
-import org.apache.iotdb.db.query.factory.AggregateResultFactory;
-import org.apache.iotdb.db.query.filter.TsFileFilter;
-import org.apache.iotdb.db.utils.FilePathUtils;
-import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
-import org.apache.iotdb.tsfile.read.common.Field;
-import org.apache.iotdb.tsfile.read.common.Path;
-import org.apache.iotdb.tsfile.read.common.RowRecord;
-import org.apache.iotdb.tsfile.read.filter.basic.Filter;
-import org.apache.iotdb.tsfile.read.query.dataset.QueryDataSet;
-import org.apache.iotdb.tsfile.utils.Binary;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import java.io.IOException;
-import java.util.*;
-
-import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_COLUMN;
-import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_COUNT;
-
-public class GroupByLevelDataSet extends QueryDataSet {
-
- private static final Logger logger = LoggerFactory
- .getLogger(GroupByLevelDataSet.class);
-
- private List<RowRecord> records = new ArrayList<>();
- private int index = 0;
-
- protected long queryId;
- private GroupByTimePlan groupByTimePlan;
- private QueryContext context;
-
- private Map<Path, GroupByExecutor> pathExecutors = new HashMap<>();
- private Map<Path, List<Integer>> resultIndexes = new HashMap<>();
-
- public GroupByLevelDataSet(QueryContext context, GroupByTimePlan plan, GroupByEngineDataSet dataSet)
- throws QueryProcessException, StorageEngineException, IOException {
- this.queryId = context.getQueryId();
- this.paths = plan.getPaths();
- this.dataTypes = plan.getDataTypes();
- this.groupByTimePlan = plan;
- this.context = context;
-
- if (logger.isDebugEnabled()) {
- logger.debug("paths " + this.paths + " level:" + plan.getLevel());
- }
-
- Map<Integer, String> pathIndex = new HashMap<>();
- Map<String, Long> finalPaths = FilePathUtils.getPathByLevel(plan.getPaths(), plan.getLevel(), pathIndex);
-
- if (!plan.isByTime()) {
- // does not has time interval,
- // so we could group by time interval [MIN_VALUE, MAX_VALUE] to get the total number
- initGroupByLevel();
- RowRecord record = mergeRecordByPath(getRecordWithoutTimeInterval(), finalPaths, pathIndex);
- if (record != null) {
- records.add(record);
- }
- } else {
- // get all records from GroupByDataSet, then we merge every record
- if (logger.isDebugEnabled()) {
- logger.debug("only group by level, paths:" + groupByTimePlan.getPaths());
- }
- while (dataSet != null && dataSet.hasNextWithoutConstraint()) {
- RowRecord curRecord = mergeRecordByPath(dataSet.nextWithoutConstraint(), finalPaths, pathIndex);
- if (curRecord != null) {
- records.add(curRecord);
- }
- }
- }
-
- this.dataTypes = new ArrayList<>();
- this.paths = new ArrayList<>();
- for (int i = 0; i < finalPaths.size(); i++) {
- this.dataTypes.add(TSDataType.INT64);
- }
- }
-
- @Override
- protected boolean hasNextWithoutConstraint() throws IOException {
- return index < records.size();
- }
-
- @Override
- protected RowRecord nextWithoutConstraint() {
- return records.get(index++);
- }
-
- private void initGroupByLevel()
- throws QueryProcessException, StorageEngineException {
- // get all aggregation results, then we package them to one record
- for (int i = 0; i < paths.size(); i++) {
- Path path = paths.get(i);
- if (!pathExecutors.containsKey(path)) {
- //init GroupByExecutor
- pathExecutors.put(path,
- getGroupByExecutor(path, groupByTimePlan.getAllMeasurementsInDevice(path.getDevice()), dataTypes.get(i), this.context, null, null));
- resultIndexes.put(path, new ArrayList<>());
- } else {
- throw new QueryProcessException("duplicated path found, path:" + path);
- }
- resultIndexes.get(path).add(i);
- AggregateResult aggrResult = AggregateResultFactory
- .getAggrResultByName(groupByTimePlan.getDeduplicatedAggregations().get(i), dataTypes.get(i));
- pathExecutors.get(path).addAggregateResult(aggrResult);
- }
- }
-
- private GroupByExecutor getGroupByExecutor(Path path, Set<String> allSensors, TSDataType dataType,
- QueryContext context, Filter timeFilter, TsFileFilter fileFilter)
- throws StorageEngineException, QueryProcessException {
- return new LocalGroupByExecutor(path, allSensors, dataType, context, timeFilter, fileFilter);
- }
-
- private RowRecord getRecordWithoutTimeInterval()
- throws IOException {
- RowRecord record = new RowRecord(0);
- AggregateResult[] fields = new AggregateResult[paths.size()];
-
- try {
- for (Map.Entry<Path, GroupByExecutor> pathToExecutorEntry : pathExecutors.entrySet()) {
- GroupByExecutor executor = pathToExecutorEntry.getValue();
- List<AggregateResult> aggregations = executor.calcResult(Long.MIN_VALUE, Long.MAX_VALUE);
- for (int i = 0; i < aggregations.size(); i++) {
- int resultIndex = resultIndexes.get(pathToExecutorEntry.getKey()).get(i);
- fields[resultIndex] = aggregations.get(i);
- }
- }
- } catch (QueryProcessException e) {
- logger.error("GroupByWithoutValueFilterDataSet execute has error", e);
- throw new IOException(e.getMessage(), e);
- }
-
- for (AggregateResult res : fields) {
- if (res == null) {
- record.addField(null);
- continue;
- }
- record.addField(res.getResult(), res.getResultDataType());
- }
- return record;
- }
-
- /**
- * merge the raw record by level, for example
- * raw record [timestamp, root.sg1.d1.s0, root.sg1.d1.s1, root.sg1.d2.s2], level=1
- * and newRecord data is [100, 1, 2]
- * return [100, 3]
- * @param newRecord
- * @param finalPaths
- * @param pathIndex
- * @return
- */
- private RowRecord mergeRecordByPath(RowRecord newRecord,
- Map<String, Long> finalPaths,
- Map<Integer, String> pathIndex) {
- if (paths.size() != newRecord.getFields().size()) {
- logger.error("bad record, result size not equal path size");
- return null;
- }
-
- // reset final paths
- for (Map.Entry<String, Long> entry : finalPaths.entrySet()) {
- entry.setValue(0L);
- }
-
- RowRecord tmpRecord = new RowRecord(newRecord.getTimestamp());
-
- for (int i = 0; i < newRecord.getFields().size(); i++) {
- if (newRecord.getFields().get(i) != null) {
- finalPaths.put(pathIndex.get(i),
- finalPaths.get(pathIndex.get(i)) + newRecord.getFields().get(i).getLongV());
- }
- }
-
- for (Map.Entry<String, Long> entry : finalPaths.entrySet()) {
- tmpRecord.addField(Field.getField(entry.getValue(), TSDataType.INT64));
- }
-
- return tmpRecord;
- }
-
-}
diff --git a/server/src/main/java/org/apache/iotdb/db/query/dataset/groupby/GroupByTimeDataSet.java b/server/src/main/java/org/apache/iotdb/db/query/dataset/groupby/GroupByTimeDataSet.java
new file mode 100644
index 0000000..b655709
--- /dev/null
+++ b/server/src/main/java/org/apache/iotdb/db/query/dataset/groupby/GroupByTimeDataSet.java
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.query.dataset.groupby;
+
+import org.apache.iotdb.db.exception.StorageEngineException;
+import org.apache.iotdb.db.exception.query.QueryProcessException;
+import org.apache.iotdb.db.qp.physical.crud.GroupByTimePlan;
+import org.apache.iotdb.db.query.aggregation.AggregateResult;
+import org.apache.iotdb.db.query.context.QueryContext;
+import org.apache.iotdb.db.query.factory.AggregateResultFactory;
+import org.apache.iotdb.db.query.filter.TsFileFilter;
+import org.apache.iotdb.db.utils.FilePathUtils;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.read.common.Field;
+import org.apache.iotdb.tsfile.read.common.Path;
+import org.apache.iotdb.tsfile.read.common.RowRecord;
+import org.apache.iotdb.tsfile.read.filter.basic.Filter;
+import org.apache.iotdb.tsfile.read.query.dataset.QueryDataSet;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.*;
+
+public class GroupByTimeDataSet extends QueryDataSet {
+
+ private static final Logger logger = LoggerFactory
+ .getLogger(GroupByTimeDataSet.class);
+
+ private List<RowRecord> records = new ArrayList<>();
+ private int index = 0;
+
+ protected long queryId;
+ private GroupByTimePlan groupByTimePlan;
+ private QueryContext context;
+
+ public GroupByTimeDataSet(QueryContext context, GroupByTimePlan plan, GroupByEngineDataSet dataSet)
+ throws QueryProcessException, StorageEngineException, IOException {
+ this.queryId = context.getQueryId();
+ this.paths = plan.getDeduplicatedPaths();
+ this.dataTypes = plan.getDeduplicatedDataTypes();
+ this.groupByTimePlan = plan;
+ this.context = context;
+
+ if (logger.isDebugEnabled()) {
+ logger.debug("paths " + this.paths + " level:" + plan.getLevel());
+ }
+
+ Map<Integer, String> pathIndex = new HashMap<>();
+ Map<String, Long> finalPaths = FilePathUtils.getPathByLevel(plan.getPaths(), plan.getLevel(), pathIndex);
+
+ // get all records from GroupByDataSet, then we merge every record
+ if (logger.isDebugEnabled()) {
+ logger.debug("only group by level, paths:" + groupByTimePlan.getPaths());
+ }
+ while (dataSet != null && dataSet.hasNextWithoutConstraint()) {
+ RowRecord curRecord = FilePathUtils.mergeRecordByPath(dataSet.nextWithoutConstraint(), finalPaths, pathIndex);
+ if (curRecord != null) {
+ records.add(curRecord);
+ }
+ }
+
+ this.dataTypes = new ArrayList<>();
+ this.paths = new ArrayList<>();
+ for (int i = 0; i < finalPaths.size(); i++) {
+ this.dataTypes.add(TSDataType.INT64);
+ }
+ }
+
+ @Override
+ protected boolean hasNextWithoutConstraint() throws IOException {
+ return index < records.size();
+ }
+
+ @Override
+ protected RowRecord nextWithoutConstraint() {
+ return records.get(index++);
+ }
+}
diff --git a/server/src/main/java/org/apache/iotdb/db/query/executor/AggregationExecutor.java b/server/src/main/java/org/apache/iotdb/db/query/executor/AggregationExecutor.java
index 04e80f2..e7f51cd 100644
--- a/server/src/main/java/org/apache/iotdb/db/query/executor/AggregationExecutor.java
+++ b/server/src/main/java/org/apache/iotdb/db/query/executor/AggregationExecutor.java
@@ -36,6 +36,7 @@ import org.apache.iotdb.db.query.reader.series.IReaderByTimestamp;
import org.apache.iotdb.db.query.reader.series.SeriesAggregateReader;
import org.apache.iotdb.db.query.reader.series.SeriesReaderByTimestamp;
import org.apache.iotdb.db.query.timegenerator.ServerTimeGenerator;
+import org.apache.iotdb.db.utils.FilePathUtils;
import org.apache.iotdb.db.utils.QueryUtils;
import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
import org.apache.iotdb.tsfile.file.metadata.statistics.Statistics;
@@ -96,7 +97,7 @@ public class AggregationExecutor {
}
}
- return constructDataSet(Arrays.asList(aggregateResultList));
+ return constructDataSet(Arrays.asList(aggregateResultList), aggregationPlan);
}
/**
@@ -273,7 +274,7 @@ public class AggregationExecutor {
aggregateResults.add(result);
}
aggregateWithValueFilter(aggregateResults, timestampGenerator, readersOfSelectedSeries);
- return constructDataSet(aggregateResults);
+ return constructDataSet(aggregateResults, queryPlan);
}
protected TimeGenerator getTimeGenerator(QueryContext context, RawDataQueryPlan queryPlan) throws StorageEngineException {
@@ -318,15 +319,34 @@ public class AggregationExecutor {
*
* @param aggregateResultList aggregate result list
*/
- private QueryDataSet constructDataSet(List<AggregateResult> aggregateResultList) {
+ private QueryDataSet constructDataSet(List<AggregateResult> aggregateResultList, RawDataQueryPlan plan) {
RowRecord record = new RowRecord(0);
for (AggregateResult resultData : aggregateResultList) {
TSDataType dataType = resultData.getResultDataType();
record.addField(resultData.getResult(), dataType);
}
- SingleDataSet dataSet = new SingleDataSet(selectedSeries, dataTypes);
- dataSet.setRecord(record);
+ SingleDataSet dataSet = null;
+ if (((AggregationPlan)plan).getLevel() >= 0) {
+ // current only support count operation
+ Map<Integer, String> pathIndex = new HashMap<>();
+ Map<String, Long> finalPaths = FilePathUtils.getPathByLevel(plan.getDeduplicatedPaths(), ((AggregationPlan)plan).getLevel(), pathIndex);
+
+ RowRecord curRecord = FilePathUtils.mergeRecordByPath(record, finalPaths, pathIndex);
+
+ List<Path> paths = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ for (int i = 0; i < finalPaths.size(); i++) {
+ dataTypes.add(TSDataType.INT64);
+ }
+
+ dataSet = new SingleDataSet(paths, dataTypes);
+ dataSet.setRecord(curRecord);
+ } else {
+ dataSet = new SingleDataSet(selectedSeries, dataTypes);
+ dataSet.setRecord(record);
+ }
+
return dataSet;
}
diff --git a/server/src/main/java/org/apache/iotdb/db/query/executor/IQueryRouter.java b/server/src/main/java/org/apache/iotdb/db/query/executor/IQueryRouter.java
index 9082900..d04a24f 100644
--- a/server/src/main/java/org/apache/iotdb/db/query/executor/IQueryRouter.java
+++ b/server/src/main/java/org/apache/iotdb/db/query/executor/IQueryRouter.java
@@ -58,7 +58,7 @@ public interface IQueryRouter {
/**
* Execute group by fill query
*/
- QueryDataSet groupByFill(GroupByFillTimePlan groupByFillPlan, QueryContext context)
+ QueryDataSet groupByFill(GroupByTimeFillPlan groupByFillPlan, QueryContext context)
throws QueryFilterOptimizationException, StorageEngineException,
QueryProcessException, IOException;
diff --git a/server/src/main/java/org/apache/iotdb/db/query/executor/QueryRouter.java b/server/src/main/java/org/apache/iotdb/db/query/executor/QueryRouter.java
index 4823cc6..4d2070f 100644
--- a/server/src/main/java/org/apache/iotdb/db/query/executor/QueryRouter.java
+++ b/server/src/main/java/org/apache/iotdb/db/query/executor/QueryRouter.java
@@ -23,6 +23,7 @@ import org.apache.iotdb.db.exception.StorageEngineException;
import org.apache.iotdb.db.exception.query.QueryProcessException;
import org.apache.iotdb.db.qp.physical.crud.*;
import org.apache.iotdb.db.query.context.QueryContext;
+import org.apache.iotdb.db.query.dataset.SingleDataSet;
import org.apache.iotdb.db.query.dataset.groupby.*;
import org.apache.iotdb.db.query.executor.fill.IFill;
import org.apache.iotdb.tsfile.exception.filter.QueryFilterOptimizationException;
@@ -88,6 +89,13 @@ public class QueryRouter implements IQueryRouter {
throws QueryFilterOptimizationException, StorageEngineException, QueryProcessException,
IOException {
+ if (logger.isDebugEnabled()) {
+ logger.debug("paths:" + aggregationPlan.getPaths()
+ + " level:" + aggregationPlan.getLevel()
+ + " duplicatePaths:" + aggregationPlan.getDeduplicatedPaths()
+ + " deduplicatePaths:" + aggregationPlan.getDeduplicatedAggregations());
+ }
+
IExpression expression = aggregationPlan.getExpression();
List<Path> deduplicatedPaths = aggregationPlan.getDeduplicatedPaths();
@@ -100,12 +108,16 @@ public class QueryRouter implements IQueryRouter {
AggregationExecutor engineExecutor = getAggregationExecutor(aggregationPlan);
+ QueryDataSet dataSet = null;
+
if (optimizedExpression != null
&& optimizedExpression.getType() != ExpressionType.GLOBAL_TIME) {
- return engineExecutor.executeWithValueFilter(context, aggregationPlan);
+ dataSet = engineExecutor.executeWithValueFilter(context, aggregationPlan);
+ } else {
+ dataSet = engineExecutor.executeWithoutValueFilter(context, aggregationPlan);
}
- return engineExecutor.executeWithoutValueFilter(context, aggregationPlan);
+ return dataSet;
}
protected AggregationExecutor getAggregationExecutor(AggregationPlan aggregationPlan) {
@@ -116,39 +128,42 @@ public class QueryRouter implements IQueryRouter {
public QueryDataSet groupBy(GroupByTimePlan groupByTimePlan, QueryContext context)
throws QueryFilterOptimizationException, StorageEngineException, QueryProcessException, IOException {
- logger.debug("paths:" + groupByTimePlan.getPaths() + " level:" + groupByTimePlan.getLevel() + " byTime:" + groupByTimePlan.isByTime());
+ if (logger.isDebugEnabled()) {
+ logger.debug("paths:" + groupByTimePlan.getPaths() + " level:" + groupByTimePlan.getLevel());
+ }
GroupByEngineDataSet dataSet = null;
- if (groupByTimePlan.isByTime()) {
- long unit = groupByTimePlan.getInterval();
- long slidingStep = groupByTimePlan.getSlidingStep();
- long startTime = groupByTimePlan.getStartTime();
- long endTime = groupByTimePlan.getEndTime();
-
- IExpression expression = groupByTimePlan.getExpression();
- List<Path> selectedSeries = groupByTimePlan.getDeduplicatedPaths();
-
- GlobalTimeExpression timeExpression = new GlobalTimeExpression(
- new GroupByFilter(unit, slidingStep, startTime, endTime));
-
- if (expression == null) {
- expression = timeExpression;
- } else {
- expression = BinaryExpression.and(expression, timeExpression);
- }
-
- // optimize expression to an executable one
- IExpression optimizedExpression = ExpressionOptimizer.getInstance()
- .optimize(expression, selectedSeries);
- groupByTimePlan.setExpression(optimizedExpression);
-
- if (optimizedExpression.getType() == ExpressionType.GLOBAL_TIME) {
- dataSet = getGroupByWithoutValueFilterDataSet(context, groupByTimePlan);
- } else {
- dataSet = getGroupByWithValueFilterDataSet(context, groupByTimePlan);
- }
+ long unit = groupByTimePlan.getInterval();
+ long slidingStep = groupByTimePlan.getSlidingStep();
+ long startTime = groupByTimePlan.getStartTime();
+ long endTime = groupByTimePlan.getEndTime();
+
+ IExpression expression = groupByTimePlan.getExpression();
+ List<Path> selectedSeries = groupByTimePlan.getDeduplicatedPaths();
+
+ GlobalTimeExpression timeExpression = new GlobalTimeExpression(
+ new GroupByFilter(unit, slidingStep, startTime, endTime));
+
+ if (expression == null) {
+ expression = timeExpression;
+ } else {
+ expression = BinaryExpression.and(expression, timeExpression);
+ }
+
+ // optimize expression to an executable one
+ IExpression optimizedExpression = ExpressionOptimizer.getInstance()
+ .optimize(expression, selectedSeries);
+ groupByTimePlan.setExpression(optimizedExpression);
+
+ if (optimizedExpression.getType() == ExpressionType.GLOBAL_TIME) {
+ dataSet = getGroupByWithoutValueFilterDataSet(context, groupByTimePlan);
+ } else {
+ dataSet = getGroupByWithValueFilterDataSet(context, groupByTimePlan);
}
+ // we support group by level for count operation
+ // details at https://issues.apache.org/jira/browse/IOTDB-622
+ // and UserGuide/Operation Manual/DML
if (groupByTimePlan.getLevel() >= 0) {
return groupByLevelWithoutTimeIntervalDataSet(context, groupByTimePlan, dataSet);
}
@@ -165,10 +180,10 @@ public class QueryRouter implements IQueryRouter {
return new GroupByWithValueFilterDataSet(context, plan);
}
- protected GroupByLevelDataSet groupByLevelWithoutTimeIntervalDataSet(QueryContext context, GroupByTimePlan plan,
- GroupByEngineDataSet dataSet)
+ protected GroupByTimeDataSet groupByLevelWithoutTimeIntervalDataSet(QueryContext context, GroupByTimePlan plan,
+ GroupByEngineDataSet dataSet)
throws StorageEngineException, QueryProcessException, IOException {
- return new GroupByLevelDataSet(context, plan, dataSet);
+ return new GroupByTimeDataSet(context, plan, dataSet);
}
@Override
@@ -192,7 +207,7 @@ public class QueryRouter implements IQueryRouter {
}
@Override
- public QueryDataSet groupByFill(GroupByFillTimePlan groupByFillPlan, QueryContext context)
+ public QueryDataSet groupByFill(GroupByTimeFillPlan groupByFillPlan, QueryContext context)
throws QueryFilterOptimizationException, StorageEngineException, QueryProcessException, IOException {
GroupByEngineDataSet groupByEngineDataSet = (GroupByEngineDataSet) groupBy(groupByFillPlan, context);
return new GroupByFillDataSet(groupByFillPlan.getDeduplicatedPaths(), groupByFillPlan.getDeduplicatedDataTypes(),
diff --git a/server/src/main/java/org/apache/iotdb/db/service/TSServiceImpl.java b/server/src/main/java/org/apache/iotdb/db/service/TSServiceImpl.java
index 0acb9de..0d5db6b 100644
--- a/server/src/main/java/org/apache/iotdb/db/service/TSServiceImpl.java
+++ b/server/src/main/java/org/apache/iotdb/db/service/TSServiceImpl.java
@@ -19,7 +19,6 @@
package org.apache.iotdb.db.service;
import static org.apache.iotdb.db.conf.IoTDBConfig.PATH_PATTERN;
-import static org.apache.iotdb.db.qp.logical.Operator.OperatorType.GROUPBYTIME;
import static org.apache.iotdb.db.qp.physical.sys.ShowPlan.ShowContentType.TIMESERIES;
import java.io.IOException;
@@ -54,13 +53,8 @@ import org.apache.iotdb.db.qp.executor.IPlanExecutor;
import org.apache.iotdb.db.qp.executor.PlanExecutor;
import org.apache.iotdb.db.qp.logical.Operator.OperatorType;
import org.apache.iotdb.db.qp.physical.PhysicalPlan;
-import org.apache.iotdb.db.qp.physical.crud.AlignByDevicePlan;
+import org.apache.iotdb.db.qp.physical.crud.*;
import org.apache.iotdb.db.qp.physical.crud.AlignByDevicePlan.MeasurementType;
-import org.apache.iotdb.db.qp.physical.crud.DeletePlan;
-import org.apache.iotdb.db.qp.physical.crud.InsertPlan;
-import org.apache.iotdb.db.qp.physical.crud.InsertTabletPlan;
-import org.apache.iotdb.db.qp.physical.crud.LastQueryPlan;
-import org.apache.iotdb.db.qp.physical.crud.QueryPlan;
import org.apache.iotdb.db.qp.physical.sys.AuthorPlan;
import org.apache.iotdb.db.qp.physical.sys.CreateTimeSeriesPlan;
import org.apache.iotdb.db.qp.physical.sys.DeleteStorageGroupPlan;
@@ -544,7 +538,7 @@ public class TSServiceImpl implements TSIService.Iface, ServerContext {
if (plan.getOperatorType() == OperatorType.FILL) {
throw new QueryProcessException("Fill doesn't support disable align clause.");
}
- if (plan.getOperatorType() == GROUPBYTIME) {
+ if (plan.getOperatorType() == OperatorType.GROUPBYTIME) {
throw new QueryProcessException("Group by doesn't support disable align clause.");
}
}
@@ -693,8 +687,8 @@ public class TSServiceImpl implements TSIService.Iface, ServerContext {
// Last Query should return different respond instead of the static one
// because the query dataset and query id is different although the header of last query is same.
return StaticResps.LAST_RESP.deepCopy();
- } else if (plan.getOperatorType() == GROUPBYTIME && plan.getLevel() >= 0) {
- Map<String, Long> finalPaths = FilePathUtils.getPathByLevel(plan.getPaths(), plan.getLevel(), null);
+ } else if (plan instanceof AggregationPlan && ((AggregationPlan)plan).getLevel() >= 0) {
+ Map<String, Long> finalPaths = FilePathUtils.getPathByLevel(((AggregationPlan)plan).getDeduplicatedPaths(), ((AggregationPlan)plan).getLevel(), null);
for (Map.Entry<String, Long> entry : finalPaths.entrySet()) {
respColumns.add("count(" + entry.getKey() + ")");
columnsTypes.add(TSDataType.INT64.toString());
diff --git a/server/src/main/java/org/apache/iotdb/db/utils/FilePathUtils.java b/server/src/main/java/org/apache/iotdb/db/utils/FilePathUtils.java
index 3c54541..bb4ff27 100644
--- a/server/src/main/java/org/apache/iotdb/db/utils/FilePathUtils.java
+++ b/server/src/main/java/org/apache/iotdb/db/utils/FilePathUtils.java
@@ -24,7 +24,11 @@ import java.util.Map;
import java.util.TreeMap;
import org.apache.iotdb.db.engine.storagegroup.TsFileResource;
+import org.apache.iotdb.db.metadata.MetaUtils;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.read.common.Field;
import org.apache.iotdb.tsfile.read.common.Path;
+import org.apache.iotdb.tsfile.read.common.RowRecord;
public class FilePathUtils {
@@ -65,7 +69,7 @@ public class FilePathUtils {
int i = 0;
for (Path value : rawPaths) {
- String[] tmpPath = value.getFullPath().split("\\.");
+ String[] tmpPath = MetaUtils.getNodeNames(value.getFullPath());
String key;
if (tmpPath.length <= level) {
@@ -90,4 +94,43 @@ public class FilePathUtils {
return finalPaths;
}
+ /**
+ * merge the raw record by level, for example
+ * raw record [timestamp, root.sg1.d1.s0, root.sg1.d1.s1, root.sg1.d2.s2], level=1
+ * and newRecord data is [100, 1, 1, 1]
+ * return [100, 3]
+ *
+ * @param newRecord
+ * @param finalPaths
+ * @param pathIndex
+ * @return
+ */
+ public static RowRecord mergeRecordByPath(RowRecord newRecord,
+ Map<String, Long> finalPaths,
+ Map<Integer, String> pathIndex) {
+ if (newRecord.getFields().size() < finalPaths.size()) {
+ return null;
+ }
+
+ // reset final paths
+ for (Map.Entry<String, Long> entry : finalPaths.entrySet()) {
+ entry.setValue(0L);
+ }
+
+ RowRecord tmpRecord = new RowRecord(newRecord.getTimestamp());
+
+ for (int i = 0; i < newRecord.getFields().size(); i++) {
+ if (newRecord.getFields().get(i) != null) {
+ finalPaths.put(pathIndex.get(i),
+ finalPaths.get(pathIndex.get(i)) + newRecord.getFields().get(i).getLongV());
+ }
+ }
+
+ for (Map.Entry<String, Long> entry : finalPaths.entrySet()) {
+ tmpRecord.addField(Field.getField(entry.getValue(), TSDataType.INT64));
+ }
+
+ return tmpRecord;
+ }
+
}
diff --git a/server/src/test/java/org/apache/iotdb/db/qp/plan/PhysicalPlanTest.java b/server/src/test/java/org/apache/iotdb/db/qp/plan/PhysicalPlanTest.java
index c424292..acd55a1 100644
--- a/server/src/test/java/org/apache/iotdb/db/qp/plan/PhysicalPlanTest.java
+++ b/server/src/test/java/org/apache/iotdb/db/qp/plan/PhysicalPlanTest.java
@@ -239,10 +239,10 @@ public class PhysicalPlanTest {
if (!plan.isQuery()) {
fail();
}
- if (!(plan instanceof GroupByFillTimePlan)) {
+ if (!(plan instanceof GroupByTimeFillPlan)) {
fail();
}
- GroupByFillTimePlan groupByFillPlan = (GroupByFillTimePlan) plan;
+ GroupByTimeFillPlan groupByFillPlan = (GroupByTimeFillPlan) plan;
assertEquals(3L, groupByFillPlan.getInterval());
assertEquals(3L, groupByFillPlan.getSlidingStep());
assertEquals(8L, groupByFillPlan.getStartTime());
@@ -268,10 +268,10 @@ public class PhysicalPlanTest {
if (!plan.isQuery()) {
fail();
}
- if (!(plan instanceof GroupByFillTimePlan)) {
+ if (!(plan instanceof GroupByTimeFillPlan)) {
fail();
}
- GroupByFillTimePlan groupByFillPlan = (GroupByFillTimePlan) plan;
+ GroupByTimeFillPlan groupByFillPlan = (GroupByTimeFillPlan) plan;
assertEquals(3L, groupByFillPlan.getInterval());
assertEquals(3L, groupByFillPlan.getSlidingStep());
assertEquals(8L, groupByFillPlan.getStartTime());
@@ -299,10 +299,10 @@ public class PhysicalPlanTest {
if (!plan.isQuery()) {
fail();
}
- if (!(plan instanceof GroupByFillTimePlan)) {
+ if (!(plan instanceof GroupByTimeFillPlan)) {
fail();
}
- GroupByFillTimePlan groupByFillPlan = (GroupByFillTimePlan) plan;
+ GroupByTimeFillPlan groupByFillPlan = (GroupByTimeFillPlan) plan;
assertEquals(3L, groupByFillPlan.getInterval());
assertEquals(3L, groupByFillPlan.getSlidingStep());
assertEquals(8L, groupByFillPlan.getStartTime());
@@ -383,7 +383,7 @@ public class PhysicalPlanTest {
if (!plan.isQuery()) {
fail();
}
- if (!(plan instanceof GroupByFillPlan)) {
+ if (!(plan instanceof GroupByTimeFillPlan)) {
fail();
}
} catch (Exception e) {
diff --git a/server/src/test/java/org/apache/iotdb/db/query/dataset/GroupByLevelDataSetTest.java b/server/src/test/java/org/apache/iotdb/db/query/dataset/GroupByLevelDataSetTest.java
index e473a5a..3e8cae0 100644
--- a/server/src/test/java/org/apache/iotdb/db/query/dataset/GroupByLevelDataSetTest.java
+++ b/server/src/test/java/org/apache/iotdb/db/query/dataset/GroupByLevelDataSetTest.java
@@ -31,6 +31,7 @@ import org.junit.Before;
import org.junit.Test;
import static org.junit.Assert.*;
+import static org.junit.Assert.assertEquals;
public class GroupByLevelDataSetTest {
private IPlanExecutor queryExecutor = new PlanExecutor();
@@ -42,6 +43,8 @@ public class GroupByLevelDataSetTest {
"CREATE TIMESERIES root.vehicle.d0.s1 WITH DATATYPE=TEXT, ENCODING=PLAIN",
"CREATE TIMESERIES root.test.d0.s0 WITH DATATYPE=INT32, ENCODING=RLE",
"CREATE TIMESERIES root.test.d0.s1 WITH DATATYPE=TEXT, ENCODING=PLAIN",
+ "CREATE TIMESERIES root.test.d0.s3 WITH DATATYPE=TEXT, ENCODING=PLAIN",
+ "CREATE TIMESERIES root.test.d1.\"s3.xy\" WITH DATATYPE=TEXT, ENCODING=PLAIN",
"insert into root.vehicle.d0(timestamp,s0) values(10,100)",
"insert into root.vehicle.d0(timestamp,s0,s1) values(12,101,'102')",
"insert into root.vehicle.d0(timestamp,s1) values(19,'103')",
@@ -74,7 +77,9 @@ public class GroupByLevelDataSetTest {
"insert into root.test.d0(timestamp,s1) values(30,'139')",
"insert into root.test.d0(timestamp,s0) values(1900,1316)",
"insert into root.test.d0(timestamp,s0,s1) values(700,1307,'1038')",
- "insert into root.test.d0(timestamp,s1) values(3000,'1309')"};
+ "insert into root.test.d0(timestamp,s1) values(3000,'1309')",
+ "insert into root.test.d0(timestamp,s3) values(10,'100')",
+ "insert into root.test.d1(timestamp, \"s3.xy\") values(10, 'text')"};
static {
MManager.getInstance().init();
@@ -105,71 +110,47 @@ public class GroupByLevelDataSetTest {
assertTrue(dataSet.hasNext());
assertEquals("0\t12", dataSet.next().toString());
- // level 0
queryPlan = (QueryPlan) processor
- .parseSQLToPhysicalPlan("select count(s1) from root.*.* group by level=0");
+ .parseSQLToPhysicalPlan("select count(s1) from root.test.* group by level=0");
dataSet = queryExecutor.processQuery(queryPlan, EnvironmentUtils.TEST_QUERY_CONTEXT);
assertTrue(dataSet.hasNext());
- assertEquals("0\t22", dataSet.next().toString());
+ assertEquals("0\t12", dataSet.next().toString());
- // level large than series path
queryPlan = (QueryPlan) processor
- .parseSQLToPhysicalPlan("select count(s1) from root.test.* group by level=5");
+ .parseSQLToPhysicalPlan("select count(s1) from root.test.* group by level=6");
dataSet = queryExecutor.processQuery(queryPlan, EnvironmentUtils.TEST_QUERY_CONTEXT);
assertTrue(dataSet.hasNext());
assertEquals("0\t12", dataSet.next().toString());
- // with time interval
+ // multi paths
queryPlan = (QueryPlan) processor
- .parseSQLToPhysicalPlan("select count(s1) from root.test.* group by ([0,20), 1ms), level=1");
+ .parseSQLToPhysicalPlan("select count(s1) from root.test.*,root.vehicle.* group by level=1");
dataSet = queryExecutor.processQuery(queryPlan, EnvironmentUtils.TEST_QUERY_CONTEXT);
assertTrue(dataSet.hasNext());
- assertEquals("0\t0", dataSet.next().toString());
- assertTrue(dataSet.hasNext());
- assertEquals("1\t0", dataSet.next().toString());
- assertTrue(dataSet.hasNext());
- assertEquals("2\t1", dataSet.next().toString());
+ assertEquals("0\t12\t10", dataSet.next().toString());
+ // with multi result
queryPlan = (QueryPlan) processor
- .parseSQLToPhysicalPlan("select count(s1) from root.test.* group by ([0,20), 1ms), level=0");
+ .parseSQLToPhysicalPlan("select count(s1), count(s0) from root.test.* group by level=6");
dataSet = queryExecutor.processQuery(queryPlan, EnvironmentUtils.TEST_QUERY_CONTEXT);
assertTrue(dataSet.hasNext());
- assertEquals("0\t0", dataSet.next().toString());
- assertTrue(dataSet.hasNext());
- assertEquals("1\t0", dataSet.next().toString());
- assertTrue(dataSet.hasNext());
- assertEquals("2\t1", dataSet.next().toString());
+ assertEquals("0\t12\t12", dataSet.next().toString());
+ // with multi result
queryPlan = (QueryPlan) processor
- .parseSQLToPhysicalPlan("select count(s1) from root.test.* group by ([0,20), 1ms), level=6");
+ .parseSQLToPhysicalPlan("select count(s1), count(s3) from root.test.* group by level=2");
dataSet = queryExecutor.processQuery(queryPlan, EnvironmentUtils.TEST_QUERY_CONTEXT);
assertTrue(dataSet.hasNext());
- assertEquals("0\t0", dataSet.next().toString());
- assertTrue(dataSet.hasNext());
- assertEquals("1\t0", dataSet.next().toString());
- assertTrue(dataSet.hasNext());
- assertEquals("2\t1", dataSet.next().toString());
+ assertEquals("0\t13", dataSet.next().toString());
- // multi paths
+ // with double quotation mark
queryPlan = (QueryPlan) processor
- .parseSQLToPhysicalPlan("select count(s1) from root.test.*,root.vehicle.* group by ([0,20), 1ms), level=1");
- dataSet = queryExecutor.processQuery(queryPlan, EnvironmentUtils.TEST_QUERY_CONTEXT);
-
- assertTrue(dataSet.hasNext());
- assertEquals("0\t0\t0", dataSet.next().toString());
- assertTrue(dataSet.hasNext());
- assertEquals("1\t0\t0", dataSet.next().toString());
- assertTrue(dataSet.hasNext());
- assertEquals("2\t1\t0", dataSet.next().toString());
-
- // with sliding step
- queryPlan = (QueryPlan) processor
- .parseSQLToPhysicalPlan("select count(s1) from root.test.* group by ([0,20), 3ms, 10ms), level=6");
+ .parseSQLToPhysicalPlan("select count(\"s3.xy\") from root.test.* group by level=2");
dataSet = queryExecutor.processQuery(queryPlan, EnvironmentUtils.TEST_QUERY_CONTEXT);
assertTrue(dataSet.hasNext());
@@ -178,12 +159,11 @@ public class GroupByLevelDataSetTest {
// not count
try {
queryPlan = (QueryPlan) processor
- .parseSQLToPhysicalPlan("select sum(s0) from root.test.* group by ([0,200), 1ms), level=6");
+ .parseSQLToPhysicalPlan("select sum(s0) from root.test.* group by level=6");
dataSet = queryExecutor.processQuery(queryPlan, EnvironmentUtils.TEST_QUERY_CONTEXT);
fail();
} catch (Exception e) {
- assertEquals("group by level only support count", e.getMessage());
+ assertEquals("group by level only support count now.", e.getMessage());
}
}
-
}
diff --git a/server/src/test/java/org/apache/iotdb/db/query/dataset/GroupByLevelDataSetTest.java b/server/src/test/java/org/apache/iotdb/db/query/dataset/GroupByTimeDataSetTest.java
similarity index 89%
copy from server/src/test/java/org/apache/iotdb/db/query/dataset/GroupByLevelDataSetTest.java
copy to server/src/test/java/org/apache/iotdb/db/query/dataset/GroupByTimeDataSetTest.java
index e473a5a..09b31fd 100644
--- a/server/src/test/java/org/apache/iotdb/db/query/dataset/GroupByLevelDataSetTest.java
+++ b/server/src/test/java/org/apache/iotdb/db/query/dataset/GroupByTimeDataSetTest.java
@@ -32,7 +32,7 @@ import org.junit.Test;
import static org.junit.Assert.*;
-public class GroupByLevelDataSetTest {
+public class GroupByTimeDataSetTest {
private IPlanExecutor queryExecutor = new PlanExecutor();
private Planner processor = new Planner();
private String[] sqls = {
@@ -42,6 +42,7 @@ public class GroupByLevelDataSetTest {
"CREATE TIMESERIES root.vehicle.d0.s1 WITH DATATYPE=TEXT, ENCODING=PLAIN",
"CREATE TIMESERIES root.test.d0.s0 WITH DATATYPE=INT32, ENCODING=RLE",
"CREATE TIMESERIES root.test.d0.s1 WITH DATATYPE=TEXT, ENCODING=PLAIN",
+ "CREATE TIMESERIES root.test.d1.\"s3.xy\" WITH DATATYPE=TEXT, ENCODING=PLAIN",
"insert into root.vehicle.d0(timestamp,s0) values(10,100)",
"insert into root.vehicle.d0(timestamp,s0,s1) values(12,101,'102')",
"insert into root.vehicle.d0(timestamp,s1) values(19,'103')",
@@ -74,13 +75,14 @@ public class GroupByLevelDataSetTest {
"insert into root.test.d0(timestamp,s1) values(30,'139')",
"insert into root.test.d0(timestamp,s0) values(1900,1316)",
"insert into root.test.d0(timestamp,s0,s1) values(700,1307,'1038')",
- "insert into root.test.d0(timestamp,s1) values(3000,'1309')"};
+ "insert into root.test.d0(timestamp,s1) values(3000,'1309')",
+ "insert into root.test.d1(timestamp, \"s3.xy\") values(10, 'text')"};
static {
MManager.getInstance().init();
}
- public GroupByLevelDataSetTest() throws QueryProcessException {
+ public GroupByTimeDataSetTest() throws QueryProcessException {
}
@Before
@@ -97,34 +99,11 @@ public class GroupByLevelDataSetTest {
}
@Test
- public void testGroupByLevel() throws Exception {
- QueryPlan queryPlan = (QueryPlan) processor
- .parseSQLToPhysicalPlan("select count(s1) from root.test.* group by level=1");
- QueryDataSet dataSet = queryExecutor.processQuery(queryPlan, EnvironmentUtils.TEST_QUERY_CONTEXT);
-
- assertTrue(dataSet.hasNext());
- assertEquals("0\t12", dataSet.next().toString());
-
- // level 0
- queryPlan = (QueryPlan) processor
- .parseSQLToPhysicalPlan("select count(s1) from root.*.* group by level=0");
- dataSet = queryExecutor.processQuery(queryPlan, EnvironmentUtils.TEST_QUERY_CONTEXT);
-
- assertTrue(dataSet.hasNext());
- assertEquals("0\t22", dataSet.next().toString());
-
- // level large than series path
- queryPlan = (QueryPlan) processor
- .parseSQLToPhysicalPlan("select count(s1) from root.test.* group by level=5");
- dataSet = queryExecutor.processQuery(queryPlan, EnvironmentUtils.TEST_QUERY_CONTEXT);
-
- assertTrue(dataSet.hasNext());
- assertEquals("0\t12", dataSet.next().toString());
-
+ public void testGroupByTimeAndLevel() throws Exception {
// with time interval
- queryPlan = (QueryPlan) processor
+ QueryPlan queryPlan = (QueryPlan) processor
.parseSQLToPhysicalPlan("select count(s1) from root.test.* group by ([0,20), 1ms), level=1");
- dataSet = queryExecutor.processQuery(queryPlan, EnvironmentUtils.TEST_QUERY_CONTEXT);
+ QueryDataSet dataSet = queryExecutor.processQuery(queryPlan, EnvironmentUtils.TEST_QUERY_CONTEXT);
assertTrue(dataSet.hasNext());
assertEquals("0\t0", dataSet.next().toString());
@@ -175,6 +154,24 @@ public class GroupByLevelDataSetTest {
assertTrue(dataSet.hasNext());
assertEquals("0\t1", dataSet.next().toString());
+ // with multi result
+ queryPlan = (QueryPlan) processor
+ .parseSQLToPhysicalPlan("select count(s1), count(s1) from root.test.* group by ([0,20), 3ms, 10ms), level=6");
+ dataSet = queryExecutor.processQuery(queryPlan, EnvironmentUtils.TEST_QUERY_CONTEXT);
+
+ assertTrue(dataSet.hasNext());
+ assertEquals("0\t1", dataSet.next().toString());
+
+ // with double quotation mark
+ queryPlan = (QueryPlan) processor
+ .parseSQLToPhysicalPlan("select count(\"s3.xy\") from root.test.* group by ([0,20), 3ms, 10ms), level=2");
+ dataSet = queryExecutor.processQuery(queryPlan, EnvironmentUtils.TEST_QUERY_CONTEXT);
+
+ assertTrue(dataSet.hasNext());
+ assertEquals("0\t0", dataSet.next().toString());
+ assertTrue(dataSet.hasNext());
+ assertEquals("10\t1", dataSet.next().toString());
+
// not count
try {
queryPlan = (QueryPlan) processor
@@ -182,7 +179,7 @@ public class GroupByLevelDataSetTest {
dataSet = queryExecutor.processQuery(queryPlan, EnvironmentUtils.TEST_QUERY_CONTEXT);
fail();
} catch (Exception e) {
- assertEquals("group by level only support count", e.getMessage());
+ assertEquals("group by level only support count now.", e.getMessage());
}
}