You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2020/03/03 13:32:49 UTC
[GitHub] [incubator-iotdb] liutaohua opened a new pull request #877: add
groupByExecutor main method doc
liutaohua opened a new pull request #877: add groupByExecutor main method doc
URL: https://github.com/apache/incubator-iotdb/pull/877
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-iotdb] qiaojialin merged pull request #877: add
groupByExecutor main method doc
Posted by GitBox <gi...@apache.org>.
qiaojialin merged pull request #877: add groupByExecutor main method doc
URL: https://github.com/apache/incubator-iotdb/pull/877
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-iotdb] qiaojialin commented on a change in pull request
#877: add groupByExecutor main method doc
Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #877: add groupByExecutor main method doc
URL: https://github.com/apache/incubator-iotdb/pull/877#discussion_r387075600
##########
File path: docs/Documentation-CHN/SystemDesign/5-DataQuery/5-GroupByQuery.md
##########
@@ -111,17 +114,87 @@ private void calcFromBatch(BatchData batchData) throws IOException;
//使用 Page 或 Chunk 的 Statistics 信息直接计算结果
private void calcFromStatistics(Statistics statistics) throws QueryProcessException;
-//从 reader 中读取数据，并计算
-private List<Pair<AggregateResult, Integer>> calcResult() throws IOException, QueryProcessException;
-
//清空所有计算结果
private void resetAggregateResults();
//遍历并计算 page 中的数据
private boolean readAndCalcFromPage() throws IOException, QueryProcessException;
```
-`GroupByExecutor` 中因为相同 `path` 的不同聚合函数使用的数据是相同的，所以 `calcFromBatch` 方法就是完成遍历所有聚合函数对 `BatchData` 的计算:
+
+`GroupByExecutor` 中因为相同 `path` 的不同聚合函数使用的数据是相同的，所以在入口方法 `calcResult` 中负责读取该 `Path` 的所有数据,
+取出来的数据再调用 `calcFromBatch` 方法完成遍历所有聚合函数对 `BatchData` 的计算。
+
+`calcResult` 方法的主要逻辑：
+
+```
+//把上次遗留的数据先做计算，如果能直接获得结果就结束计算
+if (calcFromCacheData()) {
+ return results;
+}
+
+//因为一个chunk是包含多个page的，那么必须先使用完当前chunk的page，再打开下一个chunk
+if (readAndCalcFromPage()) {
+ return results;
+}
+
+//遗留的数据如果计算完了就要打开新的chunk继续计算
+while (reader.hasNextChunk()) {
+ Statistics chunkStatistics = reader.currentChunkStatistics();
+ // 判断能否使用pageStatistics，并执行计算
+ ....
+ // 跳过当前chunk
+ reader.skipCurrentChunk();
+ // 如果已经获取到了所有结果就结束计算
+ if (isEndCalc()) {
+ return true;
+ }
+ continue;
+ }
+ //如果不能使用 chunkStatistics 就需要使用page数据计算
+ if (readAndCalcFromPage()) {
+ return results;
+ }
+}
+```
+
+`readAndCalcFromPage` 方法是从当前打开的chunk中获取page的数据，并计算聚合结果，主要逻辑：
Review comment:
加一个返回值的介绍
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-iotdb] liutaohua commented on a change in pull request
#877: add groupByExecutor main method doc
Posted by GitBox <gi...@apache.org>.
liutaohua commented on a change in pull request #877: add groupByExecutor main method doc
URL: https://github.com/apache/incubator-iotdb/pull/877#discussion_r387083853
##########
File path: docs/Documentation-CHN/SystemDesign/5-DataQuery/5-GroupByQuery.md
##########
@@ -111,17 +114,87 @@ private void calcFromBatch(BatchData batchData) throws IOException;
//使用 Page 或 Chunk 的 Statistics 信息直接计算结果
private void calcFromStatistics(Statistics statistics) throws QueryProcessException;
-//从 reader 中读取数据，并计算
-private List<Pair<AggregateResult, Integer>> calcResult() throws IOException, QueryProcessException;
-
//清空所有计算结果
private void resetAggregateResults();
//遍历并计算 page 中的数据
private boolean readAndCalcFromPage() throws IOException, QueryProcessException;
```
-`GroupByExecutor` 中因为相同 `path` 的不同聚合函数使用的数据是相同的，所以 `calcFromBatch` 方法就是完成遍历所有聚合函数对 `BatchData` 的计算:
+
+`GroupByExecutor` 中因为相同 `path` 的不同聚合函数使用的数据是相同的，所以在入口方法 `calcResult` 中负责读取该 `Path` 的所有数据,
+取出来的数据再调用 `calcFromBatch` 方法完成遍历所有聚合函数对 `BatchData` 的计算。
+
+`calcResult` 方法的主要逻辑：
+
+```
+//把上次遗留的数据先做计算，如果能直接获得结果就结束计算
+if (calcFromCacheData()) {
+ return results;
+}
+
+//因为一个chunk是包含多个page的，那么必须先使用完当前chunk的page，再打开下一个chunk
+if (readAndCalcFromPage()) {
+ return results;
+}
+
+//遗留的数据如果计算完了就要打开新的chunk继续计算
+while (reader.hasNextChunk()) {
+ Statistics chunkStatistics = reader.currentChunkStatistics();
+ // 判断能否使用pageStatistics，并执行计算
+ ....
+ // 跳过当前chunk
+ reader.skipCurrentChunk();
+ // 如果已经获取到了所有结果就结束计算
+ if (isEndCalc()) {
+ return true;
+ }
+ continue;
+ }
+ //如果不能使用 chunkStatistics 就需要使用page数据计算
+ if (readAndCalcFromPage()) {
+ return results;
+ }
+}
+```
+
+`readAndCalcFromPage` 方法是从当前打开的chunk中获取page的数据，并计算聚合结果，主要逻辑：
Review comment:
yes, thanks for you advice
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-iotdb] liutaohua commented on a change in pull request
#877: add groupByExecutor main method doc
Posted by GitBox <gi...@apache.org>.
liutaohua commented on a change in pull request #877: add groupByExecutor main method doc
URL: https://github.com/apache/incubator-iotdb/pull/877#discussion_r387026026
##########
File path: docs/Documentation-CHN/SystemDesign/5-DataQuery/5-GroupByQuery.md
##########
@@ -111,17 +115,83 @@ private void calcFromBatch(BatchData batchData) throws IOException;
//使用 Page 或 Chunk 的 Statistics 信息直接计算结果
private void calcFromStatistics(Statistics statistics) throws QueryProcessException;
-//从 reader 中读取数据，并计算
-private List<Pair<AggregateResult, Integer>> calcResult() throws IOException, QueryProcessException;
-
//清空所有计算结果
private void resetAggregateResults();
//遍历并计算 page 中的数据
private boolean readAndCalcFromPage() throws IOException, QueryProcessException;
```
-`GroupByExecutor` 中因为相同 `path` 的不同聚合函数使用的数据是相同的，所以 `calcFromBatch` 方法就是完成遍历所有聚合函数对 `BatchData` 的计算:
+`GroupByExecutor` 中因为相同 `path` 的不同聚合函数使用的数据是相同的，所以在入口方法 `calcResult` 中负责读取该 `Path` 的所有数据,
+取出来的数据再调用 `calcFromBatch` 方法完成遍历所有聚合函数对 `BatchData` 的计算。
+
+`calcResult` 方法的主要逻辑：
+```
Review comment:
fixed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-iotdb] qiaojialin commented on a change in pull request
#877: add groupByExecutor main method doc
Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #877: add groupByExecutor main method doc
URL: https://github.com/apache/incubator-iotdb/pull/877#discussion_r387024224
##########
File path: docs/Documentation-CHN/SystemDesign/5-DataQuery/5-GroupByQuery.md
##########
@@ -111,17 +115,83 @@ private void calcFromBatch(BatchData batchData) throws IOException;
//使用 Page 或 Chunk 的 Statistics 信息直接计算结果
private void calcFromStatistics(Statistics statistics) throws QueryProcessException;
-//从 reader 中读取数据，并计算
-private List<Pair<AggregateResult, Integer>> calcResult() throws IOException, QueryProcessException;
-
//清空所有计算结果
private void resetAggregateResults();
//遍历并计算 page 中的数据
private boolean readAndCalcFromPage() throws IOException, QueryProcessException;
```
-`GroupByExecutor` 中因为相同 `path` 的不同聚合函数使用的数据是相同的，所以 `calcFromBatch` 方法就是完成遍历所有聚合函数对 `BatchData` 的计算:
+`GroupByExecutor` 中因为相同 `path` 的不同聚合函数使用的数据是相同的，所以在入口方法 `calcResult` 中负责读取该 `Path` 的所有数据,
+取出来的数据再调用 `calcFromBatch` 方法完成遍历所有聚合函数对 `BatchData` 的计算。
+
+`calcResult` 方法的主要逻辑：
+```
Review comment:
add a blank line before ```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services