You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2022/10/20 11:21:44 UTC
[GitHub] [iotdb] Cpaulyz opened a new pull request, #7673: [IOTDB-4675] Use createMultiTimeseries to optimize MLogLoader
Cpaulyz opened a new pull request, #7673:
URL: https://github.com/apache/iotdb/pull/7673
## Description
When replaying CreateTimeseriesPlan, use buffer for caching and use createMultiTimeseries interface for execution when necessary, thus saving PRC overhead.
There are three cases to flush buffer:threshold
1. The number of time series in the buffer reaches the threshold. (default = 1000)
1. CreateMultiTimeseriesReq's predicted memory size reaches the threshold. (default = 10MB)
2. Other physical plans that depend on timeseries prepared to be replayed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] qiaojialin commented on a diff in pull request #7673: [To rel/0.13][IOTDB-4675] Use createMultiTimeseries to optimize MLogLoader
Posted by GitBox <gi...@apache.org>.
qiaojialin commented on code in PR #7673:
URL: https://github.com/apache/iotdb/pull/7673#discussion_r1002351926
##########
server/src/main/java/org/apache/iotdb/db/tools/mlog/MLogLoader.java:
##########
@@ -252,19 +272,12 @@ public static void parseFileAndLoad() throws Exception {
fillTagsAndOffset(createTimeSeriesPlan);
}
}
- session.createTimeseries(
- createTimeSeriesPlan.getPath().getFullPath(),
- createTimeSeriesPlan.getDataType(),
- createTimeSeriesPlan.getEncoding(),
- createTimeSeriesPlan.getCompressor(),
- createTimeSeriesPlan.getProps(),
- createTimeSeriesPlan.getTags(),
- createTimeSeriesPlan.getAttributes(),
- createTimeSeriesPlan.getAlias());
+ addBatchAndCheck(createTimeSeriesPlan);
break;
case CREATE_ALIGNED_TIMESERIES:
CreateAlignedTimeSeriesPlan createAlignedTimeSeriesPlan =
(CreateAlignedTimeSeriesPlan) plan;
+ logger.info("create aligned");
Review Comment:
debug
##########
server/src/main/java/org/apache/iotdb/db/tools/mlog/MLogLoader.java:
##########
@@ -381,12 +399,13 @@ public static void parseFileAndLoad() throws Exception {
deactivateTemplatePlan.getPrefixPath().getFullPath());
break;
default:
- logger.warn("Skip load plan {}", plan);
+ // logger.warn("Skip load plan {}", plan);
Review Comment:
remove
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] Cpaulyz commented on a diff in pull request #7673: [To rel/0.13][IOTDB-4675] Use createMultiTimeseries to optimize MLogLoader
Posted by GitBox <gi...@apache.org>.
Cpaulyz commented on code in PR #7673:
URL: https://github.com/apache/iotdb/pull/7673#discussion_r1002668901
##########
server/src/main/java/org/apache/iotdb/db/tools/mlog/MLogLoader.java:
##########
@@ -381,12 +399,13 @@ public static void parseFileAndLoad() throws Exception {
deactivateTemplatePlan.getPrefixPath().getFullPath());
break;
default:
- logger.warn("Skip load plan {}", plan);
+ // logger.warn("Skip load plan {}", plan);
Review Comment:
done
##########
server/src/main/java/org/apache/iotdb/db/tools/mlog/MLogLoader.java:
##########
@@ -252,19 +272,12 @@ public static void parseFileAndLoad() throws Exception {
fillTagsAndOffset(createTimeSeriesPlan);
}
}
- session.createTimeseries(
- createTimeSeriesPlan.getPath().getFullPath(),
- createTimeSeriesPlan.getDataType(),
- createTimeSeriesPlan.getEncoding(),
- createTimeSeriesPlan.getCompressor(),
- createTimeSeriesPlan.getProps(),
- createTimeSeriesPlan.getTags(),
- createTimeSeriesPlan.getAttributes(),
- createTimeSeriesPlan.getAlias());
+ addBatchAndCheck(createTimeSeriesPlan);
break;
case CREATE_ALIGNED_TIMESERIES:
CreateAlignedTimeSeriesPlan createAlignedTimeSeriesPlan =
(CreateAlignedTimeSeriesPlan) plan;
+ logger.info("create aligned");
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] qiaojialin merged pull request #7673: [To rel/0.13][IOTDB-4675] Use createMultiTimeseries to optimize MLogLoader
Posted by GitBox <gi...@apache.org>.
qiaojialin merged PR #7673:
URL: https://github.com/apache/iotdb/pull/7673
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org