You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2021/10/12 02:13:55 UTC
[GitHub] [iotdb] LebronAl commented on a change in pull request #4073: [ISSUE-4072] Parallel insert records in Session
LebronAl commented on a change in pull request #4073:
URL: https://github.com/apache/iotdb/pull/4073#discussion_r726706159
##########
File path: session/src/main/java/org/apache/iotdb/session/Session.java
##########
@@ -2096,6 +2010,71 @@ private TSCreateSchemaTemplateReq getTSCreateSchemaTemplateReq(
return request;
}
+ /**
+ * @param recordsGroup connection to record map
+ * @param insertConsumer insert function
+ * @param <T>
+ * <ul>
+ * <li>{@link TSInsertRecordsReq}
+ * <li>{@link TSInsertStringRecordsReq}
+ * <li>{@link TSInsertTabletsReq}
+ * </ul>
+ *
+ * @throws IoTDBConnectionException
+ * @throws StatementExecutionException
+ */
+ private <T> void insertRecordsGroup(
Review comment:
In our dialect, `record` is a row writing interface and `tablet` is a column writing interface. It is not recommended to use `record` or `tablet` as part of the function name. How about call it `insertByGroups` or `insertByGroupsParallel`
##########
File path: session/src/main/java/org/apache/iotdb/session/Session.java
##########
@@ -2096,6 +2010,71 @@ private TSCreateSchemaTemplateReq getTSCreateSchemaTemplateReq(
return request;
}
+ /**
+ * @param recordsGroup connection to record map
+ * @param insertConsumer insert function
+ * @param <T>
+ * <ul>
+ * <li>{@link TSInsertRecordsReq}
+ * <li>{@link TSInsertStringRecordsReq}
+ * <li>{@link TSInsertTabletsReq}
+ * </ul>
+ *
+ * @throws IoTDBConnectionException
+ * @throws StatementExecutionException
+ */
+ private <T> void insertRecordsGroup(
+ Map<SessionConnection, T> recordsGroup, InsertConsumer<T> insertConsumer)
+ throws IoTDBConnectionException, StatementExecutionException {
+ List<CompletableFuture<Void>> completableFutures =
+ recordsGroup.entrySet().stream()
+ .map(
+ entry -> {
Review comment:
Since this is an IO intensive loads, it is recommended not to use the default thread pool `ForkJoinPool.commonPool()`, which is suitable for CPU intensive loads. We can use our own thread pool (coreSize set to 2x cpus)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org