You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2022/10/25 17:30:29 UTC
[GitHub] [iotdb] yschengzi opened a new pull request, #7725: [IOTDB-4681] speed up mpp load
yschengzi opened a new pull request, #7725:
URL: https://github.com/apache/iotdb/pull/7725
problem:
load is slower than write in 3C3D env, should be faster in theory.
solution:
- [x] only read TsFile once
- [x] only serialize TsFile once
- [x] send data immediately after parse TsFile
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yschengzi commented on a diff in pull request #7725: [IOTDB-4681] speed up mpp load
Posted by GitBox <gi...@apache.org>.
yschengzi commented on code in PR #7725:
URL: https://github.com/apache/iotdb/pull/7725#discussion_r1015109341
##########
server/src/main/java/org/apache/iotdb/db/mpp/plan/scheduler/load/LoadTsFileDispatcherImpl.java:
##########
@@ -82,26 +82,46 @@ public void setUuid(String uuid) {
this.uuid = uuid;
}
+ // @Override
+ // public Future<FragInstanceDispatchResult> dispatch(List<FragmentInstance> instances) {
+ // return executor.submit(
+ // () -> {
+ // for (FragmentInstance instance : instances) {
+ // try (SetThreadName threadName =
+ // new SetThreadName(
+ // LoadTsFileScheduler.class.getName() + instance.getId().getFullId())) {
+ // dispatchOneInstance(instance);
+ // } catch (FragmentInstanceDispatchException e) {
+ // return new FragInstanceDispatchResult(e.getFailureStatus());
+ // } catch (Throwable t) {
+ // logger.error("cannot dispatch FI for load operation", t);
+ // return new FragInstanceDispatchResult(
+ // RpcUtils.getStatus(
+ // TSStatusCode.INTERNAL_SERVER_ERROR, "Unexpected errors: " +
+ // t.getMessage()));
+ // }
+ // }
Review Comment:
done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] HTHou commented on a diff in pull request #7725: [IOTDB-4681] speed up mpp load
Posted by GitBox <gi...@apache.org>.
HTHou commented on code in PR #7725:
URL: https://github.com/apache/iotdb/pull/7725#discussion_r1012707407
##########
server/src/main/java/org/apache/iotdb/db/mpp/plan/scheduler/load/LoadTsFileDispatcherImpl.java:
##########
@@ -82,26 +82,46 @@ public void setUuid(String uuid) {
this.uuid = uuid;
}
+ // @Override
+ // public Future<FragInstanceDispatchResult> dispatch(List<FragmentInstance> instances) {
+ // return executor.submit(
+ // () -> {
+ // for (FragmentInstance instance : instances) {
+ // try (SetThreadName threadName =
+ // new SetThreadName(
+ // LoadTsFileScheduler.class.getName() + instance.getId().getFullId())) {
+ // dispatchOneInstance(instance);
+ // } catch (FragmentInstanceDispatchException e) {
+ // return new FragInstanceDispatchResult(e.getFailureStatus());
+ // } catch (Throwable t) {
+ // logger.error("cannot dispatch FI for load operation", t);
+ // return new FragInstanceDispatchResult(
+ // RpcUtils.getStatus(
+ // TSStatusCode.INTERNAL_SERVER_ERROR, "Unexpected errors: " +
+ // t.getMessage()));
+ // }
+ // }
Review Comment:
Remove this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] HTHou merged pull request #7725: [IOTDB-4681] speed up mpp load
Posted by GitBox <gi...@apache.org>.
HTHou merged PR #7725:
URL: https://github.com/apache/iotdb/pull/7725
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org