[GitHub] [pinot] walterddr commented on a diff in pull request #10791: [multistage] Refactor StageMetadata from pinot-query-planner to pinot-query-runner module

["walterddr (via GitHub)" <gi...@apache.org>]

walterddr commented on code in PR #10791:
URL: https://github.com/apache/pinot/pull/10791#discussion_r1201465552


##########
pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/plan/serde/QueryPlanSerDeUtils.java:
##########
@@ -51,12 +53,23 @@ public static DistributedStagePlan deserialize(Worker.StagePlan stagePlan) {
     return distributedStagePlan;
   }
 
-  public static Worker.StagePlan serialize(DistributedStagePlan distributedStagePlan) {
+  public static List<DistributedStagePlan> deserialize(Worker.QueryRequest request) {
+    List<DistributedStagePlan> distributedStagePlans = new ArrayList<>();
+    for (Worker.StagePlan stagePlan : request.getStagePlanList()) {
+      distributedStagePlans.add(deserialize(stagePlan));
+    }
+    return distributedStagePlans;
+  }
+
+  public static Worker.StagePlan serialize(DispatchableSubPlan dispatchableSubPlan, int stageId,
+      VirtualServerAddress serverAddress) {

Review Comment:
   @ankitsultana i remember you mentioned to me that dispatching 1-worker per request is extremely heavy in GRPC payload gen. 
   does this approach help? 
   e.g. 
   1. the dispatchable plan with the planFragment(stage) + all the worker metadata will be **serialized** together in 1 GRPC request (1 request for all worker)
   2. the request will be **deserialized** into a list of distributed stage plans (e.g. 1 plan per worker)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org