[GitHub] [pinot] mqliang commented on a change in pull request #7724: Split thread cpu time into three metrics

[GitBox <gi...@apache.org>]

mqliang commented on a change in pull request #7724:
URL: https://github.com/apache/pinot/pull/7724#discussion_r750626812



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/InstanceResponseOperator.java
##########
@@ -44,26 +45,61 @@ public InstanceResponseOperator(BaseCombineOperator combinedOperator, List<Index
     _fetchContextSize = fetchContexts.size();
   }
 
+  /*
+   * Derive systemActivitiesCpuTimeNs from totalWallClockTimeNs, multipleThreadCpuTimeNs, singleThreadCpuTimeNs,
+   * and numServerThreads.
+   *
+   * For example, let's divide query processing into 4 phases:
+   * - phase 1: single thread preparing. Time used: T1
+   * - phase 2: N threads processing segments in parallel, each thread use time T2
+   * - phase 3: system activities (GC/OS paging). Time used: T3
+   * - phase 4: single thread merging intermediate results blocks. Time used: T4
+   *
+   * Then we have following equations:
+   * - singleThreadCpuTimeNs = T1 + T4
+   * - multipleThreadCpuTimeNs = T2 * N
+   * - totalWallClockTimeNs = T1 + T2 + T3 + T4 = singleThreadCpuTimeNs + T2 + T3
+   * - totalThreadCpuTimeNsWithoutSystemActivities = T1 + T2 * N + T4 = singleThreadCpuTimeNs + T2 * N
+   * - systemActivitiesCpuTimeNs = T3 = totalWallClockTimeNs - singleThreadCpuTimeNs - T2
+   */
+  public static long calSystemActivitiesCpuTimeNs(long totalWallClockTimeNs, long multipleThreadCpuTimeNs,
+      long singleThreadCpuTimeNs, int numServerThreads) {
+    double perMultipleThreadCpuTimeNs = multipleThreadCpuTimeNs * 1.0 / numServerThreads;
+    double systemActivitiesCpuTimeNs = (totalWallClockTimeNs - singleThreadCpuTimeNs - perMultipleThreadCpuTimeNs);
+    return Math.round(systemActivitiesCpuTimeNs);
+  }
+
   @Override
   protected InstanceResponseBlock getNextBlock() {
     if (ThreadTimer.isThreadCpuTimeMeasurementEnabled()) {
+      ThreadTimer mainThreadTimer = new ThreadTimer();
+      mainThreadTimer.start();
+
       long startWallClockTimeNs = System.nanoTime();
       IntermediateResultsBlock intermediateResultsBlock = getCombinedResults();
       InstanceResponseBlock instanceResponseBlock = new InstanceResponseBlock(intermediateResultsBlock);
       long totalWallClockTimeNs = System.nanoTime() - startWallClockTimeNs;
 
+      long mainThreadCpuTimeNs = mainThreadTimer.stopAndGetThreadTimeNs();
       /*
        * If/when the threadCpuTime based instrumentation is done for other parts of execution (planning, pruning etc),
        * we will have to change the wallClockTime computation accordingly. Right now everything under
        * InstanceResponseOperator is the one that is instrumented with threadCpuTime.
        */
       long multipleThreadCpuTimeNs = intermediateResultsBlock.getExecutionThreadCpuTimeNs();
       int numServerThreads = intermediateResultsBlock.getNumServerThreads();
-      long totalThreadCpuTimeNs =
-          calTotalThreadCpuTimeNs(totalWallClockTimeNs, multipleThreadCpuTimeNs, numServerThreads);
+      long singleThreadCpuTimeNs = mainThreadCpuTimeNs - multipleThreadCpuTimeNs / numServerThreads;

Review comment:
       ```
   long singleThreadCpuTimeNs = mainThreadCpuTimeNs;
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org