Re: [PR] SuperSorter: direct merging, increased parallelism. (druid)

["clintropolis (via GitHub)" <gi...@apache.org>]

clintropolis commented on code in PR #16775:
URL: https://github.com/apache/druid/pull/16775#discussion_r1705984098


##########
processing/src/main/java/org/apache/druid/frame/processor/SuperSorter.java:
##########
@@ -761,12 +865,42 @@ private long getTotalMergersInLevel(final int level)
     } else if (level >= totalMergingLevels) {
       throw new ISE("Invalid level %d", level);
     } else if (level == totalMergingLevels - 1) {
-      return outputPartitionsFuture.isDone() ? getOutputPartitions().size() : UNKNOWN_TOTAL;
+      if (outputPartitionsFuture.isDone()) {
+        return totalInputFrames == 0 ? 0 : getOutputPartitions().size();
+      } else {
+        return UNKNOWN_TOTAL;
+      }
+    } else if (level > 0 && level == totalMergingLevels - 2) {

Review Comment:
   nit: i wonder if it might be a bit easier to follow if we moved some of these penultimate and ultimate level checks that do math stuff into named functions to be like `level == penultimateLevel(totalMergingLevels)` or `isPenultimateLevel(level)` or something since i see them repeated in places, or i guess could also maybe precompute?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org