[GitHub] [druid] zachjsh commented on a diff in pull request #14307: Account for data format and compression in MSQ auto taskAssignment

["zachjsh (via GitHub)" <gi...@apache.org>]

zachjsh commented on code in PR #14307:
URL: https://github.com/apache/druid/pull/14307#discussion_r1213583203


##########
processing/src/main/java/org/apache/druid/data/input/impl/FlatTextInputFormat.java:
##########
@@ -139,6 +140,16 @@ public boolean equals(Object o)
            Objects.equals(delimiter, that.delimiter);
   }
 
+  @Override
+  public long getWeightedSize(String path, long size)
+  {
+    CompressionUtils.Format compressionFormat = CompressionUtils.Format.fromFileName(path);
+    if (CompressionUtils.Format.GZ == compressionFormat) {
+      return size * CompressionUtils.COMPRESSED_TEXT_WEIGHT_FACTOR;

Review Comment:
   its not, gzip is just the most common, and one that we have data for. We can add others as need arises.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org