[GitHub] [iceberg] rdblue commented on a change in pull request #3292: Spark: Compact Medium Size Files (#460)

[GitBox <gi...@apache.org>]

rdblue commented on a change in pull request #3292:
URL: https://github.com/apache/iceberg/pull/3292#discussion_r780817433



##########
File path: core/src/main/java/org/apache/iceberg/BaseFileScanTask.java
##########
@@ -224,5 +217,51 @@ public Expression residual() {
     public Iterable<FileScanTask> split(long splitSize) {
       throw new UnsupportedOperationException("Cannot split a task which is already split");
     }
+
+    public boolean isAdjacent(SplitScanTask other) {
+      return (other != null) &&
+          (this.file().equals(other.file())) &&
+          (this.offset + this.len == other.offset);
+    }
+  }
+
+  static FileScanTask[] combineSimilarTasks(List<FileScanTask> tasks) {
+    if (tasks.isEmpty()) {
+      return new FileScanTask[0];
+    }
+
+    List<FileScanTask> combinedScans = Lists.newArrayList();
+    SplitScanTask lastSplit = null;
+
+    for (FileScanTask fileScanTask : tasks) {
+      if (!(fileScanTask instanceof SplitScanTask)) {
+        // We do not know how to combine anything but SplitScanTasks

Review comment:
       Nit: personal pronouns don't usually add much to comments or documentation. It's unclear who "we" is and you can usually be more direct. Here I think it is sufficient to say "only SplitScanTasks produced by split are supported".




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org