[GitHub] [iceberg] aokolnychyi commented on a diff in pull request #8157: Core: Optimize DeleteFileIndex

["aokolnychyi (via GitHub)" <gi...@apache.org>]

aokolnychyi commented on code in PR #8157:
URL: https://github.com/apache/iceberg/pull/8157#discussion_r1275442437


##########
core/src/main/java/org/apache/iceberg/DeleteFileIndex.java:
##########
@@ -569,4 +570,153 @@ private Iterable<CloseableIterable<ManifestEntry<DeleteFile>>> deleteManifestRea
                   .liveEntries());
     }
   }
+
+  // a group of indexed delete files sorted by the sequence number they apply to
+  private static class DeleteFileGroup {
+    private final long[] seqs;
+    private final IndexedDeleteFile[] files;
+
+    DeleteFileGroup(IndexedDeleteFile[] files) {
+      this.seqs = Arrays.stream(files).mapToLong(IndexedDeleteFile::applySequenceNumber).toArray();
+      this.files = files;
+    }
+
+    DeleteFileGroup(long[] seqs, IndexedDeleteFile[] files) {
+      this.seqs = seqs;
+      this.files = files;
+    }
+
+    public Stream<IndexedDeleteFile> limit(long seq) {
+      return limitBySequenceNumber(seq, seqs, files);
+    }
+
+    public Iterable<DeleteFile> referencedDeleteFiles() {
+      return Arrays.stream(files).map(IndexedDeleteFile::wrapped).collect(Collectors.toList());
+    }
+  }
+
+  // a delete file wrapper that caches the converted boundaries for faster boundary checks
+  // this class is not meant to be exposed beyond the delete file index
+  private static class IndexedDeleteFile {

Review Comment:
   Here is how I found the issue:
   <img width="2560" alt="image" src="https://github.com/apache/iceberg/assets/6235869/737d089a-7152-47fa-99a1-c319d375388e">
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org