You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/05/18 21:32:09 UTC

[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #4395: Spark: Add custom metric for number of file splits read by a SparkScan

RussellSpitzer commented on code in PR #4395:
URL: https://github.com/apache/iceberg/pull/4395#discussion_r876400294


##########
spark/v3.2/spark/src/main/java/org/apache/iceberg/spark/source/SparkScan.java:
##########
@@ -202,21 +202,13 @@ public boolean supportColumnarReads(InputPartition partition) {
     }
   }
 
-  static long numFilesToScan(CombinedScanTask scanTask) {
-    long fileCount = 0L;
-    for (FileScanTask file : scanTask.files()) {
-      fileCount += 1L;
-    }
-    return fileCount;
-  }
-
   private static class RowReader extends RowDataReader implements PartitionReader<InternalRow> {
     private long numSplits;
 
     RowReader(ReadTask task) {
       super(task.task, task.table(), task.expectedSchema(), task.isCaseSensitive());
-      numSplits = numFilesToScan(task.task);
-      LOG.debug("reading {} file splits for table {} using RowReader", numSplits, task.table().name());
+      numSplits = (long) task.task.files().size();

Review Comment:
   I don't think it really makes a difference, the implicit cast should be just as good so I would probably just remove the cast



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org