[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #7636: Support Aggregate push down for incremental scan

["RussellSpitzer (via GitHub)" <gi...@apache.org>]

RussellSpitzer commented on code in PR #7636:
URL: https://github.com/apache/iceberg/pull/7636#discussion_r1198001598


##########
spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java:
##########
@@ -221,15 +222,26 @@ public boolean pushAggregation(Aggregation aggregation) {
       return false;
     }
 
-    TableScan scan = table.newScan().includeColumnStats();
-    Snapshot snapshot = readSnapshot();
-    if (snapshot == null) {
-      LOG.info("Skipping aggregate pushdown: table snapshot is null");
-      return false;
+    org.apache.iceberg.Scan scan;

Review Comment:
   minor style thought?
   
   Could we do something like
   ```java
   Scan scan = null;
   if (readConf.startSnapshotId() == null) {
         TableScan tableScan = table.newScan()
         Snapshot snapshot = readSnapshot();
         if (snapshot == null) {
           LOG.info("Skipping aggregate pushdown: table snapshot is null");
           return false;
         }
         tableScan = tableScan.useSnapshot(snapshot.snapshotId());
         scan = tableScan
    } else {
         IncrementalAppendScan incrementalScan = table.newIncrementalAppendScan();
         incrementalScan = incrementalScan.fromSnapshotExclusive(readConf.startSnapshotId());
         Long endSnapshotId = readConf.endSnapshotId();
         if (endSnapshotId != null) {
           incrementalScan = incrementalScan.toSnapshot(endSnapshotId);
         }
         scan = incrementalScan;
     }
     scan = scan.filter(filterExpression()).includeColumnStats();
   ```
   
   I could also see extracting that branching part into another function. My main suggestion is basically to structure it more like
   
   ```
   if (tableScan) {
     TableScanStuff
     }
   else () {
      IncrementalStuff
   }
   Common Stuff
   ```
   
   So we can remove all the casting and keep all the common code in one place just incase we need to add things in the future



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org