[GitHub] [iceberg] kbendick commented on a change in pull request #2617: Refine error message on most incremental scans on metadata tables

[GitBox <gi...@apache.org>]

kbendick commented on a change in pull request #2617:
URL: https://github.com/apache/iceberg/pull/2617#discussion_r637799928



##########
File path: core/src/main/java/org/apache/iceberg/StaticTableScan.java
##########
@@ -25,16 +25,33 @@
 
 class StaticTableScan extends BaseTableScan {
   private final Function<StaticTableScan, DataTask> buildTask;
+  // Metadata table name that the buildTask that this StaticTableScan will return data for.
+  private final String scannedTableName;
 
-  StaticTableScan(TableOperations ops, Table table, Schema schema, Function<StaticTableScan, DataTask> buildTask) {
+  StaticTableScan(TableOperations ops, Table table, Schema schema,  String scannedTableName,
+                  Function<StaticTableScan, DataTask> buildTask) {
     super(ops, table, schema);
     this.buildTask = buildTask;
+    this.scannedTableName = scannedTableName;
   }
 
-  private StaticTableScan(TableOperations ops, Table table, Schema schema,
+  private StaticTableScan(TableOperations ops, Table table, Schema schema,  String scannedTableName,
                           Function<StaticTableScan, DataTask> buildTask, TableScanContext context) {
     super(ops, table, schema, context);
     this.buildTask = buildTask;
+    this.scannedTableName = scannedTableName;
+  }
+
+  @Override
+  public TableScan appendsBetween(long fromSnapshotId, long toSnapshotId) {
+    throw new UnsupportedOperationException(
+        String.format("Incremental scan is not supported for metadata table %s", scannedTableName));
+  }
+
+  @Override
+  public TableScan appendsAfter(long fromSnapshotId) {
+    throw new UnsupportedOperationException(
+        String.format("Incremental scan is not supported for metadata table %s", scannedTableName));

Review comment:
       I think so. Same thing for `EntriesTableScan`.
   
   Currently, `FilesTableScan` and `EntriesTableScan` inherit directly from `BaseTableScan`. However, they could easily be made to inherit from `StaticTableScan` and then just override the methods that they need.
   
   I wanted to keep this PR small so that it would be easy to digest, but I think that if we refactor, we could easily make `StaticTableScan` into the base class for all metadata table scans other than the aggregate metadata table scans (the ones that derive from `BaseAllMetadataTableScan`) - essentially making it into `BaseMetadataTableScan` to go along with `BaseAllMetadataTableScan`.
   
   Currently, `FilesTableScan` and `EntriesTableScan` inherit directly from `BaseTableScan`. However, they could easily be made to inherit from `StaticTableScan` and then just override the methods that they need. This way, the overrides like `useSnapshot` and `asOfTime` which are currently handled in a few places could all be handled in one place.
   
   Then, if we chose to implement that functionality for a particular metadata table type, we could just override in the actual implementing scan class.
   
   I can update this PR so that `FilesTableScan` and `EntiresTableScan` inherit from `StaticTableScan`, or I can add a similar update to `FilesTableScan` (and possibly `EntriesTableScan`) or handle them in a follow up PR.
   
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org