[GitHub] [iceberg] szehon-ho commented on a change in pull request #2608: Core : Repair manifests

[GitBox <gi...@apache.org>]

szehon-ho commented on a change in pull request #2608:
URL: https://github.com/apache/iceberg/pull/2608#discussion_r636016830



##########
File path: spark/src/main/java/org/apache/iceberg/spark/actions/BaseRewriteManifestsSparkAction.java
##########
@@ -86,26 +91,29 @@
   private static final String USE_CACHING = "use-caching";
   private static final boolean USE_CACHING_DEFAULT = true;
 
-  private final Encoder<ManifestFile> manifestEncoder;
+  private final Encoder<RepairManifestHelper.RepairedManifestFile> manifestEncoder;
   private final Table table;
   private final int formatVersion;
   private final FileIO fileIO;
+  private final SerializableConfiguration hadoopConf;
   private final long targetManifestSizeBytes;
 
   private PartitionSpec spec = null;
   private Predicate<ManifestFile> predicate = manifest -> true;
   private String stagingLocation = null;
+  private RepairMode mode = RepairMode.NONE;
 
   public BaseRewriteManifestsSparkAction(SparkSession spark, Table table) {
     super(spark);
-    this.manifestEncoder = Encoders.javaSerialization(ManifestFile.class);
+    this.manifestEncoder = Encoders.javaSerialization(RepairManifestHelper.RepairedManifestFile.class);

Review comment:
       OK yes I was trying to think something useful to return to user.  But maybe its not very useful as its not specific as per discussion with @flyrain  earlier in the review, and returning all patched manifest-entries is a bit overkill.  I'm ok for just return the list of repaired ManifestFile 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org