You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "stevenzwu (via GitHub)" <gi...@apache.org> on 2023/06/01 03:24:03 UTC

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #7685: Spark: Positional deletes creates partitioned path on unpartitioned tables

stevenzwu commented on code in PR #7685:
URL: https://github.com/apache/iceberg/pull/7685#discussion_r1212522237


##########
core/src/main/java/org/apache/iceberg/io/RollingFileWriter.java:
##########
@@ -109,7 +109,7 @@ protected void openCurrentWriter() {
   }
 
   private EncryptedOutputFile newFile() {
-    if (partition == null) {
+    if (spec.isUnpartitioned() || partition == null) {

Review Comment:
   this check needs to be added to other writers, like `FanoutWriter`, `ClusteredWriter`. They all have the same pattern. 
   ```
       return partition == null
           ? fileFactory.newOutputFile()
           : fileFactory.newOutputFile(spec, partition);
     }
   ```
   
   Originally I was wondering if this check should be done in the method below of `OutputFileFactory` class. But I see it is not appropriate, as the Java doc clearly indicates that this method should be used for partitioned writes. Hence caller needs to separate out unpartitioned vs partitioned writes.
   ```
     /** Generates an {@link EncryptedOutputFile} for partitioned writes in a given spec. */
     public EncryptedOutputFile newOutputFile(PartitionSpec spec, StructLike partition) 
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org