[GitHub] [iceberg] idealspark commented on issue #4802: Set file sizing properties for iceberg tables?

[GitBox <gi...@apache.org>]

idealspark commented on issue #4802:
URL: https://github.com/apache/iceberg/issues/4802#issuecomment-1167080004

   I encountered the same error in version 0.12.1，it fixed in version 0.13.2。
   
   in version 0.12.1，this code has bug
   `
    # BaseTaskWriter
    private boolean shouldRollToNewFile() {
         return !format.equals(FileFormat.ORC) &&
             currentRows % ROWS_DIVISOR == 0 && 
             //check file should roll
             length(currentWriter) >= targetFileSize;  
       }
   `
   
   
   `
    //ParquetWriter,  shouldRollToNewFile code here
    @Override
     public long length() {
       try {
         if (closed) {
           return writer.getPos();
         } else {
           //this is a bug
           return writer.getPos() + (writeStore.isColumnFlushNeeded() ? writeStore.getBufferedSize() : 0);
         }
       } catch (IOException e) {
         throw new RuntimeIOException(e, "Failed to get file length");
       }
     }
   `


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org