[GitHub] [incubator-iceberg] vgankidi commented on a change in pull request #254: Truncate stats from Parquet files

[GitBox <gi...@apache.org>]

vgankidi commented on a change in pull request #254: Truncate stats from Parquet files
URL: https://github.com/apache/incubator-iceberg/pull/254#discussion_r300812596
 
 

 ##########
 File path: parquet/src/main/java/org/apache/iceberg/parquet/ParquetUtil.java
 ##########
 @@ -47,21 +48,33 @@
 import org.apache.parquet.schema.MessageType;
 
 import static org.apache.iceberg.parquet.ParquetConversions.fromParquetPrimitive;
+import static org.apache.iceberg.util.BinaryUtil.truncateBinaryMax;
+import static org.apache.iceberg.util.BinaryUtil.truncateBinaryMin;
+import static org.apache.iceberg.util.UnicodeUtil.truncateStringMax;
+import static org.apache.iceberg.util.UnicodeUtil.truncateStringMin;
 
 public class ParquetUtil {
   // not meant to be instantiated
   private ParquetUtil() {
   }
 
   public static Metrics fileMetrics(InputFile file) {
+    return fileMetrics(file, TableProperties.WRITE_METADATA_TRUNCATE_BYTES_DEFAULT);
+  }
+
+  public static Metrics fileMetrics(InputFile file, int statsTruncateLength) {
     try (ParquetFileReader reader = ParquetFileReader.open(ParquetIO.file(file))) {
-      return footerMetrics(reader.getFooter());
+      return footerMetrics(reader.getFooter(), statsTruncateLength);
     } catch (IOException e) {
       throw new RuntimeIOException(e, "Failed to read footer of file: %s", file);
     }
   }
 
   public static Metrics footerMetrics(ParquetMetadata metadata) {
+    return footerMetrics(metadata, TableProperties.WRITE_METADATA_TRUNCATE_BYTES_DEFAULT);
 
 Review comment:
   Yes, would be good to get rid of this. I updated the PR with these changes. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org