[GitHub] [parquet-mr] sunchao commented on a change in pull request #925: PARQUET-2078: Failed to read parquet file after writing with the same …

[GitBox <gi...@apache.org>]

sunchao commented on a change in pull request #925:
URL: https://github.com/apache/parquet-mr/pull/925#discussion_r700416277



##########
File path: parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java
##########
@@ -201,9 +201,23 @@ public FileMetaData toParquetMetadata(int currentVersion, ParquetMetadata parque
     List<BlockMetaData> blocks = parquetMetadata.getBlocks();
     List<RowGroup> rowGroups = new ArrayList<RowGroup>();
     long numRows = 0;
+    long preBlockStartPos = 0;
+    long preBlockCompressedSize = 0;
     for (BlockMetaData block : blocks) {
       numRows += block.getRowCount();
+      long blockStartPos = block.getStartingPos();
+      // first block
+      if (blockStartPos == 4) {
+        preBlockStartPos = 0;
+        preBlockCompressedSize = 0;
+      }
+      if (preBlockStartPos != 0) {
+        assert blockStartPos >= preBlockStartPos + preBlockCompressedSize;

Review comment:
       I'm not sure if we should use `assert` here since it's not always turned on in production. Perhaps use `Preconditions.checkState`?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org