You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@parquet.apache.org by "annimesh2809 (via GitHub)" <gi...@apache.org> on 2025/11/13 17:28:32 UTC

[PR] GH-3356: Add buffers allocated by vectored IO for releasing [parquet-java]

annimesh2809 opened a new pull request, #3357:
URL: https://github.com/apache/parquet-java/pull/3357

   <!--
   Thanks for opening a pull request!
   
   If you're new to Parquet-Java, information on how to contribute can be found here: https://parquet.apache.org/docs/contribution-guidelines/contributing
   
   Please open a GitHub issue for this pull request: https://github.com/apache/parquet-java/issues/new/choose
   and format pull request title as below:
   
       GH-${GITHUB_ISSUE_ID}: ${SUMMARY}
   
   or simply use the title below if it is a minor issue:
   
       MINOR: ${SUMMARY}
   
   -->
   
   ### Rationale for this change
   
   A couple of test cases of TestParquetReader suite started failing with errors like:
   ```
   - testRangeFiltering[0] *** FAILED ***
     org.apache.parquet.bytes.TrackingByteBufferAllocator$LeakedByteBufferException: 24 ByteBuffer object(s) is/are remained unreleased after closing this allocator.
     at org.apache.parquet.bytes.TrackingByteBufferAllocator.close(TrackingByteBufferAllocator.java:161)
     at org.apache.parquet.hadoop.TestParquetReader.closeAllocator(TestParquetReader.java:175)
     at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
     at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.base/java.lang.reflect.Method.invoke(Method.java:569)
   ```
   with hadoop 3.4.2 and parquet-mr 1.16.0
   The leaks are happening when reading using vectored IO because we never pass the buffers to the releaser.
   
   ### What changes are included in this PR?
   Added a custom allocator that adds all allocated buffers to the releaser. This way all classes using the allocator for allocating buffers (like `ChecksumFileSystem`) will also be cleaned up.
   
   ### Are these changes tested?
   TestParquetReader suite passes with these changes.
   
   ### Are there any user-facing changes?
   No
   
   <!-- Please uncomment the line below and replace ${GITHUB_ISSUE_ID} with the actual Github issue id. -->
   <!-- Closes #${GITHUB_ISSUE_ID} -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3356: Add buffers allocated by vectored IO for releasing [parquet-java]

Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on code in PR #3357:
URL: https://github.com/apache/parquet-java/pull/3357#discussion_r2525619486


##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileReader.java:
##########
@@ -1354,7 +1382,7 @@ private void readVectored(List<ConsecutivePartList> allParts, ChunkListBuilder b
     }
     LOG.debug("Reading {} bytes of data with vectored IO in {} ranges", totalSize, ranges.size());
     // Request a vectored read;
-    f.readVectored(ranges, options.getAllocator());
+    f.readVectored(ranges, new ReleasingAllocator(options.getAllocator(), builder.releaser));

Review Comment:
   I'm not sure if this is the right direction. Is it better to make it a contract for ByteBufferAllocator implementations to take this responsibility?
   
   WDYT? @gszadovszky 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3356: Add buffers allocated by vectored IO for releasing [parquet-java]

Posted by "gszadovszky (via GitHub)" <gi...@apache.org>.
gszadovszky commented on code in PR #3357:
URL: https://github.com/apache/parquet-java/pull/3357#discussion_r2536826920


##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileReader.java:
##########
@@ -1354,7 +1382,7 @@ private void readVectored(List<ConsecutivePartList> allParts, ChunkListBuilder b
     }
     LOG.debug("Reading {} bytes of data with vectored IO in {} ranges", totalSize, ranges.size());
     // Request a vectored read;
-    f.readVectored(ranges, options.getAllocator());
+    f.readVectored(ranges, new ReleasingAllocator(options.getAllocator(), builder.releaser));

Review Comment:
   Thanks for the context, @annimesh2809.
   Why do you need to track the allocated buffers to be released later instead of simply giving the `allocate` and `release` methods of the `ByteBufferAllocator` instance to the related Hadoop API via the implementations of `SeekableInputStream.readVectored`? I assume the Hadoop code would release the allocated buffers as soon as they are not needed anymore.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3356: Add buffers allocated by vectored IO for releasing [parquet-java]

Posted by "annimesh2809 (via GitHub)" <gi...@apache.org>.
annimesh2809 commented on code in PR #3357:
URL: https://github.com/apache/parquet-java/pull/3357#discussion_r2533562864


##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileReader.java:
##########
@@ -1354,7 +1382,7 @@ private void readVectored(List<ConsecutivePartList> allParts, ChunkListBuilder b
     }
     LOG.debug("Reading {} bytes of data with vectored IO in {} ranges", totalSize, ranges.size());
     // Request a vectored read;
-    f.readVectored(ranges, options.getAllocator());
+    f.readVectored(ranges, new ReleasingAllocator(options.getAllocator(), builder.releaser));

Review Comment:
   When I try to build parquet-mr with hadoop 3.4.2 without any additional changes, I see `testRangeFiltering` test case (and some others) of `TestParquetReader` suite fail. The `TrackingByteBufferAllocator` reveals that the **unreleased allocation** happens in:
   ```
     Cause: org.apache.parquet.bytes.TrackingByteBufferAllocator$ByteBufferAllocationStacktraceException: Allocation stacktrace of the first ByteBuffer:
     at org.apache.parquet.bytes.TrackingByteBufferAllocator$ByteBufferAllocationStacktraceException.create(TrackingByteBufferAllocator.java:96)
     at org.apache.parquet.bytes.TrackingByteBufferAllocator.allocate(TrackingByteBufferAllocator.java:136)
     at org.apache.hadoop.fs.impl.VectorIOBufferPool.getBuffer(VectorIOBufferPool.java:65)
     at org.apache.hadoop.fs.RawLocalFileSystem$AsyncHandler.initiateRead(RawLocalFileSystem.java:400)
     at org.apache.hadoop.fs.RawLocalFileSystem$AsyncHandler.access$000(RawLocalFileSystem.java:360)
     at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileInputStream.readVectored(RawLocalFileSystem.java:345)
     at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileInputStream.readVectored(RawLocalFileSystem.java:324)
     at org.apache.hadoop.fs.BufferedFSInputStream.readVectored(BufferedFSInputStream.java:183)
     at org.apache.hadoop.fs.FSDataInputStream.readVectored(FSDataInputStream.java:308)
     at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.readVectored(ChecksumFileSystem.java:474)
     at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.readVectored(ChecksumFileSystem.java:463)
   ```
   The root cause here seems to be that ChecksumFileSystem (coming from hadoop) starts supporting `readVectored`: https://github.com/apache/hadoop/blob/branch-3.4.2/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L460-L513.
   `ChecksumFileSystem.readVectored` internally does more allocations like:
   ```
   sums.readVectored(checksumRanges, allocate, release);
   datas.readVectored(dataRanges, allocate, release);
   ```
   which are not marked for release by `ByteBufferReleaser`.
   
   Also with vectored reads, it is not sufficient to mark the buffers returned by the allocator for release, as they are sliced internally and the returned buffer object is different even though the underlying memory remains the same.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3356: Add buffers allocated by vectored IO for releasing [parquet-java]

Posted by "gszadovszky (via GitHub)" <gi...@apache.org>.
gszadovszky commented on code in PR #3357:
URL: https://github.com/apache/parquet-java/pull/3357#discussion_r2525997787


##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileReader.java:
##########
@@ -1354,7 +1382,7 @@ private void readVectored(List<ConsecutivePartList> allParts, ChunkListBuilder b
     }
     LOG.debug("Reading {} bytes of data with vectored IO in {} ranges", totalSize, ranges.size());
     // Request a vectored read;
-    f.readVectored(ranges, options.getAllocator());
+    f.readVectored(ranges, new ReleasingAllocator(options.getAllocator(), builder.releaser));

Review Comment:
   I'm not sure I get the concept of this PR. `ByteBufferAllocator` actually has the contract of releasing the `ByteBuffer` allocated by it. The only thing we need to do is to invoke this method at the right time when the related buffer is not needed anymore.
   
   The `ByteBufferReleaser` concept came into the scope only to easily postpone the release invocation to the time we really can release the related `ByteBuffer`s. (By using `BytesInput` we may pass the related buffers around and it is not always clear when to release them.)
   
   @annimesh2809, I would suggest you to implement a unit test to reproduce the issue first. You may use `TrackingByteBufferAllocator` to fail if any allocated buffer is not released during the execution. You may find examples of its usage among the unit tests. If you find the issue, you'll need to ensure that the related allocated buffers are get back to their allocator to release them. You may use the existing patterns we already have or invent new ones if necessary.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org


Re: [PR] GH-3356: Add buffers allocated by vectored IO for releasing [parquet-java]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #3357:
URL: https://github.com/apache/parquet-java/pull/3357#issuecomment-4292660504

   This pull request has been automatically marked as stale because it has had no activity for at least 2 months. If you are still working on this change or plan to move it forward, please leave a comment or push a new commit so we know to keep it open. Otherwise, this PR will be closed automatically in about one month. Thank you for your contribution to Apache Parquet!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org