You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@parquet.apache.org by "Fokko (via GitHub)" <gi...@apache.org> on 2026/05/06 21:22:41 UTC
[PR] GH-3549: Leverage JDK11 `readNBytes` [parquet-java]
Fokko opened a new pull request, #3550:
URL: https://github.com/apache/parquet-java/pull/3550
Introduced in JDK11 and is more efficient.
Closes #3549
<!--
Thanks for opening a pull request!
If you're new to Parquet-Java, information on how to contribute can be found here: https://parquet.apache.org/docs/contribution-guidelines/contributing
Please open a GitHub issue for this pull request: https://github.com/apache/parquet-java/issues/new/choose
and format pull request title as below:
GH-${GITHUB_ISSUE_ID}: ${SUMMARY}
or simply use the title below if it is a minor issue:
MINOR: ${SUMMARY}
-->
### Rationale for this change
### What changes are included in this PR?
### Are these changes tested?
### Are there any user-facing changes?
<!-- Please uncomment the line below and replace ${GITHUB_ISSUE_ID} with the actual Github issue id. -->
<!-- Closes #${GITHUB_ISSUE_ID} -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] GH-3549: Leverage JDK11 `readNBytes` [parquet-java]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on PR #3550:
URL: https://github.com/apache/parquet-java/pull/3550#issuecomment-4414766521
There seems to be an issue with `NonBlockedDecompressorStream` reads, which renders this PR more-or-less useless :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] GH-3549: Leverage JDK11 `readNBytes` [parquet-java]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on code in PR #3550:
URL: https://github.com/apache/parquet-java/pull/3550#discussion_r3213845157
##########
parquet-common/src/main/java/org/apache/parquet/bytes/BytesInput.java:
##########
@@ -395,8 +404,11 @@ void writeInto(ByteBuffer buffer) {
public byte[] toByteArray() throws IOException {
LOG.debug("read all {} bytes", byteCount);
- byte[] buf = new byte[byteCount];
- new DataInputStream(in).readFully(buf);
+ byte[] buf = in.readNBytes(byteCount);
Review Comment:
I think it is nice to avoid the wrapper `DataInputStream` here. The biggest win here is in `writeAllTo` where we don't allocate all the bytes directly into memory, but respecting `byteCount`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] GH-3549: Leverage JDK11 `readNBytes` [parquet-java]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko closed pull request #3550: GH-3549: Leverage JDK11 `readNBytes`
URL: https://github.com/apache/parquet-java/pull/3550
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] GH-3549: Leverage JDK11 `readNBytes` [parquet-java]
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on code in PR #3550:
URL: https://github.com/apache/parquet-java/pull/3550#discussion_r3202032030
##########
parquet-common/src/main/java/org/apache/parquet/bytes/BytesInput.java:
##########
@@ -395,8 +404,11 @@ void writeInto(ByteBuffer buffer) {
public byte[] toByteArray() throws IOException {
LOG.debug("read all {} bytes", byteCount);
- byte[] buf = new byte[byteCount];
- new DataInputStream(in).readFully(buf);
+ byte[] buf = in.readNBytes(byteCount);
Review Comment:
hadn't noticed this before. Doesn't look like anything in the JDK has actually subclassed it ... that is there's no AFAIK there's no faster native code version. That's probably because java.nio is the way to go there
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org