You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@parquet.apache.org by "joyCurry30 (via GitHub)" <gi...@apache.org> on 2024/07/17 15:31:51 UTC
[PR] GH-2959: Optimize the test case of parquet rewriter. [parquet-java]
joyCurry30 opened a new pull request, #2960:
URL: https://github.com/apache/parquet-java/pull/2960
<!--
Thanks for opening a pull request!
If you're new to Parquet-Java, information on how to contribute can be found here: https://parquet.apache.org/docs/contribution-guidelines/contributing
Please open a GitHub issue for this pull request: https://github.com/apache/parquet-java/issues/new/choose
and format pull request title as below:
GH-${GITHUB_ISSUE_ID}: ${SUMMARY}
or simply use the title below if it is a minor issue:
MINOR: ${SUMMARY}
-->
### Rationale for this change
Refer to #2959
### What changes are included in this PR?
Optimize the test case of parquet rewriter.
### Are these changes tested?
Yes.
### Are there any user-facing changes?
No,
<!-- Please uncomment the line below and replace ${GITHUB_ISSUE_ID} with the actual Github issue id. -->
<!-- Closes #${GITHUB_ISSUE_ID} -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] GH-2959: Optimize the test case of parquet rewriter. [parquet-java]
Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac merged PR #2960:
URL: https://github.com/apache/parquet-java/pull/2960
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] GH-2959: Optimize the test case of parquet rewriter. [parquet-java]
Posted by "joyCurry30 (via GitHub)" <gi...@apache.org>.
joyCurry30 commented on PR #2960:
URL: https://github.com/apache/parquet-java/pull/2960#issuecomment-2236857469
Before:
<img width="851" alt="image" src="https://github.com/user-attachments/assets/ad44598c-eb3b-4050-97a9-fc8f8ed424d2">
After:
<img width="838" alt="image" src="https://github.com/user-attachments/assets/78ffc0c6-a381-46d0-b666-4b8fd7140333">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] GH-2959: Optimize the test case of parquet rewriter. [parquet-java]
Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on code in PR #2960:
URL: https://github.com/apache/parquet-java/pull/2960#discussion_r1682766359
##########
parquet-hadoop/src/test/java/org/apache/parquet/hadoop/rewrite/ParquetRewriterTest.java:
##########
@@ -342,7 +377,12 @@ public void testPruneEncryptTranslateCodecSingleFile() throws Exception {
@Test
public void testPruneEncryptTranslateCodecTwoFiles() throws Exception {
- testMultipleInputFilesSetup();
+ if (!inputFiles.contains(gzipEncryptionTestFileWithoutBloomFilterColumn)) {
Review Comment:
nit: perhaps we can wrap them in `ensureUncompressedFile()` and `ensureGzipFile()`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] GH-2959: Optimize the test case of parquet rewriter. [parquet-java]
Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on PR #2960:
URL: https://github.com/apache/parquet-java/pull/2960#issuecomment-2236389233
Thanks for the improvement! Could you please also show the elapsed time of the improved test before and after?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org