You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by GitBox <gi...@apache.org> on 2022/08/08 03:01:22 UTC
[GitHub] [parquet-mr] zhongyujiang opened a new pull request, #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
zhongyujiang opened a new pull request, #982:
URL: https://github.com/apache/parquet-mr/pull/982
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Parquet Jira](https://issues.apache.org/jira/browse/PARQUET-2160) issues and references them in the PR title. For example, "PARQUET-1234: My Parquet PR"
- https://issues.apache.org/jira/browse/PARQUET-2160
- In case you are adding a dependency, check if the license complies with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for this extremely good reason:
### Commits
- [ ] My commits all reference Jira issues in their subject lines. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes how to use it.
- All the public functions and the classes in the PR contain Javadoc that explain what it does
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] zhongyujiang commented on a diff in pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by GitBox <gi...@apache.org>.
zhongyujiang commented on code in PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#discussion_r962724424
##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java:
##########
@@ -109,7 +110,17 @@ public BytesInput decompress(BytesInput bytes, int uncompressedSize) throws IOEx
decompressor.reset();
}
InputStream is = codec.createInputStream(bytes.toInputStream(), decompressor);
- decompressed = BytesInput.from(is, uncompressedSize);
+
+ // We need to explicitly close the ZstdDecompressorStream here to release the resources it holds to avoid
+ // off-heap memory fragmentation issue, see https://issues.apache.org/jira/browse/PARQUET-2160.
+ // This change will load the decompressor stream into heap a little earlier, since the problem it solves
+ // only happens in the ZSTD codec, so this modification is only made for ZSTD streams.
+ if (codec instanceof ZstandardCodec) {
+ decompressed = BytesInput.copy(BytesInput.from(is, uncompressedSize));
Review Comment:
Hi @shangxinli , very sorry about the big delay, I was a little busy last week. The benchmark result and detailed data has been posted in the PR describe block, also cc @sunchao.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] zhongyujiang commented on a diff in pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by GitBox <gi...@apache.org>.
zhongyujiang commented on code in PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#discussion_r945428783
##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java:
##########
@@ -109,7 +110,12 @@ public BytesInput decompress(BytesInput bytes, int uncompressedSize) throws IOEx
decompressor.reset();
}
InputStream is = codec.createInputStream(bytes.toInputStream(), decompressor);
- decompressed = BytesInput.from(is, uncompressedSize);
+ if (codec instanceof ZstandardCodec) {
Review Comment:
Added comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] camper42 commented on pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by GitBox <gi...@apache.org>.
camper42 commented on PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#issuecomment-1378181577
same problem with @alexeykudinkin
currently we replace paruqet jar with patched one in our image, waiting for release
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] alexeykudinkin commented on pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#issuecomment-1376498280
@gszadovszky @ggershinsky @shangxinli
Folks, do we have an approximate timeline for the next patch release that will be including this patch?
This is a severe problem that does affect our ability to use Parquet w/ Zstd and i'm aware of at least of handful of similar occasions and issues occurring to others.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] shangxinli commented on pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by GitBox <gi...@apache.org>.
shangxinli commented on PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#issuecomment-1237287264
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] shangxinli commented on pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by GitBox <gi...@apache.org>.
shangxinli commented on PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#issuecomment-1377598327
Thanks @alexeykudinkin for the explanation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] zhongyujiang commented on a diff in pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by GitBox <gi...@apache.org>.
zhongyujiang commented on code in PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#discussion_r939789492
##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java:
##########
@@ -109,7 +110,12 @@ public BytesInput decompress(BytesInput bytes, int uncompressedSize) throws IOEx
decompressor.reset();
}
InputStream is = codec.createInputStream(bytes.toInputStream(), decompressor);
- decompressed = BytesInput.from(is, uncompressedSize);
+ if (codec instanceof ZstandardCodec) {
Review Comment:
Maybe we can consider closing the decompressed stream after it has been read:
https://github.com/apache/parquet-mr/blob/0819356a9dafd2ca07c5eab68e2bffeddc3bd3d9/parquet-common/src/main/java/org/apache/parquet/bytes/BytesInput.java#L283-L288
But I'm not sure if there is a situation where the decompressed stream is read more than once.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] shangxinli commented on pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by GitBox <gi...@apache.org>.
shangxinli commented on PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#issuecomment-1376750703
@alexeykudinkin We might release a new patch in the next 2 or 3 months.
Can you elaborate why "this is a severe problem that does affect our ability to use Parquet w/ Zstd"? I understand it is an issue but we have been running Parquet w/ Zstd for 2+ years in production.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] wangzzu commented on pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by "wangzzu (via GitHub)" <gi...@apache.org>.
wangzzu commented on PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#issuecomment-1612400508
@zhongyujiang
We also encountered the same problem, here I have another question, why this problem only occurs in the parquet/zstd read
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] shangxinli commented on a diff in pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by GitBox <gi...@apache.org>.
shangxinli commented on code in PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#discussion_r950883464
##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java:
##########
@@ -109,7 +110,17 @@ public BytesInput decompress(BytesInput bytes, int uncompressedSize) throws IOEx
decompressor.reset();
}
InputStream is = codec.createInputStream(bytes.toInputStream(), decompressor);
- decompressed = BytesInput.from(is, uncompressedSize);
+
+ // We need to explicitly close the ZstdDecompressorStream here to release the resources it holds to avoid
+ // off-heap memory fragmentation issue, see https://issues.apache.org/jira/browse/PARQUET-2160.
+ // This change will load the decompressor stream into heap a little earlier, since the problem it solves
+ // only happens in the ZSTD codec, so this modification is only made for ZSTD streams.
+ if (codec instanceof ZstandardCodec) {
+ decompressed = BytesInput.copy(BytesInput.from(is, uncompressedSize));
Review Comment:
I understand we had the discussion in the Jira that ByteInput.copy() just loads into a heap in advance but not add extra overall. Can we have a benchmark on the heap/GC(Heap size, GC time etc). I just want to make sure we fix one problem while introducing another problem.
Other than that, the ZSTD is treated especially might be OK since we had pretty decent coments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] zhongyujiang commented on a diff in pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by GitBox <gi...@apache.org>.
zhongyujiang commented on code in PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#discussion_r939787073
##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java:
##########
@@ -109,7 +110,12 @@ public BytesInput decompress(BytesInput bytes, int uncompressedSize) throws IOEx
decompressor.reset();
}
InputStream is = codec.createInputStream(bytes.toInputStream(), decompressor);
- decompressed = BytesInput.from(is, uncompressedSize);
+ if (codec instanceof ZstandardCodec) {
Review Comment:
This looks a little weird, but considering doing so will load the decompressor stream into heap in advance, and only zstd has this problem currently, so I made this modification only for zstd stream.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] shangxinli commented on a diff in pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by GitBox <gi...@apache.org>.
shangxinli commented on code in PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#discussion_r963046106
##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java:
##########
@@ -109,7 +110,17 @@ public BytesInput decompress(BytesInput bytes, int uncompressedSize) throws IOEx
decompressor.reset();
}
InputStream is = codec.createInputStream(bytes.toInputStream(), decompressor);
- decompressed = BytesInput.from(is, uncompressedSize);
+
+ // We need to explicitly close the ZstdDecompressorStream here to release the resources it holds to avoid
+ // off-heap memory fragmentation issue, see https://issues.apache.org/jira/browse/PARQUET-2160.
+ // This change will load the decompressor stream into heap a little earlier, since the problem it solves
+ // only happens in the ZSTD codec, so this modification is only made for ZSTD streams.
+ if (codec instanceof ZstandardCodec) {
+ decompressed = BytesInput.copy(BytesInput.from(is, uncompressedSize));
Review Comment:
Thanks for working on it!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] shangxinli commented on a diff in pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by GitBox <gi...@apache.org>.
shangxinli commented on code in PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#discussion_r950883464
##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java:
##########
@@ -109,7 +110,17 @@ public BytesInput decompress(BytesInput bytes, int uncompressedSize) throws IOEx
decompressor.reset();
}
InputStream is = codec.createInputStream(bytes.toInputStream(), decompressor);
- decompressed = BytesInput.from(is, uncompressedSize);
+
+ // We need to explicitly close the ZstdDecompressorStream here to release the resources it holds to avoid
+ // off-heap memory fragmentation issue, see https://issues.apache.org/jira/browse/PARQUET-2160.
+ // This change will load the decompressor stream into heap a little earlier, since the problem it solves
+ // only happens in the ZSTD codec, so this modification is only made for ZSTD streams.
+ if (codec instanceof ZstandardCodec) {
+ decompressed = BytesInput.copy(BytesInput.from(is, uncompressedSize));
Review Comment:
I understand we had the discussion in the Jira that ByteInput.copy() just loads into a heap in advance but not add extra overall. Can we have a benchmark on the heap/GC(Heap size, GC time etc). I just want to make sure we fix one problem while introducing another problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] alexeykudinkin commented on pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#issuecomment-1377589036
Totally @shangxinli
We have running Spark clusters in production _ingesting_ from 100s of Apache Hudi tables (using Parquet and Zstd) and writing into other ones. We switched from gzip to zstd slightly over a month ago and we started to have OOM issues almost immediately. It took us a bit of triaging to zero in on zstd, but now we're confident that it's not mis-calibration of our configs but slow-bleeding leak of the native memory.
The crux of the problem is very particular type of the job -- one that reads a lot of Zstd compressed Parquet (therefore triggering the affected path). Other jobs not reading Parquet are not affected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] shangxinli merged pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by GitBox <gi...@apache.org>.
shangxinli merged PR #982:
URL: https://github.com/apache/parquet-mr/pull/982
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] camper42 commented on pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by GitBox <gi...@apache.org>.
camper42 commented on PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#issuecomment-1253370864
wondering why this PR has not been merged
we have face this problem and are looking for a solution (spark 3.2.2+zstd-parquet)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] pan3793 commented on pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#issuecomment-1435989086
I also encountered this memory leak when migrating data from parquet/snappy to parquet/zstd, Spark executors always occupy unreasonable off-heap memory and have a high risk of being killed by NM, this patch can solve this problem. It's a critical issue in my case, wish the upstream publish a patched version so that we don't need to maintain the internal version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] Kimahriman commented on pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by GitBox <gi...@apache.org>.
Kimahriman commented on PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#issuecomment-1379431375
same, we have certain jobs that can't function without a patched jar. Seems to get worse with the more columns you read. Our worst offender (table with thousands of columns), can easily hit 90+ GiB of resident memory on a 25 GiB heap executor in an hour or so (running on Yarn without memory limit enforcement)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] zhongyujiang commented on a diff in pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by GitBox <gi...@apache.org>.
zhongyujiang commented on code in PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#discussion_r953866331
##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java:
##########
@@ -109,7 +110,17 @@ public BytesInput decompress(BytesInput bytes, int uncompressedSize) throws IOEx
decompressor.reset();
}
InputStream is = codec.createInputStream(bytes.toInputStream(), decompressor);
- decompressed = BytesInput.from(is, uncompressedSize);
+
+ // We need to explicitly close the ZstdDecompressorStream here to release the resources it holds to avoid
+ // off-heap memory fragmentation issue, see https://issues.apache.org/jira/browse/PARQUET-2160.
+ // This change will load the decompressor stream into heap a little earlier, since the problem it solves
+ // only happens in the ZSTD codec, so this modification is only made for ZSTD streams.
+ if (codec instanceof ZstandardCodec) {
+ decompressed = BytesInput.copy(BytesInput.from(is, uncompressedSize));
Review Comment:
> Can we have a benchmark on the heap/GC(Heap size, GC time etc).
Sure, will do a benchmark.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] sunchao commented on pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by GitBox <gi...@apache.org>.
sunchao commented on PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#issuecomment-1253950567
cc @ggershinsky @gszadovszky @shangxinli
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] zhongyujiang commented on pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by "zhongyujiang (via GitHub)" <gi...@apache.org>.
zhongyujiang commented on PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#issuecomment-1612535139
@wangzzu I guess it's because ZSTD allocates and uses off-heap memory very frequently, and before this PR the release of those native resources is subject to Java's GC, so I tried to relieve off-heap memory pressure by releasing them in time.
I didn't dig any deeper as my knowledge of these codec is very limited.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] sunchao commented on a diff in pull request #982: PARQUET-2160: Close ZstdInputStream to free off-heap memory in time.
Posted by GitBox <gi...@apache.org>.
sunchao commented on code in PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#discussion_r944925723
##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java:
##########
@@ -109,7 +110,12 @@ public BytesInput decompress(BytesInput bytes, int uncompressedSize) throws IOEx
decompressor.reset();
}
InputStream is = codec.createInputStream(bytes.toInputStream(), decompressor);
- decompressed = BytesInput.from(is, uncompressedSize);
+ if (codec instanceof ZstandardCodec) {
Review Comment:
The change looks OK to me, we probably should add some comments explaining why ZSTD deserves the special treatment here.
The change on `BytesInput` looks more intrusive since it is used not only for decompression but other places like compression. For instance, `BytesInput.copy` calls `toByteArray` underneath, and after the call the original object should still be valid.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org