You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/04/08 21:27:32 UTC
[GitHub] [iceberg] Fokko opened a new pull request #2441: Move to Parquet 1.12.0
Fokko opened a new pull request #2441:
URL: https://github.com/apache/iceberg/pull/2441
Changelog:
https://github.com/apache/parquet-mr/blob/master/CHANGES.md#version-1120
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] jshmchenxi commented on pull request #2441: Move to Parquet 1.12.0
Posted by GitBox <gi...@apache.org>.
jshmchenxi commented on pull request #2441:
URL: https://github.com/apache/iceberg/pull/2441#issuecomment-854321485
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] jshmchenxi commented on pull request #2441: Move to Parquet 1.12.0
Posted by GitBox <gi...@apache.org>.
jshmchenxi commented on pull request #2441:
URL: https://github.com/apache/iceberg/pull/2441#issuecomment-848627087
@kbendick Hi, you mentioned in the [comment](https://github.com/apache/iceberg/pull/2582#issuecomment-846325291) that spark versions should be concerned with parquet version upgrade. Who do you think would be appropriate to review this? The tests all passed though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] jshmchenxi commented on pull request #2441: Move to Parquet 1.12.0
Posted by GitBox <gi...@apache.org>.
jshmchenxi commented on pull request #2441:
URL: https://github.com/apache/iceberg/pull/2441#issuecomment-848620457
For #2582 add support for parquet bloom filter, we need to bump parquet version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko commented on pull request #2441: Move to Parquet 1.12.0
Posted by GitBox <gi...@apache.org>.
Fokko commented on pull request #2441:
URL: https://github.com/apache/iceberg/pull/2441#issuecomment-860461604
@rdblue Sure thing!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] jshmchenxi commented on pull request #2441: Move to Parquet 1.12.0
Posted by GitBox <gi...@apache.org>.
jshmchenxi commented on pull request #2441:
URL: https://github.com/apache/iceberg/pull/2441#issuecomment-854321485
@kbendick Sorry for the late reply, I've been busy with work. I'm new to Iceberg and should learn more. And thanks for telling me the right persons to review these!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #2441: Move to Parquet 1.12.0
Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #2441:
URL: https://github.com/apache/iceberg/pull/2441#issuecomment-861850669
Looks like there is a problem finding `parquet-avro`:
```
Execution failed for task ':iceberg-parquet:compileJava'.
> Could not resolve all files for configuration ':iceberg-parquet:compileClasspath'.
> Could not find org.apache.parquet:parquet-avro:1.12.1.
Searched in the following locations:
- https://repo.maven.apache.org/maven2/org/apache/parquet/parquet-avro/1.12.1/parquet-avro-1.12.1.pom
- https://repo.maven.apache.org/maven2/org/apache/parquet/parquet-avro/1.12.1/parquet-avro-1.12.1.jar
- file:/home/runner/.m2/repository/org/apache/parquet/parquet-avro/1.12.1/parquet-avro-1.12.1.pom
- file:/home/runner/.m2/repository/org/apache/parquet/parquet-avro/1.12.1/parquet-avro-1.12.1.jar
Required by:
project :iceberg-parquet
```
On maven central, it looks like 1.12.0 is the latest Parquet version. I misinterpreted @jshmchenxi's comment and though that it was released. Can you roll back to 1.12.0?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #2441: Move to Parquet 1.12.0
Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #2441:
URL: https://github.com/apache/iceberg/pull/2441#issuecomment-860126273
This looks fine to me, but there's a conflict in versions.props and I guess 1.12.1 is out. @Fokko, can you update this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] kbendick commented on pull request #2441: Move to Parquet 1.12.0
Posted by GitBox <gi...@apache.org>.
kbendick commented on pull request #2441:
URL: https://github.com/apache/iceberg/pull/2441#issuecomment-850117839
> @kbendick Hi, you mentioned in the [comment](https://github.com/apache/iceberg/pull/2582#issuecomment-846325291) that spark versions should be concerned with parquet version upgrade. Who do you think would be appropriate to review this? The tests all passed though.
@jshmchenxi - I mostly meant that it should be considered by some of the people who work closest with parquet (and also it should be ensured that it works with the more recent Spark 2.4.x and Spark 3.x versions we support).
Fortunately, @\ggershinsky is a very active Parquet committer (or possibly PMC now?) and has been doing some of the very hard work in parquet column level encryption, so if Gideon says we need this, then that's likely enough people to speak to. @\Fokko is also a parquet commiter.
Additionally, one of the PMCs / Vice President of this project, @rdblue is also a parquet PMC (in addition to a PMC or committer on what feels like every project and ), so we're in good hands in my book @jshmchenxi 👍. But thank you for considering my input. Just trying to make sure things stay sane, since it seems that some of the changes you've been putting up might not have been discussed on the dev mailing list (though I could totally have missed it). So I'm just ensuring that the right folks see potentially large changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] jshmchenxi commented on pull request #2441: Move to Parquet 1.12.0
Posted by GitBox <gi...@apache.org>.
jshmchenxi commented on pull request #2441:
URL: https://github.com/apache/iceberg/pull/2441#issuecomment-854324041
@ggershinsky @Fokko @rdblue Hi, shold we be moving on to parquet 1.12.0 for iceberg now? Or maybe wait for a new version like parquet 1.12.1?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ggershinsky commented on pull request #2441: Move to Parquet 1.12.0
Posted by GitBox <gi...@apache.org>.
ggershinsky commented on pull request #2441:
URL: https://github.com/apache/iceberg/pull/2441#issuecomment-848787433
we also need this for #2639
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org