You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "liferoad (via GitHub)" <gi...@apache.org> on 2025/05/23 13:53:39 UTC
[PR] upgrade org.apache.parquet:parquet-avro to to 1.15.2 [beam]
liferoad opened a new pull request, #35037:
URL: https://github.com/apache/beam/pull/35037
Fixes CVE-2025-46762
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] upgrade org.apache.parquet:parquet-avro to to 1.15.2 [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #35037:
URL: https://github.com/apache/beam/pull/35037#issuecomment-2960051647
it appears Java11 introduced new compiler check, and due to `-Werror` it fails the compilation
```
/runner/_work/beam/beam/sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/IcebergUtils.java:72: warning: [MissingCasesInEnumSwitch] Non-exhaustive switch; either add a default or handle the remaining cases: TIMESTAMP_NANO, GEOMETRY, GEOGRAPHY, VARIANT, UNKNOWN
switch (type.typeId()) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] upgrade org.apache.parquet:parquet-avro to to 1.15.2 [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #35037:
URL: https://github.com/apache/beam/pull/35037#discussion_r2114261459
##########
sdks/java/io/iceberg/build.gradle:
##########
@@ -37,8 +37,8 @@ def hadoopVersions = [
hadoopVersions.each {kv -> configurations.create("hadoopVersion$kv.key")}
-def iceberg_version = "1.6.1"
-def parquet_version = "1.15.1"
+def iceberg_version = "1.9.0"
Review Comment:
#35064 may help. After it gets checked in one can try to initiate an IcebergIO (w/wo managed) pipeline using Java8 to Dataflow. If it works we can bump the version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] upgrade org.apache.parquet:parquet-avro to to 1.15.2 [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on code in PR #35037:
URL: https://github.com/apache/beam/pull/35037#discussion_r2122409042
##########
sdks/java/io/iceberg/build.gradle:
##########
@@ -37,8 +37,8 @@ def hadoopVersions = [
hadoopVersions.each {kv -> configurations.create("hadoopVersion$kv.key")}
-def iceberg_version = "1.6.1"
-def parquet_version = "1.15.1"
+def iceberg_version = "1.9.0"
Review Comment:
switched back to Iceberg 1.9.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] upgrade org.apache.parquet:parquet-avro to to 1.15.2 [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35037:
URL: https://github.com/apache/beam/pull/35037#issuecomment-2958978969
Reminder, please take a look at this pr: @Abacn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] upgrade org.apache.parquet:parquet-avro to to 1.15.2 [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #35037:
URL: https://github.com/apache/beam/pull/35037#discussion_r2114254920
##########
sdks/java/io/iceberg/build.gradle:
##########
@@ -37,8 +37,8 @@ def hadoopVersions = [
hadoopVersions.each {kv -> configurations.create("hadoopVersion$kv.key")}
-def iceberg_version = "1.6.1"
-def parquet_version = "1.15.1"
+def iceberg_version = "1.9.0"
Review Comment:
there are other components depend on IcebergIO (e.g. managed) we may need to bump requireJavaVersion for all of them which makes the scope of change larger. If the old version is still compatible let's pin to i for now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] upgrade org.apache.parquet:parquet-avro to to 1.15.2 [beam]
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on code in PR #35037:
URL: https://github.com/apache/beam/pull/35037#discussion_r2114389488
##########
sdks/java/io/iceberg/build.gradle:
##########
@@ -37,8 +37,8 @@ def hadoopVersions = [
hadoopVersions.each {kv -> configurations.create("hadoopVersion$kv.key")}
-def iceberg_version = "1.6.1"
-def parquet_version = "1.15.1"
+def iceberg_version = "1.9.0"
Review Comment:
Managed does not depend on Iceberg btw. But I agree, we can wait until #35064 gets in
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] upgrade org.apache.parquet:parquet-avro to to 1.15.2 [beam]
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on code in PR #35037:
URL: https://github.com/apache/beam/pull/35037#discussion_r2104669414
##########
sdks/java/io/iceberg/build.gradle:
##########
@@ -37,8 +37,8 @@ def hadoopVersions = [
hadoopVersions.each {kv -> configurations.create("hadoopVersion$kv.key")}
-def iceberg_version = "1.6.1"
-def parquet_version = "1.15.1"
+def iceberg_version = "1.9.0"
Review Comment:
1.9.0 drops Java 8 support. Can we add `requireJavaVersion: JavaVersion.VERSION_11` to the `applyJavaNature()` at the top of this file?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] upgrade org.apache.parquet:parquet-avro to to 1.15.2 [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on code in PR #35037:
URL: https://github.com/apache/beam/pull/35037#discussion_r2138364186
##########
sdks/java/io/iceberg/build.gradle:
##########
@@ -37,8 +37,8 @@ def hadoopVersions = [
hadoopVersions.each {kv -> configurations.create("hadoopVersion$kv.key")}
-def iceberg_version = "1.6.1"
-def parquet_version = "1.15.1"
+def iceberg_version = "1.9.0"
Review Comment:
Make Java 11 required with Beam 2.66.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] upgrade org.apache.parquet:parquet-avro to to 1.15.2 [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on code in PR #35037:
URL: https://github.com/apache/beam/pull/35037#discussion_r2105654533
##########
sdks/java/io/iceberg/build.gradle:
##########
@@ -37,8 +37,8 @@ def hadoopVersions = [
hadoopVersions.each {kv -> configurations.create("hadoopVersion$kv.key")}
-def iceberg_version = "1.6.1"
-def parquet_version = "1.15.1"
+def iceberg_version = "1.9.0"
Review Comment:
https://lists.apache.org/thread/ot1xvcfmz35mdwh61fsx4pdhbm2697m6 let us wait for @Abacn to do this upgrade. let me try to pin the old version for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] upgrade org.apache.parquet:parquet-avro to to 1.15.2 [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on code in PR #35037:
URL: https://github.com/apache/beam/pull/35037#discussion_r2104651681
##########
sdks/java/io/iceberg/build.gradle:
##########
@@ -37,8 +37,8 @@ def hadoopVersions = [
hadoopVersions.each {kv -> configurations.create("hadoopVersion$kv.key")}
-def iceberg_version = "1.6.1"
-def parquet_version = "1.15.1"
+def iceberg_version = "1.9.0"
Review Comment:
iceberg 1.6.1 pins parquet-avro to 1.13.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] upgrade org.apache.parquet:parquet-avro to to 1.15.2 [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35037:
URL: https://github.com/apache/beam/pull/35037#issuecomment-2904855422
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] upgrade org.apache.parquet:parquet-avro to to 1.15.2 [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35037:
URL: https://github.com/apache/beam/pull/35037#issuecomment-2906346000
Assigning reviewers:
R: @Abacn for label java.
Note: If you would like to opt out of this review, comment `assign to next reviewer`.
Available commands:
- `stop reviewer notifications` - opt out of the automated review tooling
- `remind me after tests pass` - tag the comment author after tests pass
- `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
The PR bot will only process comments in the main thread (not review comments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] upgrade org.apache.parquet:parquet-avro to to 1.15.2 [beam]
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on PR #35037:
URL: https://github.com/apache/beam/pull/35037#issuecomment-2904537250
Can we also trigger iceberg [integration tests](https://github.com/apache/beam/blob/master/.github/trigger_files/IO_Iceberg_Integration_Tests.json)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] upgrade org.apache.parquet:parquet-avro to to 1.15.2 [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on PR #35037:
URL: https://github.com/apache/beam/pull/35037#issuecomment-2967030036
Plan to merge this after the tests are green for Beam 2.66.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] upgrade org.apache.parquet:parquet-avro to to 1.15.2 [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on PR #35037:
URL: https://github.com/apache/beam/pull/35037#issuecomment-2960896147
@Abacn looks like more modules have trouble to be compatible with Java 11 requirements for Iceberg IO.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] upgrade org.apache.parquet:parquet-avro to to 1.15.2 [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad merged PR #35037:
URL: https://github.com/apache/beam/pull/35037
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org