You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "SanjayMarreddi (via GitHub)" <gi...@apache.org> on 2025/03/17 16:05:38 UTC
[PR] AWS: Update the aws-bundle with latest dependencies [iceberg]
SanjayMarreddi opened a new pull request, #12553:
URL: https://github.com/apache/iceberg/pull/12553
This PR updates the `aws-bundle` with latest dependencies and the corresponding `LICENSE` and `NOTICE` files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] AWS: Update the aws-bundle with latest dependencies [iceberg]
Posted by "SanjayMarreddi (via GitHub)" <gi...@apache.org>.
SanjayMarreddi commented on code in PR #12553:
URL: https://github.com/apache/iceberg/pull/12553#discussion_r2016797737
##########
aws-bundle/build.gradle:
##########
@@ -36,6 +38,9 @@ project(":iceberg-aws-bundle") {
implementation "software.amazon.awssdk:sts"
implementation "software.amazon.awssdk:dynamodb"
implementation "software.amazon.awssdk:lakeformation"
+ implementation "software.amazon.awssdk:url-connection-client"
Review Comment:
Yeah, I agree. Removed it now..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] AWS: Update the aws-bundle with latest dependencies [iceberg]
Posted by "SanjayMarreddi (via GitHub)" <gi...@apache.org>.
SanjayMarreddi commented on code in PR #12553:
URL: https://github.com/apache/iceberg/pull/12553#discussion_r2021575246
##########
aws-bundle/build.gradle:
##########
@@ -25,8 +25,11 @@ project(":iceberg-aws-bundle") {
dependencies {
implementation platform(libs.awssdk.bom)
+ implementation libs.awssdk.s3accessgrants
+ implementation libs.awssdk.crt
Review Comment:
- CRT is needed to make use of the `analytics-accelerator-s3` library integration as the library itself won't include CRT in its JAR. So, I added `libs.awssdk.crt` earlier. But after your comment, I realised that we already have `software.amazon.awssdk:http-auth-aws-crt` which is sufficient too. So including `libs.awssdk.crt` is not compulsory. Removed it now.
- `libs.awssdk.s3accessgrants` and `software.amazon.awssdk:crt-core` are being used earlier also but not present in `aws-bundle`. We have decided to add missing dependencies too when I do a follow-up PR [here](https://github.com/apache/iceberg/pull/12299#discussion_r1989985092). I thought we should make `aws-bundle` include all necessary AWS vendor dependencies for user to have the best experience.
Please let me know if you have any questions/suggestions..
##########
gradle/libs.versions.toml:
##########
@@ -97,6 +98,7 @@ arrow-vector = { module = "org.apache.arrow:arrow-vector", version.ref = "arrow"
avro-avro = { module = "org.apache.avro:avro", version.ref = "avro" }
awssdk-bom = { module = "software.amazon.awssdk:bom", version.ref = "awssdk-bom" }
awssdk-s3accessgrants = { module = "software.amazon.s3.accessgrants:aws-s3-accessgrants-java-plugin", version.ref = "awssdk-s3accessgrants" }
+awssdk-crt = { module = "software.amazon.awssdk.crt:aws-crt", version.ref = "awssdk-crt" }
Review Comment:
I've addressed this above and removed now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] AWS: Update the aws-bundle with latest dependencies [iceberg]
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on code in PR #12553:
URL: https://github.com/apache/iceberg/pull/12553#discussion_r2013850856
##########
aws-bundle/build.gradle:
##########
@@ -36,6 +38,9 @@ project(":iceberg-aws-bundle") {
implementation "software.amazon.awssdk:sts"
implementation "software.amazon.awssdk:dynamodb"
implementation "software.amazon.awssdk:lakeformation"
+ implementation "software.amazon.awssdk:url-connection-client"
Review Comment:
Thats a compile only addition in `iceberg-aws` to specifically not include it in the runtime. Here we are including in the runtime. So, I am inclined towards removing this just here.
@jackye1995, @nastra : WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] AWS: Update the aws-bundle with latest dependencies [iceberg]
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on code in PR #12553:
URL: https://github.com/apache/iceberg/pull/12553#discussion_r2013850856
##########
aws-bundle/build.gradle:
##########
@@ -36,6 +38,9 @@ project(":iceberg-aws-bundle") {
implementation "software.amazon.awssdk:sts"
implementation "software.amazon.awssdk:dynamodb"
implementation "software.amazon.awssdk:lakeformation"
+ implementation "software.amazon.awssdk:url-connection-client"
Review Comment:
Thats a compile only addition in `iceberg-aws` to specifically not include it in the runtime. Here we are including in the runtime. So, I am inclined towards removing this.
@jackye1995, @nastra : WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] AWS: Update the aws-bundle with latest dependencies [iceberg]
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on code in PR #12553:
URL: https://github.com/apache/iceberg/pull/12553#discussion_r2013613026
##########
aws-bundle/LICENSE:
##########
@@ -443,6 +443,12 @@ License: Apache License, Version 2.0 - https://aws.amazon.com/apache2.0
--------------------------------------------------------------------------------
+Group: software.amazon.awssdk Name: url-connection-client Version: 2.30.31
Review Comment:
#12299 also added this dependency to for kafka connect module?
We should update the respective notice and license file in
a) https://github.com/apache/iceberg/blob/main/kafka-connect/kafka-connect-runtime/main/NOTICE
b) https://github.com/apache/iceberg/blob/main/kafka-connect/kafka-connect-runtime/hive/NOTICE
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] AWS: Update the aws-bundle with latest dependencies [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #12553:
URL: https://github.com/apache/iceberg/pull/12553#discussion_r2021223227
##########
gradle/libs.versions.toml:
##########
@@ -97,6 +98,7 @@ arrow-vector = { module = "org.apache.arrow:arrow-vector", version.ref = "arrow"
avro-avro = { module = "org.apache.avro:avro", version.ref = "avro" }
awssdk-bom = { module = "software.amazon.awssdk:bom", version.ref = "awssdk-bom" }
awssdk-s3accessgrants = { module = "software.amazon.s3.accessgrants:aws-s3-accessgrants-java-plugin", version.ref = "awssdk-s3accessgrants" }
+awssdk-crt = { module = "software.amazon.awssdk.crt:aws-crt", version.ref = "awssdk-crt" }
Review Comment:
why is this one needed now?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] AWS: Update the aws-bundle with latest dependencies [iceberg]
Posted by "jackye1995 (via GitHub)" <gi...@apache.org>.
jackye1995 commented on code in PR #12553:
URL: https://github.com/apache/iceberg/pull/12553#discussion_r1999408540
##########
aws-bundle/NOTICE:
##########
@@ -44,6 +45,8 @@ NOTICE for Group: software.amazon.awssdk Name: sdk-core Version: 2.30.31
NOTICE for Group: software.amazon.awssdk Name: sso Version: 2.30.31
NOTICE for Group: software.amazon.awssdk Name: sts Version: 2.30.31
NOTICE for Group: software.amazon.awssdk Name: utils Version: 2.30.31
+NOTICE for Group: software.amazon.s3.accessgrants Name: aws-s3-accessgrants-java-plugin Version: 2.3.0
+NOTICE for Group: software.amazon.s3.analyticsaccelerator Name: analyticsaccelerator-s3 Version: 1.0.0
Review Comment:
missing CRT?
##########
aws-bundle/LICENSE:
##########
@@ -443,6 +443,12 @@ License: Apache License, Version 2.0 - https://aws.amazon.com/apache2.0
--------------------------------------------------------------------------------
+Group: software.amazon.awssdk Name: url-connection-client Version: 2.30.31
+Project URL: https://aws.amazon.com/sdkforjava
+License: Apache License, Version 2.0 - https://aws.amazon.com/apache2.0
+
+--------------------------------------------------------------------------------
Review Comment:
missing CRT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] AWS: Update the aws-bundle with latest dependencies [iceberg]
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on PR #12553:
URL: https://github.com/apache/iceberg/pull/12553#issuecomment-2762966314
cc: @nastra, @danielcweeks, @jackye1995
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] AWS: Update the aws-bundle with latest dependencies [iceberg]
Posted by "SanjayMarreddi (via GitHub)" <gi...@apache.org>.
SanjayMarreddi commented on code in PR #12553:
URL: https://github.com/apache/iceberg/pull/12553#discussion_r1999793485
##########
aws-bundle/NOTICE:
##########
@@ -44,6 +45,8 @@ NOTICE for Group: software.amazon.awssdk Name: sdk-core Version: 2.30.31
NOTICE for Group: software.amazon.awssdk Name: sso Version: 2.30.31
NOTICE for Group: software.amazon.awssdk Name: sts Version: 2.30.31
NOTICE for Group: software.amazon.awssdk Name: utils Version: 2.30.31
+NOTICE for Group: software.amazon.s3.accessgrants Name: aws-s3-accessgrants-java-plugin Version: 2.3.0
+NOTICE for Group: software.amazon.s3.analyticsaccelerator Name: analyticsaccelerator-s3 Version: 1.0.0
Review Comment:
Actually it is already present: https://github.com/apache/iceberg/blob/main/aws-bundle/NOTICE#L20
##########
aws-bundle/build.gradle:
##########
@@ -36,6 +38,9 @@ project(":iceberg-aws-bundle") {
implementation "software.amazon.awssdk:sts"
implementation "software.amazon.awssdk:dynamodb"
implementation "software.amazon.awssdk:lakeformation"
+ implementation "software.amazon.awssdk:url-connection-client"
Review Comment:
Sure, I'm happy to remove it - I have added because it's present [here](https://github.com/apache/iceberg/blob/main/build.gradle#L462): Should we consider removing there too?
##########
aws-bundle/LICENSE:
##########
@@ -443,6 +443,12 @@ License: Apache License, Version 2.0 - https://aws.amazon.com/apache2.0
--------------------------------------------------------------------------------
+Group: software.amazon.awssdk Name: url-connection-client Version: 2.30.31
+Project URL: https://aws.amazon.com/sdkforjava
+License: Apache License, Version 2.0 - https://aws.amazon.com/apache2.0
+
+--------------------------------------------------------------------------------
Review Comment:
Actually, it is already present: https://github.com/apache/iceberg/blob/main/aws-bundle/LICENSE#L358
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] AWS: Update the aws-bundle with latest dependencies [iceberg]
Posted by "jackye1995 (via GitHub)" <gi...@apache.org>.
jackye1995 commented on code in PR #12553:
URL: https://github.com/apache/iceberg/pull/12553#discussion_r1999408056
##########
aws-bundle/build.gradle:
##########
@@ -36,6 +38,9 @@ project(":iceberg-aws-bundle") {
implementation "software.amazon.awssdk:sts"
implementation "software.amazon.awssdk:dynamodb"
implementation "software.amazon.awssdk:lakeformation"
+ implementation "software.amazon.awssdk:url-connection-client"
Review Comment:
I think the sync client defaults to apache client, so we don't need to add this URL connection client
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] AWS: Update the aws-bundle with latest dependencies [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #12553:
URL: https://github.com/apache/iceberg/pull/12553#discussion_r2021218257
##########
aws-bundle/build.gradle:
##########
@@ -25,8 +25,11 @@ project(":iceberg-aws-bundle") {
dependencies {
implementation platform(libs.awssdk.bom)
+ implementation libs.awssdk.s3accessgrants
+ implementation libs.awssdk.crt
Review Comment:
can you please elaborate why those dependencies were not needed earlier and are needed now? I don't see any CI failing without those
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] AWS: Update the aws-bundle with latest dependencies [iceberg]
Posted by "SanjayMarreddi (via GitHub)" <gi...@apache.org>.
SanjayMarreddi commented on code in PR #12553:
URL: https://github.com/apache/iceberg/pull/12553#discussion_r2016798381
##########
aws-bundle/LICENSE:
##########
@@ -443,6 +443,12 @@ License: Apache License, Version 2.0 - https://aws.amazon.com/apache2.0
--------------------------------------------------------------------------------
+Group: software.amazon.awssdk Name: url-connection-client Version: 2.30.31
Review Comment:
Yeah, updated it now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] AWS: Update the aws-bundle with latest dependencies [iceberg]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko merged PR #12553:
URL: https://github.com/apache/iceberg/pull/12553
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org